Pricing

Pay per usage

Go to Store

Diff Datasets

Try for free

Developed by

Paulo Cesar

Take one dataset on Apify platform, compare it to another, and output the missing ones. This can also be used to output only changed items, using a compound key

0.0 (0)

Pricing

Pay per usage

Monthly users

Runs succeeded

50%

Last modified

3 years ago

Automation

Open source

Take one dataset on Apify platform, compare to another, and output the missing ones. This can also be used to output only changed items, using a compound key.

Supports using whole nested objects as value, they are JSON.stringify'd before being turned into a small non-cryptographic space efficient hash

Example

1await Apify.call('pocesar/diff-datasets', {
2    baseDatasetId: 'LdNAlaOY1aKGhwAah', // place the datasets here. The order of "base" and "other" matters
3    otherDatasetId: 'Bzu1pgOjenN43VhPY', // existing items in "base" are not output from "other"
4    uniqueFields: [
5        // simple primitive field value, like string, number, boolean
6        "pageUrl",
7
8        // you can use lodash.get notation to get nested items,
9        // in this case `sub.fields.0` works like `sub.fields[0]` and the object looks like
10        // {
11        //    pageUrl: "https//pageurl",
12        //    sub: {
13        //      fields: [
14        //        {...},
15        //        {...}
16        //      ]
17        //    }
18        //  }
19        "sub.fields.0",
20
21        // you can also use .length to count arrays or string characters, as in
22        "sub.fields.length",
23        "pageUrl.length"
24    ],
25});

Limitations

Every value is kept in memory while reading from the base dataset, more items more memory needed.
The key value store might choke when trying to save the in-memory Set with too many items

License

Apache 2.0

Pricing

Pricing model

Pay per usage

This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage.

Append to dataset

valek.josef/append-to-dataset

Utility actor that allows you to build a single large dataset from individual default datasets of other actor runs.

Josef Válek

LLM Dataset Processor

dusan.vystrcil/llm-dataset-processor

Allows you to process output of other actors or stored dataset with single LLM prompt. It's useful if you need to enrich data, summarize content, extract specific information, or manipulate data in a structured way using AI.

Dušan Vystrčil

Apify Actors

prog-party/apify-actors

This Apify Actors Actor retrieves data from Apify, allowing to filter, and returns a list of actors as a Dataset.

Prog Party

Delete Builds

useful-tools/delete-builds

Actor that deletes untagged builds of other actors on Apify platform.

Useful tools

CSV File to Dataset

lukaskrivka/csv-file-to-dataset

Upload a local or remote CSV/text file and convert it to Apify Dataset for further use.

Lukáš Křivka

DALL·E 3 AI Image Generator

scraping_samurai/dall-e-3-ai-image-generator

The DALL-E AI Image Generator lets you generate images from text prompts using the powerful DALL-E models. Simply describe the image you want, select the model, and generate high-quality visuals. Easily create anything from simple pictures to complex scenes with only $0,069 per Image!

Samurai

Wikipedia Scraper

kilerhg/Wikipedia

Wikipedia Scraper, designed and developed to make your life easier on searching for answers.

Lucas Nunes

240

Forward dataset as POST data

anchor/forward-dataset-webhook

This actor forwards the results of an Actor to an endpoint, instead of having to fetch the results manually. It will download the dataset and attach it to the body of a POST request you will specify. It acts as a new webhook. Simplify your Actor process !!!

Anchor

Zip Download of KV Store

useful-tools/downloadKvStoreZip

Creates a zip file from all items in the key-value store, zips them, and downloads them in a unique file. On input, you can specify the number of the files or keep it null to download them all. The key-value store must be under your Apify account.

Useful tools