No credit card required

Crawler Results Deduplicate

petr_cermak/crawler-results-deduplicate

No credit card required

This acts takes crawler execution results and deduplicates them.

You can access the Crawler Results Deduplicate programmatically from your own applications by using the Apify API. You can choose the language preference from below. To use the Apify API, you’ll need an Apify account and your API token, found in Integrations settings in Apify Console.

1echo '{}' |
2apify call petr_cermak/crawler-results-deduplicate --silent --output-dataset

Crawler Results Deduplicate API through CLI

The Apify CLI is the official tool that allows you to use Crawler Results Deduplicate locally, providing convenience functions and automatic retries on errors.

Install the Apify CLI

1npm i -g apify-cli
2apify login

Other API clients include:

Crawler Results Deduplicate API in Python

Crawler Results Deduplicate API in JavaScript

Crawler Results Deduplicate API

Developer

Petr Cermak

Actor metrics

1 monthly user
1 star
Created in Oct 2017
Modified over 3 years ago

Categories

Developer tools

Google Maps Scraper Orchestrator

lukaskrivka/google-maps-scraper-orchestrator

Run multiple locations and search terms together with parallel runs for maximum speed.

Lukáš Křivka

100

Google Maps Scraper

compass/crawler-google-places

Extract data from hundreds of Google Maps locations and businesses. Get Google Maps data including reviews, images, contact info, opening hours, location, popular times, prices & more. Export scraped data, run the scraper via API, schedule and monitor runs, or integrate with other tools.

Compass

74.7k

409

Website Content Crawler

apify/website-content-crawler

Crawl websites and extract text content to feed AI models, LLM applications, vector databases, or RAG pipelines. The Actor supports rich formatting using Markdown, cleans the HTML, downloads files, and integrates well with 🦜🔗LangChain, LlamaIndex, and the wider LLM ecosystem.

Apify

21.2k

472

Amazon Bestsellers Scraper

junglee/amazon-bestsellers

Scrape the Amazon Best Sellers categories and extract details on top 100 most popular items on Amazon. Download product name, price, URL, and thumbnail image. Best used on .com, .co.uk, .de, .fr, .es, and .it domains. Download your data in various formats: HTML table, JSON, CSV, Excel, and more.

Junglee

626

Web Scraper

apify/web-scraper

Crawls arbitrary websites using the Chrome browser and extracts data from pages using JavaScript code. The Actor supports both recursive crawling and lists of URLs and automatically manages concurrency for maximum performance. This is Apify's basic tool for web crawling and scraping.

Apify

68.6k

170

Cheerio Scraper

apify/cheerio-scraper

Crawls websites using raw HTTP requests, parses the HTML with the Cheerio library, and extracts data from the pages using a Node.js code. Supports both recursive crawling and lists of URLs. This actor is a high-performance alternative to apify/web-scraper for websites that do not require JavaScript.

Apify

5.2k

Puppeteer Scraper

apify/puppeteer-scraper

Crawls websites with the headless Chrome and Puppeteer library using a provided server-side Node.js code. This crawler is an alternative to apify/web-scraper that gives you finer control over the process. Supports both recursive crawling and list of URLs. Supports login to website.

Apify

4.2k

Legacy PhantomJS Crawler

apify/legacy-phantomjs-crawler

Replacement for the legacy Apify Crawler product with a backward-compatible interface. The actor uses PhantomJS headless browser to recursively crawl websites and extract data from them using a piece of front-end JavaScript code.

Apify

1.6k

Extended GPT Scraper

drobnikj/extended-gpt-scraper

Extract data from any website and feed it into GPT via the OpenAI API. Use ChatGPT to proofread content, analyze sentiment, summarize reviews, extract contact details, and much more.