Web Scraper
No credit card required
Web Scraper
No credit card required
Crawls arbitrary websites using the Chrome browser and extracts data from pages using JavaScript code. The Actor supports both recursive crawling and lists of URLs and automatically manages concurrency for maximum performance. This is Apify's basic tool for web crawling and scraping.
Do you want to learn more about this Actor?
Get a democrawl yields no useful results
Open
I am trying to crawl the webpage in the run for all the end user manuals. Most of the messages coming back are this: 2024-08-20T15:14:30.889Z WARN PuppeteerCrawler: Reclaiming failed request back to the list or queue. requestHandler timed out after 60 seconds.
Can you recommend any settings I should try on the crawler to effectively grab the user manuals? I tried do scrape first with Website Content Crawler. This crawler appears to at least see more URLs than Website Content Crawler did and have asked if there are better settings on should try on that crawler as well.
Apify- 2.7k monthly users
- 167 stars
- 99.9% runs succeeded
- 6.9 days response time
- Created in Mar 2019
- Modified about 1 month ago
