Web Scraper
No credit card required
Web Scraper
No credit card required
Crawls arbitrary websites using the Chrome browser and extracts data from pages using JavaScript code. The Actor supports both recursive crawling and lists of URLs and automatically manages concurrency for maximum performance. This is Apify's basic tool for web crawling and scraping.
Do you want to learn more about this Actor?
Get a demoScrape ignores all my startUrls except the first one
Closed
I'm passing a list of urls in "startUrls" but the scraper just stops after hitting the first one. Any idea why?
Hello, and thanks for reporting this!
The scraper is likely deduplicating the URLs since they only differ in the #fragment part.
To fix this, enable the "URL #fragments identify unique pages" option (or set keepUrlFragments to true if you run the Actor via the API). This should resolve the issue.
I'll close this issue now, but feel free to reach out if you need further help. Cheers!
Apify- 2.7k monthly users
- 167 stars
- 99.9% runs succeeded
- 6.9 days response time
- Created in Mar 2019
- Modified about 1 month ago
