Website Content Crawler
No credit card required
Website Content Crawler
No credit card required
Crawl websites and extract text content to feed AI models, LLM applications, vector databases, or RAG pipelines. The Actor supports rich formatting using Markdown, cleans the HTML, downloads files, and integrates well with 🦜🔗LangChain, LlamaIndex, and the wider LLM ecosystem.
Do you want to learn more about this Actor?
Get a demoScrolling functionality breaking our scrape job
Open
Hi there,
We utilize the scroll down function to ensure all JS and content loads on the web page, but we are running into an issue where the nav bar on certain pages is blocking the visibile text that we use for AI analysis. Could you guys add a setting to scroll back up to the top of the page after zooming down, if screenshot mode is enabled please?
Hi there, thanks for reaching out. To understand this correctly, you're using Website Content Crawler to capture screenshots of webpages for the analysis, instead of the text content? Please can you share a link to the web page where this is an issue? What happens if you set higher scroll height?
Apify- 3k monthly users
- 465 stars
- 99.9% runs succeeded
- 3.1 days response time
- Created in Mar 2023
- Modified 10 days ago
