Website Content Crawler
No credit card required
Website Content Crawler
No credit card required
Crawl websites and extract text content to feed AI models, LLM applications, vector databases, or RAG pipelines. The Actor supports rich formatting using Markdown, cleans the HTML, downloads files, and integrates well with 🦜🔗LangChain, LlamaIndex, and the wider LLM ecosystem.
Do you want to learn more about this Actor?
Get a demoWebsite crawler ends successfully but crawls nothing
Opened 14 hours ago by innovum_admin, last comment 12 hours ago by innovum_admin
apify_client._errors.ApifyApiError: You must rent a paid Actor in order to run it.
Opened 18 hours ago by dresma_apify, last comment 13 hours ago by Jiří Spilka (jiri.spilka)
Use of sitemap makes crawler fail
Opened 2 days ago by fsmedat, last comment 2 days ago by fsmedat
scraper started failing
Opened 2 days ago by fsmedat, last comment 2 days ago by fsmedat
Failed crawling with 403 code
Opened 4 days ago by ballerine, last comment 4 days ago by ballerine
Can't crawl while logged in
Opened 5 days ago by rust_chimta, last comment 5 days ago by rust_chimta
Fails to scrape useful data from AWS documentation page
Opened 6 days ago by methodical, last comment 5 days ago by methodical
Are they any settings that help out with cookie prompts?
Opened 7 days ago by pumpkin_protractor, last comment 6 days ago by Oscar Rodriguez (Oscardz)
Scrolling functionality breaking our scrape job
Opened 8 days ago by developprotege, last comment 4 days ago by Jan Čurn (jancurn)
Malformed sitemap content and 429s
Opened 8 days ago by MavenAGI, last comment 8 days ago by MavenAGI
Hi my run didnt work
Opened 10 days ago by ballerine, last comment 7 days ago by ballerine
request for html output to keep only essential <img> outputs
Opened 10 days ago by confident_socket, last comment 10 days ago by Oscar Rodriguez (Oscardz)
Data not being pushed to Pinecone from WCC
Opened 12 days ago by Custombizio, last comment 10 days ago by Oscar Rodriguez (Oscardz)
My run doesn't enqueue any URLs
Opened 12 days ago by bhupeshchandra, last comment 12 days ago by bhupeshchandra
Limiting scraped pages (e.g. maxCrawlPages = 30) doesn't work
Opened 14 days ago by beaming_gauge, last comment 14 days ago by Jan Buchar (janbuchar)
My run doesn't work. I have 0 results
Opened 18 days ago by contact_plune, last comment 18 days ago by Jan Buchar (janbuchar)
Poor CPU utilization due to low usage limit
Opened 21 days ago by write2souvik, last comment 10 days ago by write2souvik
Crawling subdomains
Opened 22 days ago by gainful_governor, last comment 22 days ago by gainful_governor
Crawling takes longer when calling API vs on site
Opened 24 days ago by adi-kamaraj, last comment 24 days ago by Jan Buchar (janbuchar)
Correct Json body (API)
Opened 24 days ago by glovebubble, last comment 24 days ago by glovebubble
Apify- 3k monthly users
- 465 stars
- 99.9% runs succeeded
- 3.1 days response time
- Created in Mar 2023
- Modified 10 days ago
