Pricing

Pay per usage

Go to Store

BeautifulSoup Scraper

Try for free

Developed by

Apify

Crawls websites using raw HTTP requests. It parses the HTML with the BeautifulSoup library and extracts data from the pages using Python code. Supports both recursive crawling and lists of URLs. This Actor is a Python alternative to Cheerio Scraper.

4.4 (5)

Pricing

Pay per usage

Monthly users

Runs succeeded

>99%

Last modified

5 months ago

Developer tools

Back to issues Create new issue

Shopee scraper error

Closed

lojahub opened this issue

I'm trying to use the scraper to extract information about searching for items in the store, but without success. After some research, I noticed that the information on the page only loads after a few seconds. Any tips on how to resolve this?

URL example: https://shopee.com.br/search?keyword=chinelo

Jindřich Bär (jindrich.bar)

Hello and thank you for your interest in this Actor!

Actually, you just figured it out - this page is loading the content dynamically (with asynchronous requests through Javascript), which is something this scraper cannot do - that's also the reason why it's so fast (it doesn't execute client-side Javascript).

If you're looking for an Actor that can execute client-side scripts, check out our Web Scraper (https://console.apify.com/actors/moJRLRc85AitArpNN/). Cheers!

Add comment

Pricing

Pricing model

Pay per usage

This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage.

Cheerio Scraper

apify/cheerio-scraper

Crawls websites using raw HTTP requests, parses the HTML with the Cheerio library, and extracts data from the pages using a Node.js code. Supports both recursive crawling and lists of URLs. This actor is a high-performance alternative to apify/web-scraper for websites that do not require JavaScript.

Apify

7.1k

Web Scraper

apify/web-scraper

Crawls arbitrary websites using a web browser and extracts structured data from web pages using a provided JavaScript function. The Actor supports both recursive crawling and lists of URLs, and automatically manages concurrency for maximum performance.

Apify

80.4k

Python Example

apify/python-example

Example Actor written in Python, showing how to read the Actor input and push to the Actor's default dataset.

Apify

100

Example Selenium

apify/example-selenium

Example of loading a web page in headless Chrome using Selenium Webdriver.

Apify

278

Puppeteer Scraper

apify/puppeteer-scraper

Crawls websites with the headless Chrome and Puppeteer library using a provided server-side Node.js code. This crawler is an alternative to apify/web-scraper that gives you finer control over the process. Supports both recursive crawling and list of URLs. Supports login to website.

Apify

6.2k

Facebook Profile Scraper

lazyscraper/facebook-profile-scraper

This tool efficiently extracts public profile data from specified Facebook accounts, including: Email addresses (if publicly listed) 、Bio、User ID、Profile URL. Key Advantage: For public profiles, it delivers accurate follower/fan counts – a critical metric most competitors fail to provide reliably.

Lazy Scraper

Vanilla JS Scraper

mstephen190/vanilla-js-scraper

Scrape the web using familiar JavaScript methods! Crawls websites using raw HTTP requests, parses the HTML with the JSDOM package, and extracts data from the pages using Node.js code. Supports both recursive crawling and lists of URLs. This actor is a non jQuery alternative to CheerioScraper.

Matthias Stephens

457

Meta Data Extractor

dainty_screw/metadata-extractor-reliable-web-page-metadata-extraction

Metadata Extractor is your go-to tool for extracting meta-data from web pages. Using Cheerio, it parses HTML to extract titles, descriptions, authors, and more.Perfect for content managers and SEO experts.

codemaster devops

Selenium Custom Firefox POC

jancurn/selenium-custom-firefox

Uses Selenium to run custom build of Firefox that is harder to detect. The actor just saves a screenshot and snapshot of the HTML.

Jan Čurn

Send HTTP requests

reezuan/send-http-requests

Send HTTP requests (GET, POST, PUT, PATCH, DELETE) to any API endpoint with customizable headers, parameters, and body data. Perfect for Clay users seeking API functionality without the Explorer plan upgrade, or anyone needing simple HTTP request capabilities.