1 day trial then $25.00/month - No credit card required now

Github Search Scraper

saswave/github-search-scraper

Try for free

1 day trial then $25.00/month - No credit card required now

Github search scraper. Get all data from search results list

Back to issues Create new issue

Fails on page 10 every time

Closed

aleksandrmoshkov opened this issue

Please see the actor run. I made a resurrect attempt, but it fails at the same page.

SASWAVE (saswave)

We are looking into it, probably anti bot protections

aleksandrmoshkov

I successfully added my cookies, which improved the process, but there were still some errors. In the end, I managed to run 100 pages. I think it would be helpful to include a line in the README stating that the antibot sometimes fails and that it's better to add a cookie.

SASWAVE (saswave)

We found the issue, nothing related to anti bot protections:

When not providing cookies, github sometimes returns 429 responses to slow the requests made to the server

Initially, we ask cookies only if you need to search a url with type=code (mandatory for the extraction to work)

Now you should be able to handle the 100 pages without cookies

Add comment

Developer

SASWAVE

Actor Metrics

3 monthly users
4 stars
89% runs succeeded
10 hours response time
Created in Mar 2024
Modified 2 months ago

Categories

Automation

Other

Lead generation

OSINT Scraper

epctex/osint-scraper

Harness the power of OSINT data with our advanced OSINT Scraper. Discover keywords and leaked information from platforms like Ideone, Dumpz, Github Gist, Pastebin, Pasteorg and Textbin. You can specify search terms, customize and retrieve OSINT data out of the box.

epctex

573

📩📍 Google Maps Email Extractor

lukaskrivka/google-maps-with-contact-details

Extract Google Maps contact details. Scrape websites of Google Maps places for contact details and get email addresses, website, location, address, zipcode, phone number, social media links. Export scraped data, run the scraper via API, schedule and monitor runs or integrate with other tools.

Lukáš Křivka

11.5k

339

Instagram Scraper

apify/instagram-scraper

Scrape and download Instagram posts, profiles, places, hashtags, photos, and comments. Get data from Instagram using one or more Instagram URLs or search queries. Export scraped data, run the scraper via API, schedule and monitor runs or integrate with other tools.

Apify

67.1k

765

Facebook Posts Scraper

apify/facebook-posts-scraper

Extract data from hundreds of Facebook posts from one or multiple Facebook pages and profiles. Get post URL, post text, page or profile URL, timestamp, number of likes, shares, comments, and more. Download the data in JSON, CSV, and Excel and use it in apps, spreadsheets, and reports.

Apify

16.8k

244

Contact Details Scraper

vdrmota/contact-info-scraper

Free email extractor and lead scraper to extract and download emails, phone numbers, Facebook, Twitter, LinkedIn, and Instagram profiles from any website. Extract contact information at scale from lists of URLs and download the data as Excel, CSV, JSON, HTML, and XML.

Vojta Drmota

26.5k

274

Google Maps Extractor

compass/google-maps-extractor

Extract data from hundreds of places fast. Scrape Google Maps by keyword, category, location, URLs & other filters. Get addresses, contact info, opening hours, popular times, prices, menus & more. Export scraped data, run the scraper via API, schedule and monitor runs, or integrate with other tools.

Compass

25k

590

Facebook Groups Scraper

apify/facebook-groups-scraper

Extract data from one or multiple public Facebook groups. Get group and post URLs, post text, comments, timestamp, likes and comments count, and basic commentator info. Download the data in JSON, CSV, and Excel and use it in apps, spreadsheets, and reports.

Apify

7.9k

Instagram Post Scraper

apify/instagram-post-scraper

Scrape Instagram posts. Just add one or more Instagram usernames and get your data in seconds including text, hashtags, mentions, comments, images, URLs, likes, locations, and metadata. Export scraped data, run the scraper via API, schedule and monitor runs or integrate with other tools.

Apify

23.6k

213

Instagram Reel Scraper

apify/instagram-reel-scraper

Scrape data from Instagram reels. Just add one or more Instagram usernames and get your data in seconds including hashtags, mentions, comments, images, likes, locations, and metadata. Export scraped data, run the scraper via API, schedule and monitor runs or integrate with other tools.

Apify

5.2k

157