Github Search Scraper
1 day trial then $25.00/month - No credit card required now
Github Search Scraper
1 day trial then $25.00/month - No credit card required now
Github search scraper. Get all data from search results list
Please see the actor run. I made a resurrect attempt, but it fails at the same page.
We are looking into it, probably anti bot protections
I successfully added my cookies, which improved the process, but there were still some errors. In the end, I managed to run 100 pages. I think it would be helpful to include a line in the README stating that the antibot sometimes fails and that it's better to add a cookie.
We found the issue, nothing related to anti bot protections:
When not providing cookies, github sometimes returns 429 responses to slow the requests made to the server
Initially, we ask cookies only if you need to search a url with type=code (mandatory for the extraction to work)
Now you should be able to handle the 100 pages without cookies
- 3 monthly users
- 3 stars
- 94.2% runs succeeded
- 0.76 hours response time
- Created in Mar 2024
- Modified 3 days ago