Chrome Extension Scraper
2 hours trial then $19.99/month - No credit card required now
Chrome Extension Scraper
2 hours trial then $19.99/month - No credit card required now
Scrape Chrome Web Store extensions data including ratings, reviews, user counts, descriptions, and manifest details. Perfect for market research, competitor analysis, and tracking extension trends.
Error while Fetching Information about invalid JSON
Open
40.433Z Saved 32 items 2025-01-17T04:43:40.722Z Scrolling time...31 2025-01-17T04:43:41.090Z Error fetching data: SyntaxError: Unexpected token '', "{ 2025-01-17T04:43:41.092Z "update"... is not valid JSON 2025-01-17T04:43:41.094Z at JSON.parse () 2025-01-17T04:43:41.095Z at processData (file:///home/myuser/src/main.js:276:37) 2025-01-17T04:43:41.097Z at processArrayWith19Elements (file:///home/myuser/src/main.js:239:35) 2025-01-17T04:43:41.099Z at processArrayWith19Elements (file:///home/myuser/src/main.js:243:40) 2025-01-17T04:43:41.101Z at processArrayWith19Elements (file:///home/myuser/src/main.js:243:40) 2025-01-17T04:43:41.103Z at processArrayWith19Elements (file:///home/myuser/src/main.js:243:40) 2025-01-17T04:43:41.104Z at processArrayWith19Elements (file:///home/myuser/src/main.js:243:40) 2025-01-17T04:43:41.106Z at processArrayWith19Elements (file:///home/myuser/src/main.js:243:40) 2025-01-17T04:43:41.108Z at processArrayWith19Elements (file:///home/myuser/src/main.js:243:40) 2025-01-17T04:43:41.110Z at processArrayWith19Elements (file:///home/myuser/src/main.js:243:40) 2025-01-17T04:43:41.533Z Scrolling time...79 2025-01-17T04:43:42.256Z Saved 32 items 2025-01-17T04:43:42.496Z Scrolling time...129 2025-01-17T04:43:43.069Z Saved 32 items 2025-01-17T04:43:44.031Z Saved 32 items 2025-01-17T04:43:44.513Z Scrolling time...32
Hi, we just fixed this issue, pls check it out.
ishangirdhar
Hi Team,
I am facing 2 different challenges with the exported data:
Challenge 1:
I ran another run but on the privacy category of web store pages. I received way too many duplicate values (6774/17910) which means for some reason we are scraping the same extension id multiple times.
duplicate key error collection: chrome.extensions index: extension_id dup key: [_fts: "bmnlcjabgnpnenekpadlanbbkooimhnj", _ftsx: 1.1 } E11000 duplicate key error collection... More errors occurred, open the error log to view.
Challenge 2:
Could be related to malformed JSON which you have already fixed, but when I am importing the output into mongoDB I am getting
Large array detected Some of the imported documents contained unbounded arrays that may degrade efficiency
Quick Clarification:
6774/17910
Means 6774 Extension ID are unique out of 17910
ishangirdhar
Thank you for the great scrapper.
Do you think we could include extension's
- First published date, and
- Date when it was last updated
Which is available on Chrome Web Store Page
Honestly, I see the value in it, and would be okay to pay 20 dollars per month on an ongoing basis.
However, I am trying to figure out a way to identify different versions Because Chrome Extension ID remains same across versions.
Hi, the time field you mentioned, there is no data in the query results, you can only get it from the detail page, we just tried to introduce these fields, but there are performance issues that will cause it to be very slow. Maybe we could provide a separate actor to get the update details.
ishangirdhar
Noted. Thanks for looking into it.
ishangirdhar
Another Idea could we to separate the actor for just fetching the details from the extension page that isn't already available in the manifest.json file.
and, Second actor to look at the detailed manifest file which I believe would involve downloading the crx and unzipping it and reading the manifest file.
EasyApiActor Metrics
6 monthly users
-
2 stars
>99% runs succeeded
Created in Nov 2024
Modified 2 days ago
