🏯 Tweet Scraper V2 (Pay Per Result) - X / Twitter Scraper
Pay $0.30 for 1,000 tweets
🏯 Tweet Scraper V2 (Pay Per Result) - X / Twitter Scraper
Pay $0.30 for 1,000 tweets
⚡️ Lightning-fast search, URL, list, and profile scraping, with customizable filters. At $0.30 per 1000 tweets, and 30-80 tweets per second, it is ideal for researchers, entrepreneurs, and businesses! Get comprehensive insights from Twitter (X) now!
Scraper a lot of duplicate tweet.
Closed
For this run:https://console.apify.com/actors/runs/hcZD7Xyb7rBkXEQj7 id 1833141861617221670 duplicated for 5 times
And for this run : https://console.apify.com/actors/61RPP7dywgiy0JPD0/runs/0GBM6p0QcgxdmVLl9 id:1833197842661576793 duplicated for 104 times.
That cause my scraper fees about 100 times higher
Attach is result of hcZD7Xyb7rBkXEQj7
Hello,
Our engineering team checked the runs and we couldn't find any issues on our end. Our scraper uses the query you give to it with Twitter and returns you whatever it can get from it. There is no additional logic, including removing duplicate. It never alters the output.
Twitter changes behaviour constantly and it is better to test your queries on twitter web UI before running the actor. Another reason can be using multiple from queries withORs. Are you sure that works with Twitter? If you are using ORs, I think the best approach would be to create separate runs for each profile.
Does that make sense?
Cheers!
relishable_finance
I tried agian with only one from, but there is many duplicate as well。 https://console.apify.com/actors/61RPP7dywgiy0JPD0/runs/8tXLNNdSegCRGCYue
relishable_finance
https://console.apify.com/actors/61RPP7dywgiy0JPD0/runs/kShmSwwrGcfhDee0V this run duplicate as well
topical_summer
results are duplicated 27 times for my last run.
Hey hey,
As I mentioned, this is something we have no control over. Our actor uses Twitter search and paginates as long as it can get the pagination. And when you try to fetch lots of tweets, this is inevitable since Twitter acts very weird with long paginations.
For fetching a profile, I suggest you to use a similar approach as we explained here:
1{ 2 "includeSearchTerms": false, 3 "onlyImage": false, 4 "onlyQuote": false, 5 "onlyTwitterBlue": false, 6 "onlyVerifiedUsers": false, 7 "onlyVideo": false, 8 "searchTerms": [ 9 "from:NASA since:2023-01-01 until:2023-02-01", 10 "from:NASA since:2023-02-01 until:2023-03-01", 11 "from:NASA since:2023-03-01 until:2023-04-01", 12 "from:NASA since:2023-04-01 until:2023-05-01", 13 "from:NASA since:2023-05-01 until:2023-06-01", 14 "from:NASA since:2023-06-01 until:2023-07-01", 15 "from:NASA since:2023-07-01 until:2023-08-01", 16 "from:NASA since:2023-08-01 until:2023-09-01", 17 "from:NASA since:2023-09-01 until:2023-10-01", 18 "from:NASA since:2023-10-01 until:2023-11-01", 19 "from:NASA since:2023-11-01 until:2023-12-01" 20 ], 21 "sort": "Latest", 22 "tweetLanguage": "en" 23}
That way, you will have less paginations and your results will have less duplicates.
Cheers!
relishable_finance
https://console.apify.com/actors/61RPP7dywgiy0JPD0/runs/sJdAPAajvSfCpQpud
I've tried this way.. but when there is no reuslt return. it still show 33 outputs.
Hello,
Yes, that is expected. When your query returns 0 results, you get a zeroResult object for us to cover the cost of the run. You can check the output of the dataset in order to seet his.
Cheers!
API Dojo- 1.4k monthly users
- 194 stars
- 97.7% runs succeeded
- 6.4 hours response time
- Created in Nov 2023
- Modified about 1 hour ago
