Twitter Profile Search Scraper

ℹ️ This Twitter scraper only collects data that’s publicly available. This means data that’s accessible without logging in to Twitter and without accepting Twitter’s terms of use. Please note that if you accepted Twitter’s terms of use, your ability to scrape Twitter data may be limited. If that’s the case, please review the terms and make an informed decision yourself.

👤 What does Twitter Profile Search Scraper do?

The Twitter Profile Search Scraper is designed to scrape Twitter profiles and their tweets based on your search query in minutes.

🔭 Extract Twitter profile data from users on X.com at scale

🧚‍♀️ Extract official Twitter pages, fakes, and fan pages in one go.

👽 Get basic profile data including screen name, follower count, profile image URLs, date of creation, etc. as well as basic tweet data including text, tweet URL, media, favorite count, reply count, etc.

☄️ Get more than 1,400 results for free

⬇️ Download Twitter profile data in Excel, CSV, JSON, and other formats

🔭 Why scrape Twitter profiles?

X.com has 220 million active daily users and is a great source of data for social media research. Here are just some of the ways you could use that data:

• Track discussions about your brand, product, topic, country, or city
• Monitor your competitors and their popularity
• Keep an eye on new trends as they emerge
• Fight fake news by identifying the spreading pattern of misinformation
• Analyze consumer habits and develop new products or target underdeveloped niches
• Track sentiment to make sure your investments are protected

🧚‍♀️ What data can Twitter Profile Search Scraper extract?

Twitter Profile Search Scraper scrapes details from specified Twitter profiles as well as their tweets and extracts:

🔧 How to scrape user profiles from Twitter/X?

You can scrape all profiles that match your query. Just follow these few steps and you'll get your data in a few minutes.

1. Find Twitter Profile Search Scraper on Apify Store.
2. Click on Try for free button.
3. Enter your search query, then specify the number of user pages and tweets you want to scrape.
4. Click "Start" and wait for the data to be extracted.
5. Preview, download your Twitter Profile dataset in JSON, XML, CSV, Excel, or HTML, or export it via API.

💸 How much will scraping Twitter/X profiles cost you?

This scraper uses the Pay-per-result pricing model, so your costs can be easily calculated: it will cost you $3.5 to scrape 1,000 user profiles, so $0.0035 per item. Apify provides you with $5 free usage credits every month on the Apify Free plan, so you can scrape over 1,400 Twitter user profiles for free for those credits.

But if you need to extract data regularly, you should get an Apify subscription. We recommend our $49/month Starter plan — that one could get you 14,000 Twitter profiles every month.

⬇️ Input

The input for this Twitter Search Scraper should be search terms, number of tweets per profile, and the number of profiles you want to extract.

You can input data by filling out fields, using JSON, or programmatically via an API. For a full explanation of an input example in JSON, see the input tab.

⬆️ Output sample

The results will be wrapped into a dataset which you can find in the Output tab. Here's an excerpt from the Twitter dataset you'd get if you apply the input parameters above:

You can preview all the fields in the Storage tab and choose the format in which to export the Twitter search results you've extracted: JSON, CSV, Excel, or HTML table. Here below is the same sample dataset in JSON:

1{
2  "user": {
3    "blocking": false,
4    "created_at": "2020-02-29T21:41:31.000Z",
5    "default_profile_image": false,
6    "description": "https://t.co/lYWLrUvxVi 📊 Goals, assists, free kicks, hat-tricks, dribbles, key passes, match ratings, more! 👋\nShow your support 🙏 https://t.co/jFtzM4sJre",
7    "fast_followers_count": 0,
8    "favourites_count": 3778,
9    "follow_request_sent": false,
10    "followed_by": false,
11    "followers_count": 83203,
12    "following": false,
13    "friends_count": 302,
14    "has_custom_timelines": false,
15    "id": 0,
16    "id_str": "1233869375691730944",
17    "is_translator": false,
18    "listed_count": 213,
19    "location": "",
20    "media_count": 4200,
21    "name": "MessivsRonaldo.app",
22    "normal_followers_count": 83203,
23    "notifications": false,
24    "profile_banner_url": "https://pbs.twimg.com/profile_banners/1233869375691730944/1583013102",
25    "profile_image_url_https": "https://pbs.twimg.com/profile_images/1233871440912490496/BmAQ1975_normal.jpg",
26    "protected": false,
27    "screen_name": "mvsrapp",
28    "show_all_inline_media": false,
29    "statuses_count": 17051,
30    "time_zone": "",
31    "translator_type": "none",
32    "url": "https://t.co/XztQOpSQIn",
33    "utc_offset": 0,
34    "verified": false,
35    "withheld_in_countries": [],
36    "withheld_scope": "",
37    "is_blue_verified": true
38  },
39  "id": "1317510880632987650",
40  "conversation_id": "1317510880632987650",
41  "full_text": "‼️🚨 NOW LIVE 🚨‼️\n\n🆕 Full, searchable match histories 🆕\n\n👉 Every game. Every goal. Every assist. Every hat-trick, free kick and penalty. Even every shot, dribble and key pass for most games.\n\n👉 Over 25 filters to find the exact stat you want\n\n🔗 LINK: https://t.co/R0mMe7hBOx https://t.co/Cmr0D8y1s7",
42  "reply_count": 181,
43  "retweet_count": 213,
44  "favorite_count": 1677,
45  "hashtags": [],
46  "symbols": [],
47  "user_mentions": [],
48  "urls": [
49    {
50      "url": "https://t.co/R0mMe7hBOx",
51      "expanded_url": "https://www.messivsronaldo.app/match-histories/",
52      "display_url": "messivsronaldo.app/match-historie…"
53    }
54  ],
55  "media": [
56    {
57      "media_url": "https://pbs.twimg.com/media/EkizGZxWMAACg1J.jpg",
58      "type": "photo"
59    },
60    {
61      "media_url": "https://pbs.twimg.com/media/Eki0vglWMAUn6fz.jpg",
62      "type": "photo"
63    },
64    {
65      "media_url": "https://pbs.twimg.com/media/Eki0-7cXUAA7fIK.jpg",
66      "type": "photo"
67    },
68    {
69      "media_url": "https://pbs.twimg.com/media/Eki1CegWkAEHmbd.png",
70      "type": "photo"
71    }
72  ],
73  "url": "https://twitter.com/mvsrapp/status/1317510880632987650",
74  "created_at": "2020-10-17T17:00:46.000Z",
75  "quote_count": 21,
76  "is_quote_tweet": false,
77  "is_retweet": false,
78  "is_pinned": false,
79  "is_truncated": false,
80  "startUrl": "https://twitter.com/mvsrapp"
81},
82{
83  "user": {
84    "blocking": false,
85    "created_at": "2020-02-29T21:41:31.000Z",
86    "default_profile_image": false,
87    "description": "https://t.co/lYWLrUvxVi 📊 Goals, assists, free kicks, hat-tricks, dribbles, key passes, match ratings, more! 👋\nShow your support 🙏 https://t.co/jFtzM4sJre",
88    "fast_followers_count": 0,
89    "favourites_count": 3778,
90    "follow_request_sent": false,
91    "followed_by": false,
92    "followers_count": 83203,
93    "following": false,
94    "friends_count": 302,
95    "has_custom_timelines": false,
96    "id": 0,
97    "id_str": "1233869375691730944",
98    "is_translator": false,
99    "listed_count": 213,
100    "location": "",
101    "media_count": 4200,
102    "name": "MessivsRonaldo.app",
103    "normal_followers_count": 83203,
104    "notifications": false,
105    "profile_banner_url": "https://pbs.twimg.com/profile_banners/1233869375691730944/1583013102",
106    "profile_image_url_https": "https://pbs.twimg.com/profile_images/1233871440912490496/BmAQ1975_normal.jpg",
107    "protected": false,
108    "screen_name": "mvsrapp",
109    "show_all_inline_media": false,
110    "statuses_count": 17051,
111    "time_zone": "",
112    "translator_type": "none",
113    "url": "https://t.co/XztQOpSQIn",
114    "utc_offset": 0,
115    "verified": false,
116    "withheld_in_countries": [],
117    "withheld_scope": "",
118    "is_blue_verified": true
119  },
120  "id": "1700072575580180834",
121  "conversation_id": "1700072575580180834",
122  "full_text": "📊 Messi since 1st Jan 2018\n\n3⃣1⃣ Penalty Goals 🥅\n3⃣1⃣ Free Kick Goals 🎯 https://t.co/evwGgnrI9N",
123  "reply_count": 286,
124  "retweet_count": 2803,
125  "favorite_count": 32302,
126  "hashtags": [],
127  "symbols": [],
128  "user_mentions": [],
129  "urls": [],
130  "media": [
131    {
132      "media_url": "https://pbs.twimg.com/media/F5fe595W0AARbSw.jpg",
133      "type": "photo"
134    }
135  ],
136  "url": "https://twitter.com/mvsrapp/status/1700072575580180834",
137  "created_at": "2023-09-08T09:04:22.000Z",
138  "quote_count": 334,
139  "is_quote_tweet": false,
140  "is_retweet": false,
141  "is_pinned": false,
142  "is_truncated": false,
143  "startUrl": "https://twitter.com/mvsrapp"
144},
145{
146  "user": {
147    "blocking": false,
148    "created_at": "2020-02-29T21:41:31.000Z",
149    "default_profile_image": false,
150    "description": "https://t.co/lYWLrUvxVi 📊 Goals, assists, free kicks, hat-tricks, dribbles, key passes, match ratings, more! 👋\nShow your support 🙏 https://t.co/jFtzM4sJre",
151    "fast_followers_count": 0,
152    "favourites_count": 3778,
153    "follow_request_sent": false,
154    "followed_by": false,
155    "followers_count": 83203,
156    "following": false,
157    "friends_count": 302,
158    "has_custom_timelines": false,
159    "id": 0,
160    "id_str": "1233869375691730944",
161    "is_translator": false,
162    "listed_count": 213,
163    "location": "",
164    "media_count": 4200,
165    "name": "MessivsRonaldo.app",
166    "normal_followers_count": 83203,
167    "notifications": false,
168    "profile_banner_url": "https://pbs.twimg.com/profile_banners/1233869375691730944/1583013102",
169    "profile_image_url_https": "https://pbs.twimg.com/profile_images/1233871440912490496/BmAQ1975_normal.jpg",
170    "protected": false,
171    "screen_name": "mvsrapp",
172    "show_all_inline_media": false,
173    "statuses_count": 17051,
174    "time_zone": "",
175    "translator_type": "none",
176    "url": "https://t.co/XztQOpSQIn",
177    "utc_offset": 0,
178    "verified": false,
179    "withheld_in_countries": [],
180    "withheld_scope": "",
181    "is_blue_verified": true
182  },
183  "id": "1596608370429300736",
184  "conversation_id": "1596608370429300736",
185  "full_text": "🎖️ Lionel Messi becomes the first man ever to assist in 5⃣ different World Cups!\n\n🏆 2022 🅰️🆕\n🏆 2018 🅰️🅰️\n🏆 2014 🅰️\n🏆 2010 🅰️\n🏆 2006 🅰️ https://t.co/XXHeSnlcHP",
186  "reply_count": 147,
187  "retweet_count": 3583,
188  "favorite_count": 22734,
189  "hashtags": [],
190  "symbols": [],
191  "user_mentions": [],
192  "urls": [],
193  "media": [
194    {
195      "media_url": "https://pbs.twimg.com/media/FihKt0bXgAID-Jd.jpg",
196      "type": "photo"
197    }
198  ],
199  "url": "https://twitter.com/mvsrapp/status/1596608370429300736",
200  "created_at": "2022-11-26T20:54:33.000Z",
201  "quote_count": 236,
202  "is_quote_tweet": false,
203  "is_retweet": false,
204  "is_pinned": false,
205  "is_truncated": false,
206  "startUrl": "https://twitter.com/mvsrapp"
207},

🐦 Want more tools for scraping Twitter?

🦄 Want to find all social media handles of the same profile?

Try Social Media Finder. Give it a search term such as a profile name, and you will be able to collect the same profile data across various social media platforms such as YouTube, Facebook, Discord, TikTok, LinkedIn, Instagram, Pinterest, Medium and more.

❓FAQ

Can I integrate Twitter Profile Search Scraper with other apps?

Yes. This Twitter Search Scraper can be connected with almost any cloud service or web app thanks to integrations on the Apify platform. You can integrate your Twitter Profiles data with Zapier, Slack, Make, Airbyte, GitHub, Google Drive, Asana, LangChain and more.

You can also use webhooks to carry out an action whenever an event occurs, e.g., get a notification whenever Twitter Profile Search Scraper successfully finishes a run.

Can I use Twitter Profile Search Scraper as its own API?

Yes, you can use the Apify API to access Twitter Profile Search Scraper programmatically. The API allows you to manage, schedule, and run Apify Actors, access datasets, monitor performance, get results, create and update Actor versions, and more.

To access the API using Node.js, you can use the apify-client NPM package. To access the API using Python, you can use the apify-client PyPI package.

For detailed information and code examples, see the API tab or refer to the Apify API documentation.

Can I use this Twitter Search API in Python?

Yes, you can use the Apify API with Python. To access the Twitter Profiles Search API with Python, use the apify-client PyPI package. You can find more details about the client in our Python Client documentation.

Is it legal to scrape X.com?

Note that personal data is protected by GDPR in the European Union and by other regulations around the world. You should not scrape personal data unless you have a legitimate reason to do so. If you're unsure whether your reason is legitimate, consult your lawyers. We also recommend that you read our blog post: Is web scraping legal?

Not your cup of tea? Build your own Twitter web scraper.

Twitter Profile Search Scraper doesn’t exactly do what you need? You can always build your own! We have various scraper templates in Python, JavaScript, and TypeScript to get you started. Alternatively, you can write it from scratch using our open-source library Crawlee. You can keep the scraper to yourself or make it public by adding it to Apify Store (and find users for it).

Your feedback

We’re always working on improving the performance of our Actors. So if you’ve got any technical feedback for Twitter Profile Search Scraper or simply found a bug, please create an issue on the Actor’s Issues tab.