Quora Search and Content Scraper

Overview

Uncover valuable insights and expert opinions across a wide range of topics! This Quora scraper extracts detailed information on questions, answers, and engagement metrics, providing a comprehensive view of discussions on the platform. Save time on research and gain deep understanding of public sentiment with ease.

How it works

This actor allows you to scrape questions and answers from Quora.com, extracting important details such as question title, creation time, answer count, and full answer content. It provides valuable insights into discussions and expert opinions across various topics.

Features

• Multiple Search Queries: Supports scraping based on multiple search URLs or topics, formatted as:
  • https://quora.com/search?q=your_query
  • https://www.quora.com/search?q=artificial%20intelligence
  • https://www.quora.com/search?q=web3
• Individual Question Scraping: Now supports scraping of individual question pages with all their answers, e.g.:
  • https://www.quora.com/What-can-you-say-about-the-current-state-of-Philippine-Education-and-why
• Detailed Question Information: Extracts comprehensive data about each question, including unique identifiers, creation time.
• Full Answer Content: Retrieves the complete text of answers, providing in-depth insights on each topic.
• Answer Metrics: Extracts engagement metrics such as upvotes and view counts for each answer.
• Author Information: Retrieves non-sensitive author details like profile URL and name.
• Required Authentication over cookies: Users must log in to access a broader range of questions and answers.
• Customizable: Flexible configuration options to suit various scraping needs.
• Data Extraction: Efficiently scrapes questions, answers and other relevant data from Quora.

How to Use

1. Set Up: Ensure you have an Apify account and access to the Apify platform.
2. Configure Input: Set up your search queries or individual question URLs (see Input Data section).
3. Authentication: Quora has made some changes so in order to access data from search you need to have account on it or to be logged in. You can input your cookies from quora.com site to access the data. Cookies are used to authorize actor with quora.com. Install EditThisCookie chrome extension. Login to your quora.com account. Click on the extension and export the quora.com cookies and insert into the input.
4. (Optional) Configure proxy settings for enhanced reliability and to avoid potential IP blocks.
5. Run the Scraper: Execute the scraper on the Apify platform.
6. Data Collection: The scraper will output data in your chosen format, which can include JSON, HTML, CSV, Excel, or others as supported by Apify.

Input Data

Here's an example of how to set up the input for the Quora scraper:

For Search Queries:

1{
2    "startUrls": [
3        {
4            "url": "https://www.quora.com/topic/Artificial-Intelligence"
5        },
6        {
7            "url": "https://www.quora.com/What-can-you-say-about-the-current-state-of-Philippine-Education-and-why"
8        }
9    ],
10    "proxyConfiguration": {
11        "useApifyProxy": true
12    }
13}

For Direct Questions:

1{
2    "startUrls": [
3        {
4            "url": "https://www.quora.com/What-can-you-say-about-the-current-state-of-Philippine-Education-and-why"
5        }
6    ],
7    "proxyConfiguration": {
8        "useApifyProxy": true
9    }
10}

Output Structure

The output data typically includes:

For search results:

1{
2    "index": 1,
3    "qid": 101059049,
4    "id": "UXVlc3Rpb25AMDoxMDEwNTkwNDk=",
5    "url": "https://www.quora.com/What-is-Web3-and-why-does-it-matter",
6    "title": "What is Web3, and why does it matter?",
7    "creationTime": "2021-09-27T06:20:50.336Z",
8    "answerCount": 19,
9    "answers": "Gavin Wood, a co-founder of Ethereum, first talked about Web3 in 2014. At the core of Web3 enthusiasts put the decentralization and democratization of the Internet. It's especially funny to watch how Web2 companies are now clamoring for Web3 and promoting fancy NFTs without changing the foundation. Although the basis of Web2 companies has always been profit in the first place. But first things first.\n\n]"
10}

For individual question pages:

1{
2    "aid": 105101498,
3    "id": "QW5zd2VyQDA6MTA1MTAxNDk4",
4    "url": "https://www.quora.com/Whats-wrong-with-our-country%E2%80%99s-education-system/answer/JK-Wijaya",
5    "content": "Nothing can ruin a country more than corrupt education system...",
6    "creationTime": "2018-10-23T09:36:18.029Z",
7    "numUpvotes": 8,
8    "numViews": 4025,
9    "qid": 14949586,
10    "isAuthorAnon": false,
11    "uid": 262922285,
12    "profileUrl": "https://www.quora.com/profile/JK-Wijaya",
13    "names": [
14        {
15            "givenName": "JK Wijaya",
16            "familyName": ""
17        }
18    ]
19}

Support

• For issues or feature requests, please use the Issues section of this actor.
• If you need customization or have questions, feel free to contact the author:
  • Author's website: https://muhamed-didovic.github.io/
  • Email: muhamed.didovic@gmail.com

Additional Services

• Request customization or whole dataset: muhamed.didovic@gmail.com
• If you need anything else scraped, or this actor customized, email: muhamed.didovic@gmail.com
• For API services of this scraper (no Apify fee, just usage fee for the API), contact: muhamed.didovic@gmail.com