# Google Review Scraper (`shahzeb-king/google-review-scraper`)

High-performance Google Reviews scraper built with Playwright & Node.js. Extract reviews, ratings, authors & dates from any Google business using place IDs. Optimized for speed with stealth mode, Docker-ready, Apify-compatible. Batch processing, JSON output, minimal delays.

- **URL**: https://beta-securitybyobscurity.apify.com/shahzeb-king/google-review-scraper.md
- **Developed by:** [Shahzeb Naveed](https://beta-securitybyobscurity.apify.com/shahzeb-king) (community)
- **Categories:** Automation, Lead generation, Agents
- **Stats:** 6 total users, 3 monthly users, 100.0% runs succeeded, NaN bookmarks
- **User rating**: No ratings yet

## Pricing

$15.00/month + usage

To use this Actor, you pay a monthly rental fee to the developer. The rent is subtracted from your prepaid usage every month after the free trial period.
You also pay for the Apify platform usage, which gets cheaper the higher Apify subscription plan you have.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#rental-actors

# README

## Google Review Scraper

A highly optimized Node.js application designed to scrape Google reviews with Playwright, built for compatibility with the Apify platform and Docker deployment. **Input-driven with no database dependencies.**

### Features

- 🚀 **High Performance**: Batch processing with single browser instance
- ⚡ **Speed Optimized**: Minimal delays (1-2s navigation, 0.1-0.8s scrolling)
- 🕸️ **Stealth Mode**: Advanced anti-detection techniques
- 🐳 **Docker Ready**: Full containerization support
- 🔄 **Apify Compatible**: Ready for Apify platform deployment
- 📥 **Input Driven**: Simply provide place IDs as input
- 📄 **JSON Output**: Results saved as structured JSON files
- 🎯 **Precise Parsing**: Extracts author, rating, date, and review text
- 🔀 **Human-like Behavior**: Randomized actions with minimal delays

### Performance Optimizations

This scraper is highly optimized for speed while maintaining reliability:

- **Reduced Delays**: Navigation delays reduced to 1-2 seconds, scroll delays to 0.3-0.8 seconds
- **Configurable Scrolling**: Default 5 scrolls (vs 8-15), customizable via `maxScrolls` input
- **Single Browser Instance**: Reuses browser context across multiple place IDs
- **Minimal Anti-Detection**: Just enough randomization to avoid detection without sacrificing speed
- **Fast Parsing**: Uses Cheerio for rapid HTML parsing
- **Batch Processing**: Processes multiple place IDs in a single session

### Prerequisites

- Node.js (version 18.0.0 or higher)
- Chrome browser (for local development)
- Docker & Docker Compose (for containerized deployment)

### Quick Start

#### Local Development

1. **Clone and setup:**

   ```bash
   git clone https://github.com/MuhammadShahzeb123/google-review-scraper-2.git
   cd google-review-scraper
   npm install
   ```

2. **Install Playwright browsers:**

   ```bash
   npx playwright install chromium
   ```

3. **Run with place IDs:**

   ```bash
   ## Method 1: Command line arguments
   node src/main.js ChIJN1t_tDeuEmsRUsoyG83frY4 ChIJGVtI4by3t4kRr51d_Qm_x58

   ## Method 2: Environment variable
   PLACE_IDS="ChIJN1t_tDeuEmsRUsoyG83frY4,ChIJGVtI4by3t4kRr51d_Qm_x58" npm start

   ## Method 3: Run example
   node example.js
   ```

#### Docker Deployment

1. **Using Docker Compose:**

   ```bash
   PLACE_IDS="ChIJN1t_tDeuEmsRUsoyG83frY4,ChIJGVtI4by3t4kRr51d_Qm_x58" docker-compose up
   ```

2. **Using Docker only:**

   ```bash
   docker build -t google-review-scraper .
   docker run --rm \
     -e PLACE_IDS="ChIJN1t_tDeuEmsRUsoyG83frY4,ChIJGVtI4by3t4kRr51d_Qm_x58" \
     -v $(pwd)/output:/app/output \
     google-review-scraper
   ```

#### Apify Platform

1. **Configure via Apify Input UI:**

   All settings are now configurable through the Apify input interface:

   - **Place IDs**: List of Google Place IDs to scrape
   - **Max Scrolls**: Number of scroll actions (1-20, default: 5)
   - **Headless Mode**: Run browser in background (default: true)
   - **Browser Timeout**: Page load timeout in ms (default: 60000)
   - **Cleanup HTML**: Delete temp files after processing (default: true)
   - **Log Level**: Logging verbosity (error/warn/info/debug)

2. **Push to Apify:**

   ```bash
   apify login
   apify push
   ```

3. **Run on Apify:**

   ```bash
   apify run
   ```

### Configuration

#### **Apify Platform (Recommended)**

All configuration is done through the **Apify Input UI** - no environment variables needed!

#### **Local/Standalone Environment Variables (Legacy)**

| Variable | Description | Default |
|----------|-------------|---------|
| `PLACE_IDS` | Comma-separated place IDs | Example IDs |
| `HEADLESS_MODE` | Browser headless mode | `false` |
| `BROWSER_TIMEOUT` | Page timeout (ms) | `60000` |
| `SCROLL_COUNT_MIN` | Min scroll actions | `8` |
| `SCROLL_COUNT_MAX` | Max scroll actions | `15` |

#### Input Methods

**1. Command Line Arguments:**

```bash
node src/main.js PLACE_ID_1 PLACE_ID_2 PLACE_ID_3
```

**2. Environment Variable:**

```bash
export PLACE_IDS="ChIJN1t_tDeuEmsRUsoyG83frY4,ChIJGVtI4by3t4kRr51d_Qm_x58"
npm start
```

**3. Programmatic Usage:**

```javascript
const { scrapeReviews } = require('./src/main');

const placeIds = ['ChIJN1t_tDeuEmsRUsoyG83frY4'];
const reviews = await scrapeReviews(placeIds);
```

**4. Apify Input:**

```json
{
  "placeIds": [
    "ChIJN1t_tDeuEmsRUsoyG83frY4",
    "ChIJGVtI4by3t4kRr51d_Qm_x58"
  ],
  "headlessMode": true,
  "scrollCount": 10
}
```

### Output Format

#### **For Apify Platform:**

Each review is stored as a separate dataset item for optimal Apify integration:

```json
{
  "reviewerName": "John Doe",
  "rating": 5,
  "reviewText": "Great place, highly recommended!",
  "reviewDate": "2 months ago",
  "placeId": "ChIJN1t_tDeuEmsRUsoyG83frY4", 
  "scrapedAt": "2025-06-16T12:00:00.000Z",
  "success": true
}
```

Summary statistics are saved in the OUTPUT key-value store:

```json
{
  "totalPlaceIds": 2,
  "totalReviews": 45,
  "successfulScrapes": 2,
  "failedScrapes": 0,
  "maxScrollsUsed": 5,
  "processedAt": "2025-06-16T12:00:00.000Z",
  "placeIds": ["ChIJN1t_tDeuEmsRUsoyG83frY4"],
  "scrapingResults": {"ChIJN1t_tDeuEmsRUsoyG83frY4": true}
}
```

#### **For Local/Standalone Usage:**

Results are saved as JSON files with nested structure:

```json
{
  "summary": {
    "totalPlaceIds": 2,
    "totalReviews": 45,
    "successfulScrapes": 2,
    "failedScrapes": 0,
    "processedAt": "2025-06-16T12:00:00.000Z",
    "placeIds": ["ChIJN1t_tDeuEmsRUsoyG83frY4"]
  },
  "reviews": [
    {
      "author": "John Doe",
      "stars": 5,
      "date": "2 months ago",
      "text": "Great place, highly recommended!",
      "placeId": "ChIJN1t_tDeuEmsRUsoyG83frY4",
      "scrapedAt": "2025-06-16T12:00:00.000Z",
      "success": true
    }
  ]
}
```

### Project Structure

```
├── src/
│   ├── main.js                 ## Main application logic
│   ├── scraper.js             ## Playwright browser automation
│   ├── google_reviews_parser.js ## HTML parsing with Cheerio
│   ├── config.js              ## Configuration management
│   └── apify-main.js          ## Apify-compatible version
├── example.js                 ## Usage example
├── Dockerfile                 ## Docker container configuration
├── docker-compose.yml         ## Container orchestration
├── package.json              ## Node.js dependencies
├── INPUT_SCHEMA.json          ## Apify input schema
└── .env.example              ## Environment template
```

### How It Works

1. **Input Processing**: Accepts place IDs from various sources
2. **Browser Automation**: Launches Chrome with stealth settings
3. **Navigation**: Visits Google Maps place pages for each place ID
4. **Review Loading**: Clicks Reviews tab, sorts by newest, scrolls to load more
5. **HTML Extraction**: Saves review container HTML to temporary files
6. **Parsing**: Uses Cheerio to extract structured review data
7. **Output**: Saves results as JSON and cleans up temporary files

### Advanced Features

#### Stealth Mode

- Custom user agent and viewport
- Disabled automation flags
- Random delays between actions
- Human-like scrolling patterns

#### Error Handling

- Graceful failure handling per place ID
- Screenshot capture on errors
- Comprehensive logging
- Automatic retry mechanisms

#### Performance Optimization

- Single browser instance for all place IDs
- Parallel file processing
- Efficient memory management
- Automatic cleanup of temporary files

### API Reference

#### `scrapeReviews(placeIds)`

Programmatically scrape reviews for given place IDs.

**Parameters:**

- `placeIds` (Array<string>): Array of Google Place IDs

**Returns:**

- `Promise<Array>`: Array of review objects with metadata

**Example:**

```javascript
const { scrapeReviews } = require('./src/main');

const reviews = await scrapeReviews([
  'ChIJN1t_tDeuEmsRUsoyG83frY4'
]);

console.log(`Found ${reviews.length} reviews`);
```

### Troubleshooting

#### Common Issues

1. **Browser not found:**

   ```bash
   npx playwright install chromium
   ```

2. **No place IDs provided:**
   - Ensure place IDs are valid Google Place IDs
   - Check input format (comma-separated for env vars)

3. **Permission errors in Docker:**
   - Application runs as non-root user for security
   - Check file permissions when mounting volumes

#### Finding Google Place IDs

1. Search for a business on Google Maps
2. Look at the URL: `...place/.../@...` or use the share link
3. Extract the place ID from the URL or use Google Places API

### Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Test thoroughly
5. Submit a pull request

### License

ISC License - see LICENSE file for details.

# Actor input Schema

## `placeIds` (type: `array`):

Array of Google Place IDs to scrape reviews from (e.g., ChIJN1t\_tDeuEmsRUsoyG83frY4)

## `sortOrder` (type: `string`):

Choose how to sort the reviews

## `maxScrolls` (type: `integer`):

Maximum number of scroll actions to load reviews (higher = more reviews but slower)

## `browserTimeout` (type: `integer`):

Maximum time to wait for pages to load (in seconds). Increase for slow websites.

## Actor input object example

```json
{
  "placeIds": [
    "ChIJN1t_tDeuEmsRUsoyG83frY4",
    "ChIJGVtI4by3t4kRr51d_Qm_x58"
  ],
  "sortOrder": "most_relevant",
  "maxScrolls": 5,
  "browserTimeout": 60
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "placeIds": [
        "ChIJN1t_tDeuEmsRUsoyG83frY4",
        "ChIJGVtI4by3t4kRr51d_Qm_x58"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("shahzeb-king/google-review-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "placeIds": [
        "ChIJN1t_tDeuEmsRUsoyG83frY4",
        "ChIJGVtI4by3t4kRr51d_Qm_x58",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("shahzeb-king/google-review-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "placeIds": [
    "ChIJN1t_tDeuEmsRUsoyG83frY4",
    "ChIJGVtI4by3t4kRr51d_Qm_x58"
  ]
}' |
apify call shahzeb-king/google-review-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=shahzeb-king/google-review-scraper",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

```json
{
    "openapi": "3.0.1",
    "info": {
        "title": "Google Review Scraper",
        "description": "High-performance Google Reviews scraper built with Playwright & Node.js. Extract reviews, ratings, authors & dates from any Google business using place IDs. Optimized for speed with stealth mode, Docker-ready, Apify-compatible. Batch processing, JSON output, minimal delays.",
        "version": "0.0",
        "x-build-id": "Xwi3gFOIC1VNrhPsB"
    },
    "servers": [
        {
            "url": "https://api.apify.com/v2"
        }
    ],
    "paths": {
        "/acts/shahzeb-king~google-review-scraper/run-sync-get-dataset-items": {
            "post": {
                "operationId": "run-sync-get-dataset-items-shahzeb-king-google-review-scraper",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK"
                    }
                }
            }
        },
        "/acts/shahzeb-king~google-review-scraper/runs": {
            "post": {
                "operationId": "runs-sync-shahzeb-king-google-review-scraper",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor and returns information about the initiated run in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK",
                        "content": {
                            "application/json": {
                                "schema": {
                                    "$ref": "#/components/schemas/runsResponseSchema"
                                }
                            }
                        }
                    }
                }
            }
        },
        "/acts/shahzeb-king~google-review-scraper/run-sync": {
            "post": {
                "operationId": "run-sync-shahzeb-king-google-review-scraper",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK"
                    }
                }
            }
        }
    },
    "components": {
        "schemas": {
            "inputSchema": {
                "type": "object",
                "required": [
                    "placeIds"
                ],
                "properties": {
                    "placeIds": {
                        "title": "Google Place IDs",
                        "minItems": 1,
                        "maxItems": 50,
                        "type": "array",
                        "description": "Array of Google Place IDs to scrape reviews from (e.g., ChIJN1t_tDeuEmsRUsoyG83frY4)",
                        "items": {
                            "type": "string"
                        }
                    },
                    "sortOrder": {
                        "title": "Sort Order",
                        "enum": [
                            "most_relevant",
                            "newest",
                            "highest_rating",
                            "lowest_rating"
                        ],
                        "type": "string",
                        "description": "Choose how to sort the reviews",
                        "default": "most_relevant"
                    },
                    "maxScrolls": {
                        "title": "Maximum Scrolls",
                        "minimum": 1,
                        "maximum": 20,
                        "type": "integer",
                        "description": "Maximum number of scroll actions to load reviews (higher = more reviews but slower)",
                        "default": 5
                    },
                    "browserTimeout": {
                        "title": "Browser Timeout (seconds)",
                        "minimum": 10,
                        "maximum": 180,
                        "type": "integer",
                        "description": "Maximum time to wait for pages to load (in seconds). Increase for slow websites.",
                        "default": 60
                    }
                }
            },
            "runsResponseSchema": {
                "type": "object",
                "properties": {
                    "data": {
                        "type": "object",
                        "properties": {
                            "id": {
                                "type": "string"
                            },
                            "actId": {
                                "type": "string"
                            },
                            "userId": {
                                "type": "string"
                            },
                            "startedAt": {
                                "type": "string",
                                "format": "date-time",
                                "example": "2025-01-08T00:00:00.000Z"
                            },
                            "finishedAt": {
                                "type": "string",
                                "format": "date-time",
                                "example": "2025-01-08T00:00:00.000Z"
                            },
                            "status": {
                                "type": "string",
                                "example": "READY"
                            },
                            "meta": {
                                "type": "object",
                                "properties": {
                                    "origin": {
                                        "type": "string",
                                        "example": "API"
                                    },
                                    "userAgent": {
                                        "type": "string"
                                    }
                                }
                            },
                            "stats": {
                                "type": "object",
                                "properties": {
                                    "inputBodyLen": {
                                        "type": "integer",
                                        "example": 2000
                                    },
                                    "rebootCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "restartCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "resurrectCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "computeUnits": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            },
                            "options": {
                                "type": "object",
                                "properties": {
                                    "build": {
                                        "type": "string",
                                        "example": "latest"
                                    },
                                    "timeoutSecs": {
                                        "type": "integer",
                                        "example": 300
                                    },
                                    "memoryMbytes": {
                                        "type": "integer",
                                        "example": 1024
                                    },
                                    "diskMbytes": {
                                        "type": "integer",
                                        "example": 2048
                                    }
                                }
                            },
                            "buildId": {
                                "type": "string"
                            },
                            "defaultKeyValueStoreId": {
                                "type": "string"
                            },
                            "defaultDatasetId": {
                                "type": "string"
                            },
                            "defaultRequestQueueId": {
                                "type": "string"
                            },
                            "buildNumber": {
                                "type": "string",
                                "example": "1.0.0"
                            },
                            "containerUrl": {
                                "type": "string"
                            },
                            "usage": {
                                "type": "object",
                                "properties": {
                                    "ACTOR_COMPUTE_UNITS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_WRITES": {
                                        "type": "integer",
                                        "example": 1
                                    },
                                    "KEY_VALUE_STORE_LISTS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_INTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_EXTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_SERPS": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            },
                            "usageTotalUsd": {
                                "type": "number",
                                "example": 0.00005
                            },
                            "usageUsd": {
                                "type": "object",
                                "properties": {
                                    "ACTOR_COMPUTE_UNITS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_WRITES": {
                                        "type": "number",
                                        "example": 0.00005
                                    },
                                    "KEY_VALUE_STORE_LISTS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_INTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_EXTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_SERPS": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}
```
