# Email Scraper (`ib4ngz/email-scraper`)

This actor scrapes email addresses from a list of provided URLs. It recursively crawls pages, extracts unique emails, and stores them in a dataset. The actor supports DNS validation to ensure domain authenticity and allows filtering based on custom crawling depth.

- **URL**: https://beta-securitybyobscurity.apify.com/ib4ngz/email-scraper.md
- **Developed by:** [Iqbal R](https://beta-securitybyobscurity.apify.com/ib4ngz) (community)
- **Categories:** Automation, Developer tools, SEO tools
- **Stats:** 248 total users, 46 monthly users, 98.6% runs succeeded, 5 bookmarks
- **User rating**: No ratings yet

## Pricing

$10.00/month + usage

To use this Actor, you pay a monthly rental fee to the developer. The rent is subtracted from your prepaid usage every month after the free trial period.
You also pay for the Apify platform usage, which gets cheaper the higher Apify subscription plan you have.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#rental-actors

# README

## Email Scraper

This actor scrapes email addresses from a list of provided URLs. It recursively crawls pages, extracts unique email addresses, and stores them in a dataset. The actor supports DNS validation to ensure domain authenticity and allows filtering based on custom crawling depth. Only unique email addresses are saved, preventing duplicates during the scraping process.

### Features

- **Email Extraction**: Extracts email addresses from a list of provided URLs and recursively explores linked pages to gather additional emails.
- **Recursive Crawling**: Crawls web pages to a user-defined maximum depth, enabling thorough exploration while managing resource usage.
- **DNS Validation**: Validates email domains using DNS records to ensure authenticity and exclude invalid domains.
- **Unique Dataset**: Ensures only unique email addresses are saved by preventing duplicates during the crawling process.

### Input Schema

- **Start URLs** (required): A list of URLs to start crawling from.
- **Maximum Depth**: The maximum depth for crawling, defining how deep the crawler should explore.
- **DNS Lookup**: Option to enable or disable DNS validation for email domains.
- **Proxy Configuration**: Configuration settings for selecting and using proxies during crawling.
- **Minimum Concurrency**: The minimum number of concurrent requests or pages to process.
- **Maximum Concurrency**: The maximum number of concurrent requests or pages to process.

### Dataset Schema

- **email**: The extracted email address.
- **dnsLookup**: Indicates whether the email domain passed DNS validation.

### How to Use

1. **Set up the Actor**\
   Start by providing a list of URLs to begin the crawling process. You can either manually input the URLs or provide a list in the actor configuration.

2. **Configure the Input Parameters**

   - **Start URLs**: Provide the initial URLs from which the crawler will start.
   - **Maximum Depth**: Define how deep the crawler should explore.
   - **DNS Lookup**: Choose whether to validate email domains using DNS records.
   - **Proxy Configuration**: If necessary, configure the proxy settings for your crawler.
   - **Concurrency**: Adjust the minimum and maximum concurrency based on your needs.

3. **Run the Actor**\
   Once the input parameters are configured, run the actor to start the crawling process. The actor will crawl the pages, extract unique email addresses, and store the results in the dataset.

4. **View Results**\
   After the actor finishes running, you can view the extracted email addresses in the dataset. The data will be displayed in a table format with the following fields:

   - Email Address
   - DNS Lookup Status

5. **Export Data**\
   You can export the dataset for further processing or analysis. The results are saved in a structured format for easy integration with other tools.

6. **Modify Parameters**\
   Adjust the configuration and rerun the actor as needed to gather additional data or refine the crawling process.

### Conclusion

This actor provides an efficient solution for scraping and extracting unique email addresses from a list of URLs. It recursively crawls the provided pages, extracts emails, and stores them in a dataset. By respecting a defined maximum depth and supporting DNS validation, it ensures only authentic and relevant emails are captured.

The actor is optimized to prevent duplicates by saving only unique email addresses during the crawling process. This makes it a valuable tool for anyone looking to gather email data in a structured and efficient manner, while maintaining control over the types of emails collected.

# Actor input Schema

## `startUrls` (type: `array`):

URLs to start with.

## `maxDepth` (type: `integer`):

The maximum depth for crawling.

## `dnsLookup` (type: `boolean`):

Enable or disable DNS lookup validation for email domains.

## `crawlerType` (type: `string`):

Select the type of crawler to use.

## `proxyConfiguration` (type: `object`):

Select proxies to be used by your crawler.

## `minConcurrency` (type: `integer`):

The minimum number of concurrent requests or pages being processed.

## `maxConcurrency` (type: `integer`):

The maximum number of concurrent requests or pages being processed.

## Actor input object example

```json
{
  "startUrls": [
    {
      "url": "https://apify.com"
    }
  ],
  "maxDepth": 1,
  "dnsLookup": true,
  "crawlerType": "Crawlee + Cheerio",
  "minConcurrency": 1,
  "maxConcurrency": 10
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        {
            "url": "https://apify.com"
        }
    ],
    "crawlerType": "Crawlee + Cheerio"
};

// Run the Actor and wait for it to finish
const run = await client.actor("ib4ngz/email-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "startUrls": [{ "url": "https://apify.com" }],
    "crawlerType": "Crawlee + Cheerio",
}

# Run the Actor and wait for it to finish
run = client.actor("ib4ngz/email-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    {
      "url": "https://apify.com"
    }
  ],
  "crawlerType": "Crawlee + Cheerio"
}' |
apify call ib4ngz/email-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=ib4ngz/email-scraper",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

```json
{
    "openapi": "3.0.1",
    "info": {
        "title": "Email Scraper",
        "description": "This actor scrapes email addresses from a list of provided URLs. It recursively crawls pages, extracts unique emails, and stores them in a dataset. The actor supports DNS validation to ensure domain authenticity and allows filtering based on custom crawling depth.",
        "version": "1.1",
        "x-build-id": "8mN1czJgCP5mgFomj"
    },
    "servers": [
        {
            "url": "https://api.apify.com/v2"
        }
    ],
    "paths": {
        "/acts/ib4ngz~email-scraper/run-sync-get-dataset-items": {
            "post": {
                "operationId": "run-sync-get-dataset-items-ib4ngz-email-scraper",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK"
                    }
                }
            }
        },
        "/acts/ib4ngz~email-scraper/runs": {
            "post": {
                "operationId": "runs-sync-ib4ngz-email-scraper",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor and returns information about the initiated run in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK",
                        "content": {
                            "application/json": {
                                "schema": {
                                    "$ref": "#/components/schemas/runsResponseSchema"
                                }
                            }
                        }
                    }
                }
            }
        },
        "/acts/ib4ngz~email-scraper/run-sync": {
            "post": {
                "operationId": "run-sync-ib4ngz-email-scraper",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK"
                    }
                }
            }
        }
    },
    "components": {
        "schemas": {
            "inputSchema": {
                "type": "object",
                "required": [
                    "startUrls"
                ],
                "properties": {
                    "startUrls": {
                        "title": "Start URLs",
                        "type": "array",
                        "description": "URLs to start with.",
                        "items": {
                            "type": "object",
                            "required": [
                                "url"
                            ],
                            "properties": {
                                "url": {
                                    "type": "string",
                                    "title": "URL of a web page",
                                    "format": "uri"
                                }
                            }
                        }
                    },
                    "maxDepth": {
                        "title": "Maximum Depth",
                        "minimum": 1,
                        "type": "integer",
                        "description": "The maximum depth for crawling.",
                        "default": 1
                    },
                    "dnsLookup": {
                        "title": "DNS Lookup",
                        "type": "boolean",
                        "description": "Enable or disable DNS lookup validation for email domains.",
                        "default": true
                    },
                    "crawlerType": {
                        "title": "Crawler Type",
                        "enum": [
                            "Crawlee + Cheerio",
                            "Crawlee + Puppeteer + Chrome"
                        ],
                        "type": "string",
                        "description": "Select the type of crawler to use.",
                        "default": "Crawlee + Cheerio"
                    },
                    "proxyConfiguration": {
                        "title": "Proxy configuration",
                        "type": "object",
                        "description": "Select proxies to be used by your crawler."
                    },
                    "minConcurrency": {
                        "title": "Minimum Concurrency",
                        "minimum": 1,
                        "type": "integer",
                        "description": "The minimum number of concurrent requests or pages being processed.",
                        "default": 1
                    },
                    "maxConcurrency": {
                        "title": "Maximum Concurrency",
                        "minimum": 1,
                        "type": "integer",
                        "description": "The maximum number of concurrent requests or pages being processed.",
                        "default": 10
                    }
                }
            },
            "runsResponseSchema": {
                "type": "object",
                "properties": {
                    "data": {
                        "type": "object",
                        "properties": {
                            "id": {
                                "type": "string"
                            },
                            "actId": {
                                "type": "string"
                            },
                            "userId": {
                                "type": "string"
                            },
                            "startedAt": {
                                "type": "string",
                                "format": "date-time",
                                "example": "2025-01-08T00:00:00.000Z"
                            },
                            "finishedAt": {
                                "type": "string",
                                "format": "date-time",
                                "example": "2025-01-08T00:00:00.000Z"
                            },
                            "status": {
                                "type": "string",
                                "example": "READY"
                            },
                            "meta": {
                                "type": "object",
                                "properties": {
                                    "origin": {
                                        "type": "string",
                                        "example": "API"
                                    },
                                    "userAgent": {
                                        "type": "string"
                                    }
                                }
                            },
                            "stats": {
                                "type": "object",
                                "properties": {
                                    "inputBodyLen": {
                                        "type": "integer",
                                        "example": 2000
                                    },
                                    "rebootCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "restartCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "resurrectCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "computeUnits": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            },
                            "options": {
                                "type": "object",
                                "properties": {
                                    "build": {
                                        "type": "string",
                                        "example": "latest"
                                    },
                                    "timeoutSecs": {
                                        "type": "integer",
                                        "example": 300
                                    },
                                    "memoryMbytes": {
                                        "type": "integer",
                                        "example": 1024
                                    },
                                    "diskMbytes": {
                                        "type": "integer",
                                        "example": 2048
                                    }
                                }
                            },
                            "buildId": {
                                "type": "string"
                            },
                            "defaultKeyValueStoreId": {
                                "type": "string"
                            },
                            "defaultDatasetId": {
                                "type": "string"
                            },
                            "defaultRequestQueueId": {
                                "type": "string"
                            },
                            "buildNumber": {
                                "type": "string",
                                "example": "1.0.0"
                            },
                            "containerUrl": {
                                "type": "string"
                            },
                            "usage": {
                                "type": "object",
                                "properties": {
                                    "ACTOR_COMPUTE_UNITS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_WRITES": {
                                        "type": "integer",
                                        "example": 1
                                    },
                                    "KEY_VALUE_STORE_LISTS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_INTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_EXTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_SERPS": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            },
                            "usageTotalUsd": {
                                "type": "number",
                                "example": 0.00005
                            },
                            "usageUsd": {
                                "type": "object",
                                "properties": {
                                    "ACTOR_COMPUTE_UNITS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_WRITES": {
                                        "type": "number",
                                        "example": 0.00005
                                    },
                                    "KEY_VALUE_STORE_LISTS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_INTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_EXTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_SERPS": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}
```
