YouTube Channel Scraper

Ultra-fast YouTube channel scraper using Apify SDK v3.4.3 with advanced proxy management, memory optimization, and production-ready features.

Features

• High Performance: Concurrent scraping with configurable concurrency limits
• Full HD Thumbnails: Automatically selects highest quality thumbnails available
• Advanced Filtering: Filter by views, duration, date, and sorting options
• Proxy Support: Built-in Apify proxy integration with fallback mechanisms
• Memory Optimized: Efficient memory management for large-scale scraping
• Production Ready: Comprehensive monitoring, security, and error handling
• Clean Output: Properly formatted data with numeric views and ISO timestamps

Quick Start

Input Configuration

{
  "channelUrls": [
    "https://www.youtube.com/@MrBeast",
    "https://www.youtube.com/@PewDiePie"
  ],
  "maxVideos": 25,
  "sortBy": "newest",
  "includeMetadata": true,
  "outputOptions": {
    "includeChannelInfo": true,
    "includeThumbnails": true,
    "includeDescription": false
  }
}

Output Format

[
  {
    "url": "https://www.youtube.com/@MrBeast",
    "channelInfo": {
      "name": "MrBeast",
      "verified": true,
      "description": "Channel description...",
      "subscriberCount": "200M subscribers"
    },
    "videos": [
      {
        "id": "video123",
        "title": "Video Title",
        "url": "https://youtu.be/video123",
        "thumbnail": "https://i.ytimg.com/vi/video123/maxresdefault.jpg",
        "duration": "12:31",
        "views": "24604585",
        "publishedTime": "2025-06-27T01:40:00.849Z",
        "publishedTimeAgo": "1 month ago"
      }
    ]
  }
]

Configuration Options

Basic Options

Parameter	Type	Default	Description
channelUrls	Array	Required	List of YouTube channel URLs to scrape
maxVideos	Number	25	Maximum videos per channel
maxConcurrency	Number	10	Concurrent request limit
sortBy	String	"newest"	Sort order: newest, oldest, popular, relevance

Filtering Options

Parameter	Type	Default	Description
dateFilter	String	"all"	Filter by date: all, hour, today, week, month, year
minViews	Number	0	Minimum view count filter
maxDuration	Number	0	Maximum duration in seconds (0 = no limit)

Output Options

Parameter	Type	Default	Description
includeMetadata	Boolean	true	Include video metadata (views, duration, etc.)
outputOptions.includeChannelInfo	Boolean	true	Include channel information
outputOptions.includeThumbnails	Boolean	true	Include video thumbnails
outputOptions.includeDescription	Boolean	false	Include video descriptions

Proxy Configuration

Parameter	Type	Default	Description
proxyConfiguration.useApifyProxy	Boolean	true	Enable Apify proxy
proxyConfiguration.apifyProxyGroups	Array	["RESIDENTIAL"]	Proxy groups to use
proxyConfiguration.apifyProxyCountry	String	"US"	Proxy country code

Advanced Features

Retry Configuration

{
  "maxRetries": 3,
  "retryDelay": 1000
}

Memory Management

The scraper includes automatic memory management:

• Garbage collection triggers
• Memory usage monitoring
• Efficient data structures
• Streaming data processing

Error Handling

• Automatic retry with exponential backoff
• Proxy fallback mechanisms
• Comprehensive error logging
• Graceful degradation

Performance Optimization

Concurrency Settings

• Development: 2-5 concurrent requests
• Production: 10-20 concurrent requests
• High-scale: Up to 50 concurrent requests (with proper proxy setup)

Memory Limits

• Minimum: 1GB RAM
• Recommended: 2GB RAM
• High-scale: 4GB+ RAM

Deployment

Apify Platform

1. Upload the actor to Apify Console
2. Configure input parameters
3. Set memory limit (recommended: 2048MB)
4. Enable proxy configuration
5. Run the actor

Monitoring

The scraper includes comprehensive monitoring:

• Request success/failure rates
• Memory usage tracking
• Proxy performance metrics
• Scraping speed analytics
• Error rate monitoring

Security Features

• Input validation and sanitization
• Rate limiting protection
• Proxy rotation
• Security event logging
• Suspicious activity detection

Troubleshooting

Common Issues

1. Rate Limiting: Enable proxy configuration
2. Memory Issues: Increase memory limit or reduce concurrency
3. Timeout Errors: Adjust request timeout settings
4. Proxy Failures: Check proxy configuration and groups

Performance Tips

1. Use residential proxies for best results
2. Adjust concurrency based on available resources
3. Enable garbage collection for long-running tasks
4. Monitor memory usage and adjust limits accordingly

API Reference

Channel URL Formats

Supported YouTube channel URL formats:

• https://www.youtube.com/@username
• https://www.youtube.com/channel/CHANNEL_ID
• https://www.youtube.com/c/channelname
• https://www.youtube.com/user/username

Sort Options

• newest: Most recent videos first
• oldest: Oldest videos first
• popular: Most popular videos first
• relevance: YouTube's default relevance sorting

License

Apache-2.0

Support

For issues and questions:

1. Check the troubleshooting section
2. Review error logs in Apify Console
3. Contact support with detailed error information

---

Note: This scraper respects YouTube's terms of service and implements rate limiting to avoid overwhelming their servers. Use responsibly and in accordance with YouTube's API terms.