Changelog
v3.8.X - Final Optimizations
Reduced concurrency for better stability
Optimized delay timings
Improved memory usage with Set-based URL tracking
Enhanced logging format for production use
Added request unique keys to prevent duplicates
Added multiple email/phone extraction
Improved regex patterns for German phone numbers
Better error handling for missing elements
Added graceful degradation for failed extractions
v3.6.0 - Title Cleaning
Removed status labels from titles (Reserviert, Gelöscht, Verkauft, Pausiert)
Cleaned bullet points from titles
Improved title formatting
v3.5.0 - Data Quality Improvements
Fixed CSS styles appearing in descriptions
Added description text cleaning
Implemented proper text extraction with innerText
Limited description length to 500 characters
Support for up to 100 results
Improved next page detection
Added page number logging
Better handling of disabled pagination buttons
v3.3.0 - State Management Fix
Fixed crawler.setState function error
Proper state management implementation
Added processed URLs tracking with Set
Prevented duplicate ad processing
v3.2.0 - Direct Search URL
Removed homepage navigation dependency
Direct navigation to search results page
Reduced crawling time by 50%
Fixed timeout issues on slow connections
v3.1.0 - Navigation Fix
Fixed search input selector issues
Added multiple selector strategies
Improved page load wait conditions
Added debug screenshots for troubleshooting
Complete rewrite of navigation logic
Direct URL construction for search results
Improved selector fallbacks
Added request timeout configurations
Enhanced logging with emojis for better readability
Added email and phone number extraction from descriptions
Implemented provider information extraction
Added ad ID and posting date extraction
Improved error handling and retry logic
Added GDPR banner handling
v1.0.0 - Initial Release
Basic scraper implementation for Kleinanzeigen.de
Search functionality with query input
Extraction of basic ad information (title, description, price)
Simple pagination support
Fingerprint injection for anti-bot protection