Ultimate News API

change log for v2.0 Beta

• this scraper is still in Beta mode, so you might find some bugs or issues please don't hesitate to report them though opening an issue here is my GitHub if you need to contact me

new features added

• allow specific agencies or even 1 agency for the scraper to scrape from them (if found)
• new Date Range feature it's way better now you can view it is documents downside
• better code and better speed improvements, you can scrape way more historical data now
• API Mode is faster, you can use it to extract html on you own instead of the scraper ( it will lower your costs a lot )
• better logging system for bugs and errors detection
• better and more accurate data collection
• more scrapers added to collect a lot more data over the web

how to use v2.0 Beta new features

• Date Range
  • you can enter a date range or enter today to get today's news - all to get any date (can scrape months old articles)
  • today = today's date articles
  • yesterday = yesterday's date articles
  • to scrape a specific date range use / to separate the 2 dates (only 2 dates are counted)
  • dates are always in the format YYYY-MM-DD
  • example: 2021-12-31/2022-01-01
  • to scrape a specific date enter it like this 2021-12-31 and all articles will be in this format
  • this version also support these formats (1s, 2m, 3h, 1d, 1w, 5M, 2y)
    • so, you can use them to scrape
    • 1s for 1 second old articles
    • 1m for 1 minute old articles
    • 1h for 1 hour old articles
    • 1d for 1 day old articles
    • 1w for 1 week old articles
    • 1M for 1 month old articles
    • 1y for 1 year for old articles
• Maximum articles amount
  • you set the amount to maximize the scraper output, but it scrapes around that number not the exact number so the scraper might find less than the value you set
• API mode
  • you can lower the scraper costs by scraping the news in HTML instead of letting it extract the text, and then you can extract the text manually
• Exclude specific agencies
  • set this to exclude any agency from scraping it separate agencies like this bbc, fox
  • this feature works like this if agency in exclude_agencies.split(','): skip article so please be careful with this setting
• Allow specific agencies
  • same as exclude but works in reverse you can allow 1 or 2 or way more specific news agencies to be scraped for example
  • bbc news, fox news this will allow only bbc and Fox News to be scraped
• Proxies
  • with this option you can set your own proxies from whatever provided, or you can use Apify proxies
  • it is very optional to use proxies this scraper doesn't need any proxies but if you face an issue with no articles being scraped then allow this actor to use some proxies as a test otherwise contact me here is my GitHub

more info

• this project took me over 55 hours of pure coding to finish please if you find any bugs or errors report to me and I will fix them all
• if you have ANY suggestions or questions please contact me here is my GitHub or open an issue on the scraper I will be very happy to maintain this scraper as long as I can
• thank you for using my project