Reed.co.uk Scraper avatar
Reed.co.uk Scraper

Pricing

$30.00/month + usage

Go to Store
Reed.co.uk Scraper

Reed.co.uk Scraper

lexis-solutions/reed-co-uk-scraper

Developed by

Lexis Solutions

Maintained by Community

The Reed.co.uk scraper is a web scraping tool that retrieves job postings from Reed.co.uk, a job search website in the UK 🇬🇧

5.0 (3)

Pricing

$30.00/month + usage

10

Monthly users

6

Runs succeeded

>99%

Last modified

6 months ago

TE

Anyway in the JSON (Using API) to Increase the RAM

Closed
testtheworkout opened this issue
10 months ago

{ "proxyConfiguration": { "useApifyProxy": false }, "startUrls": [ { "url": "https://www.reed.co.uk/jobs/data-jobs?salaryFrom=90000&contract=true&dateCreatedOffSet=lastthreedays&excludeSalaryDescriptions=16%2C32%2C64" } ], "workFromHome": false }

At the moment it's only using 2gb - I'd like to increase it 8gb as it's taken 5 to 10 times longer than my other scrapers to complete (for less records) and I think it's the RAM :)

lexis-solutions avatar

Hi, you can use the "memory" parameter of the Apify API to achieve that. More info: https://docs.apify.com/api/v2?utm_source=app#/reference/actors/run-actor-synchronously/run-task

TE

testtheworkout

10 months ago

Thank you for your quick response. If I'm honest, I'm not very technical, and I'm struggling to find how to do this in the documentation. Are you able to please share a new JSON input with the memory parameter set to 8GB included I can use? Thank you for your help on this.

lexis-solutions avatar

What tools do you use to trigger a run a scrape using the API? (Postman, Insomnia, curl, else)? Or you're using the Apify dashboard?

TE

testtheworkout

10 months ago

It uses a Python Script - I copy the JSON configuration from the dashboard into a config file (JSON) and then run it using Python in VS Code, which then triggers the script to run by an API. I think I just need the updated JSON values in the first message that I can then put into the Config file when I run the Python Script.

lexis-solutions avatar

As a clarification, the JSON in the first comment controls the actor's behaviour. The actor cannot set its own RAM limits, so providing them there makes no difference. What you want to control is Apify's behaviour, which can be done by modifying the API call to Apify. You could try getting help from

Closing the issue now, as it relates to Apify rather than the actor, feel free to re-open if anything else comes up. Have a great day!

Pricing

Pricing model

Rental 

To use this Actor, you have to pay a monthly rental fee to the developer. The rent is subtracted from your prepaid usage every month after the free trial period. You also pay for the Apify platform usage.

Free trial

1 day

Price

$30.00