GPT Scraper

Input used

1{
2  "startUrls": [
3    {
4      "url": "https://play.google.com/store/apps/details?id=com.hardrockdigital.client&hl=en_US&gl=US"
5    }
6  ],
7  "maxPagesPerCrawl": 1,
8  "maxCrawlingDepth": 1,
9  "linkSelector": "a[href]",
10  "instructions": " \"I have a website link that provides information about a specific brand. \"\n          \"Please visit the link and extract detailed information about the brand.\n\"\n          \"Include the following details in your response:\n\n\"\n          \"Brand Name: The official name of the brand.\n\"\n          \"Overview: A brief description of what the brand is about, including its mission, vision, and values.\n\"\n          \"Products/Services: A list of main products or services offered by the brand.\n\"\n          \"Category of products: Provide only the name of categories of product\"\n          \"Unique selling points:\n\"\n          \"Target Audience: The primary demographic or market segment that the brand caters to.\n\"\n          \"Contact Information: How to get in touch with the brand, including customer service, social media handles, and physical addresses if available.\n\"\n          \"Website Link: The original website link provided for reference.\n\"\n          \"Social Media Handles:\n\n\"\n          Provide the output in JSON format with keys representing above points and values representing the details.\n           ",
11  "extractor": {
12    "type": "gpt",
13    "model": "gpt-4",
14    "maxTokens": 4000
15  },
16  "temperature": "0.1",
17  "top_p": "0.1",
18  "response_format": "json_object",
19  "includeUrlGlobs": [],
20  "excludeUrlGlobs": [],
21  "initialCookies": [],
22  "proxyConfiguration": {
23    "useApifyProxy": false
24  },
25  "topP": "1",
26  "frequencyPenalty": "0",
27  "presencePenalty": "0",
28  "removeElementsCssSelector": "script, style, noscript, path, svg, xlink",
29  "pageFormatInRequest": "Markdown",
30  "dynamicContentWaitSecs": 0,
31  "removeLinkUrls": false,
32  "saveSnapshots": true
33}

Hi, thanks for opening this issue!

I can see that you've found the original issue from a different user for the same website here - https://console.apify.com/actors/paOtbjvyUiNsr1Qms/issues/puLBJmEVQSvgzuXw7

Seeing more users having the same problem, we will prioritize this issue more :)

I will keep you updated here, thanks!

Hey Lukas,

Thanks for the response, can we do anything to put this on top of your priority list ?

Context: We are a bootstrapped startup just launched this product, expecting to see scrapings around 4-5K pages per day.

This a blocker for our business, we will be forced to look for alternatives if this doesn't work.

PS: We love your tool it has unlocked key capabilities for us. Would be happy to setup a chat with our engineering team, if that can help resolve the issue quicker.

I tried scraping the same url, with apify/playwright-scraper , seems to be working there.

Thanks for your insight, we will try to finish it this week :)

It's most likely just some very stupid bug in our backend that's somehow magically clashing with this very specific website, so it's just incredibly annoying to catch.