This actor checks results from Apify's scrapers or any other actor that stores its result to a dataset, and sends a notification if there are errors. It's designed to run from webhook.

• Usage
  • Actor/Task webhook
  • Actor
  • Legacy Crawler DEPRECATED
• Fields
  • actId
  • runId
  • datasetId
  • options
• Output

Usage

Actor or Task webhook

You can set up webhook for Actor or Task with URL:

https://api.apify.com/v2/acts/drobnikj~check-crawler-results/runs?token=APIFY_API_TOKEN

Then you need to set up payload template for webhook data like:

1{
2    "actId": {{resource.actId}},
3    "runId": {{resource.id}},
4    "options": {
5      "notifyTo": "jakub.drobnik@apify.com",
6      "minOutputtedPages": 10
7    }
8}

Actor

You can call it from other Actor, for example:

1await Apify.call('drobnikj/check-crawler-results', {
2    actId: 's7Jj8ik07gfV',
3    runId: 'sd86hGfHk0Uh6gF',
4    options: {
5        minOutputtedPages: 1000,
6    }
7});

Legacy Crawler DEPRECATED

For a specific crawler set the following parameters:

Finish webhook URL (finishWebhookUrl)

https://api.apify.com/v2/acts/drobnikj~check-crawler-results/runs?token=APIFY_API_TOKEN

Finish webhook data

You can set up fields from options to finish webhook data.

Fields

actId

• String
• Act ID you want to check

runId

• String
• Run ID of actor you want to check

datasetId

• String
• Dataset ID

options

• Object

options.sampleCount

• Number
• Number of results that actor checks
• Default is 100000

options.minOutputtedPages

• Number
• Indicates minimum outputted items to checks.

options.jsonSchema

• Object
• If jsonSchema is set actor check all sample results against schema.

options.compareWithPreviousExecution

• Boolean
• If compareWithPreviousExecution is set to true actor compare results with a previous execution.
• If tag for execution is set compare actor result from previous results with the same tag.
• It works only for the legacy crawler.

options.notifyTo

• String
• Mail where actor send notification if found error

options.runActOnSuccess

• Object
• If actor found errors runs this actor.
• Example:

1{
2    "id": "apify/send-mail",
3    "input": {
4        "to": "jakub.drobnik@apify.com",
5        "subject": "test on success",
6        "text": "No errors in crawler Amazon"
7    }
8}

NOTE: If you didn't set input, it set from input of main actor and errors output.

options.runActOnError

• Object
• If didn't find any errors runs this actor.
• Same format as runActOnSuccess

Output

All found errors will be in the errors field.

Example output

1{
2  "errors": [
3    "Run is not in SUCCEEDED status, act status: ABORTED",
4    "Crawler returns only 0 outputted pages and minumum is 100"
5  ],
6  "executionAttrs": []
7}