Pricing

Pay per usage

Go to Store

Scrapy Books Example

Try for free

Developed by

Vlada Dusek

Example of Python Scrapy project. It scrapes book data from https://books.toscrape.com/.

0.0 (0)

Pricing

Pay per usage

Monthly users

Runs succeeded

>99%

Last modified

8 days ago

Developer tools

Open source

Back to issues Create new issue

Works perfectly in CLI :-), but not in Apify console :-(... ?!?

Open

GC7 (GC7) opened this issue

Hello, everyone! I am currently following the tutorial: https://www.youtube.com/watch?v=4nxStxC1BJM&list=PLObrtcm1Kw6PEnu5BpeEFb8XEoQXMw0g7&index=11

And at this part, dealing with the deployment of ‘booksscraper’: 05:33 Demo: building an Actor using templates and tools

I observe the following problem: It works perfectly in CLI, but bugs as an Actor in the Apify console…

Yet, I do get: Succeeded Success! Everything went according to plan.

While the log file shows: https://prnt.sc/ub4nDgcrr3ig

Of course, no result...

Would you have any idea?

Vlada Dusek (vdusek)

Hi, on the screenshot there is only a part of the log, I don't even see the whole traceback. Could you please provide the entire traceback and your source code and Actor configuration? As a text (not a screenshot) please, so I can reproduce it. Thanks.

GC7 (GC7)

Hi, Vlada.

First, thanks for your quick answer.

And of course, here the full log:

https://api.apify.com/v2/logs/N5dmgi7HGIgFqzYuc

And the .actor/actor.json:

1{
2    "actorSpecification": 1,
3    "name": "booksscraper",
4    "title": "Getting started with Python and Scrapy",
5    "description": "Python Scrapy project wrapped using the Apify CLI.",
6    "version": "0.0",
7    "meta": {
8        "templateId": "python-scrapy"
9    },
10    "input": "./input_schema.json",
11    "dockerfile": "./Dockerfile"
12}

Note I think I have correctly followed the process as video in youtube...

To soon

Vlada Dusek (vdusek)

Oh, I see the problem :).

builtins.NameError: Module 'apify.scrapy.middlewares' doesn't define any object named 'ApifyRetryMiddleware'

A custom ApifyRetryMiddleware is not needed anymore and was removed in the later versions of SDK.

So, you should just remove it from your settings.py file - I suppose you referencing it there.

You can also compare your (settings) code with our reference scrapy project -> https://github.com/apify/actor-scrapy-books-example.

Hope it helps!

Add comment

Pricing

Pricing model

Pay per usage

This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage.

APPLE Books Scraper

jupri/apple-books

📖 Scrape Apple Books

cat

Best Linkedin Jobs Scrapy

lads.yc/easy-linkedin-jobs-scrapy

Easy way to get jobs and details

YC W

📚 Goodreads Book Scraper

easyapi/goodreads-book-scraper

Extract comprehensive book data from Goodreads search results. Get detailed information about books, authors, ratings, and more. Perfect for market research, data analysis, and building book recommendation systems. 🔍📚

EasyApi

Python Example

apify/python-example

Example Actor written in Python, showing how to read the Actor input and push to the Actor's default dataset.

Apify

100

Porter Chapter Books Scraper API

lexiehan00/porter-chapter-books-scraper-api

Porter Chapter Books Scraper API

Olga Peteva

Example Public Actor Input

lukaskrivka/public-actor-input-example

Example of input schema for public scraping actors

Lukáš Křivka

Actor in Go example

jirimoravcik/go-actor-example

Example actor written in Go.

Jiří Moravčík

Ruby Example

drobnikj/ruby-example

Example of ruby code in actor

Jakub Drobník

Example Code Runner (Python)

apify/example-code-runner-python

Python Actor to run code examples from the documentation via "Run on Apify" links.

Apify

Goodreads Scraper

epctex/goodreads-scraper

Scrape goodreads.com for data on millions of books. Crawl book details for images, ISBN, author, description, title, buy links, number of reviews, page number, language, and all other details. You can specify search terms, filters, and much more.