Forever21 Scraper

Full log:

2021-11-10T17:47:37.427Z at CheerioCrawler.handlePageFunction (/usr/src/app/src/main.js:85:52) 2021-11-10T17:47:37.427Z at CheerioCrawler._handleRequestFunction (/usr/src/app/node_modules/apify/build/crawlers/cheerio_crawler.js:434:50) 2021-11-10T17:47:37.428Z at processTicksAndRejections (internal/process/task_queues.js:97:5) 2021-11-10T17:47:37.428Z at async /usr/src/app/node_modules/apify/build/utils.js:411:20 2021-11-10T17:47:37.572Z INFO: Processing: "https://www.forever21.com/us/shop/catalog/category/f21/activewear" 2021-11-10T17:47:37.573Z ERROR: BasicCrawler: handleRequestFunction failed, reclaiming failed request back to the list or queue {"url":"https://www.forever21.com/us/shop/catalog/category/f21/activewear","retryCount":1,"id":"cpSm5YI4HuElv7r"} 2021-11-10T17:47:37.574Z TypeError: Cannot read property 'indexOf' of null 2021-11-10T17:47:37.575Z at extractSubcatPage (/usr/src/app/src/extractors.js:50:33) 2021-11-10T17:47:37.575Z at CheerioCrawler.handlePageFunction (/usr/src/app/src/main.js:85:52) 2021-11-10T17:47:37.576Z at CheerioCrawler._handleRequestFunction (/usr/src/app/node_modules/apify/build/crawlers/cheerio_crawler.js:434:50) 2021-11-10T17:47:37.576Z at processTicksAndRejections (internal/process/task_queues.js:97:5) 2021-11-10T17:47:37.577Z at async /usr/src/app/node_modules/apify/build/utils.js:411:20 2021-11-10T17:47:38.126Z INFO: Processing: "https://www.forever21.com/us/shop/catalog/category/f21/lingerie" 2021-11-10T17:47:38.132Z ERROR: BasicCrawler: handleRequestFunction failed, reclaiming failed request back to the list or queue {"url":"https://www.forever21.com/us/shop/catalog/category/f21/lingerie","retryCount":2,"id":"XPB20HPiuBbxkWV"} 2021-11-10T17:47:38.136Z TypeError: Cannot read property 'indexOf' of null 2021-11-10T17:47:38.137Z at extractSubcatPage (/usr/src/app/src/extractors.js:50:33) 2021-11-10T17:47:38.137Z at CheerioCrawler.handlePageFunction (/usr/src/app/src/main.js:85:52) 2021-11-10T17:47:38.138Z at CheerioCrawler._handleRequestFunction (/usr/src/app/node_modules/apify/build/crawlers/cheerio_crawler.js:434:50) 2021-11-10T17:47:38.138Z at processTicksAndRejections (internal/process/task_queues.js:97:5) 2021-11-10T17:47:38.139Z at async /usr/src/app/node_modules/apify/build/utils.js:411:20 2021-11-10T17:47:38.155Z INFO: Processing: "https://www.forever21.com/us/shop/catalog/category/plus/plus_size-tops" 2021-11-10T17:47:38.156Z ERROR: BasicCrawler: handleRequestFunction failed, reclaiming failed request back to the list or queue {"url":"https://www.forever21.com/us/shop/catalog/category/plus/plus_size-tops","retryCount":1,"id":"z6a1JfnDv8EKyfZ"} 2021-11-10T17:47:38.157Z TypeError: Cannot read property 'indexOf' of null 2021-11-10T17:47:38.157Z at extractSubcatPage (/usr/src/app/src/extractors.js:50:33) 2021-11-10T17:47:38.158Z at CheerioCrawler.handlePageFunction (/usr/src/app/src/main.js:85:52) 2021-11-10T17:47:38.159Z at CheerioCrawler._handleRequestFunction (/usr/src/app/node_modules/apify/build/crawlers/cheerio_crawler.js:434:50) 2021-11-10T17:47:38.159Z at processTicksAndRejections (internal/process/task_queues.js:97:5) 2021-11-10T17:47:38.160Z at async /usr/src/app/node_modules/apify/build/utils.js:411:20 2021-11-10T17:47:38.419Z INFO: Processing: "https://www.forever21.com/us/shop/catalog/category/f21/top_blouses" 2021-11-10T17:47:38.420Z ERROR: BasicCrawler: handleRequestFunction failed, reclaiming failed request back to the list or queue {"url":"https://www.forever21.com/us/shop/catalog/category/f21/top_blouses","retryCount":2,"id":"6ezVIYHB2z3p60Q"} 2021-11-10T17:47:38.421Z TypeError: Cannot read property 'indexOf' of null 2021-11-10T17:47:38.421Z at extractSubcatPage (/usr/src/app/src/extractors.js:50:33) 2021-11-10T17:47:38.422Z at CheerioCrawler.handlePageFunction (/usr/src/app/src/main.js:85:52) 2021-11-10T17:47:38.422Z at CheerioCrawler._handleRequestFunction (/usr/src/app/node_modules/apify/build/crawlers/cheerio_crawler.js:434:50) 2021-11-10T17:47:38.423Z at processTicksAndRejections (internal/process/task_queues.js:97:5) 2021-11-10T17:47:38.423Z at async /usr/src/app/node_modules/apify/build/utils.js:411:20 2021-11-10T17:47:38.739Z INFO: Processing: "https://www.forever21.com/us/shop/catalog/category/f21/sweater" 2021-11-10T17:47:38.740Z ERROR: BasicCrawler: handleRequestFunction failed, reclaiming failed request back to the list or queue {"url":"https://www.forever21.com/us/shop/catalog/category/f21/sweater","retryCount":2,"id":"DqNs4qcLsiZLXWf"} 2021-11-10T17:47:38.741Z TypeError: Cannot read property 'indexOf' of null 2021-11-10T17:47:38.741Z at extractSubcatPage (/usr/src/app/src/extractors.js:50:33) 2021-11-10T17:47:38.742Z at CheerioCrawler.handlePageFunction (/usr/src/app/src/main.js:85:52) 2021-11-10T17:47:38.743Z at CheerioCrawler._handleRequestFunction (/usr/src/app/node_modules/apify/build/crawlers/cheerio_crawler.js:434:50) 2021-11-10T17:47:38.744Z at processTicksAndRejections (internal/process/task_queues.js:97:5) 2021-11-10T17:47:38.744Z at async /usr/src/app/node_modules/apify/build/utils.js:411:20 2021-11-10T17:47:39.078Z INFO: Processing: "https://www.forever21.com/us/shop/catalog/category/plus/plus-size-new-arrivals" 2021-11-10T17:47:39.079Z ERROR: BasicCrawler: handleRequestFunction failed, reclaiming failed request back to the list or queue {"url":"https://www.forever21.com/us/shop/catalog/category/plus/plus-size-new-arrivals","retryCount":2,"id":"nIzZcqXT7uXvWjk"} 2021-11-10T17:47:39.080Z TypeError: Cannot read property 'indexOf' of null 2021-11-10T17:47:39.081Z at extractSubcatPage (/usr/src/app/src/extractors.js:50:33) 2021-11-10T17:47:39.082Z at CheerioCrawler.handlePageFunction (/usr/src/app/src/main.js:85:52) 2021-11-10T17:47:39.082Z at CheerioCrawler._handleRequestFunction (/usr/src/app/node_modules/apify/build/crawlers/cheerio_crawler.js:434:50) 2021-11-10T17:47:39.083Z at processTicksAndRejections (internal/process/task_queues.js:97:5) 2021-11-10T17:47:39.084Z at async /usr/src/app/node_modules/apify/build/utils.js:411:20 2021-11-10T17:47:39.272Z INFO: Processing: "https://www.forever21.com/us/shop/catalog/category/f21/outerwear_coats-and-jackets" 2021-11-10T17:47:39.273Z ERROR: BasicCrawler: handleRequestFunction failed, reclaiming failed request back to the list or queue {"url":"https://www.forever21.com/us/shop/catalog/category/f21/outerwear_coats-and-jackets","retryCount":2,"id":"LBOKhmNU204fmPB"} 2021-11-10T17:47:39.273Z TypeError: Cannot read property 'indexOf' of null 2021-11-10T17:47:39.274Z at extractSubcatPage (/usr/src/app/src/extractors.js:50:33) 2021-11-10T17:47:39.274Z at CheerioCrawler.handlePageFunction (/usr/src/app/src/main.js:85:52) 2021-11-10T17:47:39.275Z at CheerioCrawler._handleRequestFunction (/usr/src/app/node_modules/apify/build/crawlers/cheerio_crawler.js:434:50) 2021-11-10T17:47:39.275Z at processTicksAndRejections (internal/process/task_queues.js:97:5) 2021-11-10T17:47:39.276Z at async /usr/src/app/node_modules/apify/build/utils.js:411:20 2021-11-10T17:47:39.762Z INFO: Processing: "https://www.forever21.com/us/shop/catalog/category/f21/sets" 2021-11-10T17:47:39.764Z ERROR: BasicCrawler: handleRequestFunction failed, reclaiming failed request back to the list or queue {"url":"https://www.forever21.com/us/shop/catalog/category/f21/sets","retryCount":2,"id":"oqvTgOTkv9RbTSw"} 2021-11-10T17:47:39.764Z TypeError: Cannot read property 'indexOf' of null 2021-11-10T17:47:39.765Z at extractSubcatPage (/usr/src/app/src/extractors.js:50:33) 2021-11-10T17:47:39.767Z at CheerioCrawler.handlePageFunction (/usr/src/app/src/main.js:85:52) 2021-11-10T17:47:39.767Z at CheerioCrawler._handleRequestFunction (/usr/src/app/node_modules/apify/build/crawlers/cheerio_crawler.js:434:50) 2021-11-10T17:47:39.768Z at runMicrotasks () 2021-11-10T17:47:39.768Z at processTicksAndRejections (internal/process/task_queues.js:97:5) 2021-11-10T17:47:39.770Z at async /usr/src/app/node_modules/apify/build/utils.js:411:20 2021-11-10T17:47:39.771Z INFO: Processing: "https://www.forever21.com/us/shop/catalog/category/f21/dress" 2021-11-10T17:47:39.773Z ERROR: BasicCrawler: handleRequestFunction failed, reclaiming failed request back to the list or queue {"url":"https://www.forever21.com/us/shop/catalog/category/f21/dress","retryCount":2,"id":"hJLGCpnGiUEhRpv"} 2021-11-10T17:47:39.774Z TypeError: Cannot read property 'indexOf' of null 2021-11-10T17:47:39.775Z at extractSubcatPage (/usr/src/app/src/extractors.js:50:33) 2021-11-10T17:47:39.775Z at CheerioCrawler.handlePageFunction (/usr/src/app/src/main.js:85:52) 2021-11-10T17:47:39.776Z at CheerioCrawler._handleRequestFunction (/usr/src/app/node_modules/apify/build/crawlers/cheerio_crawler.js:434:50) 2021-11-10T17:47:39.777Z at runMicrotasks () 2021-11-10T17:47:39.778Z at processTicksAndRejections (internal/process/task_queues.js:97:5) 2021-11-10T17:47:39.781Z at async /usr/src/app/node_modules/apify/build/utils.js:411:20 2021-11-10T17:47:40.074Z INFO: Processing: "https://www.forever21.com/us/shop/catalog/category/f21/bottoms" 2021-11-10T17:47:40.076Z ERROR: BasicCrawler: handleRequestFunction failed, reclaiming failed request back to the list or queue {"url":"https://www.forever21.com/us/shop/catalog/category/f21/bottoms","retryCount":2,"id":"MKGdebDmWZYj3CP"} 2021-11-10T17:47:40.076Z TypeError: Cannot read property 'indexOf' of null 2021-11-10T17:47:40.078Z at extractSubcatPage (/usr/src/app/src/extractors.js:50:33) 2021-11-10T17:47:40.079Z at CheerioCrawler.handlePageFunction (/usr/src/app/src/main.js:85:52) 2021-11-10T17:47:40.080Z at CheerioCrawler._handleRequestFunction (/usr/src/app/node_modules/apify/build/crawlers/cheerio_crawler.js:434:50) 2021-11-10T17:47:40.081Z at runMicrotasks () 2021-11-10T17:47:40.082Z at processTicksAndRejections (internal/process/task_queues.js:97:5) 2021-11-10T17:47:40.083Z at async /usr/src/app/node_modules/apify/build/utils.js:411:20 2021-11-10T17:47:40.260Z INFO: Processing: "https://www.forever21.com/us/shop/catalog/category/f21/bottom_jeans" 2021-11-10T17:47:40.261Z ERROR: BasicCrawler: handleRequestFunction failed, reclaiming failed request back to the list or queue {"url":"https://www.forever21.com/us/shop/catalog/category/f21/bottom_jeans","retryCount":2,"id":"Q2d74LpsEA5gC3V"} 2021-11-10T17:47:40.261Z TypeError: Cannot read property 'indexOf' of null 2021-11-10T17:47:40.263Z at extractSubcatPage (/usr/src/app/src/extractors.js:50:33) 2021-11-10T17:47:40.264Z at CheerioCrawler.handlePageFunction (/usr/src/app/src/main.js:85:52) 2021-11-10T17:47:40.265Z at CheerioCrawler._handleRequestFunction (/usr/src/app/node_modules/apify/build/crawlers/cheerio_crawler.js:434:50) 2021-11-10T17:47:40.266Z at runMicrotasks () 2021-11-10T17:47:40.266Z at processTicksAndRejections (internal/process/task_queues.js:97:5) 2021-11-10T17:47:40.267Z at async /usr/src/app/node_modules/apify/build/utils.js:411:20 2021-11-10T17:47:40.546Z INFO: Processing: "https://www.forever21.com/us/shop/catalog/category/f21/top_graphic-tops" 2021-11-10T17:47:40.546Z ERROR: BasicCrawler: handleRequestFunction failed, reclaiming failed request back to the list or queue {"url":"https://www.forever21.com/us/shop/catalog/category/f21/top_graphic-tops","retryCount":2,"id":"GNaQAJG4ZRdA37y"} 2021-11-10T17:47:40.547Z TypeError: Cannot read property 'indexOf' of null 2021-11-10T17:47:40.548Z at extractSubcatPage (/usr/src/app/src/extractors.js:50:33) 2021-11-10T17:47:40.548Z at CheerioCrawler.handlePageFunction (/usr/src/app/src/main.js:85:52) 2021-11-10T17:47:40.549Z at CheerioCrawler._handleRequestFunction (/usr/src/app/node_modules/apify/build/crawlers/cheerio_crawler.js:434:50) 2021-11-10T17:47:40.550Z at runMicrotasks () 2021-11-10T17:47:40.550Z at processTicksAndRejections (internal/process/task_queues.js:97:5) 2021-11-10T17:47:40.551Z at async /usr/src/app/node_modules/apify/build/utils.js:411:20 2021-11-10T17:47:41.662Z INFO: Processing: "https://www.forever21.com/us/shop/catalog/category/f21/top_blouses-sweatshirts-hoodies" 2021-11-10T17:47:41.663Z ERROR: BasicCrawler: handleRequestFunction failed, reclaiming failed request back to the list or queue {"url":"https://www.forever21.com/us/shop/catalog/category/f21/top_blouses-sweatshirts-hoodies","retryCount":2,"id":"9jFs5POj22NfHRL"} 2021-11-10T17:47:41.664Z TypeError: Cannot read property 'indexOf' of null 2021-11-10T17:47:41.664Z at extractSubcatPage (/usr/src/app/src/extractors.js:50:33) 2021-11-10T17:47:41.665Z at CheerioCrawler.handlePageFunction (/usr/src/app/src/main.js:85:52) 2021-11-10T17:47:41.666Z at CheerioCrawler._handleRequestFunction (/usr/src/app/node_modules/apify/build/crawlers/cheerio_crawler.js:434:50) 2021-11-10T17:47:41.666Z at runMicrotasks () 2021-11-10T17:47:41.667Z at processTicksAndRejections (internal/process/task_queues.js:97:5) 2021-11-10T17:47:41.667Z at async /usr/src/app/node_modules/apify/build/utils.js:411:20 2021-11-10T17:47:42.055Z INFO: Processing: "https://www.forever21.com/us/shop/catalog/category/f21/rompers-jumpsuits" 2021-11-10T17:47:42.056Z ERROR: BasicCrawler: handleRequestFunction failed, reclaiming failed request back to the list or queue {"url":"https://www.forever21.com/us/shop/catalog/category/f21/rompers-jumpsuits","retryCount":2,"id":"7oZlozpXjb7z0L6"} 2021-11-10T17:47:42.057Z TypeError: Cannot read property 'indexOf' of null 2021-11-10T17:47:42.058Z at extractSubcatPage (/usr/src/app/src/extractors.js:50:33) 2021-11-10T17:47:42.058Z at CheerioCrawler.handlePageFunction (/usr/src/app/src/main.js:85:52) 2021-11-10T17:47:42.059Z at CheerioCrawler._handleRequestFunction (/usr/src/app/node_modules/apify/build/crawlers/cheerio_crawler.js:434:50) 2021-11-10T17:47:42.059Z at runMicrotasks () 2021-11-10T17:47:42.060Z at processTicksAndRejections (internal/process/task_queues.js:97:5) 2021-11-10T17:47:42.061Z at async /usr/src/app/node_modules/apify/build/utils.js:411:20 2021-11-10T17:47:42.168Z INFO: Processing: "https://www.forever21.com/us/shop/catalog/category/f21/activewear" 2021-11-10T17:47:42.169Z ERROR: BasicCrawler: handleRequestFunction failed, reclaiming failed request back to the list or queue {"url":"https://www.forever21.com/us/shop/catalog/category/f21/activewear","retryCount":2,"id":"cpSm5YI4HuElv7r"} 2021-11-10T17:47:42.170Z TypeError: Cannot read property 'indexOf' of null 2021-11-10T17:47:42.170Z at extractSubcatPage (/usr/src/app/src/extractors.js:50:33) 2021-11-10T17:47:42.171Z at CheerioCrawler.handlePageFunction (/usr/src/app/src/main.js:85:52) 2021-11-10T17:47:42.172Z at CheerioCrawler._handleRequestFunction (/usr/src/app/node_modules/apify/build/crawlers/cheerio_crawler.js:434:50) 2021-11-10T17:47:42.172Z at runMicrotasks () 2021-11-10T17:47:42.173Z at processTicksAndRejections (internal/process/task_queues.js:97:5) 2021-11-10T17:47:42.174Z at async /usr/src/app/node_modules/apify/build/utils.js:411:20 2021-11-10T17:47:42.560Z INFO: Processing: "https://www.forever21.com/us/shop/catalog/category/f21/lingerie" 2021-11-10T17:47:42.561Z ERROR: BasicCrawler: handleRequestFunction failed, reclaiming failed request back to the list or queue {"url":"https://www.forever21.com/us/shop/catalog/category/f21/lingerie","retryCount":3,"id":"XPB20HPiuBbxkWV"} 2021-11-10T17:47:42.562Z TypeError: Cannot read property 'indexOf' of null 2021-11-10T17:47:42.563Z at extractSubcatPage (/usr/src/app/src/extractors.js:50:33) 2021-11-10T17:47:42.563Z at CheerioCrawler.handlePageFunction (/usr/src/app/src/main.js:85:52) 2021-11-10T17:47:42.564Z at CheerioCrawler._handleRequestFunction (/usr/src/app/node_modules/apify/build/crawlers/cheerio_crawler.js:434:50) 2021-11-10T17:47:42.564Z at runMicrotasks () 2021-11-10T17:47:42.565Z at processTicksAndRejections (internal/process/task_queues.js:97:5) 2021-11-10T17:47:42.565Z at async /usr/src/app/node_modules/apify/build/utils.js:411:20 2021-11-10T17:47:42.610Z INFO: Processing: "https://www.forever21.com/us/shop/catalog/category/plus/plus_size-tops" 2021-11-10T17:47:42.611Z ERROR: BasicCrawler: handleRequestFunction failed, reclaiming failed request back to the list or queue {"url":"https://www.forever21.com/us/shop/catalog/category/plus/plus_size-tops","retryCount":2,"id":"z6a1JfnDv8EKyfZ"} 2021-11-10T17:47:42.612Z TypeError: Cannot read property 'indexOf' of null 2021-11-10T17:47:42.613Z at extractSubcatPage (/usr/src/app/src/extractors.js:50:33) 2021-11-10T17:47:42.613Z at CheerioCrawler.handlePageFunction (/usr/src/app/src/main.js:85:52) 2021-11-10T17:47:42.618Z at CheerioCrawler._handleRequestFunction (/usr/src/app/node_modules/apify/build/crawlers/cheerio_crawler.js:434:50) 2021-11-10T17:47:42.619Z at runMicrotasks () 2021-11-10T17:47:42.619Z at processTicksAndRejections (internal/process/task_queues.js:97:5) 2021-11-10T17:47:42.620Z at async /usr/src/app/node_modules/apify/build/utils.js:411:20 2021-11-10T17:47:42.700Z INFO: Processing: "https://www.forever21.com/us/shop/catalog/category/f21/top_blouses" 2021-11-10T17:47:42.702Z ERROR: BasicCrawler: handleRequestFunction failed, reclaiming failed request back to the list or queue {"url":"https://www.forever21.com/us/shop/catalog/category/f21/top_blouses","retryCount":3,"id":"6ezVIYHB2z3p60Q"} 2021-11-10T17:47:42.702Z TypeError: Cannot read property 'indexOf' of null 2021-11-10T17:47:42.703Z at extractSubcatPage (/usr/src/app/src/extractors.js:50:33) 2021-11-10T17:47:42.704Z at CheerioCrawler.handlePageFunction (/usr/src/app/src/main.js:85:52) 2021-11-10T17:47:42.704Z at CheerioCrawler._handleRequestFunction (/usr/src/app/node_modules/apify/build/crawlers/cheerio_crawler.js:434:50) 2021-11-10T17:47:42.705Z at runMicrotasks () 2021-11-10T17:47:42.705Z at processTicksAndRejections (internal/process/task_queues.js:97:5) 2021-11-10T17:47:42.706Z at async /usr/src/app/node_modules/apify/build/utils.js:411:20 2021-11-10T17:47:43.091Z INFO: Processing: "https://www.forever21.com/us/shop/catalog/category/f21/outerwear_coats-and-jackets" 2021-11-10T17:47:43.093Z ERROR: BasicCrawler: handleRequestFunction failed, reclaiming failed request back to the list or queue {"url":"https://www.forever21.com/us/shop/catalog/category/f21/outerwear_coats-and-jackets","retryCount":3,"id":"LBOKhmNU204fmPB"} 2021-11-10T17:47:43.093Z TypeError: Cannot read property 'indexOf' of null 2021-11-10T17:47:43.094Z at extractSubcatPage (/usr/src/app/src/extractors.js:50:33) 2021-11-10T17:47:43.095Z at CheerioCrawler.handlePageFunction (/usr/src/app/src/main.js:85:52) 2021-11-10T17:47:43.096Z at CheerioCrawler._handleRequestFunction (/usr/src/app/node_modules/apify/build/crawlers/cheerio_crawler.js:434:50) 2021-11-10T17:47:43.097Z at runMicrotasks () 2021-11-10T17:47:43.097Z at processTicksAndRejections (internal/process/task_queues.js:97:5) 2021-11-10T17:47:43.098Z at async /usr/src/app/node_modules/apify/build/utils.js:411:20 2021-11-10T17:47:43.113Z INFO: Processing: "https://www.forever21.com/us/shop/catalog/category/f21/sweater" 2021-11-10T17:47:43.114Z ERROR: BasicCrawler: handleRequestFunction failed, reclaiming failed request back to the list or queue {"url":"https://www.forever21.com/us/shop/catalog/category/f21/sweater","retryCount":3,"id":"DqNs4qcLsiZLXWf"} 2021-11-10T17:47:43.115Z TypeError: Cannot read property 'indexOf' of null 2021-11-10T17:47:43.116Z at extractSubcatPage (/usr/src/app/src/extractors.js:50:33) 2021-11-10T17:47:43.116Z at CheerioCrawler.handlePageFunction (/usr/src/app/src/main.js:85:52) 2021-11-10T17:47:43.117Z at CheerioCrawler._handleRequestFunction (/usr/src/app/node_modules/apify/build/crawlers/cheerio_crawler.js:434:50) 2021-11-10T17:47:43.117Z at runMicrotasks () 2021-11-10T17:47:43.118Z at processTicksAndRejections (internal/process/task_queues.js:97:5) 2021-11-10T17:47:43.119Z at async /usr/src/app/node_modules/apify/build/utils.js:411:20 2021-11-10T17:47:43.119Z INFO: Processing: "https://www.forever21.com/us/shop/catalog/category/plus/plus-size-new-arrivals" 2021-11-10T17:47:43.120Z ERROR: BasicCrawler: handleRequestFunction failed, reclaiming failed request back to the list or queue {"url":"https://www.forever21.com/us/shop/catalog/category/plus/plus-size-new-arrivals","retryCount":3,"id":"nIzZcqXT7uXvWjk"} 2021-11-10T17:47:43.121Z TypeError: Cannot read property 'indexOf' of null 2021-11-10T17:47:43.121Z at extractSubcatPage (/usr/src/app/src/extractors.js:50:33) 2021-11-10T17:47:43.122Z at CheerioCrawler.handlePageFunction (/usr/src/app/src/main.js:85:52) 2021-11-10T17:47:43.123Z at CheerioCrawler._handleRequestFunction (/usr/src/app/node_modules/apify/build/crawlers/cheerio_crawler.js:434:50) 2021-11-10T17:47:43.123Z at runMicrotasks () 2021-11-10T17:47:43.124Z at processTicksAndRejections (internal/process/task_queues.js:97:5) 2021-11-10T17:47:43.124Z at async /usr/src/app/node_modules/apify/build/utils.js:411:20 2021-11-10T17:47:43.600Z INFO: Processing: "https://www.forever21.com/us/shop/catalog/category/f21/sets" 2021-11-10T17:47:43.601Z ERROR: BasicCrawler: handleRequestFunction failed, reclaiming failed request back to the list or queue {"url":"https://www.forever21.com/us/shop/catalog/category/f21/sets","retryCount":3,"id":"oqvTgOTkv9RbTSw"} 2021-11-10T17:47:43.602Z TypeError: Cannot read property 'indexOf' of null 2021-11-10T17:47:43.602Z at extractSubcatPage (/usr/src/app/src/extractors.js:50:33) 2021-11-10T17:47:43.603Z at CheerioCrawler.handlePageFunction (/usr/src/app/src/main.js:85:52) 2021-11-10T17:47:43.603Z at CheerioCrawler._handleRequestFunction (/usr/src/app/node_modules/apify/build/crawlers/cheerio_crawler.js:434:50) 2021-11-10T17:47:43.604Z at runMicrotasks () 2021-11-10T17:47:43.605Z at processTicksAndRejections (internal/process/task_queues.js:97:5) 2021-11-10T17:47:43.605Z at async /usr/src/app/node_modules/apify/build/utils.js:411:20 2021-11-10T17:47:43.741Z INFO: Processing: "https://www.forever21.com/us/shop/catalog/category/f21/bottoms" 2021-11-10T17:47:43.742Z ERROR: BasicCrawler: handleRequestFunction failed, reclaiming failed request back to the list or queue {"url":"https://www.forever21.com/us/shop/catalog/category/f21/bottoms","retryCount":3,"id":"MKGdebDmWZYj3CP"} 2021-11-10T17:47:43.743Z TypeError: Cannot read property 'indexOf' of null 2021-11-10T17:47:43.743Z at extractSubcatPage (/usr/src/app/src/extractors.js:50:33) 2021-11-10T17:47:43.744Z at CheerioCrawler.handlePageFunction (/usr/src/app/src/main.js:85:52) 2021-11-10T17:47:43.744Z at CheerioCrawler._handleRequestFunction (/usr/src/app/node_modules/apify/build/crawlers/cheerio_crawler.js:434:50) 2021-11-10T17:47:43.745Z at runMicrotasks () 2021-11-10T17:47:43.745Z at processTicksAndRejections (internal/process/task_queues.js:97:5) 2021-11-10T17:47:43.746Z at async /usr/src/app/node_modules/apify/build/utils.js:411:20 2021-11-10T17:47:43.959Z INFO: Processing: "https://www.forever21.com/us/shop/catalog/category/f21/dress" 2021-11-10T17:47:43.960Z ERROR: BasicCrawler: handleRequestFunction failed, reclaiming failed request back to the list or queue {"url":"https://www.forever21.com/us/shop/catalog/category/f21/dress","retryCount":3,"id":"hJLGCpnGiUEhRpv"} 2021-11-10T17:47:43.961Z TypeError: Cannot read property 'indexOf' of null 2021-11-10T17:47:43.962Z at extractSubcatPage (/usr/src/app/src/extractors.js:50:33) 2021-11-10T17:47:43.963Z at CheerioCrawler.handlePageFunction (/usr/src/app/src/main.js:85:52) 2021-11-10T17:47:43.964Z at CheerioCrawler._handleRequestFunction (/usr/src/app/node_modules/apify/build/crawlers/cheerio_crawler.js:434:50) 2021-11-10T17:47:43.965Z at runMicrotasks () 2021-11-10T17:47:43.965Z at processTicksAndRejections (internal/process/task_queues.js:97:5) 2021-11-10T17:47:43.966Z at async /usr/src/app/node_modules/apify/build/utils.js:411:20 2021-11-10T17:47:44.194Z INFO: Processing: "https://www.forever21.com/us/shop/catalog/category/f21/bottom_jeans" 2021-11-10T17:47:44.195Z ERROR: BasicCrawler: handleRequestFunction failed, reclaiming failed request back to the list or queue {"url":"https://www.forever21.com/us/shop/catalog/category/f21/bottom_jeans","retryCount":3,"id":"Q2d74LpsEA5gC3V"} 2021-11-10T17:47:44.196Z TypeError: Cannot read property 'indexOf' of null 2021-11-10T17:47:44.196Z at extractSubcatPage (/usr/src/app/src/extractors.js:50:33) 2021-11-10T17:47:44.197Z at CheerioCrawler.handlePageFunction (/usr/src/app/src/main.js:85:52) 2021-11-10T17:47:44.197Z at CheerioCrawler._handleRequestFunction (/usr/src/app/node_modules/apify/build/crawlers/cheerio_crawler.js:434:50) 2021-11-10T17:47:44.198Z at runMicrotasks () 2021-11-10T17:47:44.198Z at processTicksAndRejections (internal/process/task_queues.js:97:5) 2021-11-10T17:47:44.199Z at async /usr/src/app/node_modules/apify/build/utils.js:411:20 2021-11-10T17:47:45.351Z INFO: Processing: "https://www.forever21.com/us/shop/catalog/category/f21/rompers-jumpsuits" 2021-11-10T17:47:45.352Z ERROR: BasicCrawler: handleRequestFunction failed, reclaiming failed request back to the list or queue {"url":"https://www.forever21.com/us/shop/catalog/category/f21/rompers-jumpsuits","retryCount":3,"id":"7oZlozpXjb7z0L6"} 2021-11-10T17:47:45.353Z TypeError: Cannot read property 'indexOf' of null 2021-11-10T17:47:45.353Z at extractSubcatPage (/usr/src/app/src/extractors.js:50:33) 2021-11-10T17:47:45.354Z at CheerioCrawler.handlePageFunction (/usr/src/app/src/main.js:85:52) 2021-11-10T17:47:45.355Z at CheerioCrawler._handleRequestFunction (/usr/src/app/node_modules/apify/build/crawlers/cheerio_crawler.js:434:50) 2021-11-10T17:47:45.355Z at runMicrotasks () 2021-11-10T17:47:45.356Z at processTicksAndRejections (internal/process/task_queues.js:97:5) 2021-11-10T17:47:45.356Z at async /usr/src/app/node_modules/apify/build/utils.js:411:20 2021-11-10T17:47:45.492Z INFO: Processing: "https://www.forever21.com/us/shop/catalog/category/f21/top_blouses-sweatshirts-hoodies" 2021-11-10T17:47:45.493Z ERROR: BasicCrawler: handleRequestFunction failed, reclaiming failed request back to the list or queue {"url":"https://www.forever21.com/us/shop/catalog/category/f21/top_blouses-sweatshirts-hoodies","retryCount":3,"id":"9jFs5POj22NfHRL"} 2021-11-10T17:47:45.494Z TypeError: Cannot read property 'indexOf' of null 2021-11-10T17:47:45.495Z at extractSubcatPage (/usr/src/app/src/extractors.js:50:33) 2021-11-10T17:47:45.496Z at CheerioCrawler.handlePageFunction (/usr/src/app/src/main.js:85:52) 2021-11-10T17:47:45.497Z at CheerioCrawler._handleRequestFunction (/usr/src/app/node_modules/apify/build/crawlers/cheerio_crawler.js:434:50) 2021-11-10T17:47:45.498Z at runMicrotasks () 2021-11-10T17:47:45.498Z at processTicksAndRejections (internal/process/task_queues.js:97:5) 2021-11-10T17:47:45.499Z at async /usr/src/app/node_modules/apify/build/utils.js:411:20 2021-11-10T17:47:45.500Z INFO: Processing: "https://www.forever21.com/us/shop/catalog/category/f21/top_graphic-tops" 2021-11-10T17:47:45.501Z ERROR: BasicCrawler: handleRequestFunction failed, reclaiming failed request back to the list or queue {"url":"https://www.forever21.com/us/shop/catalog/category/f21/top_graphic-tops","retryCount":3,"id":"GNaQAJG4ZRdA37y"} 2021-11-10T17:47:45.501Z TypeError: Cannot read property 'indexOf' of null 2021-11-10T17:47:45.502Z at extractSubcatPage (/usr/src/app/src/extractors.js:50:33) 2021-11-10T17:47:45.503Z at CheerioCrawler.handlePageFunction (/usr/src/app/src/main.js:85:52) 2021-11-10T17:47:45.503Z at CheerioCrawler._handleRequestFunction (/usr/src/app/node_modules/apify/build/crawlers/cheerio_crawler.js:434:50) 2021-11-10T17:47:45.505Z at runMicrotasks () 2021-11-10T17:47:45.506Z at processTicksAndRejections (internal/process/task_queues.js:97:5) 2021-11-10T17:47:45.506Z at async /usr/src/app/node_modules/apify/build/utils.js:411:20 2021-11-10T17:47:45.825Z INFO: Processing: "https://www.forever21.com/us/shop/catalog/category/f21/activewear" 2021-11-10T17:47:45.826Z ERROR: BasicCrawler: handleRequestFunction failed, reclaiming failed request back to the list or queue {"url":"https://www.forever21.com/us/shop/catalog/category/f21/activewear","retryCount":3,"id":"cpSm5YI4HuElv7r"} 2021-11-10T17:47:45.827Z TypeError: Cannot read property 'indexOf' of null 2021-11-10T17:47:45.827Z at extractSubcatPage (/usr/src/app/src/extractors.js:50:33) 2021-11-10T17:47:45.828Z at CheerioCrawler.handlePageFunction (/usr/src/app/src/main.js:85:52) 2021-11-10T17:47:45.828Z at CheerioCrawler._handleRequestFunction (/usr/src/app/node_modules/apify/build/crawlers/cheerio_crawler.js:434:50) 2021-11-10T17:47:45.829Z at runMicrotasks () 2021-11-10T17:47:45.829Z at processTicksAndRejections (internal/process/task_queues.js:97:5) 2021-11-10T17:47:45.830Z at async /usr/src/app/node_modules/apify/build/utils.js:411:20 2021-11-10T17:47:46.016Z INFO: Processing: "https://www.forever21.com/us/shop/catalog/category/f21/lingerie" 2021-11-10T17:47:46.055Z WARNING: Request https://www.forever21.com/us/shop/catalog/category/f21/lingerie failed too many times 2021-11-10T17:47:46.181Z INFO: Processing: "https://www.forever21.com/us/shop/catalog/category/plus/plus_size-tops" 2021-11-10T17:47:46.182Z ERROR: BasicCrawler: handleRequestFunction failed, reclaiming failed request back to the list or queue {"url":"https://www.forever21.com/us/shop/catalog/category/plus/plus_size-tops","retryCount":3,"id":"z6a1JfnDv8EKyfZ"} 2021-11-10T17:47:46.185Z TypeError: Cannot read property 'indexOf' of null 2021-11-10T17:47:46.190Z at extractSubcatPage (/usr/src/app/src/extractors.js:50:33) 2021-11-10T17:47:46.190Z at CheerioCrawler.handlePageFunction (/usr/src/app/src/main.js:85:52) 2021-11-10T17:47:46.191Z at CheerioCrawler._handleRequestFunction (/usr/src/app/node_modules/apify/build/crawlers/cheerio_crawler.js:434:50) 2021-11-10T17:47:46.192Z at runMicrotasks () 2021-11-10T17:47:46.193Z at processTicksAndRejections (internal/process/task_queues.js:97:5) 2021-11-10T17:47:46.194Z at async /usr/src/app/node_modules/apify/build/utils.js:411:20 2021-11-10T17:47:46.264Z INFO: Processing: "https://www.forever21.com/us/shop/catalog/category/f21/top_blouses" 2021-11-10T17:47:46.442Z WARNING: Request https://www.forever21.com/us/shop/catalog/category/f21/top_blouses failed too many times 2021-11-10T17:47:46.571Z INFO: Processing: "https://www.forever21.com/us/shop/catalog/category/f21/sweater" 2021-11-10T17:47:46.634Z WARNING: Request https://www.forever21.com/us/shop/catalog/category/f21/sweater failed too many times 2021-11-10T17:47:46.642Z INFO: Processing: "https://www.forever21.com/us/shop/catalog/category/plus/plus-size-new-arrivals" 2021-11-10T17:47:46.690Z WARNING: Request https://www.forever21.com/us/shop/catalog/category/plus/plus-size-new-arrivals failed too many times 2021-11-10T17:47:46.992Z INFO: Processing: "https://www.forever21.com/us/shop/catalog/category/f21/outerwear_coats-and-jackets" 2021-11-10T17:47:47.069Z WARNING: Request https://www.forever21.com/us/shop/catalog/category/f21/outerwear_coats-and-jackets failed too many times 2021-11-10T17:47:47.161Z INFO: Processing: "https://www.forever21.com/us/shop/catalog/category/f21/sets" 2021-11-10T17:47:47.213Z WARNING: Request https://www.forever21.com/us/shop/catalog/category/f21/sets failed too many times 2021-11-10T17:47:47.316Z INFO: Processing: "https://www.forever21.com/us/shop/catalog/category/f21/dress" 2021-11-10T17:47:47.379Z WARNING: Request https://www.forever21.com/us/shop/catalog/category/f21/dress failed too many times 2021-11-10T17:47:47.490Z INFO: Processing: "https://www.forever21.com/us/shop/catalog/category/f21/bottom_jeans" 2021-11-10T17:47:47.539Z WARNING: Request https://www.forever21.com/us/shop/catalog/category/f21/bottom_jeans failed too many times 2021-11-10T17:47:47.756Z INFO: Processing: "https://www.forever21.com/us/shop/catalog/category/f21/bottoms" 2021-11-10T17:47:47.793Z WARNING: Request https://www.forever21.com/us/shop/catalog/category/f21/bottoms failed too many times 2021-11-10T17:47:49.050Z INFO: Processing: "https://www.forever21.com/us/shop/catalog/category/f21/top_graphic-tops" 2021-11-10T17:47:49.057Z INFO: Processing: "https://www.forever21.com/us/shop/catalog/category/f21/rompers-jumpsuits" 2021-11-10T17:47:49.087Z WARNING: Request https://www.forever21.com/us/shop/catalog/category/f21/top_graphic-tops failed too many times 2021-11-10T17:47:49.149Z WARNING: Request https://www.forever21.com/us/shop/catalog/category/f21/rompers-jumpsuits failed too many times 2021-11-10T17:47:49.178Z INFO: Processing: "https://www.forever21.com/us/shop/catalog/category/f21/top_blouses-sweatshirts-hoodies" 2021-11-10T17:47:49.321Z WARNING: Request https://www.forever21.com/us/shop/catalog/category/f21/top_blouses-sweatshirts-hoodies failed too many times 2021-11-10T17:47:49.639Z INFO: Processing: "https://www.forever21.com/us/shop/catalog/category/f21/activewear" 2021-11-10T17:47:49.696Z WARNING: Request https://www.forever21.com/us/shop/catalog/category/f21/activewear failed too many times 2021-11-10T17:47:50.228Z INFO: Processing: "https://www.forever21.com/us/shop/catalog/category/plus/plus_size-tops" 2021-11-10T17:47:50.310Z WARNING: Request https://www.forever21.com/us/shop/catalog/category/plus/plus_size-tops failed too many times 2021-11-10T17:47:50.484Z INFO: BasicCrawler: All the requests from request list and/or request queue have been processed, the crawler will shut down. 2021-11-10T17:47:50.485Z INFO: Crawler final request statistics: {"avgDurationMillis":null,"perMinute":0,"finished":0,"failed":14,"retryHistogram":[null,null,null,14]} 2021-11-10T17:47:50.487Z INFO: Crawler Finished. 2021-11-10T17:47:50.642Z