No credit card required

Ultimate Screenshot

dz_omar/ultimate-screenshot

No credit card required

Ultimate Screenshot allows you to extract data in formats like JPEG, PNG, PDF, GIF, and MP4. It supports device emulation, including iPhones, Android phones, tablets, and desktops, or uses a default resolution of 1920x1080 for accurate, versatile screenshots and videos.

.actor/Dockerfile

1# Specify the base Docker image
2FROM apify/actor-node-puppeteer-chrome:20
3
4# Switch to root user for installing packages
5USER root
6
7# Install required dependencies for running Puppeteer and FFmpeg
8RUN apt-get update && apt-get install -y \
9    wget \
10    gnupg2 \
11    libx11-xcb1 \
12    libxcomposite1 \
13    libxcursor1 \
14    libxdamage1 \
15    libxrandr2 \
16    libgbm-dev \
17    libasound2 \
18    fonts-liberation \
19    libxss1 \
20    libatk-bridge2.0-0 \
21    libgtk-3-0 \
22    libnss3 \
23    libxshmfence1 \
24    ffmpeg \ 
25    --no-install-recommends \
26    && rm -rf /var/lib/apt/lists/*
27
28# Check preinstalled packages
29RUN npm ls crawlee apify puppeteer playwright
30
31# Copy just package.json and package-lock.json
32COPY --chown=myuser package*.json ./
33
34# Install NPM packages, skip optional and development dependencies to keep the image small
35RUN npm --quiet set progress=false \
36    && npm install --omit=dev --omit=optional \
37    && echo "Installed NPM packages:" \
38    && (npm list --omit=dev --all || true) \
39    && echo "Node.js version:" \
40    && node --version \
41    && echo "NPM version:" \
42    && npm --version \
43    && rm -r ~/.npm
44
45# Copy the remaining files and directories with the source code
46COPY --chown=myuser . ./
47
48# Set the environment variable for Puppeteer to use without sandboxing
49ENV PUPPETEER_SKIP_DOWNLOAD=true
50
51# Switch back to non-root user
52USER myuser
53
54# Command to run your main application
55CMD ["node", "src/main.js"]

.actor/actor.json

1{
2    "actorSpecification": 1,
3    "name": "screenshot",
4    "title": "screenshot",
5    "description": "Screenshot selenium",
6    "version": "1.0",
7    "meta": {
8        "templateId": "Screenshot"
9    },
10    "input": "./input_schema.json",
11    "dockerfile": "./Dockerfile",
12    "buildTag": "latest",
13    "minMemoryMbytes": 4096,
14    "readme": "./README.md",
15        "storages": {
16            "dataset": {
17                "actorSpecification": 1,
18                "title": "Results",
19                "views": {
20                    "results": {
21                    "title": "results to scan",
22                    "transformation": {
23                        "fields": ["screenshot_image", "content_Type", "linkUrl", "screenshot_url"]
24                    },
25                    "display": {
26                        "component": "table",
27                        "properties": {
28                        "screenshot_image": {
29                            "label": "Screenshot Url",
30                            "format": "image"
31                        },
32                        "content_Type": {
33                            "label": "Content Type",
34                            "format": "text"
35                        },
36                        "linkUrl": {
37                            "label": "link Url",
38                            "format": "link"
39                        },                        
40                        "screenshot_url": {
41                            "label": "Screenshot IF Video",
42                            "format": "link"
43                        }
44                    }
45                    }
46                }
47            }
48        }
49    }
50}

.actor/input_schema.json

1{
2  "$schema": "http://json-schema.org/draft-07/schema#",
3  "title": "Input Schema",
4  "type": "object",
5  "schemaVersion": 1,
6  "properties": {
7    "fullPage": {
8      "title": "Full Page Screenshot --- If you Select the Output Format MP4 or GIF you might want to check out (MP4 or GIF Options Section)",
9      "type": "boolean",
10      "description": "Should the screenshot capture the full length of the page?",
11      "default": false
12    },
13    "linkUrls": {
14      "title": "Link URLs",
15      "type": "array",
16      "description": "The URLs to be processed (e.g., websites to take screenshots from).",
17      "default": ["https://apify.com"],
18      "editor": "stringList"
19    },
20    "outputFormat": {
21      "title": "Output Format",
22      "type": "string",
23      "description": "The format of the screenshot output (JPEG, PNG, GIF, MP4, or PDF).",
24      "enum": ["jpeg", "png", "pdf", "gif", "mp4"],
25      "enumTitles": ["JPEG", "PNG", "PDF", "GIF", "MP4"],
26      "default": "jpeg"
27    },
28    "waitUntil": {
29      "title": "Navigation Wait Condition",
30      "type": "string",
31      "description": "Specify when the navigation should be considered finished. Options are 'load' for when the load event is fired, or 'domcontentloaded' for when the DOM has been loaded.",
32      "editor": "select",
33      "default": "networkidle0",
34      "enum": ["load", "domcontentloaded", "networkidle0", "networkidle2"],
35      "enumTitles": ["Load (all resources loaded)", "DOM Content Loaded (HTML parsed)", "Network Idle (no network connections)", "Network Idle (minimal network connections)"]
36    },  
37    "timeouT": {
38      "title": "Timeout",
39      "type": "integer",
40      "description": "Time in milliseconds to wait for the webpage to load.",
41      "default": 15,
42      "unit": "s"
43    },
44    "maxRetries": {
45      "title": "Maximum Retries",
46      "type": "integer",
47      "description": "The number of times to retry in case of a failure.",
48      "default": 3 
49    },
50    "delayBeforeScreenshot": {
51      "title": "Delay Before Screenshot",
52      "type": "integer",
53      "description": "Time in seconds to wait before taking the screenshot.",
54      "default": 1500, 
55      "unit": "ms"
56    },
57    "infiniteScroll": {
58      "sectionCaption": "MP4 or GIF Options",
59      "sectionDescription": "This section configures the output settings, specifically for MP4 or GIF formats, These options will be applied when MP4 or GIF is selected in the Output Format.",
60      "title": "Infinite Scroll - This process will not stop if the page has Infinite Scrolling",
61      "type": "boolean",
62      "description": "Determines if the page should scroll indefinitely. If true, the scrolling continues indefinitely; if false, the scrolling will stop after a specified timeout (10 seconds).",
63      "default": false
64    },
65    "timefullPagE": {
66      "title": "Time for Full Page Capture",
67      "type": "integer",
68      "description": "Time in seconds to allow for full-page capture before stopping. If infiniteScroll is true, this value is ignored.",
69      "default": 10,
70      "unit": "s"
71    },
72    "frameCounT": {
73        "title": "Frame Count",
74        "type": "integer",
75        "description": "The total number of frames to capture for the output video or GIF.",
76        "default": 10,
77        "unit": "frames"
78    },
79    "frameIntervaL": {
80        "title": "Frame Interval",
81        "type": "integer",
82        "description": "The interval in milliseconds between each captured frame.",
83        "default": 10,
84        "unit": "ms"
85    },
86    "frame": {
87        "title": "Frames per Second",
88        "type": "integer",
89        "description": "The number of frames to capture per second for the output.",
90        "default": 10,
91        "unit": "fps"
92    },
93    "scrollSteP": {
94        "title": "Scroll Step Distance This feature will only be applied if the 'Full Page Screenshot' option is selected",
95        "type": "integer",
96        "description": "Specifies the number of pixels to scroll down during full-page screenshots. This feature will only be applied if the 'Full Page Screenshot' option is selected; otherwise, it will not affect the scrolling behavior. A smaller scroll step allows for smoother scrolling and better image quality by ensuring that each frame is captured with minimal jumps. The default value is 300 pixels, but it can be adjusted for finer control over scrolling.",
97        "default": 300,
98        "unit": "px"
99
100    },
101    "printBackground": {
102      "sectionCaption": "PDF Options",
103      "sectionDescription": "This section configures the output settings, specifically for PDF format. These options will be applied when PDF is selected in the Output Format.",
104      "title": "Print Background",
105      "type": "boolean",
106      "description": "Configure whether to include the background graphics when generating a PDF.",
107      "default": true
108    },
109    "formaT": {
110      "title": "PDF Paper Format",
111      "type": "string",
112      "description": "Select the paper size to be used for generating the PDF.",
113      "editor": "select",
114      "default": "A4",
115      "enum": ["A4", "LETTER", "LEGAL", "TABLOID", "LEDGER", "A0", "A1", "A2", "A3", "A5", "A6"],
116      "enumTitles": ["A4", "Letter", "Legal", "Tabloid", "Ledger", "A0", "A1", "A2", "A3", "A5", "A6"]
117    },
118    "toP": {
119      "title": "Top Margin",
120      "type": "integer",
121      "description": "Specify the top margin for the PDF in millimeters.",
122      "default": 0,
123      "unit": "mm"
124
125
126    },
127    "righT": {
128      "title": "Right Margin",
129      "type": "integer",
130      "description": "Specify the right margin for the PDF in millimeters.",
131      "default": 0,
132      "unit": "mm"
133
134    },
135    "bottoM": {
136      "title": "Bottom Margin",
137      "type": "integer",
138      "description": "Specify the bottom margin for the PDF in millimeters.",
139      "default": 0,
140      "unit": "mm"
141
142    },
143    "lefT": {
144      "title": "Left Margin",
145      "type": "integer",
146      "description": "Specify the left margin for the PDF in millimeters.",
147      "default": 0,
148      "unit": "mm"
149
150    },    
151    "device": {
152      "sectionCaption": "Output Results: Device Emulation or Custom Window Dimensions",
153      "sectionDescription":"Configure the output display by selecting a device for emulation or manually specifying the window width and height. If no device is chosen, the default dimensions of 1920x1080 will be applied.",
154      "title": "Device",
155      "type": "string",
156      "description": "Choose a device to emulate specific dimensions and settings. If no device is selected, the default resolution of 1920x1080 will be used.",
157      "editor": "select",
158      "enum": [
159      "Blackberry PlayBook", "Blackberry PlayBook landscape", 
160      "BlackBerry Z30", "BlackBerry Z30 landscape", 
161      "Galaxy Note 3", "Galaxy Note 3 landscape", 
162      "Galaxy Note II", "Galaxy Note II landscape", 
163      "Galaxy S III", "Galaxy S III landscape", 
164      "Galaxy S5", "Galaxy S5 landscape", 
165      "Galaxy S8", "Galaxy S8 landscape", 
166      "Galaxy S9+", "Galaxy S9+ landscape", 
167      "Galaxy Tab S4", "Galaxy Tab S4 landscape", 
168      "iPad", "iPad landscape", 
169      "iPad (gen 6)", "iPad (gen 6) landscape", 
170      "iPad (gen 7)", "iPad (gen 7) landscape", 
171      "iPad Mini", "iPad Mini landscape", 
172      "iPad Pro", "iPad Pro landscape", 
173      "iPad Pro 11", "iPad Pro 11 landscape", 
174      "iPhone 4", "iPhone 4 landscape", 
175      "iPhone 5", "iPhone 5 landscape", 
176      "iPhone 6", "iPhone 6 landscape", 
177      "iPhone 6 Plus", "iPhone 6 Plus landscape", 
178      "iPhone 7", "iPhone 7 landscape", 
179      "iPhone 7 Plus", "iPhone 7 Plus landscape", 
180      "iPhone 8", "iPhone 8 landscape", 
181      "iPhone 8 Plus", "iPhone 8 Plus landscape", 
182      "iPhone SE", "iPhone SE landscape", 
183      "iPhone X", "iPhone X landscape", 
184      "iPhone XR", "iPhone XR landscape", 
185      "iPhone 11", "iPhone 11 landscape", 
186      "iPhone 11 Pro", "iPhone 11 Pro landscape", 
187      "iPhone 11 Pro Max", "iPhone 11 Pro Max landscape", 
188      "iPhone 12", "iPhone 12 landscape", 
189      "iPhone 12 Pro", "iPhone 12 Pro landscape", 
190      "iPhone 12 Pro Max", "iPhone 12 Pro Max landscape", 
191      "iPhone 12 Mini", "iPhone 12 Mini landscape", 
192      "iPhone 13", "iPhone 13 landscape", 
193      "iPhone 13 Pro", "iPhone 13 Pro landscape", 
194      "iPhone 13 Pro Max", "iPhone 13 Pro Max landscape", 
195      "iPhone 13 Mini", "iPhone 13 Mini landscape", 
196      "iPhone 14", "iPhone 14 landscape", 
197      "iPhone 14 Plus", "iPhone 14 Plus landscape", 
198      "iPhone 14 Pro", "iPhone 14 Pro landscape", 
199      "iPhone 14 Pro Max", "iPhone 14 Pro Max landscape", 
200      "iPhone 15", "iPhone 15 landscape", 
201      "iPhone 15 Plus", "iPhone 15 Plus landscape", 
202      "iPhone 15 Pro", "iPhone 15 Pro landscape", 
203      "iPhone 15 Pro Max", "iPhone 15 Pro Max landscape", 
204      "JioPhone 2", "JioPhone 2 landscape", 
205      "Kindle Fire HDX", "Kindle Fire HDX landscape", 
206      "LG Optimus L70", "LG Optimus L70 landscape", 
207      "Microsoft Lumia 550", 
208      "Microsoft Lumia 950", "Microsoft Lumia 950 landscape", 
209      "Nexus 10", "Nexus 10 landscape", 
210      "Nexus 4", "Nexus 4 landscape", 
211      "Nexus 5", "Nexus 5 landscape", 
212      "Nexus 5X", "Nexus 5X landscape", 
213      "Nexus 6", "Nexus 6 landscape", 
214      "Nexus 6P", "Nexus 6P landscape", 
215      "Nexus 7", "Nexus 7 landscape", 
216      "Nokia Lumia 520", "Nokia Lumia 520 landscape", 
217      "Nokia N9", "Nokia N9 landscape", 
218      "Pixel 2", "Pixel 2 landscape", 
219      "Pixel 2 XL", "Pixel 2 XL landscape", 
220      "Pixel 3", "Pixel 3 landscape", 
221      "Pixel 4", "Pixel 4 landscape", 
222      "Pixel 4a (5G)", "Pixel 4a (5G) landscape", 
223      "Pixel 5", "Pixel 5 landscape", 
224      "Moto G4", "Moto G4 landscape"
225      ],
226      "enumTitles": [
227      "Blackberry PlayBook", "Blackberry PlayBook (Landscape)", 
228      "BlackBerry Z30", "BlackBerry Z30 (Landscape)", 
229      "Galaxy Note 3", "Galaxy Note 3 (Landscape)", 
230      "Galaxy Note II", "Galaxy Note II (Landscape)", 
231      "Galaxy S III", "Galaxy S III (Landscape)", 
232      "Galaxy S5", "Galaxy S5 (Landscape)", 
233      "Galaxy S8", "Galaxy S8 (Landscape)", 
234      "Galaxy S9+", "Galaxy S9+ (Landscape)", 
235      "Galaxy Tab S4", "Galaxy Tab S4 (Landscape)", 
236      "iPad", "iPad (Landscape)", 
237      "iPad (Gen 6)", "iPad (Gen 6) (Landscape)", 
238      "iPad (Gen 7)", "iPad (Gen 7) (Landscape)", 
239      "iPad Mini", "iPad Mini (Landscape)", 
240      "iPad Pro", "iPad Pro (Landscape)", 
241      "iPad Pro 11", "iPad Pro 11 (Landscape)", 
242      "iPhone 4", "iPhone 4 (Landscape)", 
243      "iPhone 5", "iPhone 5 (Landscape)", 
244      "iPhone 6", "iPhone 6 (Landscape)", 
245      "iPhone 6 Plus", "iPhone 6 Plus (Landscape)", 
246      "iPhone 7", "iPhone 7 (Landscape)", 
247      "iPhone 7 Plus", "iPhone 7 Plus (Landscape)", 
248      "iPhone 8", "iPhone 8 (Landscape)", 
249      "iPhone 8 Plus", "iPhone 8 Plus (Landscape)", 
250      "iPhone SE", "iPhone SE (Landscape)", 
251      "iPhone X", "iPhone X (Landscape)", 
252      "iPhone XR", "iPhone XR (Landscape)", 
253      "iPhone 11", "iPhone 11 (Landscape)", 
254      "iPhone 11 Pro", "iPhone 11 Pro (Landscape)", 
255      "iPhone 11 Pro Max", "iPhone 11 Pro Max (Landscape)", 
256      "iPhone 12", "iPhone 12 (Landscape)", 
257      "iPhone 12 Pro", "iPhone 12 Pro (Landscape)", 
258      "iPhone 12 Pro Max", "iPhone 12 Pro Max (Landscape)", 
259      "iPhone 12 Mini", "iPhone 12 Mini (Landscape)", 
260      "iPhone 13", "iPhone 13 (Landscape)", 
261      "iPhone 13 Pro", "iPhone 13 Pro (Landscape)", 
262      "iPhone 13 Pro Max", "iPhone 13 Pro Max (Landscape)", 
263      "iPhone 13 Mini", "iPhone 13 Mini (Landscape)", 
264      "iPhone 14", "iPhone 14 (Landscape)", 
265      "iPhone 14 Plus", "iPhone 14 Plus (Landscape)", 
266      "iPhone 14 Pro", "iPhone 14 Pro (Landscape)", 
267      "iPhone 14 Pro Max", "iPhone 14 Pro Max (Landscape)", 
268      "iPhone 15", "iPhone 15 (Landscape)", 
269      "iPhone 15 Plus", "iPhone 15 Plus (Landscape)", 
270      "iPhone 15 Pro", "iPhone 15 Pro (Landscape)", 
271      "iPhone 15 Pro Max", "iPhone 15 Pro Max (Landscape)", 
272      "JioPhone 2", "JioPhone 2 (Landscape)", 
273      "Kindle Fire HDX", "Kindle Fire HDX (Landscape)", 
274      "LG Optimus L70", "LG Optimus L70 (Landscape)", 
275      "Microsoft Lumia 550", 
276      "Microsoft Lumia 950", "Microsoft Lumia 950 (Landscape)", 
277      "Nexus 10", "Nexus 10 (Landscape)", 
278      "Nexus 4", "Nexus 4 (Landscape)", 
279      "Nexus 5", "Nexus 5 (Landscape)", 
280      "Nexus 5X", "Nexus 5X (Landscape)", 
281      "Nexus 6", "Nexus 6 (Landscape)", 
282      "Nexus 6P", "Nexus 6P (Landscape)", 
283      "Nexus 7", "Nexus 7 (Landscape)", 
284      "Nokia Lumia 520", "Nokia Lumia 520 (Landscape)", 
285      "Nokia N9", "Nokia N9 (Landscape)", 
286      "Pixel 2", "Pixel 2 (Landscape)", 
287      "Pixel 2 XL", "Pixel 2 XL (Landscape)", 
288      "Pixel 3", "Pixel 3 (Landscape)", 
289      "Pixel 4", "Pixel 4 (Landscape)", 
290      "Pixel 4a (5G)", "Pixel 4a (5G) (Landscape)", 
291      "Pixel 5", "Pixel 5 (Landscape)", 
292      "Moto G4", "Moto G4 (Landscape)"
293      ]
294    },
295    "window_Width": {
296      "title": "Window Width",
297      "type": "integer",
298      "description": "The width of the browser window in pixels.",
299      "default": 1920,
300      "unit": "px"   
301    },
302    "window_Height": {
303      "title": "Window Height",
304      "type": "integer",
305      "description": "The height of the browser window in pixels.",
306      "default": 1080,
307      "unit": "px"
308    },
309    "scrollToBottom": {
310      "sectionCaption": "Scrolling Option",
311      "title": "Scroll to Bottom",
312      "type": "boolean",
313      "description": "Should the browser scroll to the bottom of the page before taking a screenshot?",
314      "default": false
315    },
316    "delayAfterScrolling": {
317      "title": "Delay After Scrolling",
318      "type": "integer",
319      "description": "Specify the delay (in milliseconds) after scrolling to the bottom before taking the screenshot. Only used if 'Wait for Network Idle After Scrolling' is not enabled.",
320      "default": 300,
321      "unit": "ms"
322
323    },
324    "cookies": {
325      "sectionCaption": "Cookies",
326      "sectionDescription": "You can use cookie editors such as [Cookie Editor](https://cookie-editor.com/) or [Copy Cookies](https://chromewebstore.google.com/detail/copy-cookies/jcbpglbplpblnagieibnemmkiamekcdg) to format cookies.",
327      "title": "Cookies",
328      "type": "array",
329      "default": [],
330      "description": "Cookies to be used for the browsing session, formatted as JSON objects.",
331      "editor": "json"
332    }
333  },
334  "required": ["linkUrls"],
335  "additionalProperties": true
336}

src/main.js

1import puppeteer from 'puppeteer';
2import path from 'path';
3import { Actor, Dataset } from 'apify';
4import fs from 'fs/promises';
5import { KnownDevices } from 'puppeteer';
6import { exec } from 'child_process';
7
8const wait = (milliseconds) => {
9    return new Promise(resolve => setTimeout(resolve, milliseconds));
10};
11
12class ScreenHot {
13    constructor(actorInput) {
14        this.linkUrls = actorInput.linkUrls;
15        this.fullPage = actorInput.fullPage;
16        this.waitUntil = actorInput.waitUntil;
17        this.device = actorInput.device;
18        this.timeouT = actorInput.timeouT;
19        this.maxRetries = actorInput.maxRetries;
20        this.window_Width = actorInput.window_Width;
21        this.window_Height = actorInput.window_Height;
22        this.delayBeforeScreenshot = actorInput.delayBeforeScreenshot;
23        this.outputFormat = actorInput.outputFormat;
24        this.scrollToBottom = actorInput.scrollToBottom;
25        this.delayAfterScrolling = actorInput.delayAfterScrolling;
26        this.cookies = actorInput.cookies;
27        this.formaT = actorInput.formaT;
28        this.printBackground = actorInput.printBackground;
29        this.toP = actorInput.toP;
30        this.righT = actorInput.righT;
31        this.bottoM = actorInput.bottoM;
32        this.lefT = actorInput.lefT;
33        this.frameCounT = actorInput.frameCounT;
34        this.frameIntervaL = actorInput.frameIntervaL;
35        this.infiniteScroll = actorInput.infiniteScroll;  // Default to false if not provided
36        this.scrollStep = actorInput.scrollSteP;
37        this.timefullPagE = actorInput.timefullPagE;
38        this.timeout = actorInput.timeouT;
39        this.proxyConfig = actorInput.proxyConfig;  // Accept proxy config in the constructor
40        this.userAgent = actorInput.userAgent || '';  // User-Agent passed by the user, if any
41
42    }
43
44    async loadUserAgentsFromFile(filePath) {
45        try {
46            const data = await fs.readFile(filePath, 'utf-8');
47            const userAgents = data.split('\n').map(line => line.trim()).filter(Boolean);
48            return userAgents;
49        } catch (error) {
50            console.error(`Error reading User-Agent file: ${error.message}`);
51            return [];
52        }
53    }
54    
55    async selectRandomUserAgent() {
56        const userAgents = await this.loadUserAgentsFromFile('user-agents.txt');
57        if (userAgents.length > 0) {
58            const randomIndex = Math.floor(Math.random() * userAgents.length);
59            return userAgents[randomIndex];
60        } else {
61            return '';
62        }
63    }
64
65    async navigatePage(page, linkUrl) {
66        let timeout = this.timeouT * 1000;
67        for (let attempt = 1; attempt <= this.maxRetries; attempt++) {
68            try {
69                await page.goto(linkUrl, { waitUntil: this.waitUntil, timeout: timeout });
70                return;  // Exit if the page loads successfully
71            } catch (error) {
72                console.error(`Error navigating to ${linkUrl}: ${error}. Attempt ${attempt} of ${this.maxRetries}.`);
73                if (attempt < this.maxRetries) {
74                    console.log(`Retrying ${linkUrl}...`);
75                } else {
76                    throw new Error(`Failed to load ${linkUrl} after ${this.maxRetries} attempts.`);
77                }
78            }
79        }
80    }
81
82    async run() {
83
84        let content_Type;
85        let Buffer;
86        let fileName;
87        let filePath;
88
89        await fs.mkdir('Output_Files', { recursive: true });
90        
91        // Initialize Apify Proxy if provided in input
92        let proxyUrl;
93        try {
94            if (this.proxyConfig) {
95                const proxyConfiguration = await Actor.createProxyConfiguration(this.proxyConfig);
96                const proxyInfo = await proxyConfiguration.newProxyInfo();
97                proxyUrl = proxyInfo.url;  // Get the proxy URL
98                console.log(`Using proxy: ${proxyUrl}`);
99            }
100        } catch (error) {
101            console.error('Error setting up proxy:', error.message);
102            proxyUrl = null;  // Proceed without proxy if there's an error
103        }
104        
105        // Set browser arguments with or without proxy
106        const browserArgs = [
107            `--window-size=${this.window_Width},${this.window_Height}`,
108            '--no-sandbox',
109            '--disable-setuid-sandbox',
110            '--ignore-certificate-errors'  // Ignore certificate errors for SSL bypass
111        ];
112        if (proxyUrl) {
113            browserArgs.push(`--proxy-server=${proxyUrl}`);
114        }
115        
116        // Select a random User-Agent if not provided
117        if (!this.userAgent) {
118            this.userAgent = await this.selectRandomUserAgent();
119        }
120        
121        let browser;
122        try {
123            // Launch the browser with proxy, SSL bypass, and User-Agent settings
124            browser = await puppeteer.launch({
125                headless: true,
126                args: browserArgs,
127                ignoreHTTPSErrors: true  // This option allows bypassing SSL certificate errors
128            });
129            console.log('Browser launched successfully with SSL bypass.');
130        } catch (error) {
131            console.error('Failed to launch the browser:', error.message);
132            throw new Error('Browser launch failed due to proxy, SSL issues, or other problems.');
133        }
134        
135        try {
136            const page = await browser.newPage();
137        
138            // Set the User-Agent
139            if (this.userAgent) {
140                await page.setUserAgent(this.userAgent);
141                console.log(`User-Agent set: ${this.userAgent}`);
142            }
143
144
145            // If a device is provided, emulate it
146            if (this.device) {
147                if (KnownDevices[this.device]) {
148                    console.log(`Emulating device: ${this.device}`);
149                    await page.emulate(KnownDevices[this.device]);
150                } else {
151                    console.log(`Setting custom viewport for ${this.device}`);
152                    await page.setViewport({ width: this.window_Width, height: this.window_Height });
153                }
154            } else {
155                await page.setViewport({ width: this.window_Width, height: this.window_Height });
156            }
157
158            // Set cookies before navigation
159            if (this.cookies.length > 0) {
160                console.log(`Setting cookies: ${this.cookies.length}`);
161                await page.setCookie(...this.cookies);
162            } else {
163                console.log(`Continuing without cookies`);
164            }
165
166            for (const linkUrl of this.linkUrls) {
167                console.log(`Processing URL: ${linkUrl}`);
168                try {
169                    await this.navigatePage(page, linkUrl);
170
171                    // Scrolling feature
172                    if (this.scrollToBottom) {
173                        await this.scrollPageToBottom(page, this.delayAfterScrolling);
174                    }
175                    // Delay before capturing output
176                    if (this.delayBeforeScreenshot > 0) {
177                        console.log(`Waiting ${this.delayBeforeScreenshot} milliseconds before capturing output.`);
178                        await wait(this.delayBeforeScreenshot);
179                    }
180                    // Handle different output formats
181                    if (this.outputFormat === 'mp4' || this.outputFormat === 'gif') {
182                        content_Type = this.outputFormat === 'gif' ? "image/gif" : "video/mp4";
183                        const frameDir = path.join('Output_Files', 'frames');
184                        await fs.mkdir(frameDir, { recursive: true });
185                        const frameCount = this.frameCounT;  // Number of frames to capture
186                        const frameInterval = this.frameIntervaL;  // Capture frame every second
187
188                        // Capture frames based on the fullPage flag
189                        if (this.fullPage === true) {
190                            // Capture frames and scroll down
191                            await this.fullPageVideo(page, frameDir, this.scrollStep);
192                        } else {
193                            for (let i = 0; i < frameCount; i++) {
194                                const framePath = path.join(frameDir, `frame${i.toString().padStart(3, '0')}.png`);
195                                await page.screenshot({ path: framePath });
196                                console.log(`Captured frame: ${framePath}`);
197                                await wait(frameInterval / 1000);  // Wait before capturing next frame
198                            }
199                        }
200
201                        // Check if frames are saved
202                        const savedFrames = await fs.readdir(frameDir);
203                        console.log(`Frames saved: ${savedFrames.length}`);
204                        if (savedFrames.length === 0) {
205                            throw new Error("No frames captured, cannot create GIF or video.");
206                        }
207
208                        // Create the video or GIF
209                        const scaleWidth = this.window_Width || -1; // Get the scale width from user input (or default to -1)
210                        const scaleHeight = this.window_Height || -1; // Get the scale height from user input (or default to -1)
211
212                        // Ensure width and height are even numbers
213                        const adjustedWidth = scaleWidth === -1 ? -1 : (scaleWidth % 2 === 0 ? scaleWidth : scaleWidth - 1);
214                        const adjustedHeight = scaleHeight === -1 ? -1 : (scaleHeight % 2 === 0 ? scaleHeight : scaleHeight - 1);
215
216                        const scaleOption = `scale=${adjustedWidth}:${adjustedHeight}`; // Scaling option for ffmpeg
217
218                        fileName = `${Array.from({ length: 16 }, () => Math.random().toString(36)[2]).join('')}.${this.outputFormat}`;
219                        filePath = path.join('Output_Files', fileName);
220
221                        const ffmpegCommand = this.outputFormat === 'gif'
222                            ? `ffmpeg -f image2 -i ${frameDir}/frame%03d.png -vf "fps=10,${scaleOption}:flags=lanczos" ${filePath}`
223                            : `ffmpeg -framerate 1 -i ${frameDir}/frame%03d.png -vf "${scaleOption},format=yuv420p" -c:v libx264 -r 30 -pix_fmt yuv420p ${filePath}`;
224
225                        await new Promise((resolve, reject) => {
226                            exec(ffmpegCommand, (error, stdout, stderr) => {
227                                if (error) {
228                                    console.error(`Error creating ${this.outputFormat}: ${error.message}`);
229                                    console.error(`FFmpeg stderr: ${stderr}`);
230                                    console.error(`FFmpeg stdout: ${stdout}`);
231                                    reject(new Error(`FFmpeg command failed with error: ${error.message}`));
232                                    return;
233                                }
234                                console.log(`Created ${this.outputFormat} at: ${filePath}`);
235                                resolve();
236                            });
237                        });
238
239                        // Clean up frames after creating the video or GIF
240                        await fs.rm(frameDir, { recursive: true });
241                        Buffer = await fs.readFile(filePath);
242                    }
243                    else if (this.outputFormat === 'pdf') {
244                            if (this.delayBeforeScreenshot > 0) {await wait(this.delayBeforeScreenshot);}
245            
246                            // Generate PDF
247                            content_Type = "application/pdf";
248                            fileName = `${Array.from({ length: 16 }, () => Math.random().toString(36)[2]).join('')}.${this.outputFormat}`;
249                            filePath = path.join('Output_Files', fileName);
250                            Buffer = await page.pdf({
251                                path: filePath,
252                                format: this.formaT,
253                                printBackground: this.printBackground,
254                                margin: {
255                                    top: this.toP,
256                                    right: this.righT,
257                                    bottom: this.bottoM,
258                                    left: this.lefT,
259                                },
260                            });
261                            console.log(`PDF saved as: ${filePath}`);
262
263                        }
264                    else if (this.outputFormat === 'jpeg' || this.outputFormat === 'png') {
265                        // Capture screenshot
266                        content_Type = this.outputFormat === 'jpeg' ? "image/jpeg" : "image/png";
267                        fileName = `${Array.from({ length: 16 }, () => Math.random().toString(36)[2]).join('')}.${this.outputFormat}`;
268                        filePath = path.join('Output_Files', fileName);
269                        Buffer = await page.screenshot({ path: filePath, type: this.outputFormat, fullPage: this.fullPage });
270                    }
271                    // Upload the buffer to Apify Key-Value Store
272                    const Apify = await Actor.openKeyValueStore();
273                    await Apify.setValue(fileName, Buffer, { contentType: content_Type });
274                    const storeId = process.env.APIFY_DEFAULT_KEY_VALUE_STORE_ID;
275                    
276                    // Generate URL
277                    const screenshot_url = `https://api.apify.com/v2/key-value-stores/${storeId}/records/${fileName}`;
278                    console.log(`- ${screenshot_url}`);
279                    const screenshot_image = screenshot_url
280                    await Dataset.pushData({ screenshot_image, content_Type, linkUrl, screenshot_url });
281
282                } catch (error) {
283                    console.error(`Error processing ${linkUrl}: ${error}`);
284                }
285            }
286
287        } finally {
288            await browser.close();
289        }
290    }
291
292    async scrollPageToBottom(page, delayAfterScrolling) {
293        console.log('Scrolling to the bottom of the page...');
294    
295        const startTime = Date.now();  // Record the start time
296        const timefullPage = this.timefullPagE*1000;  // Set the time limit to 40s (40,000 milliseconds)
297        console.log(`Time for Full Page Capture limit: ${this.timefullPagE} seconds`);
298
299
300        await page.evaluate(async (startTime, timefullPage, infiniteScroll) => {
301            await new Promise((resolve) => {
302                const distance = 100;
303                const timer = setInterval(() => {
304                    const now = Date.now();
305                    const scrollHeight = document.body.scrollHeight;
306                    window.scrollBy(0, distance);
307    
308                    // Stop if reaching the bottom or if timefullPage occurs (when infiniteScroll is false)
309                    if (window.innerHeight + window.scrollY >= scrollHeight || (!infiniteScroll && (now - startTime) >= timefullPage)) {
310                        clearInterval(timer);
311                        resolve();
312                    }
313                }, 100);
314            });
315        }, startTime, timefullPage, this.infiniteScroll);  // Pass infiniteScroll option to page context
316    
317        if (delayAfterScrolling > 0) {
318            console.log(`Waiting ${delayAfterScrolling} milliseconds after scrolling.`);
319            await wait(delayAfterScrolling);
320        }
321    }
322
323    async fullPageVideo(page, frameDir, scrollStep) {
324        let frameCount = 0;
325        const frameInterval = this.frameIntervaL;  // Interval in milliseconds between screenshots
326        let previousHeight;
327        let currentHeight = 0;
328        
329        const startTime = Date.now();  // Record the start time
330        const timefullPage = this.timefullPagE*1000;  // Set the time limit to 40s (40,000 milliseconds)
331    
332        // Scroll down, take a screenshot at each step, and stop when reaching the bottom or timefullPage
333        do {
334            const now = Date.now();
335    
336            // Check timefullPage only if infiniteScroll is false
337            if (!this.infiniteScroll && (now - startTime) >= timefullPage) {
338                console.log(`Stopping scrolling in ${this.timefullPagE} seconds due to (Time for Full Page Capture) Section.`);
339                break;
340            }
341    
342            const fileName = path.join(frameDir, `frame${frameCount.toString().padStart(3, '0')}.png`);
343            await page.screenshot({ path: fileName });
344            console.log(`Captured frame: ${fileName}`);
345            frameCount++;
346            await wait(frameInterval / 1000);  // Wait before capturing the next frame
347    
348            // Scroll down by a smaller step for smoother scrolling
349            previousHeight = await page.evaluate(() => window.scrollY);
350            await page.evaluate((step) => window.scrollBy(0, step), scrollStep);  // Scroll by scrollStep pixels
351    
352           
353            // Check if the page height has changed
354            currentHeight = await page.evaluate(() => window.scrollY);
355    
356        } while (currentHeight > previousHeight);  // Continue scrolling until no new content is loaded (bottom reached)
357    
358    }
359}
360
361(async () => {
362    await Actor.init();  // Initialize the Actor
363    const actorInput = await Actor.getInput() || {};
364    const screenHot = new ScreenHot(actorInput);
365    await screenHot.run();
366    await Actor.exit();  // Exit the Actor
367})();

.editorconfig

1root = true
2
3[*]
4indent_style = space
5indent_size = 4
6charset = utf-8
7trim_trailing_whitespace = true
8insert_final_newline = true
9end_of_line = lf

.eslintrc

1{
2    "extends": "@apify",
3    "root": true
4}

package.json

1{
2    "name": "crawlee-puppeteer-javascript",
3    "version": "0.0.1",
4    "type": "module",
5    "description": "This is an example of an Apify actor.",
6    "dependencies": {
7        "apify": "^3.1.10",
8        "crawlee": "^3.5.4",
9        "puppeteer": "^23.6.0"
10    },
11    "devDependencies": {
12        "@apify/eslint-config": "^0.4.0",
13        "eslint": "^8.50.0"
14    },
15    "scripts": {
16        "start": "node src/main.js",
17        "test": "echo \"Error: oops, the actor has no tests yet, sad!\" && exit 1"
18    },
19    "author": "It's not you it's me",
20    "license": "ISC"
21}

user-agents.txt

1Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36
2Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Firefox/102.0
3Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:112.0) Gecko/20100101 Firefox/112.0
4Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Edge/113.0.0.0 Safari/537.36
5Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36
6Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.5938.132 Safari/537.36
7Mozilla/5.0 (Macintosh; Intel Mac OS X 13_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.5938.132 Safari/537.36
8Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:118.0) Gecko/20100101 Firefox/118.0
9Mozilla/5.0 (Macintosh; Intel Mac OS X 13_4; rv:118.0) Gecko/20100101 Firefox/118.0
10Mozilla/5.0 (Macintosh; Intel Mac OS X 13_4_1) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.6 Safari/605.1.15
11Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.5938.132 Safari/537.36 Edg/117.0.2045.55
12Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.5938.132 Safari/537.36 OPR/101.0.4843.43
13Mozilla/5.0 (Macintosh; Intel Mac OS X 13_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.5938.132 Safari/537.36 Edg/117.0.2045.55
14Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.5938.132 Safari/537.36 Vivaldi/5.7.2921.66
15Mozilla/5.0 (Windows NT 11.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.5938.132 Safari/537.36
16Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.5938.132 Safari/537.36 Brave/1.58.113
17Mozilla/5.0 (Macintosh; Intel Mac OS X 13_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.5938.132 Safari/537.36 Brave/1.58.113
18Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.5845.110 Safari/537.36
19Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.6 Safari/605.1.15
20Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:114.0) Gecko/20100101 Firefox/114.0
21Mozilla/5.0 (Macintosh; Intel Mac OS X 12_5_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.5790.102 Safari/537.36
22Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Edge/116.0.1938.69
23Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.6 Safari/605.1.15
24Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.5845.97 Safari/537.36
25Mozilla/5.0 (Macintosh; Intel Mac OS X 12_4_0) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.5 Safari/605.1.15
26Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.5790.171 Safari/537.36
27Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36
28Mozilla/5.0 (Macintosh; Intel Mac OS X 13_3_1) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.4 Safari/605.1.15
29Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Edge/116.0.1938.81 Safari/537.36
30Mozilla/5.0 (Macintosh; Intel Mac OS X 13_5_1) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.6 Safari/605.1.15
31Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:116.0) Gecko/20100101 Firefox/116.0
32Mozilla/5.0 (Macintosh; Intel Mac OS X 12_6_8) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.6.8 Safari/605.1.15
33Mozilla/5.0 (Macintosh; Intel Mac OS X 13_4_1) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.4.1 Safari/605.1.15
34Mozilla/5.0 (Macintosh; Intel Mac OS X 13_2) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.3 Safari/605.1.15
35Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.5790.110 Safari/537.36
36Mozilla/5.0 (Macintosh; Intel Mac OS X 13_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.6 Safari/605.1.15
37Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:115.0) Gecko/20100101 Firefox/115.0
38Mozilla/5.0 (Macintosh; Intel Mac OS X 13_5) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.5 Safari/605.1.15
39Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Edge/117.0.2045.47 Safari/537.36
40Mozilla/5.0 (Macintosh; Intel Mac OS X 13_1) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.1 Safari/605.1.15
41Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.5735.199 Safari/537.36
42Mozilla/5.0 (Macintosh; Intel Mac OS X 13_0_1) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.1 Safari/605.1.15
43Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Edge/114.0.1823.43 Safari/537.36
44Mozilla/5.0 (Macintosh; Intel Mac OS X 12_5_1) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.6.1 Safari/605.1.15
45Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36
46Mozilla/5.0 (Macintosh; Intel Mac OS X 12_6_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.6.6 Safari/605.1.15
47Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:113.0) Gecko/20100101 Firefox/113.0
48Mozilla/5.0 (Macintosh; Intel Mac OS X 13_3) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.3 Safari/605

Developer

Omar Abdlhakim

Actor metrics

9 monthly users
3 stars
91.7% runs succeeded
Created in Oct 2024
Modified 4 days ago

Categories

Automation

SEO tools

Instagram Scraper

apify/instagram-scraper

Scrape and download Instagram posts, profiles, places, hashtags, photos, and comments. Get data from Instagram using one or more Instagram URLs or search queries. Export scraped data, run the scraper via API, schedule and monitor runs or integrate with other tools.

Apify

58k

476

Facebook ad library scraper

curious_coder/facebook-ads-library-scraper

Scrape facebook ads search and ads run by facebook pages - Fast and lightweight

Curious Coder

1.8k

Google Search Results Scraper

apify/google-search-scraper

Scrape Google Search Engine Results Pages (SERPs). Select the country or language and extract organic and paid results, AI overviews, ads, queries, People Also Ask, prices, reviews, like a Google SERP API. Export scraped data, run the scraper via API, schedule runs, or integrate with other tools.

Apify

49.3k

230

🔥 LinkedIn Jobs Scraper

bebity/linkedin-jobs-scraper

ℹ️ Designed for both personal and professional use, simply enter your desired job title and location to receive a tailored list of job opportunities. Try it today!

Bebity

3.9k

Google Trends Scraper

emastra/google-trends-scraper

Scrape data from Google Trends by search terms or URLs. Specify locations, define time ranges, select categories to get interest by subregion and over time, related queries and topics, and more. Export scraped data, run the scraper via API, schedule and monitor runs, or integrate with other tools.

Emiliano Mastragostino

3.7k

Instagram Followers Count Scraper

apify/instagram-followers-count-scraper

Scrape the number of followers & follows from any Instagram profile. Schedule the scraper to run regularly to monitor how the numbers change. You can also export scraped data, run the scraper via API, monitor runs or integrate with other tools.

Apify

Traffic Generator (Youtube, Web, Etsy, Behance and many more!)

epctex/traffic-generator

Maximize your website's performance and visibility with our Traffic Generator. Drive targeted traffic, simulate page views, and stress-test against potential threats. With the power to generate millions of visits, it's the ultimate solution for boosting your online presence.

epctex

22.2k

161

Youtube Video Downloader

epctex/youtube-video-downloader

Effortlessly download YouTube videos of your preferred quality with our user-friendly Video Downloader. Try it now!

epctex

373

PDF Text Extractor

jirimoravcik/pdf-text-extractor

PDF Text Extractor allows you to extract text from PDF files. It also supports chunking of the text to prepare the data for usage with large language models.