Get localStorage, sessionStorage and cookies from logins for usage in other actors.

Usage

This actor can help you (re)use logged in sessions for your website and serivces, abstracting away the need for developing your own login mechanism. It uses a named session storage, so you when you request a new session, it will be readily available. It's tailored to work seamlessly on Apify platform and other actors.

It's more low-level than other actors, but it tries to cover the most common use cases like:

• Single Page Applications
• Server Side Rendered websites
• Ajax login calls
• Multi step logins such as username on one page, password on another

You may call directly from your actor, or use the INPUT.json to create a task or scheduled to keep seeding your session storage with new sessions. It cannot deal with 2FA or captchas (yet).

1// in your actor
2const storageName = 'session-example';
3
4const call = await Apify.call('pocesar/login-session', {
5    username: 'username',
6    password: 'password',
7    website: [{ url: 'http://example.com' }], // the RequestList format
8    cookieDomains: [
9        "http://example.com"
10    ],
11    sessionConfig: {
12        storageName,
13        maxAgeSecs: 3600,
14        maxUsageCount: 10,
15        maxPoolSize: 120
16    },
17    steps: [{
18        username: {
19            selector: "input#email", // the input that receives the username
20            timeoutMillis: 10000 // optional timeout in ms
21        },
22        password: {
23            selector: "input#password" // the input that receives the password
24        },
25        submit: {
26            selector: "input[type=\"submit\"]", // the button that executes the login
27        },
28        failed: {
29            selector: "[role=\"alert\"],#captcha", // usually an error that tells the login failed
30            timeoutMillis: 10000 // optional timeout in ms
31        },
32        waitForMillis: 15000 // optional "sleep" in ms to consider the page as "settled"
33    }]
34});
35
36const { session, error } = call.output;
37
38// if it fails, the error will be filled with something
39// otherwise, the session will have the Session parameters, that can be
40// instantiated manually using `new Apify.Session({ ...session, sessionPool })`
41
42// load the session pool from the storage, so it has our new
43// session. this might change in the future
44const sessionPool = await Apify.openSessionPool({
45    persistStateKeyValueStoreId: storageName
46});
47
48const sessionJustCreated = sessionPool.sessions.find(s => s.id === session.id);
49
50/**
51 * the complete Cookie string for usage on the header
52 */
53sessionJustCreated.getCookieString('http://example.com');
54
55/**
56 * contains the User-Agent used for the login request.
57 * the same userAgent must be set between uses so there's no
58 * conflict and blocks. Set this as your User-Agent header
59 **/
60sessionJustCreated.userData.userAgent;
61
62/**
63 * the proxyUrl used, can be empty.
64 * Set this as your proxyUrl parameter in crawlers.
65 *
66 * This might be undefined if you didn't use any proxies
67 */
68sessionJustCreated.userData.proxyUrl;
69
70/**
71 * object containing any sessionStorage content, useful for JWT tokens.
72 * Useful for using in PuppeteerCrawler
73 */
74sessionJustCreated.userData.sessionStorage;
75
76/**
77 * object containing any localStorage content, useful for JWT
78 * tokens. Useful for using in PuppeteerCrawler
79 */
80sessionJustCreated.userData.localStorage;

Login locally then use the session on the platform

You can login locally, executing the login-session actor on your machine, make sure you're logged in to the platform using apify login and using forceCloud input option, like this:

1{
2    "username": "username",
3    "password": "s3cr3tp4ssw0rd",
4    "website": [{ "url": "https://example.com/" }],
5    "sessionConfig": {
6        "storageName": "example-login-sessions" // need to use this
7    },
8    "steps": [
9        {
10            "username": { "selector": "#email" },
11            "password": { "selector": "#password" },
12            "submit": { "selector": "input[type=\"submit\"]" },
13            "success": { "selector": ".main-menu", "timeoutMillis": 10000 },
14            "failed": {
15                "selector": ".login.error",
16                "timeoutMillis": 10000
17            },
18            "waitForMillis": 30000
19        }
20    ],
21    "cookieDomains": ["https://example.com"],
22    "proxyConfiguration": {
23        "useApifyProxy": false
24    },
25    "forceCloud": true // this forces the login to be saved on platform Storage (https://my.apify.com/storage#/keyValueStores)
26}

Place this in your apify_storage/key_value_stores/default/INPUT.json file, then run locally:

$ apify run --purge

The session will be created in your Apify platform account, under the storageName you provided, but using your local IP. Using this, you're able to avoid PIN requests and security checkpoint screens.

Input Recipes

Here are some real-life examples of INPUT.json that you may use:

Gmail

1{
2    "username": "username",
3    "password": "password",
4    "website": [{ "url": "https://accounts.google.com/signin/v2/identifier?service=mail&passive=true&flowName=GlifWebSignIn&flowEntry=ServiceLogin" }],
5    "cookieDomains": [
6        "https://mail.google.com",
7        "https://accounts.google.com",
8        "https://google.com"
9    ],
10    "steps": [{
11        "username": {
12            "selector": "#identifierId"
13        },
14        "submit": {
15            "selector": "#identifierNext"
16        },
17        "success": {
18            "selector": "input[type=\"password\"]",
19            "timeoutMillis": 10000
20        },
21        "failed": {
22            "selector": "#identifierId[aria-invalid=\"true\"],iframe[src*=\"CheckConnection\"]"
23        },
24        "waitForMillis": 30000
25    }, {
26        "password": {
27            "selector": "input[type=\"password\"]"
28        },
29        "submit": {
30            "selector": "#passwordNext",
31            "timeoutMillis": 15000
32        },
33        "failed": {
34            "selector": "input[type=\"password\"][aria-invalid=\"true\"],iframe[src*=\"CheckConnection\"]",
35            "timeoutMillis": 5000
36        },
37        "success": {
38            "selector": "link[href*=\"mail.google.com\"]",
39            "timeoutMillis": 10000
40        },
41        "waitForMillis": 30000
42    }]
43}

Facebook

1{
2    "username": "username",
3    "password": "password",
4    "website": [{ "url": "https://www.facebook.com/" }],
5    "cookieDomains": [
6        "https://facebook.com"
7    ],
8    "steps": [{
9        "username": {
10            "selector": "#login_form [type=\"email\"]"
11        },
12        "password": {
13            "selector": "#login_form [type=\"password\"]"
14        },
15        "submit": {
16            "selector": "#login_form [type=\"submit\"]"
17        },
18        "success": {
19            "selector": "body.home",
20            "timeoutMillis": 10000
21        },
22        "failed": {
23            "selector": "body.login_page,body.UIPage_LoggedOut",
24            "timeoutMillis": 10000
25        },
26        "waitForMillis": 30000
27    }]
28}

Twitter

1{
2    "username": "username",
3    "password": "password",
4    "website": [{ "url": "https://twitter.com/login" }],
5    "cookieDomains": [
6        "https://twitter.com"
7    ],
8    "steps": [{
9        "username": {
10            "selector": "h1 ~ form [name=\"session[username_or_email]\"]",
11            "timeoutMillis": 2000
12        },
13        "password": {
14            "selector": "h1 ~ form [name=\"session[password]\"]",
15            "timeoutMillis": 2000
16        },
17        "submit": {
18            "selector": "h1 ~ form [role=\"button\"][data-focusable]"
19        },
20        "success": {
21            "selector": "h2[role=\"heading\"]",
22            "timeoutMillis": 10000
23        },
24        "failed": {
25            "selector": "h1 ~ form [role=\"button\"][disabled]",
26            "timeoutMillis": 10000
27        },
28        "waitForMillis": 30000
29    }]
30}

Instagram

1{
2    "username": "username",
3    "password": "password",
4    "website": [{ "url": "https://instagram.com" }],
5    "cookieDomains": [
6        "https://www.instagram.com"
7    ],
8    "steps": [{
9        "username": {
10            "selector": "input[name=\"username\"]",
11            "timeoutMillis": 10000
12        },
13        "password": {
14            "selector": "input[name=\"password\"]",
15            "timeoutMillis": 10000
16        },
17        "submit": {
18            "selector": "button[type=\"submit\"]"
19        },
20        "success": {
21            "selector": "img[alt=\"Instagram\"]",
22            "timeoutMillis": 10000
23        },
24        "failed": {
25            "selector": "#slfErrorAlert",
26            "timeoutMillis": 5000
27        },
28        "waitForMillis": 30000
29    }]
30}

LinkedIn

1{
2    "username": "username",
3    "password": "password",
4    "website": [
5        {
6            "url": "https://www.linkedin.com/login?fromSignIn=true&trk=guest_homepage-basic_nav-header-signin"
7        }
8    ],
9    "steps": [
10        {
11            "username": {
12                "selector": "#username"
13            },
14            "password": {
15                "selector": "#password"
16            },
17            "submit": {
18                "selector": ".login__form_action_container button"
19            },
20            "success": {
21                "selector": ".authentication-outlet,.launchpad-cp-enabled",
22                "timeoutMillis": 15000
23            },
24            "failed": {
25                "selector": ".form__input--error,.login__form,.pin-verification-form",
26                "timeoutMillis": 15000
27            },
28            "waitForMillis": 30000
29        }
30    ],
31    "cookieDomains": ["https://www.linkedin.com"]
32}

Related Content

• Log in to website by transferring cookies from web browser (legacy)
• How to log in to a website using Puppeteer
• Session Management

Caveats

• Apify proxy sessions can last at most 24h, so never set your maxAgeSecs greater than this number
• If the proxy fails, the login fails. If the proxy is banned, the login fails.

Example

Example form is in https://now-h3p8398gc.now.sh

License

Apache 2.0