TikTok Scraper

Hi, thanks for opening this issue!

Yes, this is the expected functionality. That is because TikTok is updating these links very quickly as a form of blocking. That is why we've implemented a video downloader straight into this tool. Simply enable the shouldDownloadVideos input option and the videos will get downloaded by the scraper :)

Try it out and let me know how it works, thanks!

Yes, but this makes the usage go up a lot more? You're saying the downloadAddr won't be getting fixed anytime soon?

Yes, that makes the usage go up, because it's downloading the videos.

And well, there is nothing to fix, the downloadAddr is scraped straight from TikTok, that is just how it functions. The link is only available for a short time after scrape, so you could download it quickly as the items are coming into the dataset. But that is essentially what the scraper is doing...

Let me know if this helps, thanks!

Yes, but for me the downloadAddr doesn't work at all. Even if I try to scrape 2 profiles for example, and right when the run finishes, I try to visit the downloadAddr, it still doesn't work.

Right, looking at it again, downloading the videos from the URL requires specific cookies to be set in the request. Basically those that you get from visiting the video detail page. Like ttwid, tt_csrf_token etc.

To add to the above, if you'd like to see a predictable pricing for the scrapes, you could also use TikTok Data Extractor actor here https://console.apify.com/actors/OtzYfK1ndEGdwWFKQ/input, it has a fixed price per 1000 results regardless of download options

Even if I visit the webVideoUrl first and get the ttwid, tt_csrf_token, etc, I still can't access the downloadAddr right after the run finishes. The video player is just greyed out and doesn't play. Am I not understanding something?

Okay, looking into this further, the cookies have to be from the same request as for which the video download URL was generated for. This essentially means that the raw download URL we output is useless...

I will discuss it with our team, but I guess we might as well remove it, when it's practically useless.

I downloaded the results 6 months ago and now the Apify downloadaddr and TikTok downloadaddr do not work. I cannot rerun the searches as that will change my results. How should I access those TikTok videos for the analysis?

I feel like it used to work before though so I'm not sure if it's a very new thing. Perhaps I am remembering it wrong.

I'm not sure too, but either way, if it did change, it was done so by TikTok and there is unfortunately nothing we can do about that.

@Shristi, if you want to redownload the videos, then you can use the IDs of the videos and put them into the scraper directly, without running a new search.

@Lukas, How can I do that? What column (for IDs) should I use? Do I need to rent the actor just to redownload the videos. I appreciate your help.

Simply input the video URLs you want to scrape into the postURLs (Video URLs) input field. So to make the video URLs, simply replace the IDs instead of YOUR_VIDEO_ID to make a valid TikTok URL - https://www.tiktok.com/@/video/YOUR_VIDEO_ID

Yes, you will need to have the Actor rented to use it, or you can simply use our Pay Per Result version of this scraper here - TikTok Data Extractor. The pricing there is very simple, just 4$/1000 results.

I hope this helps, thanks!

@Lukas, I appreciate your reply.

Attached is the data I scraped from TikTok scraper. I do not see a TikTok link here in the dataset that can be inputed into the TikTok Data Extractor. Can you direct me which colomn to use? Should I pay for the TikTok Data Extractor before the search or after? Let me know.

Simply take the id field of the output and create the valid video TikTok URLs with them. Then input them into the postURLs input field. Also, don't forget to turn on the shouldDownloadVideos option.

You pay for the results as they come. E.g., if you scrape 100 videos, you will pay $0.4, which will be taken from your account usage.

When I inputted authorMeta/id [ID] into the Video URL in the TikTok Data Extractor, this error was shown -- "This value does not match pattern ".*tiktok.com/".

What do you think I should do? Which TikTok URL should I input into the TikTok Data Extractor to retrieve the videos?

I appreciate your reply and your patience while helping me with this.