This is a camoufox based version of David Teather's unofficial api wrapper for TikTok.com in python. It re-implements a set of features from the original library, with a shifted focus on using browser automation to allow automatic captcha solves with a hopefully minor trade-off in performance.

pip install git+https://github.com/MEOMcGill/pytok.git@master

PyTok drives camoufox, a Firefox build hardened against fingerprinting. Its browser is downloaded on first launch, or ahead of time with:

python -m camoufox fetch

Scrape as a logged-in account, acquired from an accounts pool. Anonymous sessions barely work any more — TikTok returns empty responses to most endpoints without a login — so this is the supported way to use PyTok. Register and log in an account once (see Accounts, login, and persistent sessions below), then every run comes up already authenticated:

python -m pytok.accounts.cli add   --username you@email.com --password 'your-password'
python -m pytok.accounts.cli login --username you@email.com

Here's a quick bit of code to get the videos from a particular user on TikTok. There's more examples in the examples directory.

import asyncio

from pytok.tiktok import PyTok
from pytok.accounts import AccountsPool

async def main():
    pool = AccountsPool()
    async with await PyTok.from_pool(pool) as api:
        user = api.user(username="therock")
        user_data = await user.info()
        print(user_data)

        videos = []
        async for video in user.videos():
            video_data = await video.info()
            print(video_data)

if __name__ == "__main__":
    asyncio.run(main())

The same pattern works for the other feeds — hashtags, search, and sounds:

async for video in api.hashtag(name="funny").videos(count=100):
    ...
async for video in api.search("news").videos(count=100):
    ...
# Music id: the trailing number of a sound's URL
# (tiktok.com/music/original-sound-7016547803243022337), also in every
# video's `music` dict.
sound = api.sound(id="7016547803243022337")
print(await sound.info())
async for video in sound.videos(count=100):
    ...

Please note pulling data from TikTok takes a while! We recommend leaving the scripts running on a server for a while for them to finish downloading everything. Feel free to play around with the delay constants to either speed up the process or avoid TikTok rate limiting, like so: PyTok(request_delay=10)

PyTok supports scraping as a logged-in account, and managing multiple accounts, via an accounts pool: a SQLite-backed set of TikTok accounts, each with its own persistent Firefox profile (and the browser fingerprint it was first given) and a cookie/identity backup. You register an account and log in once (interactively), and every session afterwards comes up already authenticated from that profile, repairing itself from the cookie backup if the profile's session is lost.

The pool lives in ~/.pytok by default (override with the $PYTOK_HOME env var). The database holds credentials and cookie backups in plaintext, so keep that directory private — it is deliberately kept outside the repo.

Register an account and log in once with the CLI:

# Add the account (credentials are stored in ~/.pytok/accounts.db)
python -m pytok.accounts.cli add --username you@email.com --password 'your-password'

# Open a browser and log in. Complete any email/SMS/captcha verification in the
# window; on success PyTok captures the account identity and a cookie backup.
python -m pytok.accounts.cli login --username you@email.com

# Inspect the pool
python -m pytok.accounts.cli list -v

Then scrape as a logged-in account with PyTok.from_pool, which acquires an available account (or a specific one via username=) already signed in:

import asyncio

from pytok.tiktok import PyTok
from pytok.accounts import AccountsPool

async def main():
    pool = AccountsPool()
    async with await PyTok.from_pool(pool) as api:
        hashtag = api.hashtag(name="fyp")
        async for video in hashtag.videos(count=100):
            print(await video.info())

if __name__ == "__main__":
    asyncio.run(main())

Other useful CLI commands: info <username>, stats, activate/deactivate, release (recover an account left in-use by a crashed run), unlock, and delete. Run python -m pytok.accounts.cli --help for the full list.

Each account's browser profile lives in ~/.pytok/firefox-profiles/<username>, alongside a pytok-fingerprint.json holding the browser fingerprint that profile was first given. Every later session reuses it, so the account keeps appearing from the same device. login --seed-profile <dir> copies a warmed Firefox profile into a new account's profile dir first; the source's fingerprint file is not copied, so the new account gets its own.

Earlier versions kept a Chrome profile per account, which Firefox cannot open. The first time this version opens an accounts database, it forgets those profile dirs (leaving them on disk). Each account then starts a fresh Firefox profile and logs in from its stored cookie backup. If that backup has gone stale, log the account in again with python -m pytok.accounts.cli login --username .... An older PyTok sharing the same database keeps working, since it falls back to its default Chrome profile location.

PyTok(headless=True) runs Firefox without a window, but TikTok treats true headless browsers with more suspicion. On Linux, prefer headless="virtual", which runs a normal browser inside a virtual display (needs Xvfb installed).

WorkerPool runs many sessions at once — each worker owns one account and its own isolated Firefox profile, so N accounts means N concurrent scrapers. Tasks are plain async callables async def task(api) -> result distributed across a shared queue:

import asyncio

from pytok.accounts import AccountsPool, WorkerPool

async def scrape_user(api, handle):
    videos = []
    async for video in api.user(username=handle).videos(count=100):
        videos.append(await video.info())
    return handle, videos

async def main():
    pool = AccountsPool()
    async with WorkerPool(pool, max_workers=3) as wp:
        results = await wp.run([
            lambda api, h=h: scrape_user(api, h)
            for h in ["therock", "khaby.lame", "charlidamelio"]
        ])
    for handle, videos in results:
        print(f"@{handle}: {len(videos)} videos")

if __name__ == "__main__":
    asyncio.run(main())

max_workers is capped to the number of active accounts. Workers rotate/rest accounts and rebuild crashed sessions automatically. See examples/worker_pool_example.py.

Please do not hesitate to make an issue in this repo to get our help with this!

If you use this library in your research, please cite it using the following BibTeX entry:

@article{steel2023invasion,
  title={The invasion of ukraine viewed through tiktok: A dataset},
  author={Steel, Benjamin and Parker, Sara and Ruths, Derek},
  journal={arXiv preprint arXiv:2301.08305},
  year={2023}
}

The JSONable dictionary returned by the info() methods contains all of the data that the TikTok API returns. We have provided helper functions to parse that data into Pandas DataFrames, utils.get_comment_df(), utils.get_video_df() and utils.get_user_df() for the data from comments, videos, and users respectively.

The video dataframe will contain the following columns:

The comment dataframe will contain the following columns:

The user dataframe will contain the following columns: