tiktoken
Published by marinvitlov.com · v1.0.3 · MIT License
Tiktoken is a BPE tokeniser for use with OpenAI's models. It exposes APIs used for processing text using tokens.
#7366 overall androidioswindowslinuxmacosweb
Downloads / 30d
455
as of 2026-05-06
GitHub stars
26
★ refreshed weekly
Likes
22
on pub.dev
Pub points
160 / 160
perfect score
2-day trend
Monthly downloads, sampled daily at 03:00 UTC 2026-05-05 2026-05-06
Top packages
Other libraries in the same category.