FlutterTrends.dev
⌘K
Sponsored by- The #1 Flutter-powered AI app builder
Home/Top/Ai Ml/hf_tokenizers

hf_tokenizers

Published by 🇹🇷developeryusuf.com · v1.2.2 · released 27 days ago · MIT License

First published 3 months ago.

HuggingFace tokenizers for Dart over FFI. Load any tokenizer.json and get byte-exact BPE, WordPiece, and Unigram encoding, backed by the Rust crate.

View on pub.dev View source
#200 in Ai Ml #8094 overall windowslinuxmacos
Downloads / day
24
as of 2026-10-19
GitHub stars
0
★ refreshed weekly
Likes
1
on pub.dev
Pub points
160 / 160
perfect score
2026-09-19 2026-10-19

Repository health

Last commit
18 days ago
Commit activity
Open issues
3
Open PRs
0
Contributors
1
CI
GitHub Actions
License (repo)

Last refreshed 14 days ago.

License explained · MIT

MIT License

Use freely, even commercially. Just keep the copyright notice.

A short and simple permissive license with conditions only requiring preservation of copyright and license notices. Licensed works, modifications, and larger works may be distributed under different terms and without source code.

You can

  • Commercial use
  • Distribution
  • Modification
  • Private use

You must

  • Include the license and copyright notice

The license does not

  • No liability
  • No warranty

Plain-English summary, not legal advice. Full text on SPDX or choosealicense.com. Summaries adapted from choosealicense.com (a GitHub project, MIT-licensed).

Commit activity, past 52 weeks

hf_tokenizers

50 commits in the past year
Stable
15 8 0 52w ago36w ago18w agothis week
Weekly average
1.0 commits
Busiest week
14 commits, 10 weeks ago
Active weeks
7 of 52 (13%)
Longest quiet stretch
40 weeks with no commits

Weekly commits to the default branch on GitHub. Source: GET /stats/participation. Refreshed weekly. Excludes commits to other branches.

How hf_tokenizers compares in Ai Ml

Top of the Ai Ml leaderboard, with hf_tokenizers highlighted.

Rank Package Downloads / day Stars Likes
#1
google_mlkit_commons
flutter-ml.dev
19.1K ★ 1.3K 38
#2
google_mlkit_text_recognition
flutter-ml.dev
11.1K ★ 1.3K 413
#3
mcp_dart
🇨🇦 leehack.com
7.2K ★ 114 77
#4
marionette_flutter
🇨🇴 leancode.co
6.8K ★ 464 69
#200
hf_tokenizers
developeryusuf.com
24 ★ 0 1