FlutterTrends.dev
⌘K
Sponsored by- The #1 Flutter-powered AI app builder
Home/Top/Ai Ml/hf_tokenizers

hf_tokenizers

Published by 🇹🇷developeryusuf.com · v1.1.0 · released 14 days ago · MIT License

First published 1 month ago.

HuggingFace tokenizers for Dart over FFI. Load any tokenizer.json and get byte-exact BPE, WordPiece, and Unigram encoding, backed by the Rust crate.

View on pub.dev View source
#147 in Ai Ml #7073 overall windowslinuxmacos
Downloads / day
25
as of 2026-08-23
GitHub stars
0
★ refreshed weekly
Likes
0
on pub.dev
Pub points
160 / 160
perfect score
2026-07-25 2026-08-23

Repository health

Last commit
13 days ago
Commit activity
Open issues
0
Open PRs
0
Contributors
1
CI
GitHub Actions
License (repo)

Last refreshed 6 days ago.

License explained · MIT

MIT License

Use freely, even commercially. Just keep the copyright notice.

A short and simple permissive license with conditions only requiring preservation of copyright and license notices. Licensed works, modifications, and larger works may be distributed under different terms and without source code.

You can

  • Commercial use
  • Distribution
  • Modification
  • Private use

You must

  • Include the license and copyright notice

The license does not

  • No liability
  • No warranty

Plain-English summary, not legal advice. Full text on SPDX or choosealicense.com. Summaries adapted from choosealicense.com (a GitHub project, MIT-licensed).

Commit activity, past 52 weeks

hf_tokenizers

35 commits in the past year
Stable
15 8 0 52w ago36w ago18w agothis week
Weekly average
0.7 commits
Busiest week
14 commits, 3 weeks ago
Active weeks
4 of 52 (8%)
Longest quiet stretch
47 weeks with no commits

Weekly commits to the default branch on GitHub. Source: GET /stats/participation. Refreshed weekly. Excludes commits to other branches.

How hf_tokenizers compares in Ai Ml

Top of the Ai Ml leaderboard, with hf_tokenizers highlighted.

Rank Package Downloads / day Stars Likes
#1
google_mlkit_commons
flutter-ml.dev
14.9K ★ 1.3K 37
#2
google_mlkit_text_recognition
flutter-ml.dev
8.6K ★ 1.3K 406
#3
mcp_dart
🇨🇦 leehack.com
7.7K ★ 113 75
#4
marionette_flutter
🇨🇴 leancode.co
5.2K ★ 375 56
#147
hf_tokenizers
developeryusuf.com
25 ★ 0 0