CommonJSJVMNative
ktoken
BPE tokenizer designed for seamless integration with OpenAI models, supporting local and remote modes for encoding management. Offers multiplatform compatibility and flexible setup options.
Package | Supported platforms & targets | |
---|---|---|
com.aallam.ktoken:ktoken:0.4.0 | Common, JVM, Native, JS / (15 targets) | Details |
Metadata
Latest release
5 months ago
0.4.0
- 20 stars
- Created 1 year ago
- Latest activity: 3 months ago
License
- MIT License
Updated 5 hours ago