CommonJSJVMNative

ktoken

BPE tokenizer designed for seamless integration with OpenAI models, supporting local and remote modes for encoding management. Offers multiplatform compatibility and flexible setup options.

Kodee spinning
Package Supported platforms & targets
com.aallam.ktoken:ktoken:0.4.0 Common, JVM, Native, JS / (15 targets) Details
Metadata
Latest release 5 months ago

0.4.0

  • 20 stars
  • Created 1 year ago
  • Latest activity: 3 months ago
License
  • MIT License

Updated 5 hours ago