
High-performance, type-safe bridge to Google LiteRT enabling native-speed model inference via async-first DSL, hardware acceleration (GPU/NPU/XNNPACK), efficient zero-copy TFBuffer, and vision task APIs.
High-performance, type-safe Kotlin Multiplatform (KMP) library for Google LiteRT (TensorFlow Lite).
Bringing on-device Machine Learning to every screen, from Mobile to Web and Desktop.
KMPLiteRT is a powerful bridge between the Google AI Edge LiteRT (formerly TensorFlow Lite) ecosystem and the Kotlin Multiplatform world. While TensorFlow Lite is the industry standard for on-device AI, integrating it into a cross-platform project often involves complex native interop, memory management issues, and inconsistent behavior across platforms.
KMPLiteRT solves these problems by providing:
commonMain and run it anywhere.TFBuffer ensures minimal overhead.KMPLiteRT abstracts platform-specific runtimes into a consistent Kotlin interface. It doesn't just wrap the APIs; it harmonizes the data types and execution models.
graph TD
subgraph "Your KMP Application"
UI["Compose Multiplatform UI"]
Logic["commonMain (Shared Logic)"]
end
subgraph "KMPLiteRT Common API"
API["LiteRTCompiler & TFBuffer"]
end
subgraph "Platform Implementations"
Android["Android (Official SDK / JNI)"]
iOS["Apple Targets (C-API / Metal)"]
Desktop["Desktop (JNA / C-Interop)"]
Web["Web (JS/Wasm / @litertjs)"]
end
Logic --> API
API --> Android & iOS & Desktop & Web
subgraph "Accelerators"
NPU["NNAPI / CoreML"]
GPU["GPU (Metal/Vulkan/WebGPU)"]
CPU["XNNPACK (Optimized CPU)"]
end
Android -.-> NPU & GPU
iOS -.-> GPU
Web -.-> NPU & GPU| Platform | Target Support | Runtime Engine | CPU (XNNPACK) | GPU Accel. | NPU Accel. |
|---|---|---|---|---|---|
| Android | API 21+ | Official LiteRT SDK | ✅ | ✅ (GLES) | ✅ (NNAPI) |
| iOS | 15.0+ | Native C-API | ✅ | ✅ (Metal) | ✅ (CoreML) |
| macOS | 12.0+ | Native C-API | ✅ | ✅ (Metal) | ✅ (CoreML) |
| Windows | x64 | Native C-API | ✅ | ✅ (Vulkan) | 🚧 |
| Linux | x64 / ARM64 | Native C-API | ✅ | ✅ (OpenGL) | 🚧 |
| Web | JS / Wasm | @litertjs/core | ✅ | ✅ (WebGPU) | ✅ (WebNN) |
Add KMPLiteRT to your commonMain source set in build.gradle.kts.
val kmplitertVersion = "0.1.4" // Replace with latest
kotlin {
sourceSets {
commonMain.dependencies {
// The core ML runtime engine
implementation("io.github.leitingzi:kmplitert-core:$kmplitertVersion")
// Optional: Image processing & File utilities
implementation("io.github.leitingzi:kmplitert-tool:$kmplitertVersion")
}
}
}On Android, initialization is required to handle assets correctly:
class MyApp : Application() {
override fun onCreate() {
super.onCreate()
// Required for LiteRTFileUtils to load models from assets
LiteRTFileUtils.init(this)
}
}Dynamic linking is mandatory for iOS and Apple targets to support hardware delegates like Metal and to ensure binary compatibility.
In your shared module's build.gradle.kts, configure your framework to be dynamic and set the linker search paths.
kotlin {
val iosTargets = listOf(iosArm64(), iosSimulatorArm64())
iosTargets.forEach { target ->
target.binaries.framework {
baseName = "SharedLib"
isStatic = false // MUST BE FALSE
// Link against the LiteRT dynamic library
linkerOpts("-lLiteRt", "-lc++")
// Configure rpath for the system to find the dylib inside the framework
linkerOpts(
"-Wl,-rpath,@executable_path/Frameworks",
"-Wl,-rpath,@loader_path/Frameworks",
"-Wl,-rpath,@loader_path/../../Frameworks"
)
}
}
}The libLiteRt.dylib must be physically present in your .framework/Frameworks directory. You can automate this via Gradle:
tasks.withType<org.jetbrains.kotlin.gradle.tasks.KotlinNativeLink>().configureEach {
val target = binary.target.konanTarget
if (target.isApple) {
doLast {
val destination = destinationDirectory.get().asFile
val frameworkDir = File(destination, "${binary.baseName}.framework")
val frameworksFolder = File(frameworkDir, "Frameworks").apply { mkdirs() }
// Path to where you store the prebuilt LiteRT binaries
val libPath = File(project.rootDir, "native-libs/ios/${target.name}")
copy {
from(libPath)
include("*.dylib")
into(frameworksFolder)
}
}
}
}A simple example showing low-level buffer manipulation.
import io.github.kmplitert.core.*
import io.github.kmplitert.tool.LiteRTFileUtils
suspend fun simpleInference(modelBytes: ByteArray) {
// 1. Prepare model file (cross-platform helper)
val filePath = LiteRTFileUtils.createFileFromByteArray(modelBytes, "temp_model.tflite")
// 2. Initialize Compiler
val compiler = LiteRTCompiler(filePath, LiteRTAccelerator.CPU)
compiler.init()
// 3. Prepare I/O
val inputs = compiler.getInputBuffers()
val outputs = compiler.getOutputBuffers()
// 4. Input 100°C
inputs[0].writeFloat(floatArrayOf(100f))
// 5. Run
compiler.run(inputs, outputs)
// 6. Output result
val result = outputs[0].readFloat()
println("Result: ${result[0]}°F")
// 7. Cleanup
compiler.close()
}A complete vision pipeline example, inspired by the project's :app module.
import io.github.kmplitert.core.*
import io.github.kmplitert.tool.*
suspend fun classifyImage(imageBytes: ByteArray, modelBytes: ByteArray) {
val filePath = LiteRTFileUtils.createFileFromByteArray(modelBytes, "mobilenet.tflite")
val compiler = LiteRTCompiler(filePath, LiteRTAccelerator.GPU)
compiler.init()
// Image Preprocessing Pipeline
val processedData = LiteRtImage.fromBytes(imageBytes)
.resize(224, 224) // Resize to model requirement
.toRgb() // Ensure RGB format
.toInt8Array() // Convert to Quantized Int8
val inputs = compiler.getInputBuffers()
val outputs = compiler.getOutputBuffers()
// Write to native buffer
inputs[0].writeInt8(processedData)
// Inference
compiler.run(inputs, outputs)
// Post-processing
val scores = outputs[0].readInt8()
val topIndex = scores.indices.maxBy { scores[it] }
println("Top Class Index: $topIndex")
compiler.close()
}The heart of the library. It manages the native model lifecycle.
init(): Allocates native resources. Must be called before run().run(inputs, outputs): Executes inference.close(): Releases native memory. CRITICAL for preventing leaks.getInputBufferRequirements(name): Returns tensor metadata (shape, type, size).A type-safe wrapper around native DirectByteBuffer.
writeFloat(array) / writeInt8(array): Fast copy to native memory.readFloat() / readInt8(): Fetch results back to Kotlin.byteSize: The exact capacity of the native buffer.Utility to bridge the gap between KMP resources and LiteRT's file-based requirements.
createFileFromByteArray: Saves bytes to a temporary local file (platform-specific location).The kmplitert-tool module provides a fluent API for image transformation.
val buffer = LiteRtImage.fromBytes(bytes)
.resize(width, height)
.toRgb() // or toGrayscale()
.normalize(mean = 127.5f, std = 127.5f) // For float models
.toFloatArray() // or writeInt8Buffer(tfBuffer)Contributions are what make the open-source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.
git checkout -b feature/AmazingFeature)git commit -m 'Add some AmazingFeature')git push origin feature/AmazingFeature)Built with ❤️ for the Kotlin Multiplatform community.
High-performance, type-safe Kotlin Multiplatform (KMP) library for Google LiteRT (TensorFlow Lite).
Bringing on-device Machine Learning to every screen, from Mobile to Web and Desktop.
KMPLiteRT is a powerful bridge between the Google AI Edge LiteRT (formerly TensorFlow Lite) ecosystem and the Kotlin Multiplatform world. While TensorFlow Lite is the industry standard for on-device AI, integrating it into a cross-platform project often involves complex native interop, memory management issues, and inconsistent behavior across platforms.
KMPLiteRT solves these problems by providing:
commonMain and run it anywhere.TFBuffer ensures minimal overhead.KMPLiteRT abstracts platform-specific runtimes into a consistent Kotlin interface. It doesn't just wrap the APIs; it harmonizes the data types and execution models.
graph TD
subgraph "Your KMP Application"
UI["Compose Multiplatform UI"]
Logic["commonMain (Shared Logic)"]
end
subgraph "KMPLiteRT Common API"
API["LiteRTCompiler & TFBuffer"]
end
subgraph "Platform Implementations"
Android["Android (Official SDK / JNI)"]
iOS["Apple Targets (C-API / Metal)"]
Desktop["Desktop (JNA / C-Interop)"]
Web["Web (JS/Wasm / @litertjs)"]
end
Logic --> API
API --> Android & iOS & Desktop & Web
subgraph "Accelerators"
NPU["NNAPI / CoreML"]
GPU["GPU (Metal/Vulkan/WebGPU)"]
CPU["XNNPACK (Optimized CPU)"]
end
Android -.-> NPU & GPU
iOS -.-> GPU
Web -.-> NPU & GPU| Platform | Target Support | Runtime Engine | CPU (XNNPACK) | GPU Accel. | NPU Accel. |
|---|---|---|---|---|---|
| Android | API 21+ | Official LiteRT SDK | ✅ | ✅ (GLES) | ✅ (NNAPI) |
| iOS | 15.0+ | Native C-API | ✅ | ✅ (Metal) | ✅ (CoreML) |
| macOS | 12.0+ | Native C-API | ✅ | ✅ (Metal) | ✅ (CoreML) |
| Windows | x64 | Native C-API | ✅ | ✅ (Vulkan) | 🚧 |
| Linux | x64 / ARM64 | Native C-API | ✅ | ✅ (OpenGL) | 🚧 |
| Web | JS / Wasm | @litertjs/core | ✅ | ✅ (WebGPU) | ✅ (WebNN) |
Add KMPLiteRT to your commonMain source set in build.gradle.kts.
val kmplitertVersion = "0.1.4" // Replace with latest
kotlin {
sourceSets {
commonMain.dependencies {
// The core ML runtime engine
implementation("io.github.leitingzi:kmplitert-core:$kmplitertVersion")
// Optional: Image processing & File utilities
implementation("io.github.leitingzi:kmplitert-tool:$kmplitertVersion")
}
}
}On Android, initialization is required to handle assets correctly:
class MyApp : Application() {
override fun onCreate() {
super.onCreate()
// Required for LiteRTFileUtils to load models from assets
LiteRTFileUtils.init(this)
}
}Dynamic linking is mandatory for iOS and Apple targets to support hardware delegates like Metal and to ensure binary compatibility.
In your shared module's build.gradle.kts, configure your framework to be dynamic and set the linker search paths.
kotlin {
val iosTargets = listOf(iosArm64(), iosSimulatorArm64())
iosTargets.forEach { target ->
target.binaries.framework {
baseName = "SharedLib"
isStatic = false // MUST BE FALSE
// Link against the LiteRT dynamic library
linkerOpts("-lLiteRt", "-lc++")
// Configure rpath for the system to find the dylib inside the framework
linkerOpts(
"-Wl,-rpath,@executable_path/Frameworks",
"-Wl,-rpath,@loader_path/Frameworks",
"-Wl,-rpath,@loader_path/../../Frameworks"
)
}
}
}The libLiteRt.dylib must be physically present in your .framework/Frameworks directory. You can automate this via Gradle:
tasks.withType<org.jetbrains.kotlin.gradle.tasks.KotlinNativeLink>().configureEach {
val target = binary.target.konanTarget
if (target.isApple) {
doLast {
val destination = destinationDirectory.get().asFile
val frameworkDir = File(destination, "${binary.baseName}.framework")
val frameworksFolder = File(frameworkDir, "Frameworks").apply { mkdirs() }
// Path to where you store the prebuilt LiteRT binaries
val libPath = File(project.rootDir, "native-libs/ios/${target.name}")
copy {
from(libPath)
include("*.dylib")
into(frameworksFolder)
}
}
}
}A simple example showing low-level buffer manipulation.
import io.github.kmplitert.core.*
import io.github.kmplitert.tool.LiteRTFileUtils
suspend fun simpleInference(modelBytes: ByteArray) {
// 1. Prepare model file (cross-platform helper)
val filePath = LiteRTFileUtils.createFileFromByteArray(modelBytes, "temp_model.tflite")
// 2. Initialize Compiler
val compiler = LiteRTCompiler(filePath, LiteRTAccelerator.CPU)
compiler.init()
// 3. Prepare I/O
val inputs = compiler.getInputBuffers()
val outputs = compiler.getOutputBuffers()
// 4. Input 100°C
inputs[0].writeFloat(floatArrayOf(100f))
// 5. Run
compiler.run(inputs, outputs)
// 6. Output result
val result = outputs[0].readFloat()
println("Result: ${result[0]}°F")
// 7. Cleanup
compiler.close()
}A complete vision pipeline example, inspired by the project's :app module.
import io.github.kmplitert.core.*
import io.github.kmplitert.tool.*
suspend fun classifyImage(imageBytes: ByteArray, modelBytes: ByteArray) {
val filePath = LiteRTFileUtils.createFileFromByteArray(modelBytes, "mobilenet.tflite")
val compiler = LiteRTCompiler(filePath, LiteRTAccelerator.GPU)
compiler.init()
// Image Preprocessing Pipeline
val processedData = LiteRtImage.fromBytes(imageBytes)
.resize(224, 224) // Resize to model requirement
.toRgb() // Ensure RGB format
.toInt8Array() // Convert to Quantized Int8
val inputs = compiler.getInputBuffers()
val outputs = compiler.getOutputBuffers()
// Write to native buffer
inputs[0].writeInt8(processedData)
// Inference
compiler.run(inputs, outputs)
// Post-processing
val scores = outputs[0].readInt8()
val topIndex = scores.indices.maxBy { scores[it] }
println("Top Class Index: $topIndex")
compiler.close()
}The heart of the library. It manages the native model lifecycle.
init(): Allocates native resources. Must be called before run().run(inputs, outputs): Executes inference.close(): Releases native memory. CRITICAL for preventing leaks.getInputBufferRequirements(name): Returns tensor metadata (shape, type, size).A type-safe wrapper around native DirectByteBuffer.
writeFloat(array) / writeInt8(array): Fast copy to native memory.readFloat() / readInt8(): Fetch results back to Kotlin.byteSize: The exact capacity of the native buffer.Utility to bridge the gap between KMP resources and LiteRT's file-based requirements.
createFileFromByteArray: Saves bytes to a temporary local file (platform-specific location).The kmplitert-tool module provides a fluent API for image transformation.
val buffer = LiteRtImage.fromBytes(bytes)
.resize(width, height)
.toRgb() // or toGrayscale()
.normalize(mean = 127.5f, std = 127.5f) // For float models
.toFloatArray() // or writeInt8Buffer(tfBuffer)Contributions are what make the open-source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.
git checkout -b feature/AmazingFeature)git commit -m 'Add some AmazingFeature')git push origin feature/AmazingFeature)Built with ❤️ for the Kotlin Multiplatform community.