konaResource

Embeds resource directories into native executables via a Gradle plugin and archive format; supports incremental updates, LZ4 compression, memory-mapped reads, and a CLI.

JVM
Linux
Windows
GitHub stars0
Authorstateisu
Dependents0
LicenseApache License 2.0
Creation dateabout 1 month ago

Last activityabout 1 month ago
Latest release0.1.6 (about 1 month ago)

konaResource

konaResource provides embedded resources for Linux/x64 Kotlin/Native executables.

  • plugin is a Gradle plugin that updates embedded resources.
  • common contains the KonaArchive format and the library for reading embedded resources.
  • sample1 is a sample project that uses published artifacts.
  • sample2 is a sample project that uses sibling modules.
  • cli is a CLI tool for the KonaArchive format.

Features

  • Incremental updates. When a resource directory is converted into an archive embedded in the application, Gradle's @InputFiles is used so routine builds do no more work than necessary.
  • No additional shared-library dependencies. Binaries generated by the sample modules have the same ELF DT_NEEDED entries as an empty Kotlin/Native binary.
  • The resource portion of the executable is not copied wholesale. The implementation keeps only the required portion of the memory-mapped executable in RAM.

Limitations

  • Linux musl (e.g., Alpine Linux) is not supported for Kotlin/Native builds. The konanc toolchain and its bundled LLVM distribution depend on glibc-specific symbols (mallinfo, backtrace, backtrace_symbols_fd) that are unavailable in musl libc, making it impossible to compile or run Kotlin/Native programs on Alpine. See KT-38891 for upstream status.

plugin Configuration

Example:

  • sample1/build.gradle.kts
plugins {
    alias(libs.plugins.kotlinMultiplatform)
    // Add the konaResource plugin
    id("jp.juggler.konaResource") version "..."
}
kotlin {
    sourceSets {
        linuxX64Main.dependencies {
            // Add the konaResource common module
            implementation("jp.juggler.konaResource:common:...")
        }
    }
}
// Specify compression settings and resource directories
konaResource{
    // LZ4 compression parameters. All parameters have default values and are optional.
    // LZ4F compression level. 0 is the default fast compression, positive values use LZ4HC, and negative values use fast acceleration.
    lz4CompressionLevel = 0
    lz4BlockSizeID = 1MB
    lz4BlockMode = "LZ4F_blockIndependent"
    lz4ContentSizeFlag = true
    lz4ContentChecksumFlag	= true
    lz4blockChecksumFlag = true
    lz4AutoFlush = false
    lz4FavorDecSpeed = false

    // Skip embedding for selected Kotlin/Native targets.
    // The target name is e.g. "linuxX64" or "macosArm64".
    skipEmbedIf { targetName -> targetName == "macosArm64" }

    // Used for the .o file name and symbol name
    val name1 = "resources"
    // Input directory for the resource archive
    val inDir1 = "src/resources"
    modules.add( name1 to inDir1 )

    // Multiple name and input-directory pairs can be registered
    modules.add( "resourcesB" to "src/resourcesB" )
}

Accessing Embedded Resources

Example:

  • sample1/src/linuxX64Main/kotlin/jp/juggler/konaResource/sample/Main.kt
// Open the archive in the embedded resources.
val root = embedKonaArchive("sample").root
// Read a file.
val bytes = root.pathToFile(path)?.bytes()
val string = root.pathToFile(path)?.string()
val buffer = root.pathToFile(path)?.buffer()
// Read a directory.
for (entry in root.pathToDir(path)!!) {
    println("name=${entry.name}")
}

How Native Resources Are Embedded

The konaResource plugin embeds each configured resource directory into every Kotlin/Native executable that belongs to the target project:

  1. Each target-specific generateKonaResource<Target> task packs the directory into a KonaArchive .bin file.
  2. It generates an assembly source using .incbin, with exported start and end symbols.
  3. The assembly source is compiled into an object file and passed to the Native linker.
  4. embedKonaArchive(name) converts the name to the same safe symbol name and resolves konaResource_<name>_start and konaResource_<name>_end with kona_dlsym.
  5. EmbedRandomAccess reads that address range directly from the executable, and the common decoder reads the KonaArchive metadata from it.

The embedded range is read-only and is not copied wholesale into another buffer.

Using common

The common module provides the KonaArchive reader and writer for Linux/x64 Kotlin/Native code. Use the plugin module when embedding resource archives into an executable.

Build

# Build
./gradlew build

# detekt, kotest for some modules
./gradlew check

# Run sample1, sample2

# Build and run the sample1 that uses published artifacts
./gradlew sample1:runDebugExecutableLinuxX64
./gradlew sample1:runReleaseExecutableLinuxX64

# Build and run the sample2 that uses sibling modules
./gradlew sample2:runDebugExecutableLinuxX64
./gradlew sample2:runReleaseExecutableLinuxX64

test

reason to cross-platform, this app build separate binary to unit test.

./gradlew test:deploy
java -jar bin/konaCommonTest.jar test
./bin/konaCommonTest-linuxX64 test
# (or some binalies for each build-available arch)

Run benchmarks

# Execute benchmark on the JVM
./gradlew :benchmark:runJvm

# Execute benchmark on the host's Native target
./gradlew :benchmark:runRelease

# Build and deploy standalone benchmark artifacts
./gradlew :benchmark:deploy

Build prerequisites

The project is built and tested on JDK 21 with the C toolchain. The GitHub Actions workflows run on ubuntu-24.04 / ubuntu-24.04-arm (with gcc, g++, make preinstalled) and install Java 21 via actions/setup-java. To reproduce the same environment locally on Ubuntu:

# JDK 21 (matching the CI)
sudo apt install openjdk-21-jdk

# C toolchain (gcc, g++, make)
sudo apt install build-essential

Cross-target build

this project uses 2 kind of Native code.

  • Java JNI : used in plugin module, that need to build embed resource.
  • Kotlin/Native : used in common module and embed to user application.

Kotlin/Native compiler

JNI and resource object builds use the Kotlin/Native compiler distribution's run_konan wrapper. The build script automatically detects which targets are available from kotlinc-native -list-targets.

./gradlew :commonJni:listAvailableJniBuildTargets

JNI build option overrides

The default JNI compiler options can be overridden for one host/target pair with Gradle properties. The host and target names are the enum names in KonaBuildHost and JniBuildTarget.

./gradlew \
  -PLinuxX64_MingwX64_compileOpt=-Wall,-Wextra,-O3,-D_JNI_IMPLEMENTATION_ \
  -PLinuxX64_MingwX64_linkOpt=-shared \
  :common:jvmJar
  • {host}_{target}_compileOpt replaces the default C compiler options. Options are comma-separated.
  • {host}_{target}_linkOpt replaces the default linker options. Options are comma-separated.
  • If a property is not specified, the built-in default is used.

JNI header and dll

  • go https://learn.microsoft.com/ja-jp/java/openjdk/download#openjdk-21
  • download JDK for some arch you want.
  • put JDK content into ${project}/jdk/${ArchName}/ .
  • build script search ${project}/jdk/${ArchName}/include/jni.h.
  • if not found, your host jdk Gradle runs on is used for same arch.
  • build script enables a JNI build target if its Kotlin/Native target and jni.h are available.

Kotlin/Native

  • The common module's native targets (linuxX64 / linuxArm64 / mingwX64) are cross-compiled by Kotlin/Native itself.
  • but Alpine Linux (musl) have issue for build/run. see KT-38891.

Using the CLI

# Deploy the CLI fat JAR and launcher
./gradlew cli:deploy

# Convert a directory to an archive
java -jar bin/konaArchive.jar pack sample1Res.kona sample1/src/res

# List the contents of an archive
java -jar bin/konaArchive.jar list sample1Res.kona

# Extract an archive
java -jar bin/konaArchive.jar extract sample1Res.kona /tmp/sample1Res
JVM
Linux
Windows
GitHub stars0
Authorstateisu
Dependents0
LicenseApache License 2.0
Creation dateabout 1 month ago

Last activityabout 1 month ago
Latest release0.1.6 (about 1 month ago)

konaResource

konaResource provides embedded resources for Linux/x64 Kotlin/Native executables.

  • plugin is a Gradle plugin that updates embedded resources.
  • common contains the KonaArchive format and the library for reading embedded resources.
  • sample1 is a sample project that uses published artifacts.
  • sample2 is a sample project that uses sibling modules.
  • cli is a CLI tool for the KonaArchive format.

Features

  • Incremental updates. When a resource directory is converted into an archive embedded in the application, Gradle's @InputFiles is used so routine builds do no more work than necessary.
  • No additional shared-library dependencies. Binaries generated by the sample modules have the same ELF DT_NEEDED entries as an empty Kotlin/Native binary.
  • The resource portion of the executable is not copied wholesale. The implementation keeps only the required portion of the memory-mapped executable in RAM.

Limitations

  • Linux musl (e.g., Alpine Linux) is not supported for Kotlin/Native builds. The konanc toolchain and its bundled LLVM distribution depend on glibc-specific symbols (mallinfo, backtrace, backtrace_symbols_fd) that are unavailable in musl libc, making it impossible to compile or run Kotlin/Native programs on Alpine. See KT-38891 for upstream status.

plugin Configuration

Example:

  • sample1/build.gradle.kts
plugins {
    alias(libs.plugins.kotlinMultiplatform)
    // Add the konaResource plugin
    id("jp.juggler.konaResource") version "..."
}
kotlin {
    sourceSets {
        linuxX64Main.dependencies {
            // Add the konaResource common module
            implementation("jp.juggler.konaResource:common:...")
        }
    }
}
// Specify compression settings and resource directories
konaResource{
    // LZ4 compression parameters. All parameters have default values and are optional.
    // LZ4F compression level. 0 is the default fast compression, positive values use LZ4HC, and negative values use fast acceleration.
    lz4CompressionLevel = 0
    lz4BlockSizeID = 1MB
    lz4BlockMode = "LZ4F_blockIndependent"
    lz4ContentSizeFlag = true
    lz4ContentChecksumFlag	= true
    lz4blockChecksumFlag = true
    lz4AutoFlush = false
    lz4FavorDecSpeed = false

    // Skip embedding for selected Kotlin/Native targets.
    // The target name is e.g. "linuxX64" or "macosArm64".
    skipEmbedIf { targetName -> targetName == "macosArm64" }

    // Used for the .o file name and symbol name
    val name1 = "resources"
    // Input directory for the resource archive
    val inDir1 = "src/resources"
    modules.add( name1 to inDir1 )

    // Multiple name and input-directory pairs can be registered
    modules.add( "resourcesB" to "src/resourcesB" )
}

Accessing Embedded Resources

Example:

  • sample1/src/linuxX64Main/kotlin/jp/juggler/konaResource/sample/Main.kt
// Open the archive in the embedded resources.
val root = embedKonaArchive("sample").root
// Read a file.
val bytes = root.pathToFile(path)?.bytes()
val string = root.pathToFile(path)?.string()
val buffer = root.pathToFile(path)?.buffer()
// Read a directory.
for (entry in root.pathToDir(path)!!) {
    println("name=${entry.name}")
}

How Native Resources Are Embedded

The konaResource plugin embeds each configured resource directory into every Kotlin/Native executable that belongs to the target project:

  1. Each target-specific generateKonaResource<Target> task packs the directory into a KonaArchive .bin file.
  2. It generates an assembly source using .incbin, with exported start and end symbols.
  3. The assembly source is compiled into an object file and passed to the Native linker.
  4. embedKonaArchive(name) converts the name to the same safe symbol name and resolves konaResource_<name>_start and konaResource_<name>_end with kona_dlsym.
  5. EmbedRandomAccess reads that address range directly from the executable, and the common decoder reads the KonaArchive metadata from it.

The embedded range is read-only and is not copied wholesale into another buffer.

Using common

The common module provides the KonaArchive reader and writer for Linux/x64 Kotlin/Native code. Use the plugin module when embedding resource archives into an executable.

Build

# Build
./gradlew build

# detekt, kotest for some modules
./gradlew check

# Run sample1, sample2

# Build and run the sample1 that uses published artifacts
./gradlew sample1:runDebugExecutableLinuxX64
./gradlew sample1:runReleaseExecutableLinuxX64

# Build and run the sample2 that uses sibling modules
./gradlew sample2:runDebugExecutableLinuxX64
./gradlew sample2:runReleaseExecutableLinuxX64

test

reason to cross-platform, this app build separate binary to unit test.

./gradlew test:deploy
java -jar bin/konaCommonTest.jar test
./bin/konaCommonTest-linuxX64 test
# (or some binalies for each build-available arch)

Run benchmarks

# Execute benchmark on the JVM
./gradlew :benchmark:runJvm

# Execute benchmark on the host's Native target
./gradlew :benchmark:runRelease

# Build and deploy standalone benchmark artifacts
./gradlew :benchmark:deploy

Build prerequisites

The project is built and tested on JDK 21 with the C toolchain. The GitHub Actions workflows run on ubuntu-24.04 / ubuntu-24.04-arm (with gcc, g++, make preinstalled) and install Java 21 via actions/setup-java. To reproduce the same environment locally on Ubuntu:

# JDK 21 (matching the CI)
sudo apt install openjdk-21-jdk

# C toolchain (gcc, g++, make)
sudo apt install build-essential

Cross-target build

this project uses 2 kind of Native code.

  • Java JNI : used in plugin module, that need to build embed resource.
  • Kotlin/Native : used in common module and embed to user application.

Kotlin/Native compiler

JNI and resource object builds use the Kotlin/Native compiler distribution's run_konan wrapper. The build script automatically detects which targets are available from kotlinc-native -list-targets.

./gradlew :commonJni:listAvailableJniBuildTargets

JNI build option overrides

The default JNI compiler options can be overridden for one host/target pair with Gradle properties. The host and target names are the enum names in KonaBuildHost and JniBuildTarget.

./gradlew \
  -PLinuxX64_MingwX64_compileOpt=-Wall,-Wextra,-O3,-D_JNI_IMPLEMENTATION_ \
  -PLinuxX64_MingwX64_linkOpt=-shared \
  :common:jvmJar
  • {host}_{target}_compileOpt replaces the default C compiler options. Options are comma-separated.
  • {host}_{target}_linkOpt replaces the default linker options. Options are comma-separated.
  • If a property is not specified, the built-in default is used.

JNI header and dll

  • go https://learn.microsoft.com/ja-jp/java/openjdk/download#openjdk-21
  • download JDK for some arch you want.
  • put JDK content into ${project}/jdk/${ArchName}/ .
  • build script search ${project}/jdk/${ArchName}/include/jni.h.
  • if not found, your host jdk Gradle runs on is used for same arch.
  • build script enables a JNI build target if its Kotlin/Native target and jni.h are available.

Kotlin/Native

  • The common module's native targets (linuxX64 / linuxArm64 / mingwX64) are cross-compiled by Kotlin/Native itself.
  • but Alpine Linux (musl) have issue for build/run. see KT-38891.

Using the CLI

# Deploy the CLI fat JAR and launcher
./gradlew cli:deploy

# Convert a directory to an archive
java -jar bin/konaArchive.jar pack sample1Res.kona sample1/src/res

# List the contents of an archive
java -jar bin/konaArchive.jar list sample1Res.kona

# Extract an archive
java -jar bin/konaArchive.jar extract sample1Res.kona /tmp/sample1Res