testballoon-addons

This project provides addons for TestBalloon, the next generation KMP-first, coroutine-first testing framework.

TestBalloon Addons started as migration helpers for people coming from Kotest. That still matters, but the newer Matrix Testing module is the greenfield version of the idea: What would data-driven testing, property testing, fixtures, and compact reports look like if they were designed directly for TestBalloon's KMP-first, coroutine-first execution model?

The answer is a single DSL that keeps Kotest's excellent assertion and generator libraries, but gives TestBalloon full control over registration, execution, concurrency, compaction, reporting and replaying.

val combinedFeaturesSuite by matrixSuite(matrixConfig { execution = ExecutionMode.Concurrent(12) }) {
    fixture { Random.nextBytes(16) } - {
        "data, properties, fixtures, and compact reports" - { freshBytes ->
            data("multiplier", listOf(1, 2, 3)) - { multiplier ->
                compact("generated checks") { report = CompactReport.FailuresOnly } - {
                    property("offset", Arb.int(0..100), iterations = 50) test { offset ->
                        val result = freshBytes.size * multiplier + offset
                        result shouldBeGreaterThan 0
                    }
                }
            }
        }
    }
}

[!TIP]
Looking for a smooth migration path from Kotest?
Check out the Coming from Kotest section!

The matrix module provides an original, advanced, next-generation testing DSL.
It may not be for the faint of heart, but it combines data-driven testing, property testing, FreeSpec-style names, fixture generation, concurrency controls, compact reports, and targeted replay of failed tests – things you will need for truly powerful, comprehensive test suites.

Matrix tests are built from composable layers. A data layer, a property layer, a generated fixture, and a plain FreeSpec-style suite can be stacked into an n-dimensional test matrix, where every leaf test runs once for each path through those layers. When that would create too many framework test elements, compact can flatten the subtree into one real test while still executing and reporting the full virtual matrix. This way, you still get full insights including clickable stacktraces taking you to the failing assertion(s)!

The top-level matrixSuite(matrixConfig { ... }) { ... } is a regular TestBalloon suite so IDE gutter actions can discover and run it. Inside the suite, every nested layer is configured first, then either opened with - { ... } for more nesting or finished with test { ... } to create row test elements.

val quickstart by matrixSuite(matrixConfig { execution = ExecutionMode.Sequential }) {
    data("input", listOf("foo", "bar")) - { input ->
        property("offset", Arb.int(0..100), iterations = 25) test { offset ->
            val result = "$input-$offset"
            result.startsWith(input) shouldBe true
        }
    }
}

A layer's leading name is just a label for a grouping node that wraps the rows it generates. Naming a layer is shorthand for wrapping a nameless one in a labeled suite by hand — these two produce the same test-tree shape:

data("numbers", listOf(1, 2, 3)) - { /* … */ }   // named layer

"numbers" - {                                    // equivalent structure
    data(listOf(1, 2, 3)) - { /* … */ }
}

Both nest the three rows under a numbers node. Drop the name entirely (data(listOf(1, 2, 3))) and the rows attach directly to the surrounding scope with no grouping node at all. The name, when present, additionally becomes the layer's label in failure/replay output ((data) numbers: …); a nameless layer just shows its kind ((data) …). The same applies to property. The sections below use named layers throughout, but every data/property form has a nameless overload.

data layers accept Iterable values, lazy Sequence values, and a Map (each entry comes through as a destructurable Pair<K, V>, named "<index>: (key: value)" by default). Use test { ... } when each row is the test, and - { ... } when each row is a dimension that contains more layers or explicit tests.

Every collection also reads fluently as a receiver via .asData(…), with the exact same options as data:

listOf(1, 2, 3).asData() test { it shouldBeGreaterThan 0 }
mapOf("dev" to 8080, "prod" to 443).asData(nameFn = { (env, _) -> env }) - { (env, port) -> /* … */ }

nameFn resolves by lambda arity: a single-parameter namer ({ v -> … }, { it }, or destructured { (k, v) -> … }) names by value alone, while the two-parameter { index, value -> … } form keeps the index.

val dataDrivenMatrix by matrixSuite(matrixConfig { execution = ExecutionMode.Sequential }) {
    data("numbers", listOf(1, 2, 3), nameFn = { index, value -> "$index: n=$value" }) test { number ->
        number shouldBeGreaterThan 0
    }

    data("lazy values", generateSequence(1) { it + 1 }, limit = 3) {
        execution = ExecutionMode.Concurrent(parallelism = 2)
    } test { number ->
        number shouldBeGreaterThan 0
    }

    data("string data", listOf("foo", "bar")) - { word ->
        "has a length" {
            word.length shouldBeGreaterThan 0
        }
    }
}

Layer config is written as the trailing lambda before test or -. For data, the most important layer option is execution, which can be ExecutionMode.Sequential or ExecutionMode.Concurrent(parallelism = ...). Concurrency bounds are per layer: nested concurrent layers can multiply the number of active tests or virtual checks. For large compacted matrices, prefer CompactConcurrency.Shared(n) to use one compact-wide worker budget.

Concurrency and TestScope. Real concurrency cannot run inside TestBalloon's virtual-time TestScope (TestBalloon forbids the combination, as it can deadlock through thread starvation). So wherever matrix parallelizes — any ExecutionMode.Concurrent layer/suite, and every compact block (which executes on real dispatchers) — matrix automatically disables the TestScope for that scope, even if the surrounding TestSession enabled it (the default). You therefore do not need testScope(isEnabled = false) to combine matrix concurrency with the default session — in fact, leave TestScope enabled (the default) and let matrix turn it off only where it parallelizes. Sequential matrix execution leaves the inherited TestScope untouched, so a mix works: sequential suites get virtual time (and its timeout), concurrent suites/compact run for real. The disable also wins over a testConfig you pass yourself, so a testScope(true) on a concurrent scope is overridden.

Timeouts: for a virtual-time (sequential) test, use testScope(isEnabled = true, timeout = …) (or the 60s default). The TestScope timeout is per test element, not per suite — a large matrix is many tests, each with its own budget, so the case count never drains one shared timeout; raise it only if a single case is genuinely slow. Compact and concurrent tests run with TestScope disabled, so its timeout doesn't apply to them at all — for a wall-clock cap there, use a real-time timeout via aroundEachTest { action -> withTimeout(…) { action() } } (per leaf) or aroundAll { action -> withTimeout(…) { action() } } (per block). That's a plain testConfig wrapper, not invocation/testScope, so it doesn't conflict with execution.

property layers use Kotest generators directly. You get Kotest's Arb ecosystem, edge cases, shrinking-friendly data models, and concise generator composition, while TestBalloon owns the test tree.

val propertyMatrix by matrixSuite(matrixConfig { defaultPropertyIterations = 100 }) {
    property("bytes", Arb.byteArray(Arb.int(1, 64), Arb.byte())) {
        seed = 0xC0FFEE
        nameFn = { index, bytes -> "$index: ${bytes.size} bytes" }
    } test { bytes ->
        bytes.toHexString(HexFormat.UpperCase).hexToByteArray() shouldBe bytes
    }
}

property supports iterations, seed, edgeConfig, nameFn, and per-layer execution. If iterations is omitted, the suite default is used.

Deep data/property nesting can generate very large test trees. compact collapses the virtual subtree into one real TestBalloon test and reports the virtual rows itself.

val compactMatrix by matrixSuite(matrixConfig { execution = ExecutionMode.Concurrent() }) {
    compact("all generated checks") {
        concurrency = CompactConcurrency.Shared(16)
        report = CompactReport.FailuresOnly
        reportRows = 256
        progressIndicator = Indicator.Heartbeat(every = 1.seconds)
    } - {
        data("word", listOf("foo", "bar", "baz")) - { word ->
            property("number", Arb.int(0..100), iterations = 100) test { number ->
                (word.length + number) shouldBeGreaterThan 0
            }
        }
    }
}

CompactReport.FailuresOnly renders only failing virtual rows, AllCases also renders successes, and SummaryOnly keeps the report short. reportRows bounds rendered row details. addSuppressedErrors controls whether retained failures are attached as suppressed exceptions. coroutineContext controls where compact virtual children run. CompactConcurrency.Layered keeps per-layer concurrency behaviour, while CompactConcurrency.Shared(n) uses one compact-wide worker budget so nested virtual layers cannot multiply coroutine counts.

Progress heartbeats are printed separately from the final failure report, for example:

all generated checks: compact progress: 512 of 900 queued completed (1200 source cases), 3 failed

[!NOTE] Compact virtual children are not real TestBalloon test elements, so virtual test / testSuite declarations and terminal data(...) test { ... } / property(...) test { ... } rows inside compact cannot honor per-child TestConfig. Put TestConfig on real matrix tests/suites outside compact, or configure the compact block itself.

Every matrix failure — a real test-tree leaf or a compacted virtual row — carries an Error replay info block whose detail lines are valid property / data arguments. Copy them back onto the failing layers and the matrix re-runs exactly those cases. Each frame is tagged with its layer kind — (property) or (data) — so the marker still pinpoints the right layer even for nameless layers (see below), where no layer name is printed.

at.asitplus.AssertionError: 360888 should be < 256000
    Error replay info: (property) first: 4: 2018089192 ↘ (property) second: 2: 2796 ↘ (data) third: 1: 2 ↘ (property) fourth: 3: 1
      - (property) first:  replay = Cases(seed = 4779463605442148766L, iter = 4L)
      - (property) second: replay = Cases(seed = -1353176301820643450L, iter = 2L)
      - (data)     third:  replay = Indexes(1L)
      - (property) fourth: replay = Cases(seed = 5014696554795393980L, iter = 3L)

Paste the text after each layer name into that layer's call:

val replay by matrixSuite {
    property("first", Arb.int(), replay = Cases(seed = 4779463605442148766L, iter = 4L)) - { first ->
        property("second", Arb.double(), replay = Cases(seed = -1353176301820643450L, iter = 2L)) - { second ->
            data("third", listOf(1, 2, 3, 4, 5), replay = Indexes(1L)) - { third ->
                property("fourth", Arb.byte(), replay = Cases(seed = 5014696554795393980L, iter = 3L)) test { fourth ->
                    // now runs only the single failing combination
                }
            }
        }
    }
}

Each pinned layer collapses to just the recorded case, so the whole matrix narrows to the failing leaf.

• Property layers take replay = Cases(seed = …, iter = …) for one case (what the report prints). Pass several Cases(Input(...), Input(...)) to reproduce several recorded cases — each with its own seed — at once, or Cases(seed = …, iter = 1L..5L) for several iterations of one seed.
• Data layers take replay = Indexes(n) (printed), Indexes(a, b, c) for several, or Indexes(0L..9L) for a range. Data is deterministic, so only the index is needed.
• A layer's replay is independent of its seed: seed pins a deterministic full run, while replay selects specific recorded cases (which carry their own seeds).

The path on the first line mirrors the compact report path in full: it includes the enclosing grouping suites ("name" - { ... }) as plain segments, with the (property) / (data) layers carrying the markers. Only the replayable layers get a - ... argument line below — grouping suites appear in the path but have nothing to paste. A compacted leaf therefore records the full chain (outer real layers included), and the data index is recorded even when a custom nameFn omits it from the displayed name.

fixture { ... } produces a fresh value, and every test or suite you declare inside its - { ... } block gains an extra lambda parameter that hands you that value. You name that parameter yourself — it is the freshly generated fixture, scoped to the element it is attached to:

fixture { Random.nextBytes(16) } - {
    //                              ^ open the fixture scope with `- { }`

    "a single test" { bytes ->   // <-- `bytes` is the fixture, freshly generated for THIS test
        bytes.size shouldBe 16
    }

    "a nested suite" - { bytes ->   // <-- `bytes` is one fixture, shared by everything in this suite
        data("word", listOf("foo", "bar")) test { word ->
            bytes.isNotEmpty() shouldBe true
        }
    }
}

The parameter appears on every nested form — "name" { value -> }, "name" - { value -> }, test(...) { value -> }, and testSuite(...) { value -> } — so the fixture threads in wherever you declare a child.

Freshness follows the element the lambda is attached to:

• a terminal test ("name" { value -> } / test { value -> }) gets its own fresh value;
• a suite ("name" - { value -> }) gets one fresh value that all of its children share.

So fixtures isolate per test by default, but give you a single shared instance when you group children under a suite.

Directly inside the fixture block you can only declare tests and suites — not data/property/compact layers. To add those, open a suite first: a "name" - { value -> ... } body is a normal matrix scope, so every layer you declare in it closes over that suite's value. Nesting composes the other direction too: a fixture inside a data row regenerates once per row, and a fixture inside another fixture's suite threads both values down to the leaves.

val fixtureMatrix by matrixSuite(matrixConfig { execution = ExecutionMode.Concurrent() }) {
    fixture { Random.nextBytes(16) } - {
        "regular test with fresh bytes" { bytes ->        // fresh value, this test only
            bytes.size shouldBe 16
        }

        "suite with a shared fixture" - { bytes ->        // one value for the whole suite; `- { }` opens a matrix scope,
            data("word", listOf("foo", "bar")) test { word ->   // so data/property/compact layers are available here
                bytes.isNotEmpty() shouldBe true
                word.length shouldBeGreaterThan 0
            }
        }
    }

    // Fixtures also work inside compact blocks — same parameter, same freshness rules:
    compact("compact checks with fresh fixture") { report = CompactReport.AllCases } - {
        fixture { Random.nextBytes(16) } - {
            "fresh bytes per row" { bytes ->
                bytes.size shouldBe 16
            }
        }
    }
}

Project-wide matrix defaults are best configured based on the configuration of a TestSession:

object ProjectTestSessionConfig : TestSession(testConfig = TestConfig.apply {
    MatrixTestDefaults {
        execution = ExecutionMode.Sequential
        defaultPropertyIterations = 250
        defaultCompactConcurrency = CompactConcurrency.Shared(16)
        defaultCompactReport = CompactReport.FailuresOnly
        defaultCompactReportRows = 128
        defaultProgressIndicator = Indicator.Heartbeat(every = 2.seconds)
    }
})

Individual suites can override those defaults with matrixSuite parameters. Layers can override their own execution or generation settings. Real tests and suites also accept TestConfig, which is chained onto the current matrix config.

test, testSuite, and their FreeSpec "name"(…) forms accept a full matrix config too — pass a matrixConfig { … } value to set concurrency (and any other matrix setting) for that subtree. Unset fields inherit the enclosing scope, and testConfig is one of the fields (so matrixConfig { testConfig = … } covers aroundAll etc.). The testConfig-only forms shown above are just a shorthand wrapping matrixConfig { testConfig = … }. The fixture-scope test / testSuite (and their FreeSpec forms, inside fixture { … } - { … }) accept matrixConfig { … } the same way.

⚠️ Set concurrency with execution, not a testConfig. A testConfig is for aroundAll / aroundEach / context / timeout wrappers. Never put TestConfig.invocation(...) in a matrix testConfig — matrix derives invocation from execution, can't strip a conflicting one out of an (opaque) TestConfig, and the clash can break test discovery ("did not discover any tests"). Likewise, a virtual-time testScope(...) only applies to sequential execution (matrix disables it when concurrent), so enable TestScope on the TestSession, not per matrix scope. Wrong: matrixSuite(matrixConfig { testConfig = TestConfig.invocation(Sequential) }). Right: matrixSuite(matrixConfig { execution = ExecutionMode.Sequential }).

val perScopeConfig by matrixSuite(matrixConfig { execution = ExecutionMode.Sequential }) {
    // run just this group's children concurrently (also auto-disables the TestScope — see the concurrency note above)
    testSuite("heavy group", matrixConfig { execution = ExecutionMode.Concurrent(8) }) {
        data(1..1000) test { /* … */ }
    }
    "freespec group"(matrixConfig { execution = ExecutionMode.Concurrent(4) }) - {
        "child" { /* … */ }
    }
    test("one heavy test", matrixConfig { execution = ExecutionMode.Concurrent(2) }) { /* … */ }
}

val configuredMatrix by matrixSuite(matrixConfig {
    execution = ExecutionMode.Concurrent(parallelism = 4)
    defaultPropertyIterations = 50
}) {
    test("explicit test", testConfig = TestConfig) {
        // green code
    }

    testSuite("explicit suite", testConfig = TestConfig) {
        "bare FreeSpec-style test"(testConfig = TestConfig) {
            // green code
        }
    }

    "!temporarily disabled" {
        error("will not run")
    }
}

Layers (data / property) don't take a TestConfig themselves — their config block only carries layer settings (execution, nameMaxLength, seed, replay, …). To apply a TestConfig such as aroundAll, aroundEach, or a coroutineContext to a layer's cases, wrap the layer in a suite that does. Pair it with a nameless layer so no extra node appears:

testSuite("with shared setup", testConfig = TestConfig.aroundAll { action ->
    // one-time setup around every case of this layer
    action()
}) {
    data(listOf("foo", "bar", "baz")) test { word ->
        word.length shouldBeGreaterThan 0
    }
}

[!NOTE] compact virtual children are not real TestBalloon test elements, so this wrapping (and per-child TestConfig in general) cannot be honored inside compact. Wrap real (non-compact) layers, or configure the compact block itself.

Prefix any matrix name with ! to disable it. This works for test, testSuite, bare FreeSpec-style strings, data, property, and compact. Generated row names from nameFn don't get disabled when they start with a bang.

• data(...) test { ... } / property(...) test { ... } creates row test elements; data(...) - { ... } / property(...) - { ... } creates row suite or dimension test elements.
• Forgetting test { ... } or - { ... } leaves a configured layer unopened, so no child tests are registered. The layer itself does nothing until its trailing lambda receives content; either tests or more rows.
• data and property can each appear multiple times in the same suite.
• Terminal row tests are named by the layer nameFn. Use - { ... } plus an explicit "name" { ... } leaf when the invariant itself needs a separate name.
• IDE gutter actions. matrixSuite, and a data / property / compact / test / testSuite placed at the top level or under purely static nesting (only test / testSuite / named suites in between), get a working run-gutter: clicking a data("name", …) / property("name", …) / compact("name") line runs the whole layer (all its cases). A test / testSuite nested inside a data / property / compact layer does not get a usable gutter, because that element's path includes a per-case segment (nameFn(index, value), e.g. 0: x) that is generated at runtime and is invisible to the IDE's static analysis. To run such an inner element, click the enclosing layer's gutter (runs all cases) or set a filter by hand, e.g. TESTBALLOON_INCLUDE_PATTERNS='*name' (patterns are anchored full-match with * → .* across the ↘ path separator, so a leading * matches an arbitrarily deep element). Nameless data(…) / property(…) layers carry no gutter (they have no name to select on).
• Deep nesting can still create many real test elements. Use compact when the test tree itself becomes too large.

If you want APIs that mirror Kotest more closely, the original addon modules are still available:

• datatest replicates Kotest's data-driven testing features for TestBalloon
• property brings Kotest's property testing to TestBalloon
• fixturegen introduces per-test fixture generation for TestBalloon without boilerplate, and beyond TestBalloon's current fixture generation capabilities
• freespec emulates Kotest's FreeSpec test style for TestBalloon

[!TIP]
freespec and fixturegen are modulated into the fixturegen-freespec module. This means: if you add the at.asitplus.modulator Gradle plugin to any project that uses both, you can automagically combine FreeSpec syntax and per-test fixture generation.

If you don't want to use modulator, you can add the at.asitplus.testballoon:fixturegen-freespec:$version dependency manually to your project.

[!CAUTION]
TestBalloon jumps through quite some hoops to avoid the shortcomings of the underlying Gradle-based test infrastructure and file system limitations eating your cat. However, deep nesting and exceptionally long test names (both of which are easily produced when using data-driven testing or property testing) can still cause errors or even crashes.

This is especially true for Android device/emulator-based test execution, which is a wondrous mess!

Because TestBalloon can only shorten test names (not suite names), truncation becomes useful.

All modules allow setting global defaults with regard to test name truncation. These properties are called:

• defaultTestNameLength

The former generally defaults to 64 characters (15 on Android). Display names are not truncated by default.

Both properties can be set in two ways:

• globally (e.g., TestBalloonAddons.defaultTestNameLength = 15)
• per test style (e.g., FreeSpec.defaultTestNameLength = 10)

Per-style configuration takes precedence over global configuration. Hence, per-style configuration property setters are nullable, even though their getters will never return null, as they fall back to the global configuration properties automatically.

It is also possible to set test name length for individual tests by passing the maxLength arameter. Truncated names are ellipsised in the middle, not just cut off at the end.

→ Check out the full API docs for each test style for all configuration options!

TestBalloon Addons use sane default stringification for test names of collection and array types inside data-driven tests and property tests:

• All primitive arrays are correctly joined to string (i.e. [-1, 4, -643, 34310])
• All unsigned arrays are correctly joined to string (i.e. [9, 76, 145, 9365])
• ByteArray and UByteArray use hex uppercase notation (i.e. CA:FE:BA:BE)

Data-driven testing and property testing can easily produce millions of individual cases being tested. To avoid making the test runner's heap explode in such cases, the datatest and property modules allow for compacting test series.

Just pass the compact = true parameter when creating data-driven tests or property tests (see examples in the module descriptions for data-driven testing and property testing).

The names of compacted test series consist of an uppercase sigma (Σ) followed by the test series' datatype (e.g., ΣULong, ΣByteArray, …).

To still get intelligible output about which precise data point(s) caused failing tests, the error message of the resulting failed assertion contains a compact summary and then lists the relevant child rows:

java.lang.AssertionError: ΣString
Summary: 1 OK, 7 failed
Error: 1: 4: expected:<three> but was:<4>
Error: 2: one: expected:<three> but was:<one>
Error: 3: null: Expected "three" but actual was null
Error: 4: null: Expected "three" but actual was null
Error: 5: null: Expected "three" but actual was null
Error: 6: two: expected:<three> but was:<two>
OK:    7: three
Error: 8: four: expected:<three> but was:<four>
----------------------------------------

If you only care about failing cases, set suppressCompactSuccesses = true. This can be configured globally through TestBalloonAddons.suppressCompactSuccesses, per module through DataTest.suppressCompactSuccesses or PropertyTest.suppressCompactSuccesses, and per terminal withData / checkAll call. Successful cases are still counted in the summary, but individual OK rows are omitted.

The stack trace of the thrown exception is the stack trace of the first error (which is equal to the stack traces of all failed assertions). As such, you can directly navigate to the error with the same convenience as ever!

By default, compacted reports keep only the first failure as the cause. To attach all failed child exceptions as suppressed exceptions, set addSuppressedErrorsToCompactedFailures = true globally, per module, or per compacted run where the API offers the override.

Suspending terminal compacted withData and checkAll leaves run child bodies sequentially by default. Set compactConcurrent = true globally through TestBalloonAddons.compactConcurrent, per module through DataTest.compactConcurrent or PropertyTest.compactConcurrent, or per terminal call if concurrent execution is desired for that series. Intermediate suite builders keep their existing suite registration behaviour.

Long-running compacted terminal leaves also print periodic progress heartbeats while the compacted body is still running, for example:

ΣByteArray: compact progress: 124/1000000 completed, 3 failed

This output is intentionally separate from the compacted failure report.

To globally enable compacting test series for data-driven testing and property testing, set DataTest.compactByDefault = true and PropertyTest.compactByDefault = true, respectively.

Compacting works on test and suite level!

In addition, it is possible to specify a prefix parameter when defining data-driven tests or property tests. The prefix is prepended to generated test names (in front of the sigma), which helps navigate large test graphs.

→ Check out the full API docs for each test style for all configuration options!

[!NOTE]
Deep nesting will produce a large number of tests, making the heap explode. Either manually compact tests as in the example below (works for both withData test series and withData suite series), or set the global DataTest.compactByDefault = true to automatically compact all data-driven tests.

TestBalloon makes it ridiculously easy to roll your own data-driven testing wrapper with just a couple of lines of code. So we did, by replicating Kotest's data-driven testing API:

val aDataDrivenSuite by testSuite {
    
    // -> NOTE the minus ↙↙↙, it creates a suite
    withData(1, 2, 3, 4) - { number ->
        // Will create only a single test, but the error will contain all failed inputs
        withData("one", "two", "three", "four", compact = true) { word ->
            //your test logic being run 16 times
        }
    }
}

It is possible to specify a prefix parameter when defining data-driven tests and suites. The prefix is prepended to generated test names, which helps navigate large test reports.

Running individual tests from the gutter is not possible, as the test suite structure and the names of suites and tests are computed at runtime. Hence, you must run the entire suite (but you can manually filter using wildcards).

[!NOTE]
Deep nesting will produce a large number of tests, making the heap explode. Either manually compact tests as in the first example below (works for both checkAll test series and checkAll suite series), or set the global PropertyTest.compactByDefault = true to automatically compact all data-driven tests.

Although it comes with some warts, kotest-property is still extremely helpful for generating a large corpus of test data—especially as it covers many edge cases out of the box. Again, since TestBalloon has been specifically crafted to be flexible and extensible, we did just that:

val propertySuite by testSuite {
    // DON'T generate a suite for each item. Instead: aggregate >->-->------↘↘↘↘↘↘↘↘↘↘↘↘
    checkAll(iterations = 100, Arb.byteArray(Arb.int(100, 200), Arb.byte()), compact = true) - { byteArray ->
        checkAll(iterations = 10, Arb.uLong()) { number ->
            //test with byte arrays and number for fun and profit
        }
    }

    checkAll(iterations = 100, Arb.byteArray(Arb.int(100, 200), Arb.byte())) - { byteArray ->
        checkAll(iterations = 10, Arb.uLong()) { number ->
            //test with byte arrays and number for fun and profit
        }
    }
}

It is possible to specify a prefix parameter when defining property tests and suites. The prefix is prepended to generated test names, which helps navigate large test reports.

Running individual tests from the gutter is not possible, as the test suite structure and the names of suites and tests are computed at runtime. Hence, you must run the entire suite (but you can manually filter using wildcards).

TestBalloon enforces a strict separation between blue code and green code. This is a good thing—especially for deeply nested test suites—and it supports deep concurrency. Hence, ye olde JUnit4-style @Before and @After hacks mutating global state are deliberately not supported.

Sometimes, though, you really want fresh data for every test or suite—in effect, you want to generate a fresh test fixture for every test/suite.

[!NOTE]
Fixture generation as provided by the addons does not use TestBalloon's native fixtures, as those only work in green code. The flavour of fixture generation provided by TestBalloon Addons works for suites (blue code) and tests (green code), as shown below.

import at.asitplus.testballoon.withFixtureGenerator //<- Look ma, only a single import!
import de.infix.testBalloon.framework.core.testSuite
import kotlin.random.Random
import kotlinx.coroutines.delay //just to get some suspending demo generator

val aGeneratingSuite by testSuite {

    //seed before the generator function, not inside!
    val byteRNG = Random(42);
    //We want to test with fresh randomness, so we generate a fresh fixture for each test
    withFixtureGenerator { byteRNG.nextBytes(32) } - {

        repeat(5) {
            test("Generated test with fresh randomness") { freshFixture ->
                //your test logic here
            }
        }

        testSuite("Generated Suite with fresh randomness") { freshFixture ->
            test("using the outer fixture") {
                //your logic based on freshFixture here
            }
        }
        repeat(5) {
            //✨ it ✨ just ✨ werks ✨
            test("Test with implicit fixture name `it`") {
                //do something with `it`, it contains fresh randomness!
            }
        }
    }

    //seed the RNG for reproducible tests
    val random = Random(42)

    //reference function to be called for each test inside withFixtureGenerator
    withFixtureGenerator(random::nextFloat) - {
        repeat(10) {
            test("Generated test with random float") {
                //test something floaty!
            }
        }
        test("And some other test that des not conform to the shema from the loop") {
            //test something different, with a fresh float
        }
    }

    //always-the-same fixtures also work, of course
    withFixtureGenerator {
        object {
            var a: Int = 1
            val b: Int = 2
        }
    } - {
        test("one") {
            it.a++ //and we can even modify them in one test
            println("a=${it.a}, b=${it.b}") //a=2, b=2
        }
        test("two") {
            //without affecting the other!
            println("a=${it.a}, b=${it.b}") //a=1, b=2
        }
    }

    //Let's test some nasty bug that shows itself only sometimes functionality
    val ageRNG = Random(seed = 26)
    withFixtureGenerator {
        class ABuggyImplementation(val age: Int) {
            fun restrictedAction(): Boolean =
                if (age < 18) false
                else if (age > 18) true
                else Random.nextBoolean() //introduce jitter to simulate a faulty implementation
        }

        //create new object for each test
        ABuggyImplementation(ageRNG.nextInt(0, 99))
    } - {
        repeat(1000) {
            test("Generated test accessing restricted resources") {
                //test `restrictedAction` across a wide age range
                //a thousand times to unveil the bug
            }
        }
    }
}

[!WARNING]
A fixture-generating scope is intended to be consumed by the scope directly below it (i.e. the outermost test suite, or directly by a test). Programmatically, you can mix this up and it will compile, but it will not run!

The following is an antipattern:

val outermostSuite by testSuite {
  withFixtureGenerator(random::nextFloat) - {
    testSuite("outer") { /*fixture implicitly available as `it`*/
      test("nested") { float -> /**`it` is not available, explicit parameter specification messes things up*/
        //This will throw a runtime error, because "nested" will be erroneously wired directly below the outermos suite
      }
    }
  }
}

At A-SIT Plus, we've been using Kotest's FreeSpec for its expressiveness, as it allows modeling tests and test dependencies close to natural language.

TestBalloon is flexible enough to emulate FreeSpec with very little code, if you have context parameters enabled for your codebase:

// build.gradle.kts
kotlin {
    compilerOptions {
        freeCompilerArgs.add("-Xcontext-parameters")
    }
}

import at.asitplus.testballoon.invoke
import at.asitplus.testballoon.minus
import de.infix.testBalloon.framework.core.TestConfig
import de.infix.testBalloon.framework.core.TestInvocation
import de.infix.testBalloon.framework.core.invocation
import de.infix.testBalloon.framework.core.singleThreaded
import de.infix.testBalloon.framework.core.testSuite

val aFreeSpecSuite by testSuite {
    //testConfigs are supported for suites
    "The outermost blue code"(testConfig = TestConfig.singleThreaded()) - {
        "contains some more blue code" - {
            ", some green code inside the lambda" {
                // your test logic here
            }
            //testConfigs are supported for Tests
            ", and some more green code inside the second lambda"(testConfig = TestConfig.invocation(TestInvocation.SEQUENTIAL)) {
                // more test logic here
            }
        }
        "And finally some more blue code" - {
            "!With some final disabled green code in this lambda" {
                //additional, disabled test logic here
            }
        }
    }
}

Running individual tests from the gutter is not (yet) possible, due to the intricacies of how code analysis works. Hence, you must run the entire suite (but you can manually filter using wildcards).
(You can, of course, just migrate off FreeSpec and use TestBalloon's native functions to create suites and tests.)

import at.asitplus.testballoon.withFixtureGenerator //   <- Look ma, only regular generatingFixture import!
import at.asitplus.testballoon.invoke //                 <- Look ma, only regular freespec import!
import at.asitplus.testballoon.minus  //                 <- Look ma, only regular freespec import!
import de.infix.testBalloon.framework.core.testSuite
import kotlin.random.Random

val aGeneratingFreeSpecSuite by testSuite {

    //any lambda with any return type is a fixture generator. Type is reified.
    withFixtureGenerator { Random.nextBytes(32) } - {

        "A Suite with fresh randomness" - { freshFixture ->
            "Consuming outer fixture" {
                //your freshFixture-based test logic here
            }

            withFixtureGenerator { Random.nextBytes(32) } - {
                "With fresh inner fixture" { inner ->
                    //your test logic here with always fresh inner
                    //and fixed freshFixture from outer scope
                }
            }

        }

        repeat(100) {
            "Generated test with fresh randomness" { freshFixture ->
                //some more test logic; each call gets fresh randomness
            }
        }

        //parameter must be explicitly specified to disambiguate
        "Test with fixture name `it`" { it ->
            //no need for an explicit parameter name here, just use `it`
        }

        "And we can even nest!" - {
            withFixtureGenerator { Random.nextBytes(16) } - {
                repeat(10) {
                    "pure, high-octane magic going on" { it ->
                        //Woohoo! more randomness each run
                    }
                }
            }
        }
    }
}

External contributions are greatly appreciated! Just be sure to observe the contribution guidelines (see CONTRIBUTING.md).

The Apache License does not apply to the logos, (including the A-SIT logo) and the project/module name(s), as these are the sole property of A-SIT/A-SIT Plus GmbH and may not be used in derivative works without explicit permission!