Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 7 additions & 3 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -162,10 +162,14 @@ unmangled method name, so a mangled internal name compiles, links, and dies on t
- **Link flags live in `KonanDistribution.linkArguments`**: `-fuse-ld=lld` for Linux (the GCC toolchain's
`ld.bfd` is a Linux binary; a cross link from macOS fails without it), NDK compiler-rt and
`--unwindlib=none` for Android, Xcode's compiler-rt for macOS. The wrappers add them only when linking.
- **`androidNdk` falls back, last, to the module's AGP NDK** (`AndroidComponents.sdkComponents.ndkDirectory`,
- **`androidNdk` falls back to the module's AGP NDK before `ANDROID_NDK_HOME`** (`AndroidComponents.sdkComponents.ndkDirectory`,
set into `KonanConfig.androidGradlePluginNdk` by `AgpWiring.ndkDirectory`, source `ANDROID_GRADLE_PLUGIN`).
Not a search: it is the project's own `android.ndkVersion`. AGP throws when that NDK is not installed, so the
provider catches it and reads as unset, leaving the plugin's own "not set" error.
Not a search: it is the project's own `android.ndkVersion`. The order is DSL, Gradle property, AGP, environment:
GitHub's runners preset `ANDROID_NDK_HOME` to NDK 27, which silently shipped 4 KB-aligned libraries from a
project on NDK 28 (#19). AGP throws when its NDK is not installed, so the provider catches it and reads as
unset, leaving the environment variable, then the plugin's own "not set" error. An `ANDROID_NDK_HOME` that lost
rides in `ConfigValue.overridden`; `AndroidNdkNotice`, a build service, warns once per build from a `doFirst`
of each task using the NDK — at configuration it would vanish on a configuration cache hit.
- **Android requires an NDK (`konan.config.androidNdk`), and everything Android uses it** — `android/AndroidNdk`:
the default library build (`CompileNativeLibraryTask`), the JNI and FFM links (`LinkSharedLibraryTask`), and
external builds on `AUTO` (`AndroidNdkToolchain`). Kotlin/Native's Android toolchain is clang 21 driving NDK
Expand Down
13 changes: 8 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,7 @@ from the build script, else the Gradle property `konan.config.<field>` (the buil
| `make` | `konan.config.make` | `KONAN_MAKE` | `make { }` |
| `cargo` | `konan.config.cargo` | `KONAN_CARGO` | `cargo { }` |
| `zig` | `konan.config.zig` | `KONAN_ZIG` | `zig { }` |
| `androidNdk` | `konan.config.androidNdk` | `ANDROID_NDK_HOME` | every Android target: the plugin's own compiles and links, and external builds on `AUTO` (below). In an Android module, last of all the NDK the Android Gradle plugin uses |
| `androidNdk` | `konan.config.androidNdk` | `ANDROID_NDK_HOME` | every Android target: the plugin's own compiles and links, and external builds on `AUTO` (below). In an Android module, the NDK the Android Gradle plugin uses comes before the environment variable |

```kotlin
konan {
Expand Down Expand Up @@ -614,9 +614,12 @@ with no stub. The app needs:
Every Android library the plugin builds — the default library build, the JNI library, the library FFM
loads — is compiled and linked with the Android NDK in `konan.config.androidNdk` (r23 or newer), which
Android targets require. In an Android module it is the NDK the Android Gradle plugin uses — the one
`android { ndkVersion = "…" }` names, else AGP's default, once it is installed — so nothing needs setting;
`konan.config.androidNdk=<Android SDK>/ndk/<version>` or `ANDROID_NDK_HOME` choose another, and are what a
module without AGP sets. `konanConfig` shows which one a build uses. One NDK for everything, so an archive
`android { ndkVersion = "…" }` names, else AGP's default, once it is installed — so nothing needs setting.
`konan { config { androidNdk } }` or `konan.config.androidNdk=<Android SDK>/ndk/<version>` choose another.
`ANDROID_NDK_HOME` comes last, after AGP's NDK: CI images preset it, and a variable nobody chose must not
split a module across two NDKs. It serves modules without AGP, and those whose AGP NDK is not installed;
a build that sets it to a different NDK and goes unheard gets a warning naming both, once.
`konanConfig` shows which one a build uses. One NDK for everything, so an archive
your own build made with it links against the same libc++, and `androidApiLevel` reaches every API level
the NDK has.

Expand Down Expand Up @@ -692,7 +695,7 @@ The DSL is new and there is no compatibility shim.
- JNI bindings are named after the C functions (`my_add`), not `kniBridge0`; tasks are `generateJni…` and
`linkJni…`.
- Android needs an Android NDK (r23 or newer), which builds every Android library: in an Android module the
one the Android Gradle plugin uses, else `konan.config.androidNdk` or `ANDROID_NDK_HOME`. Kotlin/Native's
one the Android Gradle plugin uses unless `konan.config.androidNdk` says otherwise, else `ANDROID_NDK_HOME`. Kotlin/Native's
own Android toolchain is no longer used, and `androidApiLevel` is no longer capped at 29.
- A library's build may leave several archives in `KONAN_OUTPUT_DIR`; they are merged into `lib<name>.a`.

Expand Down
6 changes: 5 additions & 1 deletion build.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -40,13 +40,15 @@ gradlePlugin {
}

val kgpForTests: Configuration = configurations.create("kgpForTests")
val agpForTests: Configuration = configurations.create("agpForTests")

dependencies {
// -PkgpTestVersion runs the KGP fixtures on another Kotlin version; -PtestGradleVersion on another Gradle.
"kgpForTests"(
providers.gradleProperty("kgpTestVersion").orElse(providers.gradleProperty("kotlinNativeVersion"))
.map { "org.jetbrains.kotlin:kotlin-gradle-plugin:$it" }
)
"agpForTests"("com.android.tools.build:gradle:9.2.1")
// KonanTarget, for NativeTarget.of(konanTarget) and the KGP adapter — both reached only where KGP, which
// brings it, is applied. The DSL speaks the plugin's own NativeTarget, so nothing decorated needs it.
compileOnly(providers.gradleProperty("kotlinNativeVersion").map { "org.jetbrains.kotlin:kotlin-native-utils:$it" })
Expand All @@ -72,9 +74,11 @@ tasks.test {
// proving the plugin loads without KGP.
val kgp: FileCollection = kgpForTests
inputs.files(kgp).withPropertyName("kgpForTests").withNormalizer(ClasspathNormalizer::class.java)
val agp: FileCollection = agpForTests
inputs.files(agp).withPropertyName("agpForTests").withNormalizer(ClasspathNormalizer::class.java)
val gradleVersion = providers.gradleProperty("testGradleVersion").orElse("")
inputs.property("testGradleVersion", gradleVersion)
jvmArgumentProviders.add(CommandLineArgumentProvider { listOf("-Dkgp.classpath=${kgp.asPath}", "-Dtest.gradle.version=${gradleVersion.get()}") })
jvmArgumentProviders.add(CommandLineArgumentProvider { listOf("-Dkgp.classpath=${kgp.asPath}", "-Dagp.classpath=${agp.asPath}", "-Dtest.gradle.version=${gradleVersion.get()}") })
}

// A version already on the Plugin Portal must never be published again, not even to mavenLocal: a local
Expand Down
2 changes: 1 addition & 1 deletion examples/android/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ cpp.cpp_median(arena.allocateFrom(JAVA_INT, 5, 1, 4, 2), 4) // C++, FFM
```

Everything Android is built with the NDK the Android Gradle plugin uses (`android.ndkVersion`, else its
default; `ANDROID_NDK_HOME` or `konan.config.androidNdk` choose another): the C and
default; `konan.config.androidNdk` chooses another): the C and
C++ compiles, cargo's C compilers, and the links. The Rust and C++ libraries need runtimes Android does not
give apps — Rust's std its unwinder, C++ libc++ and the unwinder — which the links put inside each library
(`-static-libstdc++`), failing the build if anything is left undefined.
Expand Down
4 changes: 2 additions & 2 deletions examples/kmp/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,8 +75,8 @@ docker run --rm --platform linux/amd64 -v "$PWD/shared/build/bin/linuxX64":/kn u
```

Needs Xcode for the iOS and macOS targets, an Android SDK (`local.properties` or `ANDROID_HOME`), and the
NDK the Android Gradle plugin uses installed (SDK Manager → NDK (Side by side)); `ANDROID_NDK_HOME` or
`konan.config.androidNdk` choose another.
NDK the Android Gradle plugin uses installed (SDK Manager → NDK (Side by side)); `konan.config.androidNdk`
chooses another.

From Swift, the common API is in the framework:

Expand Down
46 changes: 46 additions & 0 deletions src/main/kotlin/io/github/lemcoder/android/AndroidNdkNotice.kt
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
package io.github.lemcoder.android

import io.github.lemcoder.dsl.config.ConfigValue
import java.util.concurrent.ConcurrentHashMap
import org.gradle.api.Action
import org.gradle.api.Project
import org.gradle.api.Task
import org.gradle.api.logging.Logger
import org.gradle.api.provider.Provider
import org.gradle.api.services.BuildService
import org.gradle.api.services.BuildServiceParameters

/**
* Says that `ANDROID_NDK_HOME` lost to the Android Gradle plugin's NDK (#19), once per build however many
* tasks use the NDK. Tasks ask as they run: a warning at configuration would not repeat when the
* configuration cache is reused, and would reach modules that build nothing for Android.
*/
internal abstract class AndroidNdkNotice : BuildService<BuildServiceParameters.None> {
private val warned = ConcurrentHashMap.newKeySet<String>()

fun warnOnce(logger: Logger, warning: String) {
if (warned.add(warning)) logger.warn(warning)
}

companion object {
fun register(project: Project): Provider<AndroidNdkNotice> =
project.gradle.sharedServices.registerIfAbsent("konanAndroidNdkNotice", AndroidNdkNotice::class.java) {}
}
}

/** This task builds with [ndk]: warn when that left an `ANDROID_NDK_HOME` unused. */
internal fun Task.warnsOfUnusedNdkHome(ndk: Provider<ConfigValue>) {
val notice = AndroidNdkNotice.register(project)
usesService(notice)
doFirst(WarnOfUnusedNdkHome(ndk, notice))
}

/** A class, not a lambda: Gradle will not call a task up to date whose action is a lambda. */
private class WarnOfUnusedNdkHome(
private val ndk: Provider<ConfigValue>,
private val notice: Provider<AndroidNdkNotice>,
) : Action<Task> {
override fun execute(task: Task) {
ndk.get().overriddenWarning()?.let { notice.get().warnOnce(task.logger, it) }
}
}
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
package io.github.lemcoder.dsl.bindings.ffm

import io.github.lemcoder.android.warnsOfUnusedNdkHome
import io.github.lemcoder.dsl.KonanExtension
import io.github.lemcoder.dsl.bindings.BindingSpec
import io.github.lemcoder.dsl.bindings.LinkSharedLibraryTask
Expand Down Expand Up @@ -44,7 +45,10 @@ internal fun registerFfmLink(
description = "Link the '$libraryName' library for ${target.konanName}, for the FFM bindings."
dependsOn(artifacts.builtBy(target))
androidApiLevel.set(konan.androidApiLevel)
if (target.isAndroid) androidNdk.set(konan.config.readConfigField(ConfigField.ANDROID_NDK, path))
if (target.isAndroid) {
androidNdk.set(konan.config.readConfigField(ConfigField.ANDROID_NDK, path))
warnsOfUnusedNdkHome(androidNdk)
}
this.target.set(target)
libraryBaseName.set(libraryName)
staticLibrary.set(artifacts.archive(target))
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
package io.github.lemcoder.dsl.bindings.jni

import io.github.lemcoder.android.warnsOfUnusedNdkHome
import io.github.lemcoder.dsl.KonanExtension
import io.github.lemcoder.dsl.bindings.BindingSpec
import io.github.lemcoder.dsl.bindings.DefFile
Expand Down Expand Up @@ -38,7 +39,10 @@ internal fun registerKonanLink(
artifacts?.let { dependsOn(it.builtBy(target)) }
mustRunAfter(project.cleanTask())
androidApiLevel.set(konan.androidApiLevel)
if (target.isAndroid) androidNdk.set(konan.config.readConfigField(ConfigField.ANDROID_NDK, path))
if (target.isAndroid) {
androidNdk.set(konan.config.readConfigField(ConfigField.ANDROID_NDK, path))
warnsOfUnusedNdkHome(androidNdk)
}
this.target.set(target)
libraryBaseName.set(jni.stubLibraryName)
stubCFile.set(generate.flatMap { it.outputDirectory.file(jni.stubLibraryName.map { name -> "${GeneratorRequest.Jni.C_DIR}/$name.c" }) })
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
package io.github.lemcoder.dsl.builds

import io.github.lemcoder.android.AndroidNdk
import io.github.lemcoder.android.warnsOfUnusedNdkHome
import io.github.lemcoder.dsl.KonanExtension
import io.github.lemcoder.dsl.config.ConfigField
import io.github.lemcoder.kotlin.KonanToolchain
Expand Down Expand Up @@ -73,9 +74,10 @@ internal sealed class ExternalBuild(private val taskPrefix: String, private val
// explicit KONAN keeps Kotlin/Native's own Android toolchain (NDK r20).
val ndk = target.isAndroid && toolchain == ExternalToolchain.AUTO
val konanToolchain = toolchain?.takeIf { !ndk && it.usesKonan(target) }?.let { KonanToolchain.of(project, target) }
val toolchainFiles = if (ndk) {
val ndkValue = if (ndk) konan.config.readConfigField(ConfigField.ANDROID_NDK, taskPath) else null
val toolchainFiles = if (ndkValue != null) {
AndroidNdkToolchain.files(
konan.config.readConfigField(ConfigField.ANDROID_NDK, taskPath), target,
ndkValue, target,
konan.androidApiLevel.orElse(AndroidNdk.DEFAULT_API_LEVEL),
)
} else konanToolchain?.files
Expand All @@ -98,6 +100,7 @@ internal sealed class ExternalBuild(private val taskPrefix: String, private val
description = taskDescription
mustRunAfter(clean)
usesService(limit)
ndkValue?.let { warnsOfUnusedNdkHome(it) }
konanToolchain?.let { dependsOn(it.task) }
context.environment.forEach { (variable, value) -> this.environment.put(variable.variableName, value) }
libraryFileName.set(fileName)
Expand Down
10 changes: 9 additions & 1 deletion src/main/kotlin/io/github/lemcoder/dsl/config/ConfigField.kt
Original file line number Diff line number Diff line change
Expand Up @@ -28,10 +28,10 @@ enum class ConfigField(val dslName: String, val environmentVariable: String, val
enum class ConfigSource(val description: String) {
DSL("build script"),
GRADLE_PROPERTY("Gradle property"),
ENVIRONMENT("environment variable"),

/** `androidNdk` only: the NDK the module's Android Gradle plugin uses (`android.ndkVersion`, else its default). */
ANDROID_GRADLE_PLUGIN("the Android Gradle plugin's NDK"),
ENVIRONMENT("environment variable"),
}

/**
Expand All @@ -46,6 +46,8 @@ data class ConfigValue(
val usedBy: String,
/** Extra lines for the error, e.g. the default the build could use. */
val hint: String? = null,
/** A different value of [field]'s environment variable, which this one took precedence over. */
val overridden: String? = null,
) : Serializable {
/** The file named, checked to exist as the field's kind; otherwise an error saying how to set it. */
fun require(): File {
Expand All @@ -65,6 +67,12 @@ data class ConfigValue(
}
}

/** What to tell the build about [overridden], if anything: it set a variable that had no effect. */
fun overriddenWarning(): String? = overridden?.let {
"konan.config.${field.dslName}: using ${describe()}, not ${field.environmentVariable}=$it. " +
"To use that one, set ${field.gradleProperty}."
}

/** The current value and its origin, for reports: `/opt/x (Gradle property konan.config.x)`. */
fun describe(): String = when (source) {
null -> "unset"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,8 @@ internal abstract class KonanConfigReportTask : DefaultTask() {
else -> "NEEDED, ${problem.removePrefix("is ")}"
}
val needs = if (users.isEmpty()) "" else "\n${" ".repeat(width + 4)}needed by ${users.joinToString()}"
" ${value.field.dslName.padEnd(width)} ${value.describe()} — $status$needs"
val overridden = value.overridden?.let { "\n${" ".repeat(width + 4)}${value.field.environmentVariable}=$it is not used" }.orEmpty()
" ${value.field.dslName.padEnd(width)} ${value.describe()} — $status$overridden$needs"
}
logger.lifecycle((listOf("konan.config for ${projectPath.get()}:") + lines).joinToString("\n"))
val missing = values.get().filter { usages[it.field.name].orEmpty().isNotEmpty() && it.problem() != null }
Expand Down
19 changes: 12 additions & 7 deletions src/main/kotlin/io/github/lemcoder/dsl/config/KonanConfig.kt
Original file line number Diff line number Diff line change
Expand Up @@ -57,8 +57,8 @@ abstract class KonanConfig @Inject constructor(private val providers: ProviderFa
* Android library with it — the default library build, the JNI library, the library FFM loads — and
* external builds on the `AUTO` toolchain get its compilers. Kotlin/Native's own Android toolchain (NDK
* r20) is used only by external builds that ask for it with `KONAN`. In an Android module it defaults to
* the NDK the Android Gradle plugin uses (`android.ndkVersion`), after the build script, the Gradle
* property and `ANDROID_NDK_HOME`.
* the NDK the Android Gradle plugin uses (`android.ndkVersion`), after the build script and the Gradle
* property but before `ANDROID_NDK_HOME`: what the build declares beats what the machine presets.
*/
abstract val androidNdk: DirectoryProperty

Expand All @@ -78,8 +78,8 @@ abstract class KonanConfig @Inject constructor(private val providers: ProviderFa
private fun kotlinVersion(): String = kotlinPluginVersion ?: KonanDistribution.defaultVersion

/**
* [field] for the task [usedBy]: this block, else the Gradle property, else the environment variable,
* else unset. Always present; [ConfigValue.require] checks it when the task runs.
* [field] for the task [usedBy]: this block, else the Gradle property, else (`androidNdk`) the module's
* Android Gradle plugin NDK, else the environment variable, else unset. Always present; [ConfigValue.require] checks it when the task runs.
*/
internal fun readConfigField(field: ConfigField, usedBy: String): Provider<ConfigValue> {
usages.getOrPut(field) { sortedSetOf() } += usedBy
Expand All @@ -98,13 +98,18 @@ abstract class KonanConfig @Inject constructor(private val providers: ProviderFa
fun value(source: ConfigSource) = { raw: String -> ConfigValue(field, raw, source, usedBy, hint) }
return property(field).map { it.asFile.absolutePath }.map(value(ConfigSource.DSL))
.orElse(providers.gradleProperty(field.gradleProperty).map(value(ConfigSource.GRADLE_PROPERTY)))
.orElse(providers.environmentVariable(field.environmentVariable).map(value(ConfigSource.ENVIRONMENT)))
.let { configured ->
if (field == ConfigField.ANDROID_NDK) configured.orElse(androidGradlePluginNdk.map(value(ConfigSource.ANDROID_GRADLE_PLUGIN))) else configured
.let { declared ->
if (field != ConfigField.ANDROID_NDK) declared
else declared.orElse(androidGradlePluginNdk.map { value(ConfigSource.ANDROID_GRADLE_PLUGIN)(it).copy(overridden = differentEnvironment(field, it)) })
}
.orElse(providers.environmentVariable(field.environmentVariable).map(value(ConfigSource.ENVIRONMENT)))
.orElse(ConfigValue(field, null, null, usedBy, hint))
}

/** [field]'s environment variable, when it names something other than [raw]. */
private fun differentEnvironment(field: ConfigField, raw: String): String? =
providers.environmentVariable(field.environmentVariable).orNull?.takeIf { File(it).canonicalFile != File(raw).canonicalFile }

/** [field] as the program an external build step runs. */
internal fun executable(field: ConfigField, usedBy: String): Provider<Executable> =
readConfigField(field, usedBy).map { Executable.Configured(it) }
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
package io.github.lemcoder.dsl.libraries

import io.github.lemcoder.android.warnsOfUnusedNdkHome
import io.github.lemcoder.dsl.config.ConfigField
import io.github.lemcoder.dsl.KonanExtension
import io.github.lemcoder.model.NativeTarget
Expand Down Expand Up @@ -39,6 +40,7 @@ internal fun registerKonanClang(project: Project, konan: KonanExtension, library
libName.set(library.name)
if (target.isAndroid) {
androidNdk.set(konan.config.readConfigField(ConfigField.ANDROID_NDK, path))
warnsOfUnusedNdkHome(androidNdk)
androidApiLevel.set(konan.androidApiLevel)
}
arguments.addAll(DEFAULT_COMPILER_OPTIONS)
Expand Down
Loading
Loading