Kotlin K2 and Standalone Source Generator

Posted by Ting-Yuan Huang – Software Engineer, and Jiaxiang Chen – Software Engineer

The Kotlin Symbol Processing (KSP) tool provides a high-level API for doing meta-programming in Kotlin. Many tools have been built on KSP, enabling Kotlin code to be generated at compile time. For example, Jetpack Room uses KSP to generate code for accessing the database, based on an interface provided by the developer, like:

@Dao
interface UserDao {
    @Query("SELECT * FROM user")
    fun getAll(): List<User>
}

KSP provides the API to the Kotlin code so that Room in this case can generate the actual implementation of that interface. While KSP has become a core foundation for meta-programing in Kotlin, its current implementation has some gaps which we are aiming to resolve with a new KSP2 architecture. This blog details those architectural changes and the impact for plugins built on KSP.

In addition, KSP2 has preview support for:

• The new Kotlin compiler (code-named K2)
• A new standalone source generator that provides more flexibility and features than the current Kotlin compiler plugin

After getting feedback on the new architecture and continuing to address gaps we will work towards releasing KSP 2.0 where these changes will be the default.

Enabling the KSP2 Preview

The new preview changes can be enabled in KSP 1.0.14 or newer using a flag in gradle.properties:

Note: You might need to enlarge the heap size of the Gradle daemon now that KSP and processors run in the Gradle daemon instead of the Kotlin compiler’s daemon (which has larger default heap size), e.g. org.gradle.jvmargs=-Xmx4096M -XX:MaxMetaspaceSize=1024m

KSP2 and K2

Internally KSP2 uses the Beta Kotlin K2 compiler (which will be the default compiler in Kotlin 2.0). You can use KSP2 before switching your Kotlin compiler to K2 (via the languageVersion setting) but if you want to use K2 for compiling your code, check out: Try the K2 compiler in your Android projects.

Standalone Source Generator

KSP1 is implemented as a Kotlin 1.x compiler plugin. Running KSP requires running the compiler and specifying KSP and its plugin options. In Gradle, KSP’s tasks are customized compilation tasks, which dispatch real work to KotlinCompileDaemon by default. This makes debugging and testing somewhat difficult, because KotlinCompileDaemon runs in its own process, outside of Gradle.

In KSP2, the implementation can be thought of as a library with a main entry point. Build systems and tools can call KSP with this entry point, without setting up the compiler. This makes it very easy to call KSP programmatically and is very useful especially for debugging and testing. With KSP2 you can set breakpoints in KSP processors without having to perform any other / irregular setup tasks to enable debugging.

Everything becomes much easier because KSP2 now controls its lifecycle and can be called as a standalone program or programmatically, like:

val kspConfig = KSPJvmConfig.Builder().apply {
  // All configurations happen here.
}.build()
val exitCode = KotlinSymbolProcessing(kspConfig, listOfProcessors, kspLoggerImpl).execute()

KSP2 API Behavior Changes

With the new implementation, it is also a great opportunity to introduce some refinements in the API behavior so that developers building on KSP will be more productive, have better debuggability and error recovery. For example, when resolving Map<String, NonExistentType>, KSP1 simply returns an error type. In KSP2, Map<String, ErrorType> will be returned instead. Here is a list of the current API behavior changes we plan on making in KSP2:

1. Resolve implicit type from function call: val error = mutableMapOf<String, NonExistType>()

   KSP1: The whole type will be an error type due to failed type resolution

   KSP2: It will successfully resolve the container type, and for the non-existent type in the type argument, it will correctly report errors on the specific type argument.

2. Unbounded type parameter

   KSP1: No bounds

   KSP2: An upper bound of Any? is always inserted for consistency

3. Resolving references to type aliases in function types and annotations

   KSP1: Expanded to the underlying, non-alias type

   KSP2: Not expanded, like uses in other places.

4. Fully qualified names

   KSP1: Constructors have FQN if the constructor is from source, but not if the constructor is from a library.

   KSP2: Constructors do not have FQN

5. Type arguments of inner types

   KSP1: Inner types has arguments from outer types

   KSP2: Inner types has no arguments from outer types

6. Type arguments of star projections

   KSP1: Star projections have type arguments that are expanded to the effective variances according to the declaration sites.

   KSP2: No expansion. Star projections have nulls in their type arguments.

7. Variance of Java Array

   KSP1: Java Array has a invariant upper bound

   KSP2: Java Array has a covariant upper bound

8. Enum entries

   KSP1: An enum entry has its own subtype with a supertype of the enum class (incorrect behavior from language point of view)

   KSP2: An enum entry’s type is the type of the enclosing enum class

9. Multi-override scenario

   For example

   interface GrandBaseInterface1 {
       fun foo(): Unit
   }
   
   interface GrandBaseInterface2 {
       fun foo(): Unit
   }
   
   interface BaseInterface1 : GrandBaseInterface1 {
   }
   
   interface BaseInterface2 : GrandBaseInterface2 {
   }
   
   class OverrideOrder1 : BaseInterface1, GrandBaseInterface2 {
       override fun foo() = TODO()
   }
   class OverrideOrder2 : BaseInterface2, GrandBaseInterface1 {
       override fun foo() = TODO()
   }
   

   KSP1:

   Find overridden symbols in BFS order, first super type found on direct super type list that contains overridden symbol is returned
   For the example, KSP will say OverrideOrder1.foo() overrides GrandBaseInterface2.foo() and OverrideOrder2.foo() overrides GrandBaseInterface1.foo()

   KSP2:

   DFS order, first super type found overridden symbols (with recursive super type look up) in direct super type list is returned.

   For the example, KSP will say OverrideOrder1.foo() overrides GrandBaseInterface1.foo() and OverrideOrder2.foo() overrides GrandBaseInterface2.foo()

10. Java modifier

    KSP1: Transient/volatile fields are final by default

    KSP2: Transient/volatile fields are open by default

11. Type annotations

    KSP1: Type annotations on a type argument is only reflected on the type argument symbol

    KSP2: Type annotations on a type argument now present in the resolved type as well

12. vararg parameters

    KSP1: Considered an Array type

    KSP2: Not considered an Array type

13. Synthesized members of Enums

    KSP1: values and valueOf are missing if the enum is defined in Kotlin sources

    KSP2: values and valueOf are always present

14. Synthesized members of data classes

    KSP1: componentN and copy are missing if the data class is defined in Kotlin sources

    KSP2: componentN and copy are always present

New Multiplatform Processing Scheme

When it comes to the processing scheme, i.e. what sources are processed when, the principle of KSP is to be consistent with the build’s existing compilation scheme. In other words, what the compiler sees is what processors see, plus the source code that is generated by processors.

In KSP1’s current compilation scheme, common / shared source sets are processed and compiled multiple times, with each target. For example, commonMain is processed and compiled 3 times in the following project layout. Being able to process all the sources from dependencies is convenient with one exception: Processors don’t see the sources generated from commonMain when processing jvmMain and jsMain. Everything must be re-processed and that can be inefficient.

In KSP2, we plan to add an experimental mode that tries to align to how source sets are compiled in K2 better. All sources can be processed only once with the available new processing scheme:

Please note that Kotlin 2.0 is still in beta and the compilation model is subject to change. Please let us know how this works for you and give us feedback.

KSP2 Preview Feedback

KSP2 is in preview but there is still more work to be done before a stable release. We hope these new features will ultimately help you be more productive when using KSP! Please provide us with your feedback so we can make these improvements awesome as they progress towards being stable.