Historically, we haven’t shared much logic across platforms at Doist. Features have specs, but each team often implements them separately. Since we build two products, each released in multiple native platforms, we really benefit from sharing code.
While exploring options for multiplatform implementations, Kotlin stood out. It’s a language we already use to build our Android apps, and Kotlin Multiplatform Mobile did well in a recent experiment we conducted for iOS. The next logical step was to try it out on the Web, but it was less flawless. This post covers our experience and takeaways integrating a Kotlin/JS library into an existing Web application.
The library: Filterist
Filterist powers the natural language recognition behind Todoist’s filters. It receives a query string and returns the tasks that match the query. For instance, for a query like this:
(today | tomorrow) & @work
It returns all the tasks scheduled for today or tomorrow tagged with the work label. It’s implemented in Kotlin on Android, Swift on iOS, and JavaScript on the Web.
We decided this was a good stress test for Kotlin Multiplatform, since the filtering logic is rather complex, with a high amount of supported keywords and operators. Additionally, its domain is mostly business logic, and the output is the same independently of the platform where it runs.
Kick-off & goals
The Kotlin Multiplatform version of Filterist had been architected with the mobile platforms in mind until this point. We jumped into this initiative without knowledge of the programming language and ecosystem. It was time to roll up the sleeves and get acquainted with it. Our goals were:
- Minimal changes in native platforms
- Effortless to maintain and scale
- Small library footprint
Knowing that the Kotlin library was generating versions for Android and iOS that were (almost) fully integrated with the product’s codebases, we presumed that building the JavaScript bundle and integrating it was good enough.
We wrapped the bundle in an npm package, set it as a dependency, and created a test case to assert some natural language use cases. Unsurprisingly, the bundle wasn’t usable right out of the box.
No public API and types
We installed the npm package, but what was the API? What should the JavaScript code call?
Our Web products use TypeScript to enforce typing and establish an API for the codebase, and Kotlin generates .d.ts files containing type definitions along with the JavaScript bundle.
And yet, the file was empty.
We missed that we had to annotate Filterist’s public API functions with @JsExport. Bingo! After adding those annotations, the main library’s entry point function was listed in the .d.ts file and exported by the bundle. The test suite could finally import the appropriate library parts.
Conflicting platform architectures
We defined a test case that used Filterists entry point function, filter(). It had the following type definition:
export function filter<T>(
query: string,
projects: FilterableProjects<FilterableProject>,
sections: FilterableSections<FilterableSection>,
items: FilterableItems<T>,
user: FilterableUser,
): kotlin.collections.List<Filterist.Result<T>>
While some of the arguments had types as FilterableProject or FilterableItems, none of these were exported or listed in the type definitions file.
The Kotlin function had this structure:
fun <T : FilterableItem> filter(
query: String,
projects: FilterableProjects<FilterableProject>,
sections: FilterableSections<FilterableSection>,
items: FilterableItems<T>,
user: FilterableUser,
): List<Result<T>>
Those custom types were interfaces. Filterist expected arguments like projects, sections, or items implementing specific interfaces. For example, FilterableProject defined a few properties:
interface FilterableProject : FilterableModel {
val filterableName: String
val filterableParentId: FilterableId?
val filterableChildOrder: Int
val filterableIsShared: Boolean
}
On iOS, the Project class, which models the state of a project, then extended from FilterableProject, deriving the added properties values from properties of the base class.
extension Project: FilterableProject {
public var filterableId: FilterableId {
return globalID
}
public var filterableParentId: FilterableId? {
return parent?.globalID
}
...
}
It’s almost plug-n-play. Classes extend these interfaces and provide the necessary properties within state class objects. Since both platforms share a similar state architecture principle, where classes model main business logic entities, it’s a sensible choice.
But on the JavaScript side, we use redux. Classes do not model the state. Immutable plain objects do. In the end, there was a clash between object-oriented and functional architectures.
Adapting on the JavaScript side would require creating a class layer that extends Kotlin’s interfaces and is passed to the library, maintaining an OOP state layer parallel to the functional redux paradigm. We ruled out this option quickly as we moved away from OOP in our JavaScript codebases years ago. Also, converting each redux state property to plain objects that implement Kotlin’s interfaces would be a memory and performance nightmare since there might be hundreds, perhaps thousands, of properties to convert. Lastly, it failed our first goal.
Instead, we made Filterist more flexible and agnostic.
Some drafts later, and we had the concept of data providers. Rather than passing class modeled state to the library, its functions receive data providers containing getter functions. We defined data providers through interfaces:
interface ProjectAccessorProvider {
fun getName(projectId: String): String
fun getParentId(projectId: String): FilterableId?
fun getIsShared(projectId: String): Boolean
}
While each platform imports these interface definitions and creates class or plain object instances that implement those interfaces.
const state = getReduxState()
const projectAccessorProvider: ProjectAccessorProvider = {
getName(id: string) {
return state.projects[id].name
},
getParentId(id: string) {
return state.projects[id].parent_id
},
getIsShared(id: string) {
return state.projects[id].is_shared
},
}
Filterist no longer cares about state modeling and instead delegates that work to data provider functions. Platforms no longer have to ensure that business logic entities map to Filterist’s interfaces. They only provide getters, which is easy to do as each platform knows how its state is modeled and should be accessed.
Non-mapped Kotlin types
While the architecture puzzle got solved, it came to light that there was another bump in the road, this time with something that we had less control over.
Kotlin maps only some of its types to their JavaScript counterparts. One can call Kotlin’s JavaScript exported library functions and pass them JavaScript native types, as Number, String, or Array. Unfortunately, we implemented Filterist using a few non-mapped Kotlin types, such as List or, more importantly, interface.
Because interface doesn’t map to any JavaScript/TypeScript type, the compiler throws an error when an interface is annotated with @JsExport. Since Kotlin doesn’t export them, they don’t have TypeScript mappings. Manually adding type information is possible, but it would easily break with new changes to Filterist. Without type information altogether, the only way to venture through the library API would be to read the docs, inspect or debug the generated JavaScript bundle, or dive into the library’s Kotlin implementation.
All would be fine, but none scales well.
Meanwhile, elsewhere at Doist
In parallel, another Doist team started experimenting with a different library to manage user permissions. They found that it’s possible to declare external interfaces at the package level in the JS-specific source set, making them exportable when annotated with @JsExport. There was a catch: external interfaces cannot extend common package interfaces, and other source sets cannot access them.
Let’s see them in action.
In the project’s JavaScript source set, there’s an external interface declared for the relevant state and a function. The latter has the sole purpose of receiving the state (compliant with the external interface) from JavaScript and calling the common package relevant function, passing an object that implements the external’s interface common package counterpart:
external interface ChannelData {
val userCount: Int
val isArchived: Boolean
}
fun channelPermissionOf(data: ChannelData): (Int) -> Boolean {
val permissions = permissionsOf(
object : CommonChannelData {
override val userCount: Int get() = data.userCount
override val isArchived: Boolean get() = data.isArchived
}
)
}
That logic generates the expected TypeScript definitions for an interface and a function:
export interface ChannelData {
readonly userCount: number
readonly isArchived: boolean
}
export function channelPermissionOf(data: ChannelData): (p: number) => boolean
We learned that it isn’t impossible to overcome the non-exportable interfaces problem. After all, the permissions manager library makes it work without them just fine. However, we decided that this solution doesn’t suit Filterist.
Adopting external interfaces requires creating a one-to-one mapping between external and common package interfaces, and there are lots of them! But doing so means keeping a carbon copy of common interfaces for JavaScript specifically, which fails to meet our second goal.
List, interface… and Long
Kotlin’s number mapping is another troublesome area for which there might not be JavaScript equivalents. JavaScript’s Number is a 64-bit double-precision floating-point number, just like Kotlin’s Double. Its precision for whole numbers ranges from -253+1 to 253-1. However, it maps to Kotlin’s Int, a 32-bit integer.
In addition, there is no mapping for Long, Kotlin’s 64-bit integer, so the JavaScript bundle doesn’t make it publicly available. However, the bundle functions expect Long arguments because Filterist’s Kotlin code uses it. We experimented with three ideas to overcome this:
Use Kotlin.js standard library
The Kotlin.js standard library is publicly available as an npm package. We installed it, instantiated its Long object, and passed it to Filterist’s filter function in our JavaScript test case. It didn’t work.
In the generated bundle, Filterist called Long’s plus_wiekkq_k$() method to do an arithmetic operation. Sketchy naming, uh?
Long.prototype.plus_wiekkq_k$ = function (other) {
return add(this, other)
}
That’s because Kotlin mangles names to support function overloading, a concept that doesn’t exist in JavaScript. We had to annotate the Filterist functions exported to JavaScript with @JsName to prevent this from happening. However, we don’t control the internal Long class. Its methods’ names are mangled, and we can’t annotate it with @JsName. Alternatively, instantiating Long objects through the standard library npm package kept the object properties and methods names intact. We couldn’t use the standard library to create unmapped object types.
Change the argument type to Int
In an attempt to build a proof of concept, we converted all exported functions Long arguments to Int and added .toLong() calls where needed in Filterist’s Kotlin project.
Filterist’s logic was now dependent on JavaScript limitations. Besides, other platforms’ integers are 32-bit and not 64-bit values like JavaScript’s Number, which would lead to bugs over time. It was acceptable for exploration purposes but failed our second goal.
Create a custom type
We came up with a common custom type, InteroperableLong, allowing for type mapping and keeping the Long ordeal away from the platform codebases. InteroperableLong is defined by an expected declaration in Filterist:
expect class InteroperableLong {
fun toLong(): Long
}
With the actual implementation in each platform. For instance, in JavaScript:
actual class InteroperableLong(private val number: Number) {
actual fun toLong() = number.toLong()
}
Or iOS:
import platform.darwin.SInt64
actual class InteroperableLong(private val int64: SInt64) {
actual fun toLong() = int64
}
With all Long occurrences swapped to InteroperableLong, the solution was ready for battle testing. Using toLong() wherever InteroperableLong is used is an extra step compared to having Long natively mapped, but without a common type between all platforms, it scales well and is at least a long-term solution.
Kotlin’s date/time library
kotlinx-datetime is Kotlin’s official library to manage dates and times. It encapsulates platform-specific date/time logic in a common API, relying on native libraries under the hood. In JavaScript’s case, on js-joda.
Our JavaScript codebases depend on moment for date/time handling, so we explored if js-joda could be an alternative within our requirements. Timezone handling is where moment shines for us, through moment-timezone. Otherwise, we resort to native date helpers more often than not. js-joda includes moment-timezone’s dataset, providing the same features that we need from moment. Swapping libraries wouldn’t be a daunting task.
moment and moment-timezone out, js-joda in, problem solved. Almost.
There was one last thing. The library wraps js-joda in custom classes. This means that JavaScript date/time objects created through js-joda npm package instances have a different type than the date/time objects that the generated JavaScript bundle expects. To work around this, we added adapter functions that create instances of the custom classes. Here’s an example that returns js-joda’s Instant:
import kotlinx.datetime.Instant
import kotlin.js.JsExport
@JsExport
fun createInstant(date: String): Instant {
return Instant.parse(date)
}
Admittedly, it isn’t very refined, seems redundant, and we have to call it for every single js-joda object that’s passed to the bundle. We would also prefer to control which date/time dependency Kotlin uses, to improve the odds it fits other projects uneventfully. But for validation purposes, it works.
Bundle size
On the Web, the size of application assets matters. Fast page loads and efficient resource usage depend on it.
The generated JavaScript bundle includes the Kotlin standard library. We can rely on Kotlin’s dead code elimination feature (DCE), which performs tree shaking to build much slimmer production bundles. However, we still noticed a several-fold increase in size when comparing both implementations. The native implementation weights at around 10kb, while the bundle generated by Kotlin, with DCE enabled, measured 41.05kb (both gzipped).
DCE mitigates the bundle size problem, but it isn’t good enough to be negligible. No benefits outweigh an asset bundle that becomes too large. We expect tradeoffs, but not any tradeoffs.
Conclusions
Having one implementation shared across platforms is very tempting. It ensures that all platforms have the same requirements implemented and tested, and optimizes human effort to build and maintain. For core business logic like Filterist, Kotlin Multiplatform is very promising.
Our attempt at integrating with JavaScript led to various challenges, the most critical being the lack of support for some core Kotlin types in JavaScript, namely interface, List, and Long. Sadly, we feel that Kotlin/JS is not quite production-ready for Filterist. The workarounds required to make basic interoperability work are at odds with the goals we set at the start of the exploration. However, the permission management library, a smaller greenfield project, tailored itself to Kotlin’s JavaScript limitations and was more successful, reaffirming our conviction on multiplatform libraries.
The rest of the topics were either solved or had manageable downsides.
A failed integration doesn’t mean that all this work was in vain for us, though. At first, we built Filterist’s Kotlin library against our Android and iOS architectures and tailored its design to them. This exercise helped refactor Filterist to be more modular and state agnostic. It’s an important lesson to remember when building shared libraries. More so, this exploration allowed us to have a clear perspective of the state of Kotlin/JS. It doesn’t suit our needs at this point, but it shows promising indicators that it can become a relevant actor in our codebases.
What’s next
Kotlin is a pillar of our Android codebases and integrates well with iOS. Our Kotlin Multiplatform projects will keep evolving, even if the Web continues using native JavaScript implementations.
We weren’t the only ones that had difficulties with the JavaScript integration. Kotlin’s ecosystem has a very active community, and other developers have reported JavaScript’s mapping shortcomings on multiple occasions (1, 2, 3, 4).
The good news is that the Kotlin team marked the interface issue as fixed in the upcoming 1.6.20 release (1), which we considered to be the most pressing blocker. It’s still unknown if the remaining non-mapped types will ever be officially available on JavaScript, but the folks over Deezer have taken matters into their hands with KustomExport. We looked into it, but it was still in active development, and we were unable to use it. We’ll keep it on our radar.
As new Kotlin releases come out, we’ll re-evaluate. Other in-house libraries may start to use Kotlin Multiplatform on JavaScript, as the permission management library did. For now, we expect its usage to be sparse until it checks all the boxes for larger projects.