New Arrow Optics copy makes data transformation easier

• by Alejandro Serrano
• •
• September 14, 2022
• •
• arrow• kotlin• functional programming
• |
• 8 minutes to read.

Arrow Optics is a combination of a library and a compiler plug-in that makes manipulation of immutable values in Kotlin much easier. They are often used in combination with data classes, Kotlin’s approach to lightweight data modeling. In this post, we focus on one of its virtues–the manipulation of (deeply) nested fields–and discuss how we’ve improved the API in that regard with the help of a type-safe builder.

The feature described in this post is part of the upcoming 1.1.3 release of Arrow Optics.

To give a very quick recap of what optics aim to improve, let’s focus on a small set of classes. The key consideration here is that some values, like the name of a Street, require several hops if we start from a Company value.

data class Street(val number: Int, val name: String) {
  companion object
}
data class Address(val city: String, val street: Street) {
  companion object
}
data class Company(val name: String, val address: Address) {
  companion object
}

Imagine that we want to normalize the name of the street by making it lowercase. Since we’re taking an immutable approach to our data here – notice that we’re using val on every field – we have to generate a copy. But that gets messy!

fun Company.normalizeStreeName(): Company =
  this.copy(address =
    address.copy(street =
      address.street.copy(name =
        address.street.name.lowercase())))

Optics solve this problem by separating the definition of how we access and modify data from the query or modification itself. This means that you first build a description of where you want to focus, and only then indicate the transformation to perform. In our case, the example turns into:

fun Company.normalizeStreeName(): Company =
  Company.address.street.name.modify(this) {
    it.lowercase()
  }

The Company.address.street.name part comes from a set of definitions generated by the Arrow Optics compiler plug-in; and by itself refers to no particular object. We then ask for a modification of a particular value in a particular way.

copy on steroids

Even though modify is awesome when dealing with nested fields, it operates on a single value at once. If you need to update several fields at once, copy provides a much nicer syntax.

fun Street.nonsenseOperation(): Street =
  this.copy(
    number = number + 1,
    name = name.lowercase()
  )

A recent PR to the Arrow repository brings together both worlds: the ability of copy to modify several fields, with the ability of optics to modify nested fields. Here’s an example of its usage:

fun Company.normalizeStreeName(): Company =
  this.copy {
    Company.name transform { it.capitalize() }
    Company.address.street.name transform { it.lowercase() }
  }

The new copy method takes a block as parameter, which specifies a sequence of transformations to be applied to the data. But instead of a field name, as with the original copy, each of those transformations is specified by an optic. That way we are no longer restricted to looking only one level deep; we can access nested fields, and even collections of elements when using traversals!

A look under the hood

This new syntax is also interesting from the perspective of its implementation; it requires a really short amount of code to work. We use the type-safe builders pattern, which is often used to define domain-specific languages in Kotlin.

The crux of the idea is the following: we want the data transformation to be defined in a block because we can benefit from Kotlin not requiring parentheses on a trailing lambda. Inside this block, we want the transform keyword to be available; this goes into an interface defining such operation.

public interface Copy<A> {
  public infix fun <B> Traversal<A, B>.transform(f: (B) -> B)
}

How do we ensure that transform is available, but without the need of an additional argument? We use a function with receiver. The final signature for the copy function is thus:

public fun <A> A.copy(f: Copy<A>.() -> Unit): A

The final component is a particular implementation of Copy that we can use in the body of the new copy function. We would use calls to transform to “stack” on each other so that, at the end of the block, all the transformations are applied to the original value. This requires mutability to keep track of the current status of the value.

private class CopyImpl<A>(var current: A): Copy<A> {
  override fun <B> Traversal<A, B>.transform(f: (B) -> B) {
    current = this.modify(current, f)
  }
}

Although we often run away from mutability, this is an example of a good use case for it. The key reason for accepting that var is that mutability in this case is local: no side effect is visible outside the copy operation. Mutability is really an implementation detail.

Better with receivers

There’s one final trick we can play to remove that ugly initial Company. required at the beginning of every field. The end result is a DSL that really mimics the original copy,

fun Company.normalizeStreeName(): Company =
  this.copy {
    name transform { it.capitalize() }
    address.street.name transform { it.lowercase() }
  }

To perform this trick, we have to remember that name, address, and the rest of optics live in Company’s companion object. If we bring that object into scope, then we don’t need the Companion qualifier anymore. We have a limitation, though: the receiver function syntax only allows us to declare one receiver, and we’ve already used that slot for Copy<A>.

Fortunately, the Kotlin compiler includes an experimental feature since version 1.6.20 that drops this limitation: context receivers. We can add as many additional types as we want within context before the function type, and those will be available as part of the implicit this in the block.

data class Company(val name: String, val address: Address) {
  public fun <A> A.copy(
    f: context(Company.Companion) Copy<A>.() -> Unit
  ): A = /* autogenerated */

  companion object
}

The limitation here is that a different copy method has to be generated for each type. The reason is that “being a companion object” is not really reflected in the type system, so we cannot create a fully generic copy extension function. This is not so terrible, as the function can be automatically generated by the Arrow Optics plug-in, in the same way that we now generate the optics.

Join us on this journey

If you are interested in contributing, learning, or have any questions about Arrow Optics, or the Arrow project as a whole, don’t be shy and join us on the Kotlin Slack in the #arrow channel. We are an inclusive group of people committed to providing the best experience to our users and community, and we look forward to seeing you there!