47 Degrees joins forces with Xebia read more

Environment Configuration in Kotlin

Environment Configuration in Kotlin

When setting up a server, we often need to provide specific configurations depending on the environment we’re running on. It’s quite common to have three different environments; one for local development, another one for acceptance testing and a production environment. For these different environments, the server needs to be set up at different IP addresses and ports. For example:

DEV: 127.0.0.1:8080
ACC: 192.168.0.105:8085
PROD: 12.14.16.18:89127

Severs typically also need to integrate with other services such as a databases, distributed message systems like Kafka, caches like Redis, or other microservices. These external services also run at different network coordinates and credentials based on the environment.

For example, TestContainers, docker-compose, or a cloud provider. These could be called TEST, DEV and ACC & PROD environments respectively.

Commonly used techniques for configuring applications is to either use data class to model the environment configuration in a typed way and manually configuring frameworks, or letting frameworks automatically read configuration.

When writing your own typed environment configuration model, you can either load the config programmatically, or use formatted files with a library to convert the files into a typed model.

Let’s explore both options.

Manual configuration

First, let’s look at how we can manually configure our environment in Kotlin with a typed domain model, also used inside 47 Degrees Github Alert Project. At 47 Degrees, soon to be Xebia Functional, we like using type-safe pragmatic solutions, and with plain Kotlin, we can get pretty far.

The Github Alert Project relies on Postgres, so let’s see how we can model both our http configuration, and database configuration.

import java.lang.System.getenv

data class Env(val http: Http = Http(), val postgres: Postgres = Postgres())

data class Http(
  val host: String = getenv("HOST") ?: "0.0.0.0",
  val port: Int = getenv("PORT")?.toIntOrNull() ?: 8080,
)

data class Postgres(
  val url: String = getenv("POSTGRES_URL") ?: "jdbc:postgresql://localhost:5432/databasename",
  val username: String = getenv("POSTGRES_USER") ?: "test",
  val password: String = getenv("POSTGRES_PASSWORD") ?: "test",
)

In the example above, our typed Env type has a couple of trade-offs:

  1. It directly initializes the values in the constructor, and ignores the fact that System.getenv is a side-effect. We consider this non-problematic since we want to fail-fast when the configuration is not available but it could easily be extracted from the constructor, and we’ll see below how to do so, and how to evolve this pattern further.

  2. This technique relies on System.getenv, or defaults to a single value. So we only encode two different flavors; in testing we rely on (TestContainer) -> Postgres to build our Env.Postgres value. This is all possible since we’re working with simple data classes.

Evolve plain Kotlin

There are two things we’ve ignored in the previous section which are side effects and error tracking. System.getenv is already a side effect, but wrapping it inside suspend doesn’t offer us a lot of benefit.

If a project requires accessing a remote config, feature flags, or reading configuration from disk, then suspend might offer more benefits. For example, it could read remote configs in parallel.

suspend fun remoteEnv(): Env =
  parZip({ remoteHttp() }, { remotePostgres() }) { http, postgres -> Env(http, postgres) }

When loading configurations, you might want to know which properties were missing before crashing the application. This can be useful for debugging; a logger could then list all the missing properties with a clear message.

import java.lang.System.getenv

fun env(name: String) : ValidatedNel<String, String> =
   getenv(name)?.valid() ?: "\"$name\" configuration missing".invalidNel()

fun <A : Any> env(name: String, transform: (String) -> A?) : ValidatedNel<String, A> =
   env(name).andThen { transform(it)?.valid() ?: "\"$name\" configuration found with $it".invalidNel() }

fun http(): ValidatedNel<String, Http> =
  env("HOST").zip(env("PORT", String::toIntOrNull), ::Http)

fun postgres(): ValidatedNel<String, Postgres> =
  env("POSTGRES_URL").zip(env("POSTGRES_USER"), env("POSTGRES_PASSWORD"), ::Postgres)

fun env(): ValidatedNel<String, Env> =
  http().zip(postgres(), ::Env)

fun ValidatedNel<String, Env>.getOrThrow(): Env =
   fold({ errors ->
      val message = errors.joinToString(
         prefix = "Environment failed to load:\n",
         separator = "\n"
      )
      throw RuntimeException(message)
   }) { it }

fun main(): Unit {
   env().getOrThrow()
}

When we run the above example without any environment variables available, we will see the following output in the console:

Exception in thread "main" java.lang.RuntimeException: Environment failed to load:
"HOST" configuration missing
"PORT" configuration missing
"POSTGRES_URL" configuration missing
"POSTGRES_USER" configuration missing
"POSTGRES_PASSWORD" configuration missing
	at MainKt.getOrThrow(main.kt:..)

This clearly shows us what is going wrong with the environment, and which configurations we are missing. Arrow naturally composes with suspend, and thus this gives us all powers we typically look for when building typed configurations.

File based configuration (hocon, yml, …)

Perhaps the most common approach on JVM is to use configuration files inside the resource folder, and letting the framework read it, or use a library specifically for decoding configuration files.

In the example below, we’re going to use the Hocon format by Lightbend, a popular format for configuring servers, in combination with Hoplite to automatically read & decode the hocon file into our own data class domain.

Let’s take the same example from before, but adjust it to use Hocon.

data class Env(val http: Http, val postgres: Postgres)
data class Http(val host: String, val port: Int)
data class Postgres(val url: String, val username: String, val password: String)

fun main() {
   val env = ConfigLoader().loadConfigOrThrow<Env>("/application.conf")
}

With configuration files, the project needs to split the configuration between at least two files: The first one defining our data classes in our main code (seen in the snippet above); the second one defining our actual configuration in our main/resources/application.conf directory, like the snippet below. As you can see below, it requires learning about a new format like HOCON or any other formats that might be used.

http {
  host = "127.0.0.1"
  host = ${?HOST}
  port = 8080
  port = ${?PORT}
}

postgres {
  url = "jdbc:postgresql://localhost:5432/databasename"
  url = ${?POSTGRES_URL}
  username = "test"
  username = ${?POSTGRES_USER}
  password = "test"
  password = ${?POSTGRES_PASSWORD}
}

Since the project still needs to define multiple environments, such as TEST, DEV, and ACC & PROD, we still need a way to define default values and point to environment variables. HOCON uses a similar approach as was used above, where the values are defined through optional environment variables while providing default values.

With HOCON, it reads the other way around. The first example above used the elvis operator ?: to configure getenv("XXX") ?: "default_value", whereas HOCON defines a value url = default-value and it then attempts to override with an optional ${?XXX} environment variable.

url = "jdbc:postgresql://localhost:5432/databasename"
url = ${?POSTGRES_URL}

A library, or the framework, can read these specific configuration by providing a path to them /application-prod.conf, /application-dev.conf, or /application-acc.conf, etc.

val env = ConfigLoader().loadConfigOrThrow<Env>("/application.conf")

In the case below, none of the optional environment variables were present.

Env(http=Http(host=127.0.0.1, port=8080), postgres=Postgres(url=jdbc:postgresql://localhost:5432/alerts, username=test, password=test))

In this example, we used "com.sksamuel.hoplite:hoplite-hocon:2.5.2", but Hoplite also has support for yml, json, toml, and Java Properties.

Alternatively, you can have Ktor or Spring read in the application configuration, and they offer some utilities to access non-framework configuration values. See Ktor documentation for examples. This however takes away the ability to have typed domain models as used above, and takes away quite a bit of flexibility to setup services that are not related to the framework.

Conclusion

When using plain Kotlin, we get the most flexibility, and this solution will work on any Kotlin platform. Since Native, JVM, and NodeJS give you easy access to environment variables, it can be combined with any other technique such as Structured Concurrency, and validation as you see fit. This approach requires writing more Kotlin code, and couples the configurations inside the Kotlin codebase.

File based configuration, with excellent libraries such as Hoplite, offer a great solution. But, in comparing the two, we found that it typically requires a similar amount of code whilst also introducing more formats. This solutions is most common on the JVM, and often doesn’t work for Kotlin MPP. This approach requires writing less Kotlin code, and allows you to swap configuration simply by swapping a file in the resources before building the JAR.

Both are great solutions with their own pros and cons. Happy coding!

47 Degrees, soon to be Xebia Functional, 💙 Kotlin

We’re great fans of Kotlin at 47 Degrees (soon to be Xebia Functional), exploring the many possibilities it brings to the back-end scene. We’re proud maintainers of Arrow, a set of companion libraries to Kotlin’s standard library, coroutines, and compiler; and provide Kotlin training to become an expert Kotliner. If you’re interested in talking to us, you can use our contact form, or join us on the Kotlin Slack.

Ensure the success of your project

47 Degrees can work with you to help manage the risks of technology evolution, develop a team of top-tier engaged developers, improve productivity, lower maintenance cost, increase hardware utilization, and improve product quality; all while using the best technologies.