Mu-RPC: securing communications

Mu-RPC: securing communications

Securing Communications is the third article in an ongoing series about the Mu open source library for building purely functional microservices. Previously, Juan Pedro Moreno provided an overview of Mu and illustrated the different types of communication between services and how they are defined with Protocol Buffers, Avro, Open API, and Mu. Then, Oli Makhasoeva covered how to define messages and services and their usage.

In this post, we are going to demonstrate how to secure your RPC communications with the protocol TLS, which stands for Transport Security Layer.

Transport Security Layer

TLS is the standard protocol for providing security to HTTP/2 for encrypting communications. Mu supports ok-http and Netty as a base transport layer, but only supports Netty for encrypting.

Since HTTP/2 over TLS mandates the use of ALPN (Application-Layer Protocol Negotiation Extension), we have a few options depending on the environment we’re using. For our purposes, we’ll explore how to mount a server in a Java 9+ environment using netty-tcnative on BoringSSL. If you’d like to review other options, take a look at the security guide in GRPC-Java.

Overview

The process is relatively straightforward; we need to start a server with a certificate, configure the server to only accept connections through TLS, and get the clients to trust the server certificate.

Mu - TLS Communication

There are two approaches for establishing a TLS communication between the client and server using Mu; both of these use Netty as the transport layer (as we mentioned previously):

  1. Use a certificate that clients trust
  2. Use an SSL context with an auto-contained Certification Authority (in practice, a sub-example of the point 1)

We’ll cover both of these approaches with a streamlined example. You’ll likely need to adapt this to your current infrastructure, which is something we won’t go over this time.

Both approaches will use the same protocol, so let’s start by defining the model and the service. In case you missed it, we saw how to define messages and services and their usage in the previous post of this series.

Service Protocol Definition

Since the scope of this post is channel communication, we’ll use a simple model and service to illustrate our example:

Model

package higherkindness.mu.protocols

final case class SampleRequest(name: String)
final case class SampleResponse(greet: String)

Service

package higherkindness.mu.protocols

import higherkindness.mu.rpc.protocol._

@service(Protobuf) trait SampleService[F[_]] {
  def getGreet(request: SampleRequest): F[SampleResponse]
}

Trusted Certificate

We’re going to start a server with a certificate that the clients will trust. Consequently, we’ll have secured client-server communication. First, we need to create a certificate and a private key:

$ openssl req -newkey rsa:2048 -nodes -keyout key.pem -x509 -days 365 -out certificate.pem

Generating a 2048 bit RSA private key
......................+++
..+++
writing new private key to 'key.pem'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) []:ES
State or Province Name (full name) []:Cádiz
Locality Name (eg, city) []:San Fernando
Organization Name (eg, company) []:47 Degrees
Organizational Unit Name (eg, section) []:
Common Name (eg, fully qualified host name) []:localhost
Email Address []:

Note that we’ve specified the Common Name as localhost because we’re going to establish the connection between the client and the server in this example locally. You need to use the current server IP (or hostname) in that field to make the client trust that certificate. As we mentioned before, this will depend on your current infrastructure.

The previous command will generate two files, the private key (key.pem) and the certificate (certificate.pem). Let’s copy these files in a subfolder of the project, for example, certs:

$ mkdir certs
$ mv key.pem certificate.pem certs

Server

As we saw in Mu-RPC: defining messages and services, we need to:

  1. Add an implementation of the service
  2. Provide a runtime
  3. Add an App with the server application

First, we need to add our dependencies, specifically, server, netty, and netty-ssl:

libraryDependencies ++= Seq(
    "io.higherkindness" %% "mu-rpc-server" % "0.17.2",
    "io.higherkindness" %% "mu-rpc-netty" % "0.17.2",
    "io.higherkindness" %% "mu-rpc-netty-ssl" % "0.17.2")

Next, we’ll create an implementation of the service. For example:

package higherkindness.mu.example

import cats.effect._
import higherkindness.mu.protocols._

class SampleServiceImpl extends SampleService[IO] {
  def getGreet(request: SampleRequest): IO[SampleResponse] =
    for {
      _ <- IO(println(s"Receiving request $request"))
      response = SampleResponse(s"Hello ${request.name}")
      _ <- IO(println(s"Generating response $response"))
    } yield response
}

And finally, our server application:

package higherkindness.mu.example

import java.io.File

import cats.effect._
import higherkindness.mu.protocols._
import higherkindness.mu.rpc.server._

object RPCServer extends App {

  implicit val EC: scala.concurrent.ExecutionContext =
    scala.concurrent.ExecutionContext.Implicits.global

  implicit val timer: Timer[cats.effect.IO]     = IO.timer(EC)
  implicit val cs: ContextShift[cats.effect.IO] = IO.contextShift(EC)

  implicit val service: SampleService[IO] = new SampleServiceImpl

  val ts = UseTransportSecurity(new File("certs/certificate.pem"), new File("certs/key.pem"))

  (for {
    grpcConfig <- SampleService.bindService[IO].map(AddService(_))
    grpcServer <- GrpcServer.netty[IO](8080, ts :: grpcConfig :: Nil)
    _          <- GrpcServer.server[IO](grpcServer)
  } yield ()).unsafeRunSync()
}

And that’s it! As you can see, we’ve created a UseTransportSecurity object by passing two files, the certificate, and the private key:

val ts = UseTransportSecurity(new File("certs/certificate.pem"), new File("certs/key.pem"))

Then, we’ve created a netty server, passing the ts argument (besides the list of services).

$ sbt ";project sample-ssl-server; run"
[info] Running higherkindness.mu.example.RPCServer

Client

We’ve said the client needs to trust the server certificate which can be achieved by several different methods including, adding the certificate in a keystore and passing the keystore in the client run.

Let’s first create the keystore and we’ll learn to add it to our runtime later:

$ cd certs
$ keytool -import -alias sample-ssl -file certificate.pem -keystore sample-ssl-keystore

This will create a keystore locally called sample-ssl-keystore with the certificate.

The client will need the following dependencies:

libraryDependencies ++= Seq(
    "io.higherkindness" %% "mu-rpc-netty" % "0.17.2",
    "io.higherkindness" %% "mu-rpc-netty-ssl" % "0.17.2")

Now, we’re going to create the client program. As you’ll see, it’s pretty simple:

package higherkindness.mu.sample

import cats.effect._
import higherkindness.mu.rpc._
import higherkindness.mu.protocols._

object RPCClient extends App {

  implicit val EC: scala.concurrent.ExecutionContext =
    scala.concurrent.ExecutionContext.Implicits.global

  implicit val timer: Timer[cats.effect.IO]     = IO.timer(EC)
  implicit val cs: ContextShift[cats.effect.IO] = IO.contextShift(EC)

  val channelFor: ChannelFor = ChannelForAddress("localhost", 8080)

  val serviceClient: Resource[IO, SampleService[IO]] =
    SampleService.client[IO](channelFor, Nil)

  val request: SampleRequest = SampleRequest("fede")

  val app: IO[Unit] = for {
    _       <- IO(println(s"Calling server with request $request"))
    result  <- serviceClient.use(_.getGreet(request))
    _       <- IO(println(s"Response $result"))
  } yield ()

  app.unsafeRunSync()

}

Remember that you need to add the keystore to the trust store in the client JVM. You can do this with the java property javax.net.ssl.trustStore. So, for example, if you’re running this example with sbt, all you need to do is start the console with the command:

sbt -Djavax.net.ssl.trustStore=certs/sample-ssl-keystore ";project sample-ssl-client; run"

[info] Running higherkindness.mu.sample.RPCClient
Calling server with request SampleRequest(fede)
Response SampleResponse(Hello fede)

There’s nothing that indicates that we’re using a TLS communication except for the second argument in SampleService.client[IO](channelFor, Nil) where we’re sending a Nil instead of PlainText() (the default argument). This is because in the protocol handshake, the server will transmit the certificate to the client, and since the client trusts that certificate, the communication will be secured through TLS.

We can be more specific by indicating in the client that we want to use TLS:

package higherkindness.mu.sample

import cats.effect._
import higherkindness.mu.rpc._
import higherkindness.mu.protocols._
import higherkindness.mu.rpc.channel._
import higherkindness.mu.rpc.channel.netty._
import io.grpc.netty.NegotiationType

object RPCClient extends App {

  implicit val EC: scala.concurrent.ExecutionContext =
    scala.concurrent.ExecutionContext.Implicits.global

  implicit val timer: Timer[cats.effect.IO]     = IO.timer(EC)
  implicit val cs: ContextShift[cats.effect.IO] = IO.contextShift(EC)

  val channelInterpreter: NettyChannelInterpreter = new NettyChannelInterpreter(
    ChannelForAddress("localhost", 8080),
    Nil,
    List(NettyNegotiationType(NegotiationType.TLS))
  )

  val serviceClient: Resource[IO, SampleService[IO]] =
    SampleService.clientFromChannel[IO](IO(channelInterpreter.build))

  val request: SampleRequest = SampleRequest("fede")

  val app: IO[Unit] = for {
    _       <- IO(println(s"Calling server with request $request"))
    result  <- serviceClient.use(_.getGreet(request))
    _       <- IO(println(s"Response $result"))
  } yield ()

  app.unsafeRunSync()

}

This looks very similar to the previous code, except here we’re defining a NettyChannelInterpreter and using the client builder clientFromChannel that receives a ManagedChannel (the result of calling NettyChannelInterpreter.build).

As you probably expect, there is no difference in the output:

sbt -Djavax.net.ssl.trustStore=certs/sample-ssl-keystore ";project sample-ssl-client; run"

[info] Running higherkindness.mu.sample.RPCClient
Calling server with request SampleRequest(fede)
Response SampleResponse(Hello fede)

SSL Context

The other approach we mentioned is to create an SSL Context with an auto-contained Certification Authority. Basically, we’re providing the client and the server both with a certificate and an authority that can verify that certificate.

We’re not going to cover the creation of the certificate, the private key, and the certification authority. Instead, we’re going to use the test certificates from the grpc-java project, more concretely, the server1.pem, server1.key, and ca.pem files.

The first thing we need to do is to copy those files to the certs folder. Now, let’s see how to implement the server and the client.

Server

Our first task is to generate a SslContext for the server; we can split this process into three steps:

Step 1: Initialize the certificate files

val serverCertFile: File       = new File("certs/server1.pem")
val serverPrivateKeyFile: File = new File("certs/server1.key")

Step 2: Generate the X509 Certificate

Because we’re working with a Java library, we’ve wrapped the GRPC utility methods with some cats effect builders.

def generateTrustedCA: IO[X509Certificate] = for {
  caFactory   <- IO(CertificateFactory.getInstance("X.509"))
  certificate <- IO(caFactory.generateCertificate(new FileInputStream("certs/ca.pem")))
  x509Cert    <- IO.fromEither {
    certificate match {
      case c: X509Certificate => Right(c)
      case _                  => Left(new IllegalStateException("Invalid certificate type: " + certificate.getType))
    }
  }
} yield x509Cert

Step 3: Generate the SSL Context for the server

def generateSSLContext(cert: X509Certificate): SslContext =
  GrpcSslContexts
    .configure(
      GrpcSslContexts.forServer(serverCertFile, serverPrivateKeyFile),
      SslProvider.OPENSSL)
    .trustManager(cert)
    .clientAuth(ClientAuth.REQUIRE)
    .build()

Now that we know how to generate the SSL Context, it’s just a matter of using it in the server initialization:

(for {
  sslContext <- generateTrustedCA.map(generateSSLContext)
  grpcConfig <- SampleService.bindService[IO].map(AddService(_))
  grpcServer <- GrpcServer.netty[IO](8080, SetSslContext(sslContext) :: grpcConfig :: Nil)
  _          <- GrpcServer.server[IO](grpcServer)
} yield ()).unsafeRunSync()

As you can see, it’s very similar to our first approach only here we’re passing SetSslContext instead of UseTransportSecurity.

Now we can start the server:

$ sbt ";project sample-ssl-server; run"
[info] Running higherkindness.mu.example.RPCServer

Client

The SSL Context generation for the client is quite similar, with a few minor changes. The primary difference is that we’re using the GrpcSslContexts.forClient builder instead of GrpcSslContexts.forServer, but let’s look at the entire code:

val serverCertFile: File       = new File("certs/server1.pem")
val serverPrivateKeyFile: File = new File("certs/server1.key")

def generateTrustedCA: IO[X509Certificate] = for {
  caFactory   <- IO(CertificateFactory.getInstance("X.509"))
  certificate <- IO(caFactory.generateCertificate(new FileInputStream("certs/ca.pem")))
  x509Cert    <- IO.fromEither {
    certificate match {
      case c: X509Certificate => Right(c)
      case _                  => Left(new IllegalStateException("Invalid certificate type: " + certificate.getType))
    }
  }
} yield x509Cert

def generateSSLContext(cert: X509Certificate): SslContext =
  GrpcSslContexts.forClient
    .keyManager(serverCertFile, serverPrivateKeyFile)
    .trustManager(cert)
    .build()

The first two steps (initialize the certificate files and generate the X509 Certificate) are precisely the same as with the server, and the third step only varies slightly.

Now let’s see how to use that SSL Context for generating the client:

def client(context: SslContext): Resource[IO, SampleService[IO]] = {
  val channelInterpreter: NettyChannelInterpreter = new NettyChannelInterpreter(
    initConfig = ChannelForAddress("localhost", 8080),
    configList = List(OverrideAuthority("foo.test.google.fr")),
    nettyConfigList = List(
      NettyNegotiationType(NegotiationType.TLS),
      NettySslContext(context))
  )
  SampleService.clientFromChannel[IO](IO(channelInterpreter.build))
}

The third parameter in the NettyChannelInterpreter (nettyConfigList) is where we’re passing the SSL Context and the negotiation type. In the second parameter (configList), we need to pass the OverrideAuthority key since we’re using the sample certificate and that is the hostname that was used for generating it.

Let’s write the final client code:

val request: SampleRequest = SampleRequest("fede")

(for {
  _           <- IO(println(s"Calling server with request $request"))
  sslContext  <- generateTrustedCA.map(generateSSLContext)
  result      <- client(sslContext).use(_.getGreet(request))
  _           <- IO(println(s"Response $result"))
} yield ()).unsafeRunSync()

And execute it:

sbt ";project sample-ssl-client; run"

[info] Running higherkindness.mu.sample.RPCClient
Calling server with request SampleRequest(fede)
Response SampleResponse(Hello fede)

As you can see, we don’t need to pass the keystore when using the SSL Context. We’re providing the client with all the information it needs to trust the server, and consequently, establishing a secure connection.

Summary

We’ve covered the basic concepts for establishing a secure client-server RPC connection with Mu, and we’ve seen two approaches, one using a certificate that the clients trust (through a keystore) and another using an SSL Context (with a certificate authority).

We’re using grpc-java behind the scenes, so the security docs are a good place for getting more information and a better understanding of how it all works. You can also check the official grpc guide for Authentication for an overview of gRPC authentication.

Have questions or comments? Join us in the Mu Gitter channel. Mu is made possible by an awesome group of contributors . As with all of the open source projects under the 47 Degrees umbrella, we’re always looking for new contributors and we’re happy to help guide you towards your first contribution.

Ensure the success of your project

47 Degrees can work with you to help manage the risks of technology evolution, develop a team of top-tier engaged developers, improve productivity, lower maintenance cost, increase hardware utilization, and improve product quality; all while using the best technologies.