Mu-Scala 0.22: Distributed tracing

by Chris Birchall
•
May 05, 2020
•
scala• functional programming• functional• mu• microservices• rpc• grpc• tracing• distributed tracing• natchez
|
7 minutes to read.

We are pleased to announce the release of Mu-Scala v0.22.0.

The headline feature is support for distributed tracing, thanks to an integration with Rob Norris’s Natchez library.

This release also introduces a few breaking changes, detailed below.

Distributed tracing with Natchez

This release of Mu-Scala introduces an integration with Natchez to enable distributed tracing of gRPC calls.

Example of a distributed trace

Mu-Scala makes it easy to add tracing to RPC servers and clients.

On the server side, we provide a new MyService.bindTracingService method, corresponding to the existing MyService.bindService. This takes care of extracting the necessary trace information from request headers and setting up the span for you.

If you want to, you can make use of Natchez’s Trace[F] typeclass inside your service implementation to create child spans, attach key-value metadata to the current span and so on. For example, you might want to create child spans when your service queries the database or makes an HTTP call to a 3rd-party API.

On the client side, we provide a few new factory methods such as MyService.tracingClient, corresponding to the existing MyService.client method. This returns a MyService[Kleisli[F, Span[F], *]], i.e. a client which takes the current span as input. The client wil take care of creating a child span for the RPC call and adding the necessary trace information to the request headers.

For more details on how to use this new feature, take a look at the how-to guide on the microsite.

If you want to play with a full working example, try the tracing example in the mu-scala-examples repo.

Mu-Haskell integration tests

Since the last release, we have written a suite of tests to verify RPC communication between Mu-Scala clients and Mu-Haskell servers, and vice versa.

Testing two independent implementations against each other like this helped us uncover and fix a number of issues on both sides.

Breaking changes

This release includes a number of potentially breaking changes. Let’s look at each one in detail.

Streaming responses now wrapped in F

The gRPC spec includes support for the server sending a stream of messages in its response, rather than just a single message. For example, here is a definition for a service with a unary method and a server-streaming method:

service MyService {
  rpc SingleResponse(MyRequest) returns (MyResponse) {}
  rpc StreamOfResponses(MyRequest) returns (stream MyResponse) {}
}

Mu-Scala supports gRPC streaming, allowing users to choose between FS2 and Monix as their streaming implementation.

In previous versions of Mu-Scala the corresponding service algebra, using FS2, would look like:

@service(Protobuf)
trait MyService[F[_]] {
  def SingleResponse(req: MyRequest): F[MyResponse]
  def StreamOfResponses(req: MyRequest): Stream[F, MyResponse]
}

In version 0.22.0 and later, however, the algebra looks like this:

@service(Protobuf)
trait MyService[F[_]] {
  def SingleResponse(req: MyRequest): F[MyResponse]
  def StreamOfResponses(req: MyRequest): F[Stream[F, MyResponse]]
}

Notice how the return type of the second method is now F[Stream[F, MyResponse]].

This change was made for a couple of reasons:

Consistency. All return types now have the shape F[...], no matter whether streaming is used or not.
It was necessitated by the implementation of the distributed tracing feature.

If you are using the sbt-mu-srcgen plugin to generate service definitions in Scala from .proto files, make sure to upgrade the plugin to 0.22.0 at the same time you upgrade Mu. The next time you run the code generation task, it will generate method signatures with the correct return type.

Updating server-side code

Let’s say you have implemented an interpreter of the service algebra as follows:

class TheService[F[_]: Applicative] extends MyService[F] {
  def SingleResponse(req: MyRequest): F[MyResponse] = ???
  def StreamOfResponses(req: MyRequest): Stream[F, MyResponse] =
    Stream(...)
}

You now need to return an F[Stream[F, MyResponse]] instead of a plain Stream. The simplest way to do that is to use pure to lift the result into F:

import cats.syntax.applicative._

class TheService[F[_]: Applicative] extends MyService[F] {
  def SingleResponse(req: MyRequest): F[MyResponse] = ???
  def StreamOfResponses(req: MyRequest): F[Stream[F, MyResponse]] =
    Stream(...).pure[F]
}

The same idea applies if you are using Monix Observable as your streaming implementation.

Updating client-side code

Your code that calls the StreamOfResponses method is expecting a Stream[F, MyResponse], but it now needs to handle F[Stream[F, MyResponse]] instead.

You can turn the response back into a Stream[F, MyResponse] using fs2.Stream.force:

val effect: F[Stream[F, MyResponse]] = client.StreamOfResponses(request)
val stream: Stream[F, MyResponse] = Stream.force(effect)

If you are using Monix Observable, you can use flatten:

val effect: F[Observable[MyResponse]] = client.StreamOfResponses(request)
val observable: Observable[MyResponse] = Observable.from(effect).flatten

Reorganized modules

Mu’s codebase and module structure has grown quite organically over the years. We decided it was time for some spring cleaning, so we’ve refactored the project slightly. We renamed some modules for clarity, and in a few places we’ve combined a number of small, tightly-coupled modules into larger modules.

The only changes that should directly affect users are as follows:

The mu-rpc-netty module has been renamed to mu-rpc-client-netty, to make it clear that it’s a client-side module.
Similarly, mu-rpc-okhttp is now mu-rpc-client-okhttp.
mu-rpc-channel is gone. Please use mu-rpc-service instead.
mu-rpc-health-check-unary is now mu-rpc-health-check. It also includes streaming health check services, with FS2 and Monix implementations, which used to live in the mu-rpc-fs2 and mu-rpc-monix modules respectively.

We’ve also slimmed down the project by moving the sbt-mu-srcgen sbt plugin and the examples to their own repositories:

Removed legacy-avro-decimal-compat modules

These were added a long time ago to aid migration when we changed the Avro encoding of BigDecimal. We decided enough time has passed to make it safe to remove them.

Removed some annotations

The @option, @outputPackage and @outputName annotations have been removed.

These annotations were a relic of a previously removed feature involving generation of IDL files from Scala code. These days Mu only supports going the other way: generating service definitions in Scala from IDL files.

Final word

The active development of Mu-Scala is proudly sponsored by 47 Degrees, a Functional Programming consultancy with a focus on the Scala, Kotlin, Swift, and Haskell programming languages.