2016 Lambda World Overview
by Maureen Elsberry- •
- October 12, 2016
- •
- lambda world• functional programming• events
- |
- 19 minutes to read.
The 2nd Annual Lambda World took place on September 30th through October 1st and brought together over 270 Functional Programming enthusiasts from around the globe for two days of presentations, learning, hacking, and networking.
We kicked things off with a packed house for the Typelevel Community Conference where eight talks were selected by the audience on topics such as ScalaCheck, Haskell, data science, cofree coalgebras, and and how to speed up the scala compiler through inductive implicit resolution heuristics.
Following the Typelevel conference was a four-hour hackathon hosted by the Scala Center designed to bring people together to work on issues on projects like Scaladex, Scalafmt, Scala.meta, and Cats.
We were happy to have Scala Exercises included in the hackathon and offered a few issues as potential projects to work on. We’d like to give a special mention to Svetlana Filimonova who submitted a PR resolving Issue 8 in the ScalaCheck exercises library, updating the Grouping Properties exercise to accept three arguments. Thanks, Svetlana!
The evening concluded with a series of workshops covering a variety of programming languages and a Welcome Dinner to cap off the evening.
The second day of Lambda World consisted of sixteen talks and a killer closing party and concert at an old castle. Here are a few reviews of some of the incredible presentations covered at the event:
Erik Osheim
- Room to Grow: Evolving Functional Programming Languages
- Review by Diego Alonso
Erik devoted the opening keynote to a known language development dilemma: What is the best way to provide a new feature in a functional language? On this issue, he gave his views, drawn from his work on several Scala libraries or extensions, such as the kind-projector plugin.
In his view, a major win of functional languages is that one can write reusable code with little boilerplate. Thus, new language features should also make it easier for us to solve more complex problems by writing reusable code with little boilerplate. For a case study, he considered type classes. Unlike Haskell, Scala has no special syntax for typeclasses: these can be implemented using objects and implicits. Erik refers to this as an encoding: use the existing features in the language to write code that looks and works as if it had been written with a language having that feature. The alternative is a language extension, which consists of modifying the language compiler, so that it would accept a new special syntax for that feature.
Having explained these alternatives, Erik discussed their advantages or disadvantages, in particular, in which circumstances is each one a better choice. His conclusion was that encodings always give better returns in the short term, and should always be preferred as a first approach to implementing and using, a novel language feature. Extensions, instead, should not be used until that feature has been well tried as an encoding, and its development should be seen as a long-term infrastructure investment.
Additional Resources:
Edwin Brady
- Type - Driven development of communicating systems in Idris
- Review by Diego Alonso
Edwin’s talk introduced us to the Idris programming language, to type-driven development, and to the use of Idris to write concurrent systems (like Akka actors) that are correct by design.
Edwin first introduced type-driven-development (a pun on test-driven development), a method for writing programs. It consists on writing a program by using types as a blueprint, and using the interaction with the compiler as a guide, so the type-checker drives us to write each function in the program with the type it needs. Edwin compares this to making a puzzle: every piece is placed considering both the picture and its neighbours.
Edwin then moved to function totality checking, a feature that sets Idris apart from Haskell or Scala. Total functions are those whose execution always terminates without failures (i.e. exceptions). For him, if you care about types you should care about totality: the type of a partial function only says
what the type of the value is if the evaluation terminates and does not fail;
whereas the type of a total function says that it terminates and returns a value of that type. In the talk, Edwin showcased Idris’s total label to check function totality, and some programs it rejects.
In the last part, Edwin showed how to use totality to ensure correctness of communicating processes. Unlike normal functions, these processes are supposed to never terminate. In their case, totality is meant as productiveness: they always produce a finite answer in finite time. Using Idris, Edwin showed how to write a process and type-check it so that it responds to any given message in a finite time. He then wrote a system of two processes, whose communication was defined by a simple protocol, and he showed how Idris’s totality checking could prove that they interact and work as expected, so that the sender always sends messages that the receiver can handle, and how they can reach a final state.
Edwin’s talk was a good appetizer for Idris, requiring no previous knowledge of Idris or of its syntax, and demonstrates the features of its advanced type systems.
Additional Resources:
Iain Hull
- Adopting actors…an epic tail of loss and learning
- Review by Diego Alonso
In his talk, Iain told us about his team’s work in migrating Workday’s systems to Akka, and about the mistakes and misconceptions they fell to in moving from an an object-oriented design (Java) to an actor-based one (in Akka).
Workday is a cloud-based software company that over the last few years has grown its cloud infrastructure hardware. This called for a change in the software architecture, which Iain’s team carried out. They started from an existing Java implementation, which followed an object-oriented design, and then they chose to make the new implementation in Akka due to several reasons: In Akka every actor is a sequential and deterministic processor, without race conditions or deadlocks. Also, communication between actors easily handles their distribution across the network. However, in the process of the migration they fell into several antipatterns or false patterns.
One of the antipatterns was the movie star: an actor that receives too many messages. Such actors accrue so much of the application’s state, that error recovery becomes very hard. The lesson from this antipattern is to always design every actor “as if it were as created from error recovery”.
Another of the false patterns was the state actor, when a large actor is split into several ones carrying part of the data. This resembles classical object-oriented design patterns. However, in Akka this creates a new problem: actors’ inner states can fall out of sync due to the delay in message passing. If the data they carry has to be kept consistent, then this out-of-sync state introduces a breach in the application’s consistency. The lesson, for this one, was that any consistent or atomic set of data should be kept in a single actor.
In conclusion, Iain’s talk is a cautionary tale for experienced object-oriented developer, who should avoid the trap of seeing actors as a objects, no matter how similar the notions of encapsulated state or interface may be said to be.
Additional Resources:
Raoul-Gabriel Urma:
- Pragmatic functional refactoring with Java 8
- Review by Javier Siloniz
In his presentation, Raoul introduced several foundational concepts of Functional Programming, with multiple code examples specially tailored to Java developers. These concepts include function composition in Java 8 with lambdas, currying, and partially applied functions to increase abstraction and reusability. Also, how immutability can help our code to have tread-safe cacheable code and achieve referential transparency. He also covered optional types in Java 8 to explicitly state potential absence of data in our models; each with detailed case uses, and a discussion of its pros and cons as they’re currently implementable in Java and Java 8.
Additional resources:
Aaron Levin
- The dialectics of type-level programming or how i learned to love values
- Review by Javier Siloniz
Aaron offered an introduction to DSLs (i.e. Akka, Specs2), and how type-level programming can allow extension of DSLs. He went through an example of how these type-level DSLs can be implemented and used in Scala and Haskell for a real use-case. Taking advantage of this example, a complete run-through on how Scala performs type inductions step-by-step was also discussed. Finally he concluded the talk mentioning the problems developers can currently face when using type induction (specially non-descriptive compile error messages), and his ideas on how this could be improved in the future.
Additional resources:
Trisha Gee
- Java 8 in Anger
- Review by Javier Siloniz
Trisha offered us a live code session, implementing a real-time Twitter feed dashboard that analyzes moods in each of the messages. The purpose of this coding session was to show that, even though (as she mentioned) Java is not a Functional Programming language, the new APIs and language features Java 8 and 9 include can help developers achieve many of the benefits of FP languages (specially lambdas and the Stream API). She also showed the dos and not-to-dos while using these new features, and also used several awesome refactoring tools included in IntelliJ that amazed many in the audience.
Additional resources:
Oskar Wickström
- Oden
- Review by Javier Siloniz
Oskar shared with us his motivations to start working on Oden, a functional programming language based on Go and compatible with its ecosystem. He wants to have a type-safe FP language for writing web apps, and although he was seduced to use Haskell, he decided to go the extra-mile and take the good stuff of Go to create a new one. His project goals include several features Go is missing (i.e.: generic programming, abstraction when handling errors), and also some principles he wants to follow (Oskar mentioned some points he considered could raise some debate, like avoiding the use of macros, or taking a “clarity avoid cleverness” approach in order to avoid hiding important parts of the code under wraps). Finally he ran through the several features already available in Oden including the absence of macros, curried functions with type signatures, imports from existing Go libraries, type-classes-like protocols, and how to operate with Go channels.
Update: After the conference, Oskar made the announcement that he will cease working on Oden due to time constraints. We’d like to thank him for all the time and effort he put in to developing the language.
Additional resources:
Simon Belak
- Doing data science with Clojure: the ugly, the sad, the joyful
- Review by Alejandro Gomez
In his presentation, Simon discussed how programming and data science are both about encoding and representing information.
Traditional data analytics frameworks (ab)use the “data frame model, which confalte representation and abstractions. Clojure excels in structure manipulation and encoding, so why not use Clojure data structures directly?
He outlined how time is spent in data science explaining that most of the time is about cleaning and preparing data, a little time is spent applying models, libraries, and algorithms, and rarely, time is spent implementing new models.
He suggests using Huri for data science and clojure.spec for describing data and data manipulations.
In general, the typical data science pipeline is: events/external data -> kafka -> precomputed views (and back again). Even and attribute ontology is manual or inferred and statistical analysis is seasonality detection, and outlier removal.
He stressed the importance of thinking in distributions, not in numbers by plotting with notebooks, the difference between notebook vs repl, and gorilla repl.
Since the Cloure ecosystem is lacking libraries, call Python (scikit-learn), and mini compilers targeting a library in another language.
The takeaway? The speed of answers matters for being data-driven, there is no need to try to reinvent the wheel, data science is encoding and representing info.
Additional resources:
Christopher Meiklejohn
- Declarative, Secure, Convergent Edge Computation: An Approach for the Internet of Things
- Review by Alejandro Gomez
Christopher talked about the challenges of synchronization to enforce order and data integrity and eliminate race conditions.
He mentioned that the tools used in single-machine systems (locks, mutexes, etc.) don’t work for a distributed system.
Synchronization is difficult on distributed systems. In devices connected to the internet (IoT), low power, limited memory and connectivity makes being online constantly not feasible. On mobile devices, apps need to work offline with replicated shared state.
He also introduced the notion of “edge computing” in distributed systems for devices that are not continuously connected to a network, such as sensors, smartphones, or laptops. Edge computing pushes both computation and data to the logical extremes of the network and decentralized, replicated state across distributed devices.
Christopher talked about how CRDT (Conflict-free Replicated Data Types) enable edge computing, and how many data structures (sets, trees, etc.) have CRDT equivalents already. Its an active area of research for him.
Challenges:
- Immutability in distributed systems is expensive
- Central processing unavailable: difficult to enforce ordering, transactions
Goals:
- Mutable state is a reality, current state of distributed, shared data
- immutabiliy makes things easier, so we model mutable state as a time series of immutable events (commutative operations if we allow concurrency)
- as researchers: push the limits of whats possible
Additional resources:
Alberto Gómez
- Transient: full algebraic and monadic composability in presence of multithreading, events and distributed computing
- Review by Alejandro Gomez
Alberto talked about how some of the goals of programming such as reusability, maintanability, and expressing solutions in a high-level language conflict with real-world concurrency problems. Events/callbacks, threading, and other concurrency constructs are hard to compose, but a reality.
We want to program with “recipes” and mathematical formulas, how do we make concurrency composable and simple?
He talked about how programs started to batch process data, how unix introduced the composition of file IO and CPU processing with pipes. The console application as a single blocking IO action. But how do we compose more than one simultaneous, asynchronous IO actions?
There are some approaches to composing multiple IO actions, like continuations, but the weird syntax and hard to grasp semantics make them not very practical to use. Also, its hard to use them with more than one source of events.
He tries to solve this with his Transient library (written in Haskell), providing powerful asynchrony primitives:
- parallelism
- events
- asynchronous IO
- early termination
- non-determinism
- logging
- distributed computing
and using well-known operators to compose them:
- monoidal and applicative expressions for concurrency & parallelism
- monadic bind for sequencing
He showed plenty of code examples and the possibilities the library offers.
Additional resources:
Pascal Voitot
- Describe & Conquer (& Freek)
- Review by Alejandro Gomez
Pascal Voitot talked about the different levels of representation when programming
- human world: description of business logic
- machine world: execution of logic & resource managament
In the human representation, we use a DSL written in a host language. Constraints, logic, types, laws, and typeclasses help us represent and reason about our problem. Mathematics allows us to express better our thought. We use machine-oriented host languages.
He then talked about how we can separate DSLs from their interpretation with Free Monad + Coyoneda, constructing a grammar with the operations we want to support and combining operations with monadic sequential flow. The target monad is not known until execution of the free program.
His library, Freek, which uses cats’ Free monad under the hood, helps people use free programs to describe & separate description from execution. It provides artificial values to convince scalac that the types align and hides much of Free monads boilerplate. It also has helpers to combine DSLs and their interpreters.
One limitation of this approach is that there is one single target monad, which we can overcome using a monad transformer for stacking multiple effects (errors, non-determinism, etc.). Also, what happens when the return types of our DSL are monadic (Option, Xor) and want to preserve their semantics in Free? Freek allows us to combine multiple monadic return types by stacking all used return types in an “onion”.
Limitations:
- compiling duration with many DSLs
- Free/Coproduct wrapping cost (negligible with regards to IO)
- Garbage Collection can be an issue
- Not good for high-performance clearly, suitable for IO workloads though.
Runtime optimizations with alternate Free representations:
- Right association (cats & scalaz do this)
- O(1) appending & observation (see the Reflection without Remorse paper)
Can we eliminate free structures at compile-time?
- Free applicative is feasible, structure of the computation is static
- With free monads is a challenge (do we want it?)
We need compiler optimizations, Scalac implicit resolution by induction is very naive (Miles Sabin is working on improving this).
Free requires quite a lot of boilerplate and is a step towards better models for modeling computation:
- MTL, finaly tagless interpreters, object algebras, Van Laarhoven Free
- effect-oriented models: Freer monad, Eff
We need better support for DSLs and Free in host languages, be it through macros or extensions. Also, we need better host languages for this approachs to modeling computation.
Additional resources:
Miles Sabin
- Typelevel Scala Rebooted
- Review by Juan Méndez
In his talk, Miles explains what Typelevel is: A community of projects and people that focus on pure, typeful, functional programming in Scala.
The background of the entire story is that there was a fork of Scala initiated at the end of 2014. The development on the fork stalled at the beginning of 2015. At the same time, plugins, macros, workarounds filled some of the gaps.
The talk went on to on explain SI-2712, a bug that stayed opened for years, and what it took to fix it. The modification largely improves Type inference, and it affects libraries like ScalaZ and Cats.
Then he proceeds to explain the reasons that people have for using Typelevel Scala now.
- Releases compatible with the existing ecosystem
- The benefits must be significant
- The disruption must be minimal
- The risks should be very low
Hence, to achieve those goals, he points out the Typelevel strategy:
- Track Lightbend Scala releases
- Every fix included in a Typelevel Scala release
- Must be submitted as a PR against Lightbend Scala first.
- Binary compatible with Lightbend’s Scala (LBS)
The results of these efforts can be seen, for example:
- First release of Typelevel Scala
- Relative to LBS 2.11.8
- Includes fix for SI-2712
It’s worth it to mention his confidence of this version when he asserts that it is risky not to use it!
Using it is a simple as configuring:
scalaOrganization:= “org.typelevel”
assumes you are using LBS 2.11.8
Raul Raja asked about collaboration between Typelevel Scala and Dotty. Miles answered that it’s a problem for the future. He doesn´t think that Dotty is paying attention to what practitioners need. So, this is an opportunity for people to get introduced on low hanging fruit issues and influence on the direction of Scala.
Additional Resources:
These are just a selection of the great presentations at Lambda World. We’ll be uploading videos of all the talks in the near future.
We want to send our thanks to the formidable speakers, sponsors, attendees, and individuals who made this event sensational (and a big thanks to our internal organizational team). We’re already busy planning next year and have some big things in store to make 2017’s event even better.
Check out the Lambda World photos and stay tuned for the videos from the presentations on the Lambda World YouTube channel.
We also want to give a bonus shoutout to Lambdaman for making an appearance at the event.
Consider buying a Lambdaman or Lambdawoman t-shirt. All proceeds go to support Bootstrap, an initiative to train and provide cirriculum materials to teachers to teach Functional Programming to Middle and High School students.