Comparing performance of Scala's standard actors with Akka
UPDATE 13/12/09: Akka's performance has improved after I submitted my results to them. It is now 2-3 times faster than Scala's standard actors.
To those unfamiliar with the concept of actors, it is a programming model where subsystems communicate via messages between actors and do not share state. Each subsystem handles one message at a time. This model simplifies developing concurrent systems since you no longer need to reason about threads and locks and also unifies local and remote computing.
Scala's standard library provides a very powerful actor abstraction, using the power of the language. Akka's implementation has a more simplistic model that is supposed to be faster. Akka also provides mechanisms for transaction handling, supervision trees and more. I didn't need all of those, so the choice was based on the actors performance.
I borrowed a test from http://shootout.alioth.debian.org/ . The tests simulates N animals (chameneos) where 2 random pairs meet in a mall, exchange colors and then go on to meet others M times. Obviously exchanging colors is not a very intensive task, so most processing goes on matching the animals and in the actor's case this means passing and receiving messages.
If you'll go to the results of the test in the shootout site, you'll see the best result is achieved by C/C++, Haskell, go, Java and Scala close behind. This is understandable, because for such a simple use case, the higher level of development efficiency provided by the various languages is not accounted for nor is debugging time taken to debug complex concurrency scenarios. However, the Scala implementation is not even actor based, it is a regular thread/lock based implementation. The actor based implementation is way behind, close to Erlang. Another interesting note is that dynamic languages fall behind the static typed languages. Again, for me this doesn't mean much as such benchmarks cannot model real-life development.
OK, after this short analysis, back to the subject. Since we chose Scala as our implementation language for its advanced features and the JVM ecosystem, my concern was comparing JVM based implementations. Out of the Java based implementations, Kilim stood out as doing something different than a simple OO implementation. Unfortunately, Kilim cannot be used with Scala (since it expects all involved objects to be annotated and programming in Scala means creating a lot of function objects behind the scenes). So I chose to focus on Scala based implementations since they use Scala's features to provide a more usable API.
So I took one of the Scala implementations of the chameneos test (BTW, it is not one of those presented in the results page of the shootout site) and modified it to work with Akka's actors. Then I timed both with increasing number of meetings.
Notes: I built with the -optimise flag and used the JVM flag -XX:+DoEscapeAnalysis. I ran the benchmarks twice to load any files to the buffer cache. The scale of the graph is logarithmic.
The Akka version came out more awkward. While Scala allows to use 'react' to receive messages in a very flexible way, Akka enforces one to implement a method for receiving messages. Things like sending a message then waiting for a response, are more awkward.
Don't trust hype. Akka prouds itself for being event based and so much more efficient, this is not true. Nor it's more restrictive model offer more efficiency (at least in the current implementation)
Both benchmarks utilized only one core. I assume this is because the actual work being done was so little that the kernel chose CPU affinity over multi-core.
I've contacted Akka's creator with these results. Will update with any new findings
You can find the source code (very little of it) in here
We will contact you as soon as possible.