Scala actors thread pool pitfall

The thread pool used by the Scala actors may not work liike you think it should, causing actors to starve.


To see why, lets first define a utility method to measure how long an actor takes to act. We take a timestamp, then create an actor that just prints to the console how many milliseconds passed since the timestamp



import scala.actors.Actor._

def timeActor = {
    val start = System.currentTimeMillis
    actor {println("Took: " + (System.currentTimeMillis - start))}


Testing it:

scala> timeActor
Took: 2
res49: scala.actors.Actor = scala.actors.Actor$$anon$1@18e905


We see the actor ran within 2 milliseconds.


Now lets throw some more actors that just block and see how much the actor took:

scala> for (_ <- 1 to 4) actor{Thread.sleep(5000)}; timeActor

scala> Took: 0


Not much surprise here either.


Next I run 8 actors:

scala> for (_ <- 1 to 8) actor{Thread.sleep(5000)}; timeActor

scala> Took: 5000

Oops. Why did it take 5 seconds to run? 


If you look in scala.actors.scheduler.ThreadPoolConfig you might be tempted to set actors.maxPoolSize to some large value. This will not change the result of the above test. In fact, if the property is not set, then ThreadPoolConfig#maxPoolSize returns 256


What is happening here? 


The secret is the type of scheduler used. On Sun / Apple JVM version 1.6 the scheduler is a ForkJoinScheduler. This scheduler uses ForkJoinPool from the upcoming(?) jdk7 which tries to maintain a fixed set of concurrently running threads set by setParallelism. ForkJoinScheduler uses ThreadPoolConfig#corePoolSize to set this property. corePoolSize is by default twice the number of cpus. In my case 2 * 4 ==> 8


What this means is that no more than 8 threads will run concurrently.


For CPU intensive tasks, this is a good thing, since it means your CPUs won't be thrashed by many threads causing context switching and cache misses.


However, if your threads are blocking on IO, waiting for a service, etc. Then ForkJoinPool has now way of knowing they are not really working. All tasks will just wait for them to finish. Note that if an actor waits on another actor with the !? method, then the scheduler knows the actor is blocked and the fork-join pool will create another thread to maintain the level of parallelism.


What is the lesson? Never block inside actors if they use ForkJoinScheduler.


Alternatives are: 

  1. set the actors.enableForkJoin system property to false. This may not be good, as explained above.
  2. or, use a diffent scheduler for actors that may block. You can choose between ResizableThreadPoolScheduler that uses a ThreadPoolExecutor, or ExecutorScheduler which is initialized with an ExecutorService instance.


To demonstrate the second approach:

object ActorThatBlocks {
  lazy val scheduler = {
    val s = new ResizableThreadPoolScheduler(false)
  def actorThatBlocks(body: => Unit): Actor = {
    val a = new ActorThatBlocks {
      def act() = body
trait ActorThatBlocks extends Actor{
  override def scheduler = ActorThatBlocks.scheduler

And lets test this:

scala> for (_ <- 1 to 8) actorThatBlocks{Thread.sleep(5000)}; timeActor
Took: 1


Note one cavet: if inside an ActorThatBlocks you will use the 'actor' method to create an anonymous class actor, it will use the same scheduler, and not the ForkJoinScheduler. A workaround is to create a actorThatDoesNotBlock method...