automatic unit test generation in #java

In this article I'll go over two Java packages that allow to create specifications for automatic unit test generation. 


What does that mean? It means that instead of creating a unit test, you create a description of the properties that the unit under test must hold. A property is something like "the stack size after push is the size before plus 1". With regular unit tests we would assert this property by creating a specific test case (say, starting from an empty stack) and doing the operation. But then we might miss some corner cases. For eample, maybe our stack is backed by an array starting at size 10 and we forgot to code the array resizing. Then 'push' will work as expected for the first 10 items, but fail for the 11th. Our simple unit test might not try to push 10 items.


Instead, we want to specify some basic building blocks and then have many instances generated for us. E.g., with the example above, we would like to describe how to generate a stack from a random list of integers and then write a test that pushes an integer to that stack. While generating some of the stack instances, some will be larger than 10 and the test will fail.


Note: The original idea for this approach came from a Haskell library named quickcheck.


The first implementation is quickcheck.

for (Pair<Integer, List<Integer>> any : somePairs(integers(), lists(integers()))) {
  Integer element = any.getFirst();
  Collection<Integer> collection = any.getSecond();
  boolean changedCollection = false;
  boolean exceptionThrown = false;
  try {
    changedCollection = collection.add(element);
  } catch (Exception e) {
    exceptionThrown = true;
  assertTrue(collection.contains(element) != exceptionThrown);
  assertTrue(changedCollection == containsInstance(collection,element));

literaly it means that for any pair of integer and list, out of randomly generated pairs of integers and lists, it takes the integer and adds to the list and then makes some assertions about the modified collection.


The arbitrary instances are created by creating instance of Generator class that has a next method for returning a new value.


Another implementation is the package fj.test of functionaljava. It is more complex to understand, but much more powerful as it allows more combinations of properties and generators


In fj.test, the test creates Property instances that work on arbitrary values. fj.test will then generate many values (100 is the default) and will assert the properties hold.


Here's an example (from the demo source code) that asserts for LinkedList that l1.addAll(l2) will make l1's size the sum of the previous sizes (the code is verbose and explained below):

final Property p = property(arbLinkedList(arbInteger), arbLinkedList(arbInteger),
   new F2<LinkedList<Integer>, LinkedList<Integer>, Property>() {
   public Property f(final LinkedList<Integer> x, final LinkedList<Integer> y) {
     final LinkedList<Integer> xy = new LinkedList<Integer>(x);
     return prop(xy.size() == x.size() + y.size());


This reads as follows: creates a property that states that for two arbitrary linked lists, they have the property that the size of adding the two is the sum of sizes.


Lets disect a bit:

property is a method that accepts two Arbitrary objects and a function (F2 objct with f method) that defines a result property. In this case the result is a boolean assertion

arbLinkedList(arbInteger) creates a linked list of arbitrary size. An Arbitrary<A> object generates arbitrary A values. It does this with a Gen object that generates them. arbInteger will generate arbitrary integers and arbLinkedList will generate a list of arbitrary size filled with those integers


Here's an example of how to create arbitrary values for your own class (taken from here): MyClass is constructed with a byte and a String, so to generate an arbitrary instance, we combine two arbitraries of byte and string with a function that acts as a factory

final Arbitrary<MyClass> arbMyClass = arbitrary(arbByteR.gen.bind(arbStringR.gen, curry(new F2<Byte, String, MyClass>() {
  public MyClass f(final Byte b, final String s) {
    return new MyClass(b, s);


We can then use the arbitrary MyClass to create a Property that asserts the hashCode of two equal instances is the same (note: in order to make sure that of the arbitrary instances that are created, there will be equal ones, the arbitrary MyClass instances are created from a limited set of strings and bytes, hense the arbByteR and arbStringR uses above, the R stands for restrictive)

final Property p = property(arbMyClass, arbMyClass, new F2<MyClass, MyClass, Property>() {
  public Property f(final MyClass m1, final MyClass m2) {
    return bool(m1.equals(m2)).implies(m1.hashCode() == m2.hashCode());