1.4-Software-Development-Principles

JUnit5 - Bad Weather Tests

Requirements

In order to handle this theory, you must know what lambda expressions are, or you must at least understand Anonymous classes.

Introduction

We can now assume that you know how to design tests that confirm that your code works as expected. You will usually confirm that your code gives the correct answers when asking the right questions. This is called “Good Weather Testing” or “The Happy Path”.

You have also learned that checking for correct input and throwing exceptions is important to deliver correct results. So the question becomes, how do you test that your code rejects certain values or states. In other words, my code needs to crash with an exception in certain situations. Can I test that? This is called “Bad Weather Testing” or “The Nefarious Ravine”. (@TODO: Check this term)

In this way we test both situations:

Executable interface

To test code that has to go wrong, we need to supply a set of instructions that should cause an Exception.

To provide our assertion with such a set of instructions we need to use the Executable interface:

@FunctionalInterface
public interface Executable {
	void execute() throws Throwable;
}

The execute() method is in this case a method that requires no arguments and simply carries out some instructions.

For the assertThrows method we will use this interface through a lambda expression:

// Create an instance of the Executable interface by referencing a method.
Executable r = MyClass::doStuff;
r.execute();

assertThrows

In order to use this assertion method you must understand either lambda expressions or anonymous classes. (Although you can do without but that would make for very lengthy code.)

The assertThrows assertion checks two things:

Afterward you may further inspect the content of the Exception with the known assert methods.

The assertThrows method requires two arguments, one is the class of the Exception that should be thrown when this “mistake” is made. And the second is an instance of the Executable interface that executes an instruction that causes that particular exception.

@Test
public void AfterCallingConstructor_WhenAgeIsNegative_ThenThrowsIllegalArgument() {
    IllegalArgumentException ex = assertThrows(
		// This is the expected Exception.
        IllegalArgumentException.class,
		// This is code that should go wrong.
		() -> { Person p = new Person("John", -1); } 
    );	
	// Additional confirmation that the internal data is as expected.
	assertEquals("Expected message", ex.getMessage());
}

In this example, with a nice Given_When_Then name, the assertThrows is used to confirm that a piece of code fails. In this case the code is provided as a lambda expression: () -> { Person p = new Person("John", -1); }

You could have used an anonymous Executable class instead:

IllegalArgumentException ex = assertThrows(
    IllegalArgumentException.class, // This is the expected Exception.
    new Executable() {
        @Override
        public void execute() {
            Person p = new Person("John", -1);
        }
    }
);

Ensuring “branch coverage”

When your code has multiple different reasons to crash, it is vitally important to confirm that each of those reasons are tested.

The point is that each possible path through each method is tested. When they come up with an expected answer, but also when they should throw a different exception.

Boundary tests

When only values (lets assume numbers) within a certain range are allowed then I can come up with seven separate categories of numbers:

Category What is tested? Example range 13-25
1 Numbers below the range fail. -15, 0 & 10
2 The number on the border (but below the range) failss 12
3 The number on the border (but inside the range) succeed. 13
4 Numbers within the range succeed. 15, 17 & 20
5 Number on the other border (but inside the range) succeed. 25
6 Number on the other border (but above the range) fail. 26
7 Numbers above the range fail. 30, 100 & 1000

Besides these numbers we strongly suggest to also test strange numbers like: