Integration Test - Case Study

In the previous post I discussed an integration-test framework that I developed for testing system  interactions with a java  master controller. Recall that this system consists of separate View, Master Controller, and Model/ModelController components.

A basic flow through the system would start with an action in the View, pass through the Master Controller to the Model/Model Controller, pass back into the Master Controller, and then terminate with responses back to the view and the model controller.

For this case I configured Dagger2 to create a mock of the manager that interacts with the view (see mocked view in the diagram). I used the mock to verify that the correct method and method parameters are called at the end of the flow. I also wanted to verify that the correct methods and method parameters are called on the Model Controller, but I could not mock the Model Controller because it is integral to the flow we are testing. Instead, the test spied on the model controller to verify the expected method and method parameter calls.

Test flow (from diagram):

1. Initiate an action into the controller to mimic a user interaction from the GUI (View)
2. The Master Controller interprets the input action as an action needing interpretation and dispatch to the Model Controller and performs these actions. I used Mockito to verify that the correct Model Controller method was called and that the correct parameters were supplied.
3. The Model Controller interacts with the Model, processes the action and sends its own command back to the controller. I verify that the correct Master Controller method is called with the correct parameters. If the correct signature is seen, it means that the following components are working properly for the flow under test:
1. The JNI layer for passing between the Java Master Controller and the C Model Controller for these interactions
2. The Model Controller logic for this action 
4. The Master Controller then interprets the action from the Model Controller, dispatches an action to the View to inform it what to display, and replies to the Model Controller with a confirmation able to successfully interpret and handle the command. Both these terminal responses are also verified.

One added complexity with testing this integration is that there are several locations where the flow crosses thread boundaries. I needed a mechanism for delaying our verifies until a callback is received. The integration test will take much longer to run than a typical unit test (on the order of 15 seconds), but this does not mean that we can get sloppy and use sleep statements. To get reliable results, sleep statements would add more delay than necessary, and could still result in tests which are not repeatable. It’s almost never a good idea to sleep in an automated test...or in any code.

Mockito provides the doAnswer() method which is very helpful for mocking asynchronous responses from an object, but this is not helpful to us because we are testing objects which we can’t mock. What we need is a way of synchronizing with the asynchronous callback, because the code under test relies on that callback completing before it can proceed. Java provides such a mechanism with the CountdownLatch. The CountdownLatch will block until its counter reaches 0. We can setup the CountdownLatch in the test thread and give it a value of 1. In the callback thread, we reduce the count to 0 to unblock the test thread when we perform the callback, allowing it to proceed with its verification. We attach a timeout to the latch in case the callback is never hit (15 seconds in this case). This is much better than a sleep statement because the timeout condition is only hit on a failure condition. For this reason we can choose a large number that we know will pass under all conditions, without actually delaying the test execution time except for the exceptional condition of an actual failing test. 



This solution is still not ideal. We had to instrument the test with a handler that overrides the implementation we are testing to call the implementation’s body and then countdown the latch. I could have used DI to inject this test handler, but in this case the framework already had a mechanism for registering handlers directly, so we were able to register the test handler this way.