Theory of Operation and Goals of CUTE

CUTE leverages modern C++ libraries and features to make writing C++ unit tests easier. For example, one disadvantage of CppUnit is that you have to write a subclass in order to have your own test case. This is a lot of programming overhead, especially when you want to start small.

CUTE stands for C++ Unit Testing Easier. This article was first published by Peter Sommerlad in ACCU Overload Journal #75. It is maintained here in the wiki with updates to match the current version of CUTE.

I was inspired by Kevlin Henney's Java testing framework called JUTLAND (Java Unit Testing: Light, Adaptable 'n' Discreet), and the corresponding presentation he gave at JAOO 2005. In addition, I wondered if I could come up with a design that is similarly orthogonal, easily extendable and also much simpler to use than current C++ unit testing approaches.

Here you will learn how to use CUTE in your projects and also some of the more modern C++ coding techniques employed for its implementation. I also ask you to give feedback to help me further simplify and improve CUTE.

My Problems with CppUnit

Inheritance is a very strong coupling between classes. Requiring a test case class to inherit from a CppUnit framework class couples both closely together. The CppUnit Cookbook tutorial lists at least six classes you must deal with in order to get things up and running. You even have to decide if you want to inherit from TestCase or TestFixture for even the simplest test case. I do not want to go into more details, but this is how I want to write tests:

#include "cute.h" 

int lifeTheUniverseAndEverything = 6*7;

void mySimpleTest(){
    ASSERT_EQUAL(42, lifeTheUniverseAndEverything);
}

That's it. A simple test is a simple void function. Done. Well, almost done.

In addition, CppUnit originated in the days of non-standard C++ compilers, when the more modern features of the language were just not broadly available. From a modern perspective, this limits its design.

Today, all relevant compilers are able to compile most of standard C++ and the std::tr1 libraries from boost.

Running a Single Test

Since we lack reflection mechanisms available in other programming languages, testing with CUTE requires its own main function. For simple cases this is straightforward. As with CppUnit you instantiate a runner object and pass it your test. The simplest possible way to produce some output is:

#include "my_simple_test.h" 
#include "cute_runner.h" 
#include "ostream_listener.h" 
#include <iostream>

int main(){
    cute::ostream_listener<> listener{};
    if (cute::makeRunner(listener)(mySimpleTest)){
        std::cout << "success\n";
    } else {
        std::cout << "failure\n";
    }   
}

This may not yet be very impressive, but it is simple. Reporting test outcome is important, so CUTE provides a means to configure the runner with a listener. You may have wondered why cute::runner has template brackets. This is where you specify the listener class:

#include "my_simple_test.h" 
#include "cute_runner.h" 
#include "ostream_listener.h" 

int main(){
    cute::ostream_listener<> listener{};
    cute::makeRunner(listener)(mySimpleTest);
}

This test succeeds and outputs:

starting: void ()
void () OK

From the above test result, we know only that a void function without any arguments ran successfully. This shows the C++ introspection limitation, which only provides type information, not function names. But the preprocessor can help us. We make our test cuter by applying the macro CUTE():

cute::makeRunner(listener)(CUTE(mySimpleTest));

This test succeeds and produces the terse output "mySimpleTest OK". However, if we make the test fail by setting the variable lifeTheUniverseAndEverything to 6*6 instead of 6*7, we get:

my_simple_test.h:16: testcase failed: mySimpleTest: 42 == lifeTheUniverseAndEverything expected: 42 but was: 36 in mySimpleTest

This failure message contains everything we need to find the origin of the failure, plus some context help to hint at the problem. You probably already guessed that the preprocessor macro ASSERT_EQUAL() from cute.h contains the magic that collects this interesting information.

For a better integration into Visual Studio and Eclipse IDEs, CDT CUTE now provides yet another listener, cute::ide_listener. This listener allows navigation from failure messages to their source, and even provides a red/green bar UI within Eclipse CDT.

Have a look at the How Things Work section to gain insight into the framework.

Limitations and Outlook

One big difference between C++ and other languages is the lack of method-level introspection. The only way to create a list of tests to execute is for a programmer to specify it by registering test objects somewhere. Our CUTE Eclipse plug-in eases that task by automatically registering test functions and methods for you in the current file's suite object.

Conclusion

CUTE 2.1 and its Eclipse plug-in are already very well established. CUTE comes with a small test suite for itself, but it may still have problems that I have not yet encountered. If you have not yet written unit tests for your code, try starting now using CUTE via its Eclipse plug-in and tell us how it feels and how it works for you.

If you have more ideas for extending CUTE to make it a more convenient environment: Tell me your ideas, or just implement them. Thank you in advance.

References