Unit Testing : Making sure your bugs don’t come back

8. Unit Testing : Making sure your bugs don’t come back#

Unit testing allows to ensure that a given software behaves in the correct way, at least for the cases one is testing. Once a function is written (or even before in TTD) or a bug is fixed, it is necessary to write a test that ensures the function to work properly in limit cases or the bug to not reappear in the future. There are several levels associated with unit testing .

In this unit we will learn the general philosophy behind it and a couple of tools to implement very basic tests, althoguh the list of testing frameworks is very large. Furthermore, modularization will be very important, so you must have a clear understanding on how to split some given code into headers, source files, and how to compile objects and then link them using the linker, hopefully through a Makefile.

It is worth mentioning that catching an exception (try and catch blocks) also helps with handling runtime errors and reacting correspondingly. Also, using logging libraries like logger or loguru (python), or spdlog, https://github.com/gabime/spdlog, (c++) allows to print useful logging messages that also helps when you try to understand what is going on with your program.

https://matklad.github.io/2021/05/31/how-to-test.html

8.1. Catch2#

Our goal here is to learn to use catch2 to test a very simple function extracted from their tutorial. Later we will modularize the code to practice that and write a useful Makefile.

8.1.1. Install catch2#

If you don’t have it installed, you can use spack:

spack install catch2

8.1.2. Tutorial example: factorial#

Here we will follow the tutorial , testing a factorial function implementation. To do so, we need, first, to modularize our code:

factorial.h :

#pragma once
int factorial(int n);

factorial.cpp:

  #include "factorial.h"

  int factorial(int number)
  {
      return number <= 1 ? number : factorial(number-1)*number;
  }

main_factorial.cpp

  #include <iostream>
  #include "factorial.h"

  int main(void)
  {
  std::cout << factorial(4) << std::endl;
  return 0;
}

With these three files we have the basic utilities to use the factorial function. To compile, we must run something like

g++ -c factorial.cpp
g++ -c main_factorial.cpp
g++ factorial.o main_factorial.o -o factorial_test.x

and then run as

./factorial_test.x

The compilation can be automated with a Makefile as

all: main_factorial.x

%.x: %.o factorial.o
	g++ $^ -o $@

%.o: %.cpp
	g++ -c $<

clean:
	rm -f *.o *.x

8.1.3. Including a test using catch2#

This is the file example extracted from catch2 tutorial.

#define CATCH_CONFIG_MAIN  // This tells Catch to provide a main() - only do this in one cpp file
#include "catch2/catch_test_macros.hpp"

#include "factorial.h"

TEST_CASE( "Factorials are computed", "[factorial]" ) {
    //REQUIRE( factorial(0) == 1 );
    REQUIRE( factorial(1) == 1 );
    REQUIRE( factorial(2) == 2 );
    REQUIRE( factorial(3) == 6 );
    REQUIRE( factorial(10) == 3628800 );
}

To compile, you need to also link with the corresponging catch2 flags (if you are using spack, do not forget to load catch2, spack load catch2)

g++ -c test_factorial.cpp
g++ -c factorial.cpp
g++ test_factorial.o factorial.o -o test_factorial.x -l Catch2Main -l Catch2
./test_factorial.x

The last two flags, -l Catch2Main -l Catch2, are used to link the program with the catch implementations. Sometimes, for non standard installations, it is useful to configure the paths to find both the includes and libs, and this can be done with the pkg-config utility. For example, to get the include path one can use

$ pkg-config --cflags catch2
-I/usr/local/include # This result can change if catch2 is installed on other systems

or for the libs path

$ pkg-config --libs-only-L catch2
-L/usr/local/lib  # This result can change if catch2 is installed on other systems

So, the compilation line could

g++ $(pkg-config --cflags) $(pkg-config --libs-only-L catch2) test_factorial.o factorial.o -o test_factorial.x -l Catch2Main -l Catch2

Again, it is better to include this in a Makefile:

SHELL:=/bin/bash

all: main_factorial.x

%.x: %.o factorial.o
	g++ $^ -o $@

test_factorial.x: test_factorial.o factorial.o
	g++ $$(pkg-config --cflags) $$(pkg-config --libs-only-L catch2) $^ -o $@ -l Catch2Main -l Catch2

test: test_factorial.x
	./test_factorial.x

%.o: %.cpp
	g++ -c $<

clean:
	rm -f *.o *.x

This way, when you write

make test

it will compile and then run all the test.

Please uncomment the commented line in the test, analyze and fix the factorial function. Also implemente more tests, for large numbers, negative numbers, and so on.

NOTE: if you are using spack, you might need to add more code to your target commands

%.x: %.o factorial.o
    source $$HOME/repos/spack/share/spack/setup-env.sh; \
    spack load catch2; \
    g++ $$(pkg-config --cflags catch2) $^ -o $@

8.2. Google test#

Google test is a famous and advance unit framework that goes well beyond of what is shown here. You are invited to follow the docs to learn more.

8.2.1. Installation#

Again, we will use spack

spack install googletest
mkdir googletest

8.2.2. Example#

This is an example, already modularized.

Factorial and isprime header:

#ifndef GTEST_SAMPLES_SAMPLE1_H_
#define GTEST_SAMPLES_SAMPLE1_H_

// Returns n! (the factorial of n).  For negative n, n! is defined to be 1.
int Factorial(int n);

//// Returns true if and only if n is a prime number.
bool IsPrime(int n);

#endif  // GTEST_SAMPLES_SAMPLE1_H_

Source file

#include "factorial.h"

// Returns n! (the factorial of n).  For negative n, n! is defined to be 1.
    int Factorial(int n) {
    int result = 1;
    for (int i = 1; i <= n; i++) {
        result *= i;
    }

    return result;
  }

// Returns true if and only if n is a prime number.
bool IsPrime(int n) {
    // Trivial case 1: small numbers
    if (n <= 1) return false;

    // Trivial case 2: even numbers
    if (n % 2 == 0) return n == 2;

    // Now, we have that n is odd and n >= 3.

    // Try to divide n by every odd number i, starting from 3
    for (int i = 3; ; i += 2) {
        // We only have to try i up to the square root of n
        if (i > n/i) break;

        // Now, we have i <= n/i < n.
        // If n is divisible by i, n is not prime.
        if (n % i == 0) return false;
    }

    // n has no integer factor in the range (1, n), and thus is prime.
    return true;
}

Test source file (to be compiled as an object)

#include <limits.h>
#include "factorial.h"
#include "gtest/gtest.h"
namespace {
    // Tests factorial of negative numbers.
    TEST(FactorialTest, Negative) {
        // This test is named "Negative", and belongs to the "FactorialTest"
        // test case.
        EXPECT_EQ(1, Factorial(-5));
        EXPECT_EQ(1, Factorial(-1));
        EXPECT_GT(Factorial(-10), 0);
    }
    // Tests factorial of 0.
    TEST(FactorialTest, Zero) {
        EXPECT_EQ(1, Factorial(0));
    }

// Tests factorial of positive numbers.
    TEST(FactorialTest, Positive) {
        EXPECT_EQ(1, Factorial(1));
        EXPECT_EQ(2, Factorial(2));
        EXPECT_EQ(6, Factorial(3));
        EXPECT_EQ(40320, Factorial(8));
    }

    // Tests negative input.
    TEST(IsPrimeTest, Negative) {
        // This test belongs to the IsPrimeTest test case.

        EXPECT_FALSE(IsPrime(-1));
        EXPECT_FALSE(IsPrime(-2));
        EXPECT_FALSE(IsPrime(INT_MIN));
    }

// Tests some trivial cases.
    TEST(IsPrimeTest, Trivial) {
        EXPECT_FALSE(IsPrime(0));
        EXPECT_FALSE(IsPrime(1));
        EXPECT_TRUE(IsPrime(2));
        EXPECT_TRUE(IsPrime(3));
    }

// Tests positive input.
    TEST(IsPrimeTest, Positive) {
        EXPECT_FALSE(IsPrime(4));
        EXPECT_TRUE(IsPrime(5));
        EXPECT_FALSE(IsPrime(6));
        EXPECT_TRUE(IsPrime(23));
    }
}

Main google test file

#include <cstdio>
#include "gtest/gtest.h"

GTEST_API_ int main(int argc, char **argv) {
    printf("Running main() from %s\n", __FILE__);
    testing::InitGoogleTest(&argc, argv);
    return RUN_ALL_TESTS();
}