previous episode

CESSDA Infrastructure Training

lesson home

Using automation to build, test and deploy our code

Overview

Teaching: 20 min
Exercises: 60 min
Questions

  • How do we use CESSDA’s infrastructure to automatically build, test and deploy our code?

Objectives

  • Understand the benefits of automation in software development.

  • Use CESSDA’s Jenkins Continuous Integration server to build and test exemplar code.

  • Write unit tests for exemplar code.

  • Contribute unit tests upstream to official repository.

Building on the code we’ve written and placed into Git, in this episode we will see how CESSDA’s infrastructure can help us to to automatically build, test, and deploy the code we’ve written.

Automation in software development

Automating as much as you can in software development is a best practice goal for a number of reasons. A good rule of thumb is if you find yourself doing something manually more than once than can be automated, strongly consider automating it.

Question: What are the benefits of automation?

What do you think are the main benefits of automation?

Some general benefits include:

The main steps, as a ‘holy grail’ of what to achieve, are:

  1. Provide automated build process: far easier and quicker to validate changes, e.g. use of Make, Ant, Maven.
  2. Provide a set (or suite) of unit tests: to check if changes to your code break anything, e.g. JUnit, CPPUnit, xUnit, fUnit, …
  3. Join together - automated build & test: a ‘fail fast’ environment for community development.
  4. Use Continuous Integration (CI): automate the building, testing, and even deployment, of your code as changes are made.

How CI works in practice is that:

  1. You make and commit code changes to your repository.
  2. A CI server is configured to notice these commits and independently checks out your code from revision control and performs a number of predefined steps automatically (like build, test, deploy).
  3. Presents you with a report on the progress, along with success or failure, of these tasks as well as any errors encountered.

This is why continuous integration helps your software to always be releasable: tests are run in response to changes to the code, and you are notified quickly when tests fail so that you can correct the reason for the failure. It is easier to fix a bug in something you wrote a few minutes ago, than something you wrote yesterday (or last week, or last month).

For getting your software accepted by CESSDA, you’re expected to use CI from the start of the development process. This means you’re already thinking about and implementing code quality from the outset. This also means CESSDA can inspect test results, as everything is already within the CESSDA environment.

Jenkins within CESSDA

Jenkins is a popular, extensible, open source continuous integration server. By associating a Jenkins ‘job’ with a code repository, it can automatically build, test, and even deploy, your software in the background.

This is not intended to be a complete introduction to everything that Jenkins offers. Rather, it is hoped that it will serve to get you started, helping you through your first steps of using Jenkins, to give an idea as to its potential and usefulness and how you can use it within CESSDA.

There are many ways to use Jenkins to do things like build, test and deploy the code in your repository. For instance, you can configure Jenkins to run a specific command, e.g. ant to run the Ant commands present in a build.xml file.

Within CESSDA, a Jenkins job is preconfigured for your repository when the repository is created, and it helpfully uses a more modern Jenkins approach, which gives you more flexibility on what is done by the Jenkins job. This approach is achieved via a Jenkins Pipeline, where you define the commands you’d like Jenkins to run within a Jenkinsfile. You can either define these commands in one of two ways:

For the purposes of this training, we’ll be concentrating on the declarative approach to build and test our code. But this should give you a solid foundation for exploring the more advanced features offered by using a scripted pipeline.

Using Jenkins to build and test our code

So a Jenkinsfile isn’t used as a replacement to how we build and test our code, but rather it invokes a suitable build system underneath to do these things. For our example code, we used Ant, so we need to define these actions within our Jenkinsfile.

A Jenkinsfile is structured around a pipeline, which can include:

Let’s start by looking at the Jenkinsfile we have in the root directory of our repository:

$ cat Jenkinsfile

pipeline {
    agent any

    stages {
        stage('Build') {
            steps {
                echo 'Building..'
                sh 'mvn clean compile'
            }
        }
        stage('Test') {
            steps {
                echo 'Testing..'
                sh 'mvn test'
            }
        }
    }
}

For simplicity, here we’re specifying that we don’t mind which type of agent will run our Jenkins job. Within the stages, our ‘Build’ stage simply instructs Maven to build (compile) our Java code, and our ‘Test’ stage invokes Maven to run our tests.

Jenkins - a brief tour

So in the background, pull requests that were merged into the official repository started off a new Jenkins job within CESSDA’s Jenkins infrastructure. The Jenkins job has cloned the official repository following that change to it, and run the Jenkins job based on the pipeline specified in our Jenkinsfile.

Let’s take a look at the results of the latest Jenkins job now, by going to the official repo at https://bitbucket.org/cessda/cessda.ces2018.test, and selecting ‘Branches’ from the navigation bar.

We’ll see our two branches Master and Develop, but also note the icons in the ‘Builds’ column. Jenkins is running a build job against both of these branches in the background, and these results are linked to back here. You’ll notice our Master branch (which still contains our faulty Fibonacci code) has a failed build, as we would expect, but the Develop branch now has a successful build!

The successful Jenkins build

Let’s look at the successful build first by clicking on that green icon, where we’ll be taken to the results of that Jenkins job. We can see that the stages in our Jenkinsfile pipeline, ‘Build’ and ‘Test’, have both been successful. Below, we have access to more details.

Select ‘Build’ from the pipeline, and we can see results for the following successes:

By inspecting each, we can see the console output from each of the steps.

Our Jenkins job also runs the ‘Test’ stage. It turns out that we already have a test in our repository for our code, that we’ll look at later. So similarly, if we select ‘Test’ from the pipeline at the top, we can see results for:

The failed Jenkins build

If we now select the failed Jenkins build from the official repository branches page, it’s a similar but different story. We can see that whilst the ‘Build’ stage was successful, the ‘Test’ one failed, showing the console error we would expect in the console output.

At least our Develop branch was successful. But of course, we only have one test. Is this enough?

Unit testing

Confess!

Why don’t you write tests?

  • “I don’t write buggy code”
  • “It’s too hard”
  • “It’s not interesting”
  • “It takes too much time and I’ve research to do”

What testing gives you:

Wise words

“If it’s not tested, it’s broken” - bittermanandy, 10/09/2010

Examples of unit testing frameworks:

Most people don’t enjoy writing tests, so if we want them to actually do it, it must be easy to:

Test results must also be reliable. If a testing tool says that code is working when it’s not, or reports problems when there actually aren’t any, people will lose faith in it and stop using it.

The simplest kind of test is a unit test that checks the behavior of one component of a program. We still have to decide what to test and how many tests to run. Our best guide here is economics: we want the tests that are most likely to give us useful information that we don’t already have. Now, we should try to choose tests that are as different from each other as possible, so that we force the code we’re testing to execute in all the different ways it can - to ensure our tests have a high degree of code coverage.

Testing our code

This repository already has a test we can run that uses the jUnit unit testing framework for Java, which checks it works correctly for a single test case. In an editor, such as Nano, open src/test/java/math/FibonacciTest.java.

$ nano src/test/java/math/FibonacciTest.java

You should see:

package math;

// Copyright 2014 The University of Edinburgh.
// ...

import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import static org.junit.Assert.*;

import math.Fibonacci;

/**
 * Test class for Fibonacci class.
 */
public class FibonacciTest
{
    /** Initialise test suite - no-op. */
    @Before
    public void setUp()
    {
    }

    /** Clean up test suite - no-op. */
    @After
    public void tearDown()
    {
    }

    /** Test fib(1). */
    @Test
    public void testFib1()
    {
        assertEquals(1, Fibonacci.fib(1));
    }
}

So we can see we have a single test that checks that an argument of 1 to Fibonacci yields an answer of 1, otherwise the test will fail. So we can use this test to verify the fix we made earlier.

Notice that there are setup() and tearDown() functions, which are optionally used to setup and undo any preparation required to run any of the tests, although we don’t need them here. The prefix ‘test’ for test methods and ‘Test’ for the test class is expected convention for jUnit. jUnit will automatically run any tests marked with the @Test decorator.

Ordinarily, code should have a suite of tests which test its correct operation more completely - we’ll look into adding some more shortly.

We can build and run our test using Maven:

$ mvn test

You should see, embedded in the output:

...
-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running math.FibonacciTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.077 sec

Results :

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
...

So we get a nice summary of what happened with our tests, and that the fix to the problem was successful (as reflected in the Jenkins build on the Develop branch).

Adding unit tests to our software

Given what we’ll learnt do far, let’s put it all together and write some of our own unit tests for Fibonacci.

Exercise: Write your own unit tests

Consider what other ways the code should be tested, and write 2-3 jUnit tests, adding to the src/test/java/math/FibonacciTest class, to check your code handles them correctly. Again, build and run your tests using mvn test. Whilst writing these, also create a test that deliberately fails so you can see what happens to the results.

Committing our tests

Now we should add commit our tests to our forked repository as before. We can check our changes, as before:

$ git status

On branch develop
Your branch is up-to-date with 'origin/develop'.
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

	modified:   src/test/java/math/FibonacciTest.java

no changes added to commit (use "git add" and/or "git commit -a")

Then add and commit them:

$ git add src/test/java/math/Fibonacci.java
$ git commit src/test/java/math/Fibonacci.java

Adding in a suitable commit message.

Add test for second Fibonacci number

To more completely test the second fib() clause

Save and exit the editor, and then see this commit in our local repository log:

$ git log

commit 3b7a62d27da0876e723192be15f8b9767db09cc7
Author: bitbucket_user <bitbucket_user_email_address>
Date:   Mon Sep 24 09:47:12 2018 +0100

    Add test for second Fibonacci number

    To more completely test the second fib() clause

commit a87f5f901c4dcc3938a2ec62a54e5a389ce2fcf1

We’ll submit a pull request - along with some other commits - later.

Adding unit tests to our software (with TDD)

Unit tests are actually such a good way to define how functions ought to behave that many programmers use a practice called test-driven development (TDD). Instead of writing code, then figuring out how to test it, these programmers:

The mantra often used during TDD is “red, green, refactor”:

TDD’s proponents argue that it helps people produce better code for two reasons:

Exercise: Implement a feature using TDD

Think of a new feature you would like to add to your code, and write the unit tests for it first. Then implement the feature in the code, and rerun the tests. Once successful, refactor your code as necessary, making it more readable and commented for other developers (including yourself) - and ensure your tests still pass after refactoring!

Question: to TDD or not to TDD?

Which approach to writing unit tests did you prefer? Using TDD or not using it, and why?

Now we should add and commit our new tests to our local repository as before (see steps above).

Push changes to fork and create Pull Request

We need to push these changes to our forked repository on Bitbucket, as we did in the previous lesson:

$ git push

Then, as before, we need to create a pull request back to the official repository via the Bitbucket interface.

Select ‘Pull requests’ from the navigation bar, and select ‘Create pull request’. What we want is to submit pull requests that will merge with the Develop branch on the official repository (since that’s what we’ve committed to). Importantly, you’ll see at the top of the page on the right, that the pull request, by default, will attempt to create a pull request for the Master branch on the official repository, so change the ‘master’ branch to ‘develop’. Also make sure that the destination repository is set to ‘cessda/cessda.ces2018.test’.

Add “WIP” to the beginning of the pull request’s title, e.g. “WIP Fix issue with incorrect Fibonacci result”. When ready, select ‘Create pull request’ at the bottom right. At some point, the maintainer will go through a list of outstanding pull requests and either merge them into the official repository or decline them.

The repository maintainer now has merged two pull requests into the official repository’s Develop branch:

So what the maintainer can do now, when ready (i.e. for an upcoming release), merge them into the Master branch.

Key Points

  • Automation benefits developers, CESSDA, and anyone that uses or develops your code.

  • Use CESSDA’s Jenkins CI infrastructure when developing for CESSDA.

  • Write unit tests for your code contributions where possible.

previous episode

lesson home