Spring Boot Integration Tests with docker-compose

Hero

In this blog post, I will show you how to run Spring Boot integration tests with docker-compose file.

Previously I have written a blog showing how to do Spring Boot integration testing using Testcontainers. Using Testcontainers for integration testing one of the best approach but it requires at least JDK 1.8 and above to use it.

If you are working on a legacy project which is still using JDK 1.5 or if you are not able to use Testcontainers library due to any other limitations , you can use the docker containers directly.

Note

If you are looking to migrate docker-compose based integration tests to Testcontainers, you can start docker-compose containers with Testcontainers. Please visit below blog post for more details

https://fullstackcode.dev/2022/01/22/integration-testing-with-docker-compose-and-testcontainers/

Setting up the project

Let’s setup a sample project for testing

To create a project, go to https://start.spring.io and select following dependencies.

  • spring-boot-starter-web
  • spring-boot-starter-data-jpa
  • PostgreSQL driver

Download the project, extract and import it into your favorite IDE.

Let’s develop sample REST API for our testing

package dev.fullstackcode.eis.sb.controller;


import dev.fullstackcode.eis.sb.entity.Employee;
import dev.fullstackcode.eis.sb.entity.Gender;
import dev.fullstackcode.eis.sb.service.EmployeeService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.*;

import java.util.List;

@RestController
@RequestMapping("/employee")
public class EmployeeController {

    @Autowired
    EmployeeService employeeService;

    @GetMapping
    public List<Employee> getEmployees() {
        return employeeService.getAllEmployees();
    }

    @GetMapping(value="/{id}")
    public Employee getEmployeeById(@PathVariable Integer id) {
        return employeeService.getEmployeeById(id);
    }

    @PostMapping
    public Employee createEmployee(@RequestBody Employee employee) {
        return employeeService.createEmployee(employee);
    }

    @GetMapping(value="/gender/{gender}")
    public List<Employee> getEmployeesByGender(@PathVariable String gender) {

        return employeeService.findEmployeesByGender(Gender.valueOf(gender));
    }

    @GetMapping(value="/gender2/{gender}")
    public List<Employee> searchEmployeesByGender(@PathVariable String gender) {

        return employeeService.searchEmployeesByGender(Gender.valueOf(gender));
    }

}
Code language: Java (java)

Let’s write a integration test

@SpringBootTest(webEnvironment = RANDOM_PORT)
public class BaseIT {
    @Autowired
    protected TestRestTemplate testRestTemplate ;

}Code language: Java (java)
public class EmployeeControllerIT extends BaseIT {

    @Test
    @Sql({ "/import.sql" })
    public void testCreateEmployee() {

        Department dept = new Department();
        dept.setId(100);

        Employee emp = new Employee();

        emp.setFirst_name("abc");
        emp.setLast_name("xyz");
        emp.setDepartment(dept);
        emp.setBirth_date(LocalDate.of(1980,11,11));
        emp.setHire_date(LocalDate.of(2020,01,01));
        emp.setGender(Gender.F);

        ResponseEntity<Employee> response = testRestTemplate.postForEntity( "/employee", emp, Employee.class);

        Employee employee =  response.getBody();

        assertNotNull(employee.getId());
        assertEquals("abc", employee.getFirst_name());

    }

    @Test
    @Sql({ "/import.sql" })
    public void testGetEmployeeById() {

        ResponseEntity<Employee> response = testRestTemplate.getForEntity( "/employee/{id}",Employee.class,100);
        Employee employee =  response.getBody();

        assertEquals(100,employee.getId());
        assertEquals("Alex", employee.getFirst_name());

    }


}
Code language: Java (java)

Docker Compose

In local development for running integration test against database , I want to use a throw away database instance. Using Docker we can create throw away instance of databases. In this example, I am using PostgreSQL database for testing. You can use this approach for any database.

We can use below command to start the PostgreSQL server using Docker

docker run --name postgresql-container -p 5436:5432 -e POSTGRES_PASSWORD=secret -d postgres:13.2Code language: Shell Session (shell)

Instead of running Docker command every to create a container , I am going to use docker-compose file. In docker-compose file containers are defined as services.

We are going to use following docker-compose file for starting the Postgres database for our testing.

# A Docker Compose must always start with the version tag.
# We use '3.3' because it's the last version.
version: '3.3'

# You should know that Docker Compose works with services.
# 1 service = 1 container.
# For example, a service, a server, a client, a database...
# We use the keyword 'services' to start to create services.
services:
  # The name of our service is "postgres"
  # but you can use the name of your choice.
  # Note: This may change the commands you are going to use a little bit.
  postgres:
    # Official Postgres image from DockerHub
    image: 'postgres:13.2'

    # By default, a Postgres database is running on the 5432 port.
    # If we want to access the database from our computer (outside the container),
    # we must share the port with our computer's port.
    # The syntax is [port we want on our machine]:[port we want to retrieve in the container]
    # Note: You are free to change your computer's port,
    # but take into consideration that it will change the way
    # you are connecting to your database.
    ports:
      - 5436:5432
    networks:
      - app_net

    environment:
      POSTGRES_USER: postgres # The PostgreSQL user (useful to connect to the database)
      POSTGRES_PASSWORD: secret # The PostgreSQL password (useful to connect to the database)
      POSTGRES_DB: eis # The PostgreSQL default database (automatically created at first launch)

networks:
  app_net:
    driver: bridge
Code language: YAML (yaml)

Let’s start the Postgres database server with following command

docker-compose upCode language: Shell Session (shell)
postgres container

The PostgreSQL database is running and it is ready to accept database commands.

Maven Failsafe Plugin

We are going to use the Maven failsafe plugin to run the integration test.

Since integration tests are very time taking we are not going to run them as part of the normal build process.

I am going to use the profile concept of Maven to run the integration tests separately.

Add the following configuration to pom.xml

 <profiles>
        <profile>
            <id>manual-integration-test</id>
            <build>
                <plugins>
                    <plugin>
                        <groupId>org.apache.maven.plugins</groupId>
                        <artifactId>maven-failsafe-plugin</artifactId>
                        <version>3.0.0-M5</version>
                    </plugin>

                </plugins>
            </build>
        </profile>
</profiles>Code language: Java (java)

By default, the Failsafe Plugin will automatically include all test classes with the following wildcard patterns:

  • "**/IT*.java" – includes all of its subdirectories and all Java filenames that start with “IT”.
  • "**/*IT.java" – includes all of its subdirectories and all Java filenames that end with “IT”.
  • "**/*ITCase.java" – includes all of its subdirectories and all Java filenames that end with “ITCase”.

If the test classes do not follow any of these naming conventions, then configure Maven Failsafe Plugin and specify the tests you want to include.



<profiles>
        <profile>
            <id>manual-integration-test</id>
            <build>
                <plugins>
                    <plugin>
                        <groupId>org.apache.maven.plugins</groupId>
                        <artifactId>maven-failsafe-plugin</artifactId>
                        <version>3.0.0-M5</version>
                       <configuration>
                         <includes>
                                <include>**/*<XX>*.java</include>
                        </includes>
                       </configuration>
                    </plugin>
                </plugins>
            </build>
        </profile>
</profiles>

Code language: Java (java)

Now let’s run the integration tests with following command

mvn clean verify -P manual-integration-testCode language: Shell Session (shell)

-P option specifies the profile to run.

Spring Boot Integration Tests with docker-compose
Integration Tests

In above image we can clearly see that, integration tests ran successfully against the database.

But there is one problem, everytime you want to run the integration tests, you need to manually start the PostgresSQL database from docker-compose file and stop when you are done with.

Let’s see is there any way to automate the process of starting the containers before the integration tests run and stop the container once tests are completed.

There are 2 Maven docker compose plugins which helps us to automate the process.

Now lets see how to use them.

Syncdk Docker Compose Plugin

Syncdk Docker Compose Plugin is Maven plugin for running basic docker-compose commands with Maven.

This can be used as part of the Maven lifecycle or as a utility to bring docker-compose commands to your Maven toolkit.

This plugin is designed to be light, fast and with minimum dependencies

Currently implemented docker-compose commands (exposed as Maven goals) are:

  • up – runs docker-compose up
  • down – runs docker-compose down
  • build – runs docker-compose build
  • push – runs docker-compose push
  • pull – runs docker-compose pull
  • stop – runs docker-compose stop
  • restart – runs docker-compose restart

You can configure the Profile like below to start and stop the docker-compose services during integration testing.

<profile>
            <id>integration-test</id>
            <build>
                <plugins>
                    <plugin>
                        <groupId>org.apache.maven.plugins</groupId>
                        <artifactId>maven-failsafe-plugin</artifactId>
                        <version>3.0.0-M5</version>
                    </plugin>
                    <plugin>
                        <groupId>com.dkanejs.maven.plugins</groupId>
                        <artifactId>docker-compose-maven-plugin</artifactId>
                        <version>2.4.0</version>
                        <executions>
                            <execution>
                                <id>up</id>
                                <phase>pre-integration-test</phase>
                                <goals>
                                    <goal>down</goal>
                                    <goal>up</goal>
                                </goals>
                                <configuration>
                                    <composeFile>${project.basedir}/docker-compose.yaml</composeFile>
                                    <detachedMode>true</detachedMode>
                                </configuration>
                            </execution>
                            <execution>
                                <id>down</id>
                                <phase>post-integration-test</phase>
                                <goals>
                                    <goal>down</goal>
                                </goals>
                                <configuration>
                                    <composeFile>${project.basedir}/docker-compose.yaml</composeFile>
                                    <removeVolumes>true</removeVolumes>
                                </configuration>
                            </execution>
                        </executions>
                    </plugin>
                </plugins>
            </build>
        </profile>Code language: HTML, XML (xml)

Now let’s run the integration tests with following command

mvn clean verify -P integration-testCode language: Shell Session (shell)

In below screens, we can see that containers started before integration tests and stopped after integration test.

Note
This project is now archived

br4chu Docker Compose Plugin

It is a Maven plugin that talks to docker-compose command-line interface.

Currently implemented docker-compose commands (exposed as Maven goals) are:

  • up & down
  • start & stop

By default this plugin will start services in pre-integration-test phase and stops services in post-integration-test phase.

If you want to start and stop the containers during other phases, you can refer to sample configuration xml

You can configure the Profile like below to start and stop the docker-compose services during integration testing.

<profile>
            <id>integration-test2</id>
            <build>
                <plugins>
                    <plugin>
                        <groupId>org.apache.maven.plugins</groupId>
                        <artifactId>maven-failsafe-plugin</artifactId>
                        <version>3.0.0-M5</version>
                    </plugin>
                    <plugin>
                        <groupId>io.brachu</groupId>
                        <artifactId>docker-compose-maven-plugin</artifactId>
                        <version>0.9.0</version>
                        <configuration>
                            <projectName>myProject</projectName>
                            <file>./docker-compose.yaml</file>
                        </configuration>
                        <executions>
                            <execution>
                                <goals>
                                    <goal>up</goal>
                                    <goal>down</goal>
                                </goals>
                            </execution>
                        </executions>
                    </plugin>
                </plugins>
            </build>
        </profile>Code language: HTML, XML (xml)

Now let’s run the integration tests with following command

mvn clean verify -P integration-test2Code language: Shell Session (shell)

In below screens, we can see that containers started before integration tests and stopped after integration test.

Exec Maven Plugin

We can also use Maven exec plugin to run the docker compose commands during Maven life cycle.

With exec plugin we can run all the commands that are supported by docker-compose command

You can configure the Profile like below to start and stop the docker-compose services during integration testing.

<profile>
            <id>integration-test3</id>
            <build>
                <plugins>
                    <plugin>
                        <groupId>org.apache.maven.plugins</groupId>
                        <artifactId>maven-failsafe-plugin</artifactId>
                        <version>3.0.0-M5</version>
                    </plugin>
                    <plugin>
                        <groupId>org.codehaus.mojo</groupId>
                        <artifactId>exec-maven-plugin</artifactId>
                        <version>3.0.0</version>
                        <executions>
                            <execution>
                                <id>up</id>
                                <phase>pre-integration-test</phase>
                                <goals>
                                    <goal>exec</goal>
                                </goals>
                                <configuration>
 <!--                                    Specify the directory structure if the docker-compose file is inside the some folder-->
<!--                                    <workingDirectory></workingDirectory>-->
                                    <executable>docker-compose</executable>
                                    <arguments>
                                        <argument>up</argument>
                                        <argument>-d</argument>
                                    </arguments>
                                </configuration>
                            </execution>
                            <execution>
                                <id>down</id>
                                <phase>post-integration-test</phase>
                                <goals>
                                    <goal>exec</goal>
                                </goals>
                                <configuration>
<!--                                    Specify the directory structure if the docker-compose file is inside the some folder-->
<!--                                    <workingDirectory></workingDirectory>-->
                                    <executable>docker-compose</executable>
                                    <arguments>
                                        <argument>down</argument>
                                    </arguments>
                                </configuration>
                            </execution>
                        </executions>
                    </plugin>
                </plugins>
            </build>
        </profile>Code language: HTML, XML (xml)

Now let’s run the integration tests with following command

mvn clean verify -P integration-test3Code language: Shell Session (shell)

In below screens, we can see that containers started before integration tests and stopped after integration test.

Maven Surefire Plugin

In general Maven surefire plugin is used to run the unit tests. But we can configure it to run the integration tests also.

Surefire plugin does not have integration test phase, so we are going to use test-compile phase to start and package phase to stop the containers.

By default, the Surefire Plugin will automatically include all test classes with the following wildcard patterns:

  • "**/Test*.java" – includes all of its subdirectories and all Java filenames that start with “Test”.
  • "**/*Test.java" – includes all of its subdirectories and all Java filenames that end with “Test”.
  • "**/*Tests.java" – includes all of its subdirectories and all Java filenames that end with “Tests”.
  • "**/*TestCase.java" – includes all of its subdirectories and all Java filenames that end with “TestCase”.

As our integration tests use **IT.java convention, we have to explicitly configure the plugin to run those tests.

Syncdk Docker Compose Plugin

Let’s see how to use Syncdk docker compose maven plugin along with Surefire plugin run integration tests.

You can configure the Profile like below to run the integration tests with Maven surefire plugin.

<profile>
            <id>integration-test4</id>
            <build>
                <plugins>
                    <plugin>
                        <groupId>org.apache.maven.plugins</groupId>
                        <artifactId>maven-surefire-plugin</artifactId>
                        <version>3.0.0-M5</version>
                        <configuration>
                            <includes>**/*IT.java</includes>
                        </configuration>
                    </plugin>
                    <plugin>
                        <groupId>com.dkanejs.maven.plugins</groupId>
                        <artifactId>docker-compose-maven-plugin</artifactId>
                        <version>2.4.0</version>
                        <executions>
                            <execution>
                                <id>up</id>
                                <phase>test-compile</phase>
                                <goals>
                                    <goal>up</goal>
                                </goals>
                                <configuration>
                                    <composeFile>${project.basedir}/docker-compose.yaml</composeFile>
                                    <detachedMode>true</detachedMode>
                                </configuration>
                            </execution>
                            <execution>
                                <id>down</id>
                                <phase>package</phase>
                                <goals>
                                    <goal>down</goal>
                                </goals>
                                <configuration>
                                    <composeFile>${project.basedir}/docker-compose.yaml</composeFile>
                                    <removeVolumes>true</removeVolumes>
                                </configuration>
                            </execution>
                        </executions>
                    </plugin>

                </plugins>
            </build>
        </profile>Code language: HTML, XML (xml)

Now let’s run the integration tests with following command

mvn clean verify -P integration-test4Code language: Shell Session (shell)

In below screens, we can see that containers started before test-compile phase and stopped after package phase.

br4chu Docker Compose Plugin

Let’s see how to use br4chu docker compose maven plugin along with Surefire plugin run integration tests.

You can configure the Profile like below to run the integration tests with Maven surefire plugin.

 <profile>
            <id>integration-test5</id>
            <build>
                <plugins>
                    <plugin>
                        <groupId>org.apache.maven.plugins</groupId>
                        <artifactId>maven-surefire-plugin</artifactId>
                        <version>3.0.0-M5</version>
                        <configuration>
                            <includes>**/*IT.java</includes>
                        </configuration>
                    </plugin>

                    <plugin>
                        <groupId>io.brachu</groupId>
                        <artifactId>docker-compose-maven-plugin</artifactId>
                        <version>0.9.0</version>
                        <configuration>
                            <projectName>myProject</projectName>
                            <file>./docker-compose.yaml</file>
                            <removeOrphans>true</removeOrphans>
                        </configuration>
                        <executions>
                            <execution>
                                <id>up</id>
                                <phase>test-compile</phase>
                                <goals>
                                    <goal>up</goal>
                                </goals>
                            </execution>
                            <execution>
                                <id>down</id>
                                <phase>package</phase>
                                <goals>
                                    <goal>down</goal>
                                </goals>
                            </execution>
                        </executions>
                    </plugin>
                </plugins>
            </build>
        </profile>Code language: HTML, XML (xml)

Now let’s run the integration tests with following command.

 mvn clean verify -P integration-test5Code language: Shell Session (shell)

In below screens, we can see that containers started before test-compile phase and stopped after package phase.

Exec Maven Plugin

Let’s see how to use Maven executor plugin to run the docker compose commands during Maven life cycle.

You can configure the Profile like below to run the integration tests with Maven exec plugin.

<profile>
            <id>integration-test6</id>
            <build>
                <plugins>
                    <plugin>
                        <groupId>org.apache.maven.plugins</groupId>
                        <artifactId>maven-surefire-plugin</artifactId>
                        <version>3.0.0-M5</version>
                        <configuration>
                            <includes>**/*IT.java</includes>
                        </configuration>
                    </plugin>
                    <plugin>
                        <groupId>org.codehaus.mojo</groupId>
                        <artifactId>exec-maven-plugin</artifactId>
                        <version>3.0.0</version>
                        <executions>
                            <execution>
                                <id>up</id>
                                <phase>test-compile</phase>
                                <goals>
                                    <goal>exec</goal>
                                </goals>
                                <configuration>
                                    <executable>docker-compose</executable>
                                    <!--                                    Specify the directory structure if the docker-compose file is inside the some folder-->
                                    <!--                                    <workingDirectory></workingDirectory>-->
                                    <arguments>
                                        <argument>up</argument>
                                        <argument>-d</argument>
                                    </arguments>
                                </configuration>
                            </execution>

                            <execution>
                                <id>down</id>
                                <phase>package</phase>
                                <goals>
                                    <goal>exec</goal>
                                </goals>
                                <configuration>
                                    <executable>docker-compose</executable>
                                    <!--                                    Specify the directory structure if the docker-compose file is inside the some folder-->
                                    <!--                                    <workingDirectory></workingDirectory>-->
                                    <arguments>
                                        <argument>down</argument>
                                    </arguments>
                                </configuration>
                            </execution>
                        </executions>
                    </plugin>
                </plugins>
            </build>
        </profile>Code language: HTML, XML (xml)

Now let’s run the integration tests with following command.

 mvn clean verify -P integration-test6Code language: Shell Session (shell)

In below screens, we can see that containers started before test-compile phase and stopped after package phase.

When using Maven surefire plugin to run integration tests one problem I observed is , If build fails during integration test it will not execute other phases of life cycle so docker containers which are started are not stopped automatically.

We can overcome this problem by running maven commands from batch/shell script which handles stopping the containers when build fails.

Windows

call mvn clean
echo Exit Code = %ERRORLEVEL%
if not "%ERRORLEVEL%" == "0" exit /b

call mvn verify -P integration-test4
echo Exit Code = %ERRORLEVEL%
if not "%ERRORLEVEL%" == "0" (
  call docker-compose down
 ) ELSE (
       ECHO "No errors found"
   )
exit /bCode language: Bash (bash)

Linux

#!/bin/sh
mvn clean verify -P integration-test4
rc=$?
if [ $rc -ne 0 ] ; then
  echo Errors found during mvn clean verify, exit code [$rc];
  docker-compose down
fiCode language: Bash (bash)

Let’s run the script

./mvnverify.batCode language: Shell Session (shell)

In below image you can observe that containers are stopped even when build failed.

Maven Multi Module Project

If you are using Maven multi module project and if you configure the pom.xml to run the integration tests like above, containers will be start and stop for every child project even if they do not have any integration test.

We can overcome the problem by configuring pom.xml file like below.

Create a profile in parent pom.xml file.

 <profiles>
        <profile>
            <id>integration-test</id>
            <build>
                <plugins>
                    <plugin>
                        <groupId>org.apache.maven.plugins</groupId>
                        <artifactId>maven-failsafe-plugin</artifactId>
                        <version>3.0.0-M5</version>
                    </plugin>

                </plugins>
            </build>
        </profile>
</profiles>Code language: HTML, XML (xml)

In the child project which has integration test, define same profile but configure one of the plugin to start and stop the containers.

<profile>
            <id>integration-test</id>
            <build>
                <plugins>                   
                    <plugin>
                        <groupId>com.dkanejs.maven.plugins</groupId>
                        <artifactId>docker-compose-maven-plugin</artifactId>
                        <version>2.4.0</version>
                        <executions>
                            <execution>
                                <id>up</id>
                                <phase>pre-integration-test</phase>
                                <goals>
                                    <goal>down</goal>
                                    <goal>up</goal>
                                </goals>
                                <configuration>
                                    <composeFile>${project.basedir}/docker-compose.yaml</composeFile>
                                    <detachedMode>true</detachedMode>
                                </configuration>
                            </execution>
                            <execution>
                                <id>down</id>
                                <phase>post-integration-test</phase>
                                <goals>
                                    <goal>down</goal>
                                </goals>
                                <configuration>
                                    <composeFile>../docker-compose.yaml</composeFile>
                                    <removeVolumes>true</removeVolumes>
                                </configuration>
                            </execution>
                        </executions>
                    </plugin>
                </plugins>
            </build>
        </profile>Code language: HTML, XML (xml)

You can download entire source code for this blog from GitHub

You can also start docker compose containers with Testcontainers. You can find more details in below article.

Reference

You Never Know: Executing multiple mvn commands from a Windows bat file

Similar Posts