Spring Boot integration testing with Testcontainers

Springboot provides excellent support for unit testing and integration testing. I have covered how to do unit testing for Springboot controllers in various ways in one of my previous articles. While doing unit testing we try to test a single method while mocking all the dependencies. While doing integration testing we see that interaction between different methods and as well as different layers of the application (Controllers, Services and Repositories) is working as expected.

One of the hardest part while doing integration testing is simulating interactions with third-party applications like Databases, HTTP Servers and Caches. Imagine that we are using the `PostgreSQL’ database in our Springboot application, while running integration tests in CI/CD pipeline in general we use in-memory databases like ‘H2’ and execute our queries against it. But using the H2 database has some drawbacks.

• As H2 is not a production database it does not give high confidence
• If we have written any native queries using production database features we can not test them.

We can overcome above problems by using Testcontainers java library.

As per official documentation

Testcontainers make the following kinds of tests easier:

• Data access layer integration tests
• Application integration tests
• UI/Acceptance tests

Testconatiners provides the GenericContainer class which you can use to create containers from Docker images. When using Generic containers developers are responsible for maintaining the life cycle like start and stopping the container.

Testcontainer also provides Specific container classes for modules like Databases, Kafka, RabbitMQ, Nginx. It has supporting modules for all the major SQL and NoSQL databases.

In this blog post, I will demonstrate how we can simplify the integration testing when the database layer is involved using Testcontainers.

Required Software

• JDK 1.8+
• JUnit 5
• Docker
• Springboot

Note : Testconatiners also supports Junit 4.

Adding Testcontainers dependency

We can add Testcontainer components in pom.xml file in 2 ways.

i) Using BOM to add dependencies in pom.xml

A major advantage of using BOM is you can avoid specifying the version of each dependency

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
	......
	<properties>
		<java.version>11</java.version>
		<testcontainers.version>1.16.2</testcontainers.version>
	</properties>
	<dependencies>
		
		.....
		<dependency>
			<groupId>org.testcontainers</groupId>
			<artifactId>junit-jupiter</artifactId>
			<scope>test</scope>
		</dependency>
		<dependency>
			<groupId>org.testcontainers</groupId>
			<artifactId>postgresql</artifactId>
			<scope>test</scope>
		</dependency>
	</dependencies>
	<dependencyManagement>
		<dependencies>
			<dependency>
				<groupId>org.testcontainers</groupId>
				<artifactId>testcontainers-bom</artifactId>
				<version>${testcontainers.version}</version>
				<type>pom</type>
				<scope>import</scope>
			</dependency>
		</dependencies>
	</dependencyManagement>

	....

</project>Code language: Java (java)

ii) Adding dependencies directly

<dependency>
    <groupId>org.testcontainers</groupId>
    <artifactId>junit-jupiter</artifactId>
    <version>1.16.2</version>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>org.testcontainers</groupId>
    <artifactId>testcontainers</artifactId>
    <version>1.16.2</version>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>org.testcontainers</groupId>
    <artifactId>postgresql</artifactId>
    <version>1.16.2</version>
    <scope>test</scope>
</dependency>Code language: Java (java)

As we are using PostgreSQl in our application, we have added corresponding jdbc driver dependency in pom file.

<dependency>
	<groupId>org.postgresql</groupId>
	<artifactId>postgresql</artifactId>
	<scope>runtime</scope>
</dependency>Code language: Java (java)

Using Testcontainers in integration tests

We can start containers using Testcontainers in 3 ways.

1. Using Special JDBC URL ( This is specific to Database containers)
2. Auto handling containers using @Container (Junit 5) or @ClassRule (Junit4) annotations
3. Manual container starting

Using Special JDBC URL

This is specific to Database containers. If you are only dealing with database container you can use this method. If you are using other containers also then it is better use second or third method.

Write your regular integration test cases.

@SpringBootTest(webEnvironment = WebEnvironment.RANDOM_PORT)
@Testcontainers
public  class BaseIT {
	
	@Autowired
	protected TestRestTemplate testRestTemplate ;

}Code language: Java (java)

public class DepartmentControllerIT extends BaseIT {



    @Test
    @Sql({ "/import.sql" })
    public void testGetDepartmentById() {

        ResponseEntity<Department> response = testRestTemplate.getForEntity( "/department/{id}",Department.class,100);
        Department dept =  response.getBody();

        assertEquals(100,dept.getId());
        assertEquals("HR", dept.getName());

    }

}Code language: Java (java)

public class EmployeeControllerIT extends BaseIT {

    @Test
    @Sql({ "/import.sql" })
    public void testCreateEmployee() {

        Department dept = new Department();
        dept.setId(100);

        Employee emp = new Employee();

        emp.setFirst_name("abc");
        emp.setLast_name("xyz");
        emp.setDepartment(dept);
        emp.setBirth_date(LocalDate.of(1980,11,11));
        emp.setHire_date(LocalDate.of(2020,01,01));
        emp.setGender(Gender.F);

        ResponseEntity<Employee> response = testRestTemplate.postForEntity( "/employee", emp, Employee.class);

        Employee employee =  response.getBody();

        assertNotNull(employee.getId());
        assertEquals("abc", employee.getFirst_name());

    }

    @Test
    @Sql({ "/import.sql" })
    public void testGetEmployeeById() {

        ResponseEntity<Employee> response = testRestTemplate.getForEntity( "/employee/{id}",Employee.class,100);
        Employee employee =  response.getBody();

        assertEquals(100,employee.getId());
        assertEquals("Alex", employee.getFirst_name());

    }


}Code language: Java (java)

You can instantiate database containers using special JDBC url without touching any of the code.

In SpringBoot application with Postgres database , You need to add following properties to application.properties of test resources.

In spring.datasource.url observe that we have inserted special tc string in regular URL which hints Testcontainers to start the database container when ever application starts.

spring.datasource.url=jdbc:tc:postgresql:13.2:////<DatabaseName>
spring.datasource.driver-class-name=org.testcontainers.jdbc.ContainerDatabaseDriverCode language: Java (java)

If you want to run the init script while starting db containers, you can add them in the url

If the script is in class path

spring.datasource.url=jdbc:tc:postgresql:13.2:////<DatabaseName>?TC_INITSCRIPT=somepath/init_script.sqlCode language: Properties (properties)

If the script is in the file path

spring.datasource.url=jdbc:tc:postgresql:13.2:////<DatabaseName>?TC_INITSCRIPT=file:src/main/resources/init_script.sqlCode language: Properties (properties)

For MySQL you can use below url

jdbc:tc:mysql:5.7.34:///databasenameCode language: Java (java)

Now lets run the integration test with following command

mvn clean verifyCode language: Shell Session (shell)

You can down sample application for this approach from GitHub

Using @Container

For using Testcontainers , you need to annotate test class with @Testcontainers.

Add the following to the body of our test class:

@Container
public PostgreSQLContainer<?> postgresDB = new PostgreSQLContainer<>("postgres:13.2").withDatabaseName("eis");Code language: Java (java)

If you are using Junit 4

@ClassRule
public PostgreSQLContainer<?> postgresDB = new PostgreSQLContainer<>("postgres:13.2").withDatabaseName("eis");Code language: Java (java)

The @Container / @ClassRule annotation tells JUnit to notify this field about various events in the test lifecycle.

Adding above code snippet to every test class in your project is a tedious work.So it is good practice to develop a base test class which contains all the reusable code across the integration tests.

@SpringBootTest(webEnvironment = WebEnvironment.RANDOM_PORT)
@Testcontainers
@DirtiesContext
public  class BaseIT {
	
	@Autowired
	protected TestRestTemplate testRestTemplate ;

	@Container
	public static PostgreSQLContainer<?> postgresDB = new PostgreSQLContainer<>                
                                                             ("postgres:13.2")
			                                     .withDatabaseName("eis")												                        
                                                             .withUsername("postgres")
                                                             .withPassword("postgres")
			                                     .withInitScript("ddl.sql");


	@DynamicPropertySource
	public static void properties(DynamicPropertyRegistry registry) {
		registry.add("spring.datasource.url",postgresDB::getJdbcUrl);
		registry.add("spring.datasource.username", postgresDB::getUsername);
		registry.add("spring.datasource.password", postgresDB::getPassword);

	}


}Code language: Java (java)

We are going to use @SpringBootTest and TestRestTemplate  combination to simulate HTTP calls.

Now let’s dive into the code

@SpringBootTest – The @SpringBootTest annotation tells Spring Boot to look for the main configuration class (one with @SpringBootApplication) and use that to start a Spring application context.  webEnvironment = WebEnvironment.RANDOM_PORT parameter starts the actual HTTP web server at a random available port.

@Testcontainers – This extension provides Junit integration. The extension finds all fields that are annotated with @Container and calls their container lifecycle methods during test execution.

@DirtiesContext – Ensures that each subclass gets its own ApplicationContext with the correct dynamic properties.

TestRestTemplate – This class offers templates for common scenarios by the HTTP method to prepare web requests. If you are using the @SpringBootTest annotation with an embedded server,  TestRestTemplate class is automatically available and can be @Autowired into test

@Container – This annotation is used to mark containers that should be managed by the Testcontainers extension

Then we have declared `PostgreSQLContainer` as static field

Containers declared as static fields will be shared between test methods. They will be started only once before any test method is executed and stopped after the last test method has executed. This will reduce the test execution time drastically, but we need to take care of data isolation for different tests.

Containers declared as instance fields will be started and stopped for every test method. This will increase the test execution time drastically.

PostgreSQLContainer constructor takes the docker image name as a parameter to spin up the database. You can specify the version of DB you want to use(eg: "postgres:13.2" ). You can get available image names from the docker hub. If you want to use the latest available version always you can use the PostgreSQLContainer.IMAGE parameter.

withDatabaseName() – specify the database name

withUsername() – specify the username of database

withPassword()-specify the password of database

withInitScript() method can be used to run the database initialization scripts such as schema creation and sequence creations. In our sample application, I have used this method to create hibernate_sequence.

Note : Since I am using spring.jpa.hibernate.ddl-auto=create-drop property in the sample application, Schema is generated on the fly. So I have not included schema in the init script.

Since container spinup database dynamically, we need to get hold of JDBC url, username, and password of the database and add it to the environment variables.

We use @DynamicPropertySource annotation to get hold of required database properties and add them to environment variables.

For more information on you can refer to the spring documentation here.

If you use the Junit5 and Springboot version < 2.2.6, you can use following code

@SpringBootTest(webEnvironment = WebEnvironment.RANDOM_PORT)
@Testcontainers
@ContextConfiguration(initializers = BaseIT2.TestEnvInitializer.class)
@DirtiesContext
public class BaseIT2 {

    @Autowired
    protected TestRestTemplate testRestTemplate ;


    @Container
    private static PostgreSQLContainer<?> postgresDB = new PostgreSQLContainer<>  
                                                             ("postgres:13.2")
                                                   .withDatabaseName("testdb")
                                                     .withUsername("postgres")
                                                     .withPassword("postgres")
                                                   .withInitScript("ddl.sql");



    static class TestEnvInitializer implements ApplicationContextInitializer<ConfigurableApplicationContext> {

        @Override
        public void initialize(ConfigurableApplicationContext applicationContext) {
            TestPropertyValues values = TestPropertyValues.of(
                    "spring.datasource.url=" + postgresDB.getJdbcUrl(),
                    "spring.datasource.password=" + postgresDB.getPassword(),
                    "spring.datasource.username=" + postgresDB.getUsername()
            );
            values.applyTo(applicationContext);

        }

    }

}Code language: Java (java)

We use ApplicationContextInitializer class to add database properties to environment variables.

Using @Container annotation spins new container for each test class and handles start and stop life cycle automatically.

Since this method starts and stops container for each test class running tests will take more time .

You can down sample application for this approach from GitHub

Manual container starting

In this method we start the container manually and Testcontainers takes care of stopping the container.

In this method we use singleton pattern to share the container across the test cases. This method is very efficient for running large no of test classes but we need to take care to clear the data between testcases.

@SpringBootTest(webEnvironment = WebEnvironment.RANDOM_PORT)
@Testcontainers
@DirtiesContext
public  class BaseIT {
	
	@Autowired
	protected TestRestTemplate testRestTemplate ;

	public static PostgreSQLContainer<?> postgresDB;

	static {
	 postgresDB = new PostgreSQLContainer<>("postgres:13.2")
			.withDatabaseName("eis");

	 postgresDB.start();
	}

	@DynamicPropertySource
	public static void properties(DynamicPropertyRegistry registry) {
		registry.add("spring.datasource.url",postgresDB::getJdbcUrl);
		registry.add("spring.datasource.username", postgresDB::getUsername);
		registry.add("spring.datasource.password", postgresDB::getPassword);

	}


}Code language: Java (java)

In this method we use the static block to define start the container so that it will be available at the start of application.

You can down sample application for this approach from GitHub

You might be also interested in