How to Write Liquibase Integration Tests
- 1 Why do we have Integration Tests?
- 2 What can be tested?
- 3 What is required to make the integration tests work?
- 4 Setup and Run an Integration Test Database
- 5 Types of Integration Tests
The Liquibase code contains both unit tests and integration tests. This document explains the basics for creating Integration tests that will be used by the automated build process to validate Liquibase and reduce regressions.
Why do we have Integration Tests?
The goal of integration tests are to:
Verify that various Liquibase components work in concert with each other correctly (including end-to-end scenarios)
Verify that Liquibase interacts correctly with databases and other external systems
What can be tested?
The overall flow within Liquibase is shown on the left. Each integration test should focus on the relevant component in the flow that has been implemented or changed.
For example, to test that a particular configuration of a CreateTableChange object works, the test would start with the Change object and test the rest of the flow to ensure it correctly creates the table in the datatabase. It shouldn’t start with a changelog file, because that particular test doesn’t care how an xml/json/yaml file is converted into that Change object, it just cares that the Change object itself works.
If you are wanting to test that a particular XML <createTable> tag is being handled correctly, you can create the file in your integration test and run it though the changelog parser and assert that the created CreateTableChange object matches what you expect. Stopping the flow at that point vs. letting it run all the way to the database is generally better because it is the interaction with the database that is slow and not always available, so the more you can focus your test on the parts of the overall flow that you are interested in, the better.
What is required to make the integration tests work?
The integration test framework consists of the following components:
Liquibase Core
The Integration Test
If testing a database, you’ll need to configure a database
Setup and Run an Integration Test Database
Being able to connect to databases is central to running integration tests in Liquibase, but we also recognize that developers rarely have all possible databases running and available in their system. Our integration test system is able to find your available databases and quietly ignore tests when a database is unavailable.
The instructions below explain how to configure an integration test database and make it available to the integration test framework.
Configure Database Connection Files
Update the Primary Configuration file
This is the primary configuration file used by the integration test infrastructure:
file location: liquibase-integration-tests/src/test/resources/liquibase
file name: liquibase.integrationtest.properties
The liqubase.integrationtest.properties file contains the default configuration for the integration test database connections. It is called by the docker-compose.yml and is discussed in more detail in the Docker Databases section below.
The properties file contains the following default information:
integration.test.username=lbuser
integration.test.password=LiquibasePass1and the following database specific information:
DB_NAMEis the relevant database platform name. You can determine DB_NAME from https://www.liquibase.org/databases.htmlusernameis the username required to connect to the integration test databasepasswordis the password required to connect to the integration test databaseurlis the JDBC URL required to connect to the integration test database
integration.test.DB_NAME.username=liquibase
integration.test.DB_NAME.password=liquibase
integration.test.DB_NAME.url=jdbc:localhost:2638?ServiceName=liquibaseOverride Primary Configuration with a Local Configuration file
If you ever need to test against a connection with a different configuration, you can create a local configuration file to provide specific database overrides.
Here are the steps to create and use this file:
Create a
liquibase.integrationtest.local.propertiesfile in the same directory as the primary configuration file (liquibase-integration-tests/src/test/resources/liquibase) described above.Override ONLY the configuration values you need.
For example, if you are running against an oracle database on a different system, you can set only
integration.test.oracle.url=YOUR_URL.Do not check-in the
liquibase.integrationtest.local.propertiesfile into git. It should remain local to your environment.
Setup and Run Docker Databases
To standardize configuration/installation of different databases, we have a https://github.com/liquibase/liquibase/blob/master/liquibase-integration-tests/docker/docker-compose.yml file stored in the liquibase-integration-tests/docker directory. The databases started through this setup will match the default liquibase.integrationtest.properties settings and are used as part of the build server.
We currently only have mysql configured, but will be adding more soon.
To start the test databases:
Install Docker and docker-compose
Startup one or more test databases
To start all databases
cd to the
liquibase-integration-tests/dockerdirectoryrun
docker-compose up
To start one database - in this case, start the mysql database
cd to the
liquibase-integration-tests/dockerdirectoryrun
docker-compose up mysql
Types of Integration Tests
There are 2 high-level types of integration tests in the Liquibase codebase.
Database Integration Tests
The first type validates common/shared database functionality.
The second type validates specific database functionality across supported databases.
Interface/Integration Integration Tests
The following sections discuss the two major types of tests and the additional subsets of tests for databases.
Database Integration Tests
Tests for functionality that is the same across all databases should go in the liquibase-integration-tests/src/test/java/liquibase/dbtest/AbstractIntegrationTest class. This will ensure that the integration test is executed against all database types.
For each database we support, there is also a subclass of AbstractIntegrationTest.java in a sub-package. Here are the locations of the current integration test classes for each supported database.
All integration tests are standard JUnit classes, so new tests are created by adding new methods with the @Test annotation.
Common/Shared Integration Tests
AbstractIntegrationTest.java
The base java file (liquibase-integration-tests/src/test/java/liquibase/dbtest/AbstractIntegrationTest.java) defines the standard tests that get executed against our supported databases.
This is a large test suite that does everything from testing specific methods on the Database interface to running changelogs of different formats through the entire Liquibase process.
Database Platform | Database Type | Integration Test - Main Java Class Location | Test Count |
|---|---|---|---|
All | All | liquibase-integration-tests/src/test/java/liquibase/dbtest/AbstractIntegrationTest.java | 27 |
Example test method:
public abstract class AbstractIntegrationTest {
....
@Test
public void testGenerateChangeLogWithNoChanges() throws Exception {
assumeNotNull(this.getDatabase());
runCompleteChangeLog();
DiffResult diffResult = DiffGeneratorFactory.getInstance().compare(database, database, new CompareControl());
DiffToChangeLog changeLogWriter = new DiffToChangeLog(diffResult, new DiffOutputControl(false, false, false, null));
List<ChangeSet> changeSets = changeLogWriter.generateChangeSets();
assertEquals("generating two change logs without any changes in between should result in an empty generated " +
"differential change set.", 0, changeSets.size());
}
}The database class-level field referenced in the test will be the database under test with the live connection to the database. As each subclass of of AbstractIntegrationTest is ran, it will set up that object based on the information in liquibase.integrationtest.properties. If the connection cannot be made, ALL tests in the subclass will be skipped.
Using that database field, you can interact with any of the Liquibase classes you need to implement your test.
Explanation of a Common/Shared Integration Test
You can see the test method, testRerunDiffChangeLog, that does all those steps at https://github.com/liquibase/liquibase/blob/a930d0241b03602ef1bee30d2bd725bcd77b1665/liquibase-integration-tests/src/test/java/liquibase/dbtest/AbstractIntegrationTest.java#L660
changelogs for all of the integration tests - https://github.com/liquibase/liquibase/blob/master/liquibase-integration-tests/src/test/resources/changelogs
each changeset is an integration test
Example Tests | Liquibase Command | See the example |
|---|---|---|
To run a changelog file, use the | update | |
Snapshot the database after applying the changelog on line 669. | snapshot | |
You can see how we do a diffChangeLog with an empty database to re-create what was in the original changelog on lines 726-734. | diffChangeLog | |
You can see how we execute the generated changelog on lines 736-742. | update | |
You can see how we validate the results by comparing the snapshot from the original update with the one we from the just did from the generated changelog on lines 744-749. | N/A |
Database Specific Integration Tests
IntegrationTest Subclasses
For each database we support, there is also a subclass of AbstractIntegrationTest.java in a sub-package. Here are the locations of the current integration test classes for each supported database.
Liquibase Supported Databases
The purpose of this class is to:
Connect to the database under test
Allow overriding of tests in the AbstractIntegrationTest when a specific database works differently than the standard
Add additional database-specific test cases
Interface/Integration Integration Tests
Maven Integration Tests
The Maven integration tests use the Maven Plugin Testing Harness. The tests themselves are stored in liquibase-maven-plugin/src/test/java/org/liquibase/maven/plugins.
There should be a 1-1 correspondence between the maven plugin classes and the *Test classes for them.
Example Test File
The tests themselves are JUnit tests, so adding additional tests are a matter of adding additional test methods as needed.
Ant Integration Tests
The Ant integration tests use Apache AntUnit. The tests themselves are stored as xml files in liquibase-core/src/test/resources/liquibase/integration/ant .
The database url, username, and password are stored as properties near the top of the test file. They are defaulted to run against against h2 because these tests care more about the Ant integration and not as much about database variations
Example Test File
The available assertions are defined in liquibase-core/src/test/resources/liquibase/integration/ant/test/antlib.xml:
assertTableExists
assertTableDoesntExist
assertColumnExists
assertColumnDoesntExist
assertRowCountEquals