How to Write Liquibase Integration Tests

The Liquibase code contains both unit tests and integration tests. This document explains the basics for creating Integration tests that will be used by the automated build process to validate Liquibase and reduce regressions.

Why do we have Integration Tests?

The goal of integration tests are to:

  • Verify that various Liquibase components work in concert with each other correctly (including end-to-end scenarios)

  • Verify that Liquibase interacts correctly with databases and other external systems

What can be tested?

The overall flow within Liquibase is shown on the left. Each integration test should focus on the relevant component in the flow that has been implemented or changed.

For example, to test that a particular configuration of a CreateTableChange object works, the test would start with the Change object and test the rest of the flow to ensure it correctly creates the table in the datatabase. It shouldn’t start with a changelog file, because that particular test doesn’t care how an xml/json/yaml file is converted into that Change object, it just cares that the Change object itself works.

If you are wanting to test that a particular XML <createTable> tag is being handled correctly, you can create the file in your integration test and run it though the changelog parser and assert that the created CreateTableChange object matches what you expect. Stopping the flow at that point vs. letting it run all the way to the database is generally better because it is the interaction with the database that is slow and not always available, so the more you can focus your test on the parts of the overall flow that you are interested in, the better.

What is required to make the integration tests work?

The integration test framework consists of the following components:

Setup and Run an Integration Test Database

Being able to connect to databases is central to running integration tests in Liquibase, but we also recognize that developers rarely have all possible databases running and available in their system. Our integration test system is able to find your available databases and quietly ignore tests when a database is unavailable.

The instructions below explain how to configure an integration test database and make it available to the integration test framework.

Configure Database Connection Files

Update the Primary Configuration file

This is the primary configuration file used by the integration test infrastructure:

The liqubase.integrationtest.properties file contains the default configuration for the integration test database connections. It is called by the docker-compose.yml and is discussed in more detail in the Docker Databases section below.

The properties file contains the following default information:

1 2 integration.test.username=lbuser integration.test.password=LiquibasePass1

and the following database specific information:

  • DB_NAME is the relevant database platform name. You can determine DB_NAME from https://www.liquibase.org/databases.html

  • username is the username required to connect to the integration test database

  • password is the password required to connect to the integration test database

  • url is the JDBC URL required to connect to the integration test database

1 2 3 integration.test.DB_NAME.username=liquibase integration.test.DB_NAME.password=liquibase integration.test.DB_NAME.url=jdbc:localhost:2638?ServiceName=liquibase

Override Primary Configuration with a Local Configuration file

If you ever need to test against a connection with a different configuration, you can create a local configuration file to provide specific database overrides.

Here are the steps to create and use this file:

  1. Create a liquibase.integrationtest.local.properties file in the same directory as the primary configuration file (liquibase-integration-tests/src/test/resources/liquibase) described above.

  2. Override ONLY the configuration values you need.

    1. For example, if you are running against an oracle database on a different system, you can set only integration.test.oracle.url=YOUR_URL.

    2. Do not check-in the liquibase.integrationtest.local.properties file into git. It should remain local to your environment.

Setup and Run Docker Databases

To standardize configuration/installation of different databases, we have a https://github.com/liquibase/liquibase/blob/master/liquibase-integration-tests/docker/docker-compose.yml file stored in the liquibase-integration-tests/docker directory. The databases started through this setup will match the default liquibase.integrationtest.properties settings and are used as part of the build server.

We currently only have mysql configured, but will be adding more soon.

To start the test databases:

  1. Install Docker and docker-compose

  2. Startup one or more test databases

    1. To start all databases

      1. cd to the liquibase-integration-tests/docker directory

      2. run docker-compose up

    2. To start one database - in this case, start the mysql database

      1. cd to the liquibase-integration-tests/docker directory

      2. run docker-compose up mysql

Types of Integration Tests

There are 2 high-level types of integration tests in the Liquibase codebase.

  • Database Integration Tests

    • The first type validates common/shared database functionality.

    • The second type validates specific database functionality across supported databases.

  • Interface/Integration Integration Tests

The following sections discuss the two major types of tests and the additional subsets of tests for databases.

Database Integration Tests

Tests for functionality that is the same across all databases should go in the liquibase-integration-tests/src/test/java/liquibase/dbtest/AbstractIntegrationTest class. This will ensure that the integration test is executed against all database types.

For each database we support, there is also a subclass of AbstractIntegrationTest.java in a sub-package. Here are the locations of the current integration test classes for each supported database.

All integration tests are standard JUnit classes, so new tests are created by adding new methods with the @Test annotation.

Common/Shared Integration Tests

AbstractIntegrationTest.java

The base java file (liquibase-integration-tests/src/test/java/liquibase/dbtest/AbstractIntegrationTest.java) defines the standard tests that get executed against our supported databases.

This is a large test suite that does everything from testing specific methods on the Database interface to running changelogs of different formats through the entire Liquibase process.

Database Platform

Database Type

Integration Test - Main Java Class Location

Test Count

Database Platform

Database Type

Integration Test - Main Java Class Location

Test Count

All

All

liquibase-integration-tests/src/test/java/liquibase/dbtest/AbstractIntegrationTest.java

27

Example test method:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 public abstract class AbstractIntegrationTest { .... @Test public void testGenerateChangeLogWithNoChanges() throws Exception { assumeNotNull(this.getDatabase()); runCompleteChangeLog(); DiffResult diffResult = DiffGeneratorFactory.getInstance().compare(database, database, new CompareControl()); DiffToChangeLog changeLogWriter = new DiffToChangeLog(diffResult, new DiffOutputControl(false, false, false, null)); List<ChangeSet> changeSets = changeLogWriter.generateChangeSets(); assertEquals("generating two change logs without any changes in between should result in an empty generated " + "differential change set.", 0, changeSets.size()); } }

The database class-level field referenced in the test will be the database under test with the live connection to the database. As each subclass of of AbstractIntegrationTest is ran, it will set up that object based on the information in liquibase.integrationtest.properties. If the connection cannot be made, ALL tests in the subclass will be skipped.

Using that database field, you can interact with any of the Liquibase classes you need to implement your test.

Explanation of a Common/Shared Integration Test

You can see the test method, testRerunDiffChangeLog, that does all those steps at https://github.com/liquibase/liquibase/blob/a930d0241b03602ef1bee30d2bd725bcd77b1665/liquibase-integration-tests/src/test/java/liquibase/dbtest/AbstractIntegrationTest.java#L660

changelogs for all of the integration tests - https://github.com/liquibase/liquibase/blob/master/liquibase-integration-tests/src/test/resources/changelogs

https://github.com/liquibase/liquibase/blob/master/liquibase-integration-tests/src/test/resources/changelogs/mysql/complete/root.changelog.xml

each changeset is an integration test

https://github.com/liquibase/liquibase/blob/a930d0241b03602ef1bee30d2bd725bcd77b1665/liquibase-integration-tests/src/test/resources/changelogs/common/common.tests.changelog.xml

Example Tests

Liquibase Command

See the example

Example Tests

Liquibase Command

See the example

To run a changelog file, use the runChangeLogFile() command - this changelog can be in any changelog format on line 365.

update

AbstractIntegrationTest.java#L365

Snapshot the database after applying the changelog on line 669.

snapshot

AbstractIntegrationTest.java#L669

You can see how we do a diffChangeLog with an empty database to re-create what was in the original changelog on lines 726-734.

diffChangeLog

AbstractIntegrationTest.java#L726

You can see how we execute the generated changelog on lines 736-742.

update

AbstractIntegrationTest.java#L736

You can see how we validate the results by comparing the snapshot from the original update with the one we from the just did from the generated changelog on lines 744-749.

N/A

AbstractIntegrationTest.java#L744

Database Specific Integration Tests

IntegrationTest Subclasses

For each database we support, there is also a subclass of AbstractIntegrationTest.java in a sub-package. Here are the locations of the current integration test classes for each supported database.

Liquibase Supported Databases

Database Platform

Database Type

Integration Test - Main Java Class Location

Test Count

Database Platform

Database Type

Integration Test - Main Java Class Location

Test Count

Apache Derby

Relational Embedded (Java)

liquibase-integration-tests/src/test/java/liquibase/dbtest/derby/DerbyIntegrationTest.java

0

DB2

Relational Server

liquibase-integration-tests/src/test/java/liquibase/dbtest/db2/DB2IntegrationTest.java

0

Firebird

Relational Server

liquibase-integration-tests/src/test/java/liquibase/dbtest/firebird/FirebirdIntegrationTest.java

0

H2

Relational Embedded (Java)

liquibase-integration-tests/src/test/java/liquibase/dbtest/h2/H2IntegrationTest.java

8

HSQL

Relational Embedded (Java) & Relational Server

liquibase-integration-tests/src/test/java/liquibase/dbtest/hsqldb/HsqlIntegrationTest.java

0

Informix

Relational Server

liquibase-integration-tests/src/test/java/liquibase/dbtest/InformixIntegrationTest.java

0

MariaDB

Relational Server

liquibase-integration-tests/src/test/java/liquibase/dbtest/mariadb/MariaDBIntegrationTest.java

3

MSSQL (SQL Server)

Relational Server

  • 2

  • 0

  • 3

  • 0

MySQL

Relational Server

liquibase-integration-tests/src/test/java/liquibase/dbtest/mysql/MySQLIntegrationTest.java

3

Oracle

Relational Server

liquibase-integration-tests/src/test/java/liquibase/dbtest/oracle/OracleIntegrationTest.java

5

Postgres

Relational Server

liquibase-integration-tests/src/test/java/liquibase/dbtest/pgsql/PostgreSQLIntegrationTest.java

0

SQLite

Relational Embedded (C)

liquibase-integration-tests/src/test/java/liquibase/dbtest/SQLiteIntegrationTest.java

2

Sybase Anywhere

Relational Server

liquibase-integration-tests/src/test/java/liquibase/dbtest/asany/SybaseASAIntegrationTest.java

0

Sybase ASE

Relational Server

  • 0

  • 0

The purpose of this class is to:

  • Connect to the database under test

  • Allow overriding of tests in the AbstractIntegrationTest when a specific database works differently than the standard

  • Add additional database-specific test cases

Interface/Integration Integration Tests

Maven Integration Tests

The Maven integration tests use the Maven Plugin Testing Harness. The tests themselves are stored in liquibase-maven-plugin/src/test/java/org/liquibase/maven/plugins.

There should be a 1-1 correspondence between the maven plugin classes and the *Test classes for them.

Example Test File

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 public class LiquibaseRollbackMojoTest extends AbstractLiquibaseMojoTest { .... public void testRollbackDateNoPropertiesFile() throws Exception { LiquibaseRollback mojo = createUpdateMojo(DATE_CONFIG_FILE); // Clear out any settings for the property file that may be set setVariableValueToObject(mojo, "propertyFile", null); setVariableValueToObject(mojo, "propertyFileWillOverride", false); Map values = getVariablesAndValuesFromObject(mojo); checkValues(DATE_DEFAULT_PROPERTIES, values); } ... }

The tests themselves are JUnit tests, so adding additional tests are a matter of adding additional test methods as needed.

Ant Integration Tests

The Ant integration tests use Apache AntUnit. The tests themselves are stored as xml files in liquibase-core/src/test/resources/liquibase/integration/ant .

The database url, username, and password are stored as properties near the top of the test file. They are defaulted to run against against h2 because these tests care more about the Ant integration and not as much about database variations

Example Test File

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 <?xml version="1.0" encoding="UTF-8"?> <project name="DropAllTaskTest" basedir="." xmlns="antlib:org.apache.tools.ant" xmlns:au="antlib:org.apache.ant.antunit" xmlns:lb="antlib:liquibase.integration.ant" xmlns:db="antlib:liquibase.integration.ant.test"> <tempfile property="temp.dir" prefix="DropAllTaskTest" destDir="${java.io.tmpdir}"/> <path id="basic-classpath"> <pathelement path="."/> </path> <property name="jdbc.driver" value="org.h2.Driver"/> <property name="jdbc.url" value="jdbc:h2:mem:test;DB_CLOSE_DELAY=-1"/> <property name="db.user" value="sa"/> <property name="db.password" value=""/> <lb:database id="test-db" driver="${jdbc.driver}" url="${jdbc.url}" user="${db.user}" password="${db.password}"/> <target name="setUp"> <sql driver="${jdbc.driver}" url="${jdbc.url}" userid="${db.user}" password="${db.password}" encoding="UTF-8" src="${liquibase.test.ant.basedir}/sql/h2-setup.sql"/> <mkdir dir="${temp.dir}"/> <lb:updateDatabase databaseref="test-db" changelogfile="${liquibase.test.ant.basedir}/changelog/simple-changelog.xml"/> </target> <target name="tearDown"> <sql driver="${jdbc.driver}" url="${jdbc.url}" userid="${db.user}" password="${db.password}" encoding="UTF-8" src="${liquibase.test.ant.basedir}/sql/h2-teardown.sql"/> <delete dir="${temp.dir}"/> </target> <target name="testDropAllTaskLegacy"> <db:assertTableExists driver="${jdbc.driver}" url="${jdbc.url}" user="${db.user}" password="${db.password}" table="USERS"/> <lb:dropAllDatabaseObjects driver="${jdbc.driver}" url="${jdbc.url}" username="${db.user}" password="${db.password}" classpathref="basic-classpath"/> <db:assertTableDoesntExist driver="${jdbc.driver}" url="${jdbc.url}" user="${db.user}" password="${db.password}" table="USERS"/> </target> <target name="testDropAllTask"> <db:assertTableExists driver="${jdbc.driver}" url="${jdbc.url}" user="${db.user}" password="${db.password}" table="USERS"/> <lb:dropAllDatabaseObjects> <lb:database driver="${jdbc.driver}" url="${jdbc.url}" user="${db.user}" password="${db.password}"/> </lb:dropAllDatabaseObjects> <db:assertTableDoesntExist driver="${jdbc.driver}" url="${jdbc.url}" user="${db.user}" password="${db.password}" table="USERS"/> </target> ..... </project>

The available assertions are defined in liquibase-core/src/test/resources/liquibase/integration/ant/test/antlib.xml:

  • assertTableExists

  • assertTableDoesntExist

  • assertColumnExists

  • assertColumnDoesntExist

  • assertRowCountEquals