Kola is a framework for testing software integration in CoreOS systems across multiple platforms. It is primarily designed to operate within the CoreOS Assembler for testing software that has landed in the OS image.
Kola supports running tests on multiple platforms, currently QEMU, GCE, AWS, VMware VSphere, Packet, and OpenStack. In the future systemd-nspawn and other platforms may be added. Local platforms do not rely on access to the Internet as a design principle of kola, minimizing external dependencies. Any network services required get built directly into kola itself. Machines on cloud platforms do not have direct access to the kola so tests may depend on Internet services such as discovery.etcd.io or quay.io instead.
Kola outputs assorted logs and test data to
_kola_temp for later inspection.
Kola is still under heavy development and it is expected that its interface will continue to change.
By default, kola uses the
qemu-unprivileged platform with the most recently built image (assuming it is run from within a CoreOS Assembler working directory).
- kola run
- kola list
- kola spawn
- kola bootchart
- kola subtest parallelization
- kola test namespacing
- kola test registration
- kola test writing
- kola native code
The run command invokes the main kola test harness. It runs any tests whose registered names matches a glob pattern.
kola run <glob pattern>
--denylist-test can be used if one or more tests in the pattern should be skipped. This switch may be provided once:
kola --denylist-test linux.nfs.v3 run
kola --denylist-test linux.nfs.v3 --denylist-test linux.nfs.v4 run
and can also be used with glob patterns:
kola --denylist-test linux.nfs* --denylist-test crio.* run
Tests specified in
src/config/kola-denylist.yaml will also be skipped regardless of whether the switch
--denylist-test was provided.
Example format of the file:
- pattern: test1.blobpattern.* tracker: https://github.com/coreos/coreos-assembler/pull/123 streams: # This test will be skipped in these streams # If no streams are specified, test will be skipped on all streams - stream1 - stream2 # The test will only be skipped until this date (will resume on the date) # Format: YYYY-MM-DD snooze: 2021-07-20 arches: # This test will be skipped on these arches # If no arches are specified, test will be skipped on all arches - s390x platforms: # This test will be skipped on these platforms # If no platforms are specified, test will be skipped on all platforms - openstack - aws - pattern: test2.test ...
The list command lists all of the available tests.
The spawn command launches CoreOS instances.
The bootchart command launches an instance then generates an svg of the boot process using
Subtests can be parallelized by adding
c.H.Parallel() at the top of the inline function given to
c.Run. It is not recommended to utilize the
FailFast flag in tests that utilize this functionality as it can have unintended results.
The top-level namespace of tests should fit into one of the following categories:
- Groups of tests targeting specific packages/binaries may use that namespace (ex:
- Tests that target multiple supported distributions may use the
- Tests that target singular distributions may use the distribution’s namespace.
Registering kola tests currently requires that the tests are registered under the kola package and that the test function itself lives within the mantle codebase.
Groups of similar tests are registered in an init() function inside the kola package.
Register(*Test) is called per test. A kola
Test struct requires a unique name, and a single function that is the entry point into the test. Additionally, userdata (such as an Ignition config) can be supplied. See the
Test struct in kola/register/register.go for a complete list of options.
A kola test is a go function that is passed a
platform.TestCluster to run code against. Its signature is
func(platform.TestCluster) and must be registered and built into the kola binary.
TestCluster implements the
platform.Cluster interface and will give you access to a running cluster of CoreOS machines. A test writer can interact with these machines through this interface.
To see test examples look under kola/tests in the mantle codebase.
For a quickstart see kola/adding-tests.md.
For some tests, the
Cluster interface is limited and it is desirable to run native go code directly on one of the CoreOS machines. This is currently possible by using the
NativeFuncs field of a kola
Test struct. This like a limited RPC interface.
NativeFuncs is used similar to the
Run field of a registered kola test. It registers and names functions in nearby packages. These functions, unlike the
Run entry point, must be manually invoked inside a kola test using a
RunNative method. The function itself is then run natively on the specified running CoreOS instances.
For more examples, look at the coretest suite of tests under kola. These tests were ported into kola and make heavy use of the native code interface.
platform.Manhole() function creates an interactive SSH session which can be used to inspect a machine during a test.
--ssh-on-test-failure flag can be specified to have the kola runner automatically SSH into a machine when any
MustSSH calls fail.
kolet is run on kola instances to run native functions in tests. Generally kolet is not invoked manually.