1. Background and motivation¶
Kernel Test Framework (KTF) implements a unit test framework for the Linux kernel. There’s a wide selection of unit test frameworks available for normal user land code testing, but I have so far not seen any similar frameworks that can be used with kernel code, to test details of both exported and non-exported kernel APIs. The hope is that providing an easy to use and convenient way to write simple unit tests for kernel internals, that this can promote a more test driven approach to kernel development, where appropriate.
Test driven development¶
Unit testing is an important component of Test driven development (TDD). The idea of test driven development is that when you have some code to write, whether it is a bug to fix, or a new feature or enhancement, that you start by writing one or more tests for it, have those tests fail, and then do the actual development.
Typically a test driven development cycle would have several rounds of development and test using the new unit tests, and once the (new) tests pass, you would also run all or a suitable subset (limited by execution time) of the old tests to verify that you have not broken any old functionality by the new code.
At this stage it is important that the tests that are run can be run quickly to allow them to be actively used in the development cycle. When time comes for submission of the code, a full, extensive set of both the tests the developer thinks can touch the affected code and all the other tests should be run, and a longer time to run tests can be afforded.
KTF tries to support this by using the module system of the kernel to support modularized test suites, where a user only need to insmod the test subsets that he/she wants to use right then. Different test suites may touch and require different kernel APIs and have lots of different module and device requirements. To enable as much reuse of the functionality of code developed within KTF it is important that any piece of test code has as few dependencies as possible.
Good use cases for KTF¶
Unit testing is at it’s most valuable when the code to test is relatively error prone, but still might be difficult to test in a systematic and reproducable way from a normal application level. It can be difficult to trigger corner cases from a high abstraction layer, the code paths we want to exercise might only be used occasionally, or we want to exercise that error/exception scenarios are handled gracefully. Particularly good use cases are simple APIs with a relativ complex implementation - such as container implementations eg. scatterlists, rbtrees, list, ... are obvious candidates
When not to use KTF¶
Writing kernel code has some challenges compared to user land code. KTF is intended for the cases where it is not easy to get coverage by writing simple tests from user land.
Why you would want to write and run KTF tests¶
Besides the normal write test, write code, run test cycle of development and the obvious benefits of delivering better quality code with fewer embarrassments, there’s a few other upsides from developing unit test for a particular area of the kernel:
- A test becomes an invariant for how the code is supposed to work. If someone breaks it, they should detect it and either document the changes that caused the breakage by fixing the test or realize that their fix is broken before you even get to spend time on it.
- Kernel documentation while quite good in some places, does not always cover the full picture, or you might not find that sentence you needed while looking for it. If you want to better understand how a particular kernel module actually works, a good way is to write a test that codes your assumptions. If it passes, all is well, if not, then you have gained some understanding of the kernel.
- Sometimes you may find yourself relying on some specific feature or property of the kernel. If you encode a test that guards the assumption syou have made, you will capture if someone changes it, or if your code is ported to an older kernel which does not support it.