25 Testing
A test is a small program that checks a part of a bigger one, and
says clearly when the part is wrong. Firmware that has been tested on a
computer before it reaches a board is firmware that wastes fewer
afternoons at the bench. Zeno has tests in the language: a
test block, an assert statement, and the
command zeno test.
25.1 A first test
temperature.zeno
function to_fahrenheit(celsius: float) -> float {
return celsius * 1.8 + 32.0
}
test "water freezes at 32" {
assert to_fahrenheit(0.0) == 32.0
}
test "water boils at 212" {
assert to_fahrenheit(100.0) == 212.0
}$ zeno test temperature.zeno
temperature.zeno
ok water freezes at 32
ok water boils at 212
2 tests: 2 passed, 0 failed.
A test is the word test, a name in quotes that says in
ordinary words what it checks, and a block. It stands at the top of a
file, beside what it tests. zeno test runs the file as
zeno run would, and then each test, in the order
written.
zeno run goes past the tests, and
zeno build leaves them out altogether, so a board pays
nothing for them. But they are checked with the program by
zeno check and zeno run, so a test cannot go
stale without anybody noticing.
25.2 assert
assert and a condition say what must be so. When the
condition is a comparison, a failure shows what was on each side:
average.zeno
function average(values: list[float]) -> float {
return sum(values) / float(len(values) + 1)
}
test "the average of one number is that number" {
assert average([4.0]) == 4.0
}
test "an average lies between the smallest and the largest" {
let values = [1.0, 2.0, 6.0]
assert min(values) <= average(values)
}$ zeno test average.zeno
average.zeno
FAILED the average of one number is that number
ok an average lies between the smallest and the largest
FAILED: the average of one number is that number
error: this is false. The left of `==` is 2.0, and the right is 4.0
--> average.zeno:6:12
|
6 | assert average([4.0]) == 4.0
| ^
2 tests: 1 passed, 1 failed.
The first test failed, and the message shows that the left side was
2.0 where 4.0 was wanted: the + 1 is the bug. The second
test passed, because a wrong average can still lie in the right range. A
failed assert ends its test; the next test runs anyway. The
command fails when any test fails, so a script or a build server can
tell.
Other languages grow a family of asserts, assert_eq,
assert_ne, assertIn, each so that a failure
can show the values. Zeno has the comparison in front of it, so the one
assert can show them, and there is one thing to learn.
(Decision 10.17.)
assert is only for tests. In a program, if
and raise say what is wrong and to whom, and can be caught.
A failed assert is not an error of the program, so
except Error does not catch it.
25.3 Testing that something fails
Fault paths are half of what firmware has to get right.
assert raises checks that a block raises an error of a
kind:
speed.zeno
function check_speed(speed: int) -> int raises ValueError {
if speed < 0 or speed > 100 {
raise ValueError(f"a speed is 0 to 100, and {speed} is not")
}
return speed
}
test "speeds in range are kept" {
assert check_speed(0) == 0
assert check_speed(100) == 100
}
test "speeds out of range are refused" {
assert raises ValueError {
check_speed(101)
}
assert raises ValueError {
check_speed(-1)
}
}$ zeno test speed.zeno
speed.zeno
ok speeds in range are kept
ok speeds out of range are refused
2 tests: 2 passed, 0 failed.
The test fails if the block raises nothing, or raises another kind,
and says which. To take an error of any kind, name Error:
assert raises Error.
25.4 Where tests go
A test can stand in the file it tests, and then it sees what is
private to that file, which is usually where the things worth testing
are. A file of tests alone, such as ring_test.zeno
importing ring, is the same thing with nothing else in
it.
zeno test with a folder tests every file in it, and
below it, that has a test. With no folder it means the folder it is run
in:
$ zeno test ring
ring/ring.zeno
ok a new ring is empty
ok bytes come out in the order they went in
ok a full ring keeps what it has
ok the ring goes round
ok a ring of no size is refused
5 tests: 5 passed, 0 failed.
Each file is tested as a program of its own: its tests start from the
state the file leaves, and the tests of two files cannot disturb each
other. A file with no test in it is not run at all, so testing a folder
does not run main.zeno and blink its LED. The tests of an
imported file run when that file is tested, not each time another file
imports it.
Tests of one file share what the file declares, so a test that
changes a var at the top of the file leaves it changed for
the next. A test that needs a fresh value makes it inside itself; what a
test declares is gone when it ends.
A test with a name in plain words says what is checked, and is what a
failure is reported under. Functions whose names begin with
test_, as in Python and Go, are a rule nobody can see in
the program. Marks such as @test are a new piece of syntax
for one use. Tests only in files of their own could not test anything
private. test is not even a reserved word:
test " at the start of a statement can mean nothing else,
so programs keep the name for their own use. (Decision 10.16.)
25.5 Testing code that uses the board
Code that drives hardware can be tested on a computer too, because
zeno run simulates the board: a pin remembers whether it is
on, and a register remembers what was written to it (The Board). A test can drive a pin
and then ask it:
heater.zeno
from board import pin
let heater = pin(5)
function control(temperature: float) {
if temperature < 19.0 {
heater.on()
} else if temperature > 21.0 {
heater.off()
}
}
test "the heater comes on when it is cold" {
control(15.0)
assert heater.is_on()
}
test "it stays on until it is warm" {
control(15.0)
control(20.0)
assert heater.is_on()
control(22.0)
assert not heater.is_on()
}$ zeno test heater.zeno
heater.zeno
pin 5 on
ok the heater comes on when it is cold
pin 5 off
ok it stays on until it is warm
2 tests: 2 passed, 0 failed.
The simulated board says each change of a pin as it happens, which is
why pin 5 on appears among the results.
25.6 What tests leave out
In version 0.1 there is no message after an assert (the
test’s name and the values shown already say most of it), no
assert outside a test, no code run before and after each
test (a function that builds what the tests need does that), no choosing
tests by name on the command line, and no running of tests in a compiled
program. (Decision 10.21.)
test "what it checks" { ... } stands at the top of a
file. assert condition fails the test when the condition is
false, and shows both sides of a comparison.
assert raises Kind { ... } checks that a block raises.
zeno test runs a file, a folder or the folder it is in, and
fails if any test fails. Builds leave tests out.
Write three tests for
checksumfrom Sized Numbers and Bits, including one where the total passes 255.Fix the
averagefunction above and run its tests again. Then add a test for an empty list. What shouldaveragedo there, and what does the test say?Write a test that checks
int("fast")raises aValueError. Then change the kind toKeyErrorand read the failure.Make a test in one file change a
varat the top of the file, and see the next test find it changed.