The standard library, part 2: reading and writing
Until now a Zeno program could say things and could not be told anything. Whatever it worked on was written in the program. This page gives it the three ways in that a program on a computer has: files, the keyboard, and the words it was started with.
cargo run -- run examples/logbook.zeno bench.txt
Type three lines and close the keyboard with Ctrl-D:
reading> temp 23.5
reading> hello
reading> humidity 40
reading>
3 lines typed
4 readings in bench.txt, mean 26.75, lines skipped: 1
still there: false
FileError: there is no file `bench.txt`
examples/logbook.zeno takes the name of a
file from its command line, starts the file with two readings, adds
every line that is typed, reads the file back, and removes it. The
decisions behind each part are 6.11 to 6.20 in
decisions.md.
What was added
| Written | Gives | Can fail with | |
|---|---|---|---|
| Read a file | files.read(path) |
str |
FileError |
| Write a file | files.write(path, text) |
nothing | FileError |
| Add to a file | files.append(path, text) |
nothing | FileError |
| Is there a file? | files.exists(path) |
bool |
never |
| Remove a file | files.remove(path) |
nothing | FileError |
| The lines of a text | text.lines() |
list[str] |
never |
| A line from the keyboard | input(), input("Name: ") |
str or none |
never |
| The command line | program.arguments() |
list[str] |
never |
files and program are modules: import files, import program.
input is a plain word, like print.
The files
| File | What part 2 added |
|---|---|
src/value.rs |
Outside: the keyboard and the arguments, as both engines see them |
src/builtins.rs |
The functions, for zeno run and the VM |
src/checker.rs |
Their types, and that four of them raise FileError |
src/runtime.c |
The same functions in C, for compiled programs |
src/emit_c.rs, src/emit_llvm.rs |
A call of the C function, and a main that takes the command line |
src/main.rs |
The words after the file of zeno run go to the program |
Files
import files
files.write("notes.txt", "milk\neggs\n") # makes it, or empties it
files.append("notes.txt", "bread\n") # makes it, or adds to it
print(files.read("notes.txt")) # all of it, as one str
print(files.exists("notes.txt")) # true
files.remove("notes.txt")
A file is read whole and written whole. There is no open and no
close, so there is nothing to forget to close (6.12).
write and append put down the text they are given and nothing else.
A program that wants lines ends each with "\n" (6.13).
What can go wrong
| What happened | The message of the FileError |
|---|---|
| Nothing is at the path | there is no file `notes.txt` |
| It is a folder, or the system will not let it be read | cannot read `notes.txt` |
| It is not text | `notes.txt` does not hold text |
| It cannot be made or written, such as a folder that is not there | cannot write `notes.txt` |
| It cannot be removed | cannot remove `notes.txt` |
A function that uses a file either catches the error or says it can happen:
function first_line(path: str) -> str raises FileError {
return files.read(path).lines()[0]
}
Leave raises FileError out and zeno build (or zeno check
--strict) says:
error: `first_line` can raise FileError here but does not say so. Add `raises FileError` to it, or catch the error
This holds in a firmware build too. A bad index is a fault there and
stops the program, because the program could have checked the length.
No check rules out a missing file, so FileError travels as a value
and can always be caught (6.14).
A file holds text
files.read gives a str, and a str is characters. So the file has
to be UTF-8 with no zero byte in it. Anything else is refused:
FileError: `logo.png` does not hold text
The interpreter asks Rust (String::from_utf8). A compiled program has
its own test, zeno_is_text in runtime.c, about forty lines. The
first byte of a character says how many follow:
0xxxxxxx one byte: the ASCII characters
110xxxxx 10xxxxxx two bytes
1110xxxx 10xxxxxx 10xxxxxx three
11110xxx 10xxxxxx 10xxxxxx 10xxxxxx four
and four first bytes hold the byte after them to a tighter range. That
rules out a character written in more bytes than it needs (C0 80),
the numbers Unicode keeps free (ED A0 80), and anything past the last
character (F4 90 80 80). The test
what_is_text_is_the_same_in_every_build gives both tests nine files
and requires the same answer about each.
Files are opened as bytes ("rb", "wb", "ab"), so nothing is
changed on the way in or out. A file from Windows arrives with its
"\r\n" line ends, and lines() deals with them.
lines()
for line in files.read("notes.txt").lines() {
print(f"[{line}]")
}
| Text | .lines() |
|---|---|
"" |
[] |
"one" |
["one"] |
"one\n" |
["one"] |
"one\ntwo" |
["one", "two"] |
"one\r\ntwo\r\n" |
["one", "two"] |
"\n" |
[""] |
"a\n\nb" |
["a", "", "b"] |
A line ends at "\n". A "\r" just before that "\n" goes with it.
The last line need not end.
split("\n") is not the same: it gives "a\nb\n" a third, empty,
piece, and leaves the "\r" on. The C is a dozen lines:
static zeno_list *zeno_lines(const char *text) {
zeno_list *lines = zeno_list_new(0);
const char *start = text;
while (*start != '\0') {
const char *end = strchr(start, '\n');
size_t length = end != NULL ? (size_t)(end - start) : strlen(start);
const char *next = start + length + (end != NULL ? 1 : 0);
if (end != NULL && length > 0 && start[length - 1] == '\r') length--;
zeno_list_append(lines, (zeno_slot){.s = zeno_piece(start, length)});
start = next;
}
return lines;
}
In the interpreter it is one call, because Rust's own str::lines has
the same rules.
input()
let name = input("Name: ")
if name is none {
print("nobody there")
} else {
print(f"Hello, {name}!")
}
input() gives the next line without its line end. With a question,
it prints the question first and stays on that line. When there are no
more lines it gives none: Ctrl-D at a keyboard, or the end of a file
that was piped in (zeno run count.zeno < notes.txt).
Because the answer is str or none, the checker will not let it be
used as text until it has been tested:
error: this is str or none. Check it with `is not none` before using `.upper`
A loop that reads everything:
var count = 0
while true {
let line = input()
if line is none {
break
}
count += 1
}
print(count, "lines")
After if line is none { break } the checker knows line is a str
for the rest of the loop.
How a line is read
zeno_input in runtime.c reads one byte at a time with fgetc:
'\n' the line is finished
'\r' held back: it is dropped if '\n' comes next, kept if not
'\0' left out, because text ends at a zero byte
end the line is finished, if there was anything in it
anything added to the line
Nobody knows how long the line will be, so it grows as it is read, and
the two kinds of build grow it differently (zeno_line_add):
- With a collector the line is a block. When it is full it moves to a block twice the size, and the collector takes the old one.
- With no collector it grows in the text room, which is a stack: each byte goes on top of the one before. The line takes exactly its own length.
The interpreter's version is Outside::line in value.rs. It reads a
whole line with Rust's read_until, then takes off the line end and
any zero bytes, so that both come to the same text.
program.arguments()
import program
let words = program.arguments()
if len(words) != 2 {
print("usage: copy <from> <to>")
}
zeno run copy.zeno a.txt b.txt ["a.txt", "b.txt"]
./copy a.txt b.txt ["a.txt", "b.txt"]
The program's own name is not in the list, so words[0] is the first
thing the user wrote (6.19).
With zeno run, every word after the file is the program's, even one
that looks like an option of Zeno's. Zeno's options go before the file:
zeno run --vm copy.zeno a.txt --strict the program gets ["a.txt", "--strict"]
In a compiled program the words come from C's main. A program that
never asks keeps the main it always had:
int main(int argc, char **argv) { /* was: int main(void) */
...
zeno_program_start(argc, argv); /* the runtime keeps both */
zeno_program();
return 0;
}
define i32 @main(i32 %argc, ptr %argv) {
entry:
...
call void @zeno_program_start(i32 signext %argc, ptr %argv)
call void @zeno_program()
ret i32 0
}
zeno_program_arguments then makes a new list from argv[1] onward
at each call.
One function, through the engines
Follow files.read(path).
1. zeno run and the VM (builtins.rs). One Rust function, in the
table for import files:
fn files_read(_: &mut dyn Machine, args: Vec<Value>, span: Span) -> Result {
let path = paths("read", &args, 1, span)?[0];
match at(path) {
At::Nothing => return file_error(format!("there is no file `{path}`"), span),
At::Other => return file_error(format!("cannot read `{path}`"), span),
At::File => {}
}
let Ok(bytes) = std::fs::read(path) else {
return file_error(format!("cannot read `{path}`"), span);
};
match String::from_utf8(bytes) {
Ok(text) if !text.contains('\0') => Ok(Value::str(text)),
_ => file_error(format!("`{path}` does not hold text"), span),
}
}
2. The checker (checker.rs, builtin). The argument is a str,
the answer is a str, and the function it is called in can now raise:
if short != "exists" {
self.can_raise("FileError", span);
}
That one line is what makes a function say raises FileError.
3. The runtime (runtime.c). zeno_file_read does the work and
answers NULL, or the message of what went wrong. Around it are two
thin faces, one for each way errors travel:
#ifndef ZENO_FIRMWARE
static const char *zeno_files_read(const char *path, int line, int col); /* jumps */
#else
static bool zeno_files_read(const char *path, int line, int col, const char **result); /* says */
#endif
4. The C backend. In an ordinary build the call is an expression, and an error jumps out of it:
static const char *z_first(const char *z_path) {
return (*zeno_list_at(zeno_lines(zeno_files_read(z_path, 5, 12)), 0, 5, 12)).s;
}
In a firmware build the same line is a test, and the function passes the failure to its own caller:
static bool z_first(const char *z_path, const char **zeno_result) {
const char *t2;
if (!zeno_files_read(z_path, 5, 12, &t2)) return false;
*zeno_result = (*zeno_list_at(zeno_lines(t2), 0, 5, 12)).s;
return true;
}
5. The LLVM backend. The firmware build again:
define internal zeroext i1 @z_first(ptr %arg.path, ptr %result) {
entry:
%answer.1 = alloca ptr
%t1 = call zeroext i1 @zeno_files_read(ptr %arg.path, i32 5, i32 12, ptr %answer.1)
br i1 %t1, label %worked2, label %failed2
failed2:
ret i1 false
worked2:
%t2 = load ptr, ptr %answer.1
%t3 = call ptr @zeno_lines(ptr %t2)
...
Nothing new was needed in either backend for the firmware build: a
runtime function that says whether it worked is called the way
int(text) already was (checked_call).
With no collector
The text of a file and a line from input() are built text, like the
answer of upper(): they are in the text room until the statement
ends, and text that is kept is copied to the pool
(text.md). lines() and program.arguments() make their
items in the pool directly.
So the size of the text room is the size of the largest file that can be read. It is 1024 bytes unless the C is compiled with another:
error: a statement built more text than there is room for (ZENO_TEXT_ROOM is 1024 bytes)
zeno build --no-collector copy.zeno
cc -O2 -DZENO_TEXT_ROOM=8192 -o copy copy.c -lm
zeno build --no-collector --memory prints how much of the room a run
used. The logbook example, with three lines typed:
memory: 35 blocks made (661 bytes), at most 214 bytes in use at once, no collector, 0 blocks left at the end
pool: 384 bytes taken, in a pool of 32768
text: at most 64 bytes in use at once, in a room of 1024
On a board
These functions are the C library's: fopen, fgetc, remove. A
firmware C library usually ties standard input to a serial line, and
then input() reads a line from that. Where there is no file system,
every files call but exists raises a FileError, which the program
can catch. The simulated board (board.md) is not involved.
What it does not do
- Large files. A file is read whole. Reading in pieces needs an open file with an owner, and is left until a program needs it (6.12).
- Bytes. A file that is not text cannot be read at all yet (6.16).
- Folders. No listing, making or removing of folders.
program.exit(code)and environment variables (6.19).- Typed text that is not UTF-8.
zeno runreplaces the bad bytes with�. A compiled program passes them on as they came.
How it is tested
Every engine and every build has to do the same with the same keyboard
and the same command line, so the tests supply both. A test wraps its
programs in with_outside (value.rs):
with_outside("temp 23.5\nhello\n", &["bench.txt"], || {
no_collector_as_interpreter("logbook", source);
});
Inside it, the interpreter and the VM read the typed text in place of
the keyboard, and each built program is started with the arguments and
has the text piped to its standard input. With no with_outside,
nothing is typed and there are no arguments, so no test can sit waiting
for a keyboard.
Each program that writes a file removes it again, so the eight runs of one test (interpreter, VM, and C and LLVM in three builds) each start from the same. Every test's file has its own name in the system's temporary folder.
The tests:
files_are_the_same_in_every_build: all five functions, the error messages, caught in a function and passed on by one.an_uncaught_file_error_is_the_same_in_every_build: the same message and the same line and column.what_is_text_is_the_same_in_every_build: nine files of bytes.the_keyboard_and_the_arguments_are_the_same_in_every_build: a line of 300 characters,"\r"in every position, an empty line, a zero byte, Tamil, and no line end on the last line.the_logbook_example_is_the_same_in_every_build.main_takes_the_command_line_only_when_the_program_reads_it.- In the interpreter and the checker: the exact output and the exact messages.
There are 373 tests in all.
Things to try
-
Run
zeno run examples/logbook.zenoand type a few readings. Then put them in a file and runzeno run examples/logbook.zeno < readings.txt. Why doesreading>appear once more than there are lines? -
In
logbook.zeno, change the firstfiles.writetofiles.appendand put a#in front offiles.remove(path). Run it twice with nothing typed (< /dev/null). The second run reports 4 readings. Which decision says whywritewould have reported 2? -
Take
raises FileErroroffsummaryand runzeno check --strict examples/logbook.zeno. Read the message, then fix it the other way: keep theraisesoff and catch the error insidesummary. -
Run this, and say where the third item of the first list comes from:
print("a\nb\n".split("\n"))
print("a\nb\n".lines())
-
Write
copy.zeno: with two arguments it copies the first file to the second, with any other number it prints how to use it, and aFileErroris caught and its message printed. Build it, run./copy copy.zeno again.zeno, and compare the two withcmp. -
Build your
copy.zenowith--no-collector --memoryand copy a file of more than 1024 bytes. Read the message, then compile the C again with-DZENO_TEXT_ROOM=8192as shown above. What does thetext:line say the copy needed? -
In a shell,
printf '\xC0\x80' > odd.bin, then readodd.binwithfiles.read. Find the line ofzeno_is_textinsrc/runtime.cthat refuses it. Which bytes would the same character be, written properly? -
input()leaves out zero bytes. Find the two places that do it, one insrc/value.rsand one insrc/runtime.c. What would go wrong in a compiled program if only the Rust one did?
This page is docs/reading-and-writing.md in the language repository.