The language server
zeno lsp lets an editor show the mistakes in a Zeno file as you
type, lay the file out when you save, jump to where a name was
declared, and show what a name is. This page says what it does,
how to watch it work with no editor at all, and walks through
src/lsp.rs and src/json.rs.
Setting up your editor is in editors/README.md.
The decisions are 9.10 to 9.13 in decisions.md.
What it is
An editor does not know Zeno. What it can do is start a program, the language server, and talk to it in a fixed way, the Language Server Protocol. The editor says "this file was opened" and "this file now reads so". The server says "these are the mistakes in it". VS Code, Neovim, Helix, Zed and Emacs all speak the protocol, so one server serves them all.
Zeno's server is the zeno program itself, started as zeno lsp. It
runs the loader and the checker that zeno check runs, so what the
editor underlines is exactly what zeno check would print.
| The editor says | The server does |
|---|---|
| a file was opened, or its text changed | checks every open file, and sends the mistakes of each file whose mistakes changed |
| lay this file out | answers with the file as zeno format writes it |
| where is this name declared? | answers with the file and the line of its declaration |
| what is this name? | answers with the name and its type, such as count: int |
| anything else that wants an answer | says it has no answer to that |
The two questions about a name are answered for variables,
parameters, functions, classes, enums and the names of a module. A
field or a method (p.x) says what it is, and not yet where it was
declared (17.4).
Watching it work
You do not need an editor to see the server. editors/talk.py plays
the editor's part and prints both sides:
cargo build
python3 editors/talk.py target/debug/zeno examples/thermostat/main.zeno
--> {"jsonrpc": "2.0", "method": "initialize", "id": 1, "params": {"rootUri": "file:///.../examples/thermostat", ...}}
<-- {"jsonrpc": "2.0", "id": 1, "result": {"capabilities": {"textDocumentSync": {"openClose": true, "change": 1},
"documentFormattingProvider": true, "definitionProvider": true, "hoverProvider": true},
"serverInfo": {"name": "zeno", "version": "0.1.0"}}}
--> {"jsonrpc": "2.0", "method": "textDocument/didOpen", "params": {"textDocument": {"uri": "file:///.../main.zeno", "text": "# A program..."}}}
<-- {"jsonrpc": "2.0", "method": "textDocument/publishDiagnostics", "params": {"uri": "file:///.../main.zeno", "diagnostics": []}}
--> {"jsonrpc": "2.0", "method": "textDocument/didChange", ...}
<-- {"jsonrpc": "2.0", "method": "textDocument/publishDiagnostics", "params": {"uri": "file:///.../main.zeno", "diagnostics": [
{"range": {"start": {"line": 31, "character": 6}, "end": {"line": 31, "character": 18}},
"severity": 1, "source": "zeno", "message": "`no_such_name` is not defined"}]}}
--> {"jsonrpc": "2.0", "method": "textDocument/didChange", ...}
<-- {"jsonrpc": "2.0", "method": "textDocument/publishDiagnostics", "params": {"uri": "file:///.../main.zeno", "diagnostics": []}}
--> {"jsonrpc": "2.0", "method": "textDocument/hover", "id": 4, "params": {"textDocument": {"uri": "file:///.../main.zeno"},
"position": {"line": 32, "character": 6}}}
<-- {"jsonrpc": "2.0", "id": 4, "result": {"contents": {"kind": "plaintext", "value": "answer: int"}}}
--> {"jsonrpc": "2.0", "method": "textDocument/definition", "id": 5, "params": {... "position": {"line": 32, "character": 6}}}
<-- {"jsonrpc": "2.0", "id": 5, "result": {"uri": "file:///.../main.zeno",
"range": {"start": {"line": 31, "character": 0}, "end": {"line": 31, "character": 0}}}}
Reading it from the top:
initialize. The editor says hello, and the server answers with what it can do: it wants to be told when files open and close, it wants the whole text at every change ("change": 1), it can lay a file out, and it can say where a name was declared and what it is.didOpen. The editor sends the file's text. The server checks it and sends back its mistakes: an empty list.didChange. The script "types" a line,print(no_such_name). The server checks again and sends one mistake, with where it is: line 31, characters 6 to 18. Both count from 0, so that is line 32 in your editor. An editor draws the red line from this.didChangeagain. The script takes the mistake out and types two good lines,let answer = 6 * 7andprint(answer). The server sends an empty list, and the editor takes the red line away.hover. The script asks what stands at line 32, character 6: theanswerinsideprint(answer). The server saysanswer: int.definition. The same place, and the other question: where was it declared? On line 31, the line with thelet.
After that the script spoils the layout and asks for the file to be laid out, which the last lines of its output show.
A message with an id is a request and gets an answer with the same
id. A message with none is a notification, and gets none.
How it works
JSON
Every message is JSON. Zeno uses no library, so src/json.rs reads
and writes it. JSON has six kinds of value and the type says so:
pub enum Json {
Null,
Bool(bool),
Number(f64),
Text(String),
List(Vec<Json>),
Object(Vec<(String, Json)>),
}
Reading is a function for each kind, picked by the first character:
" begins a text, [ a list, { an object. Writing is the same
match the other way round. The fiddly part is text: \n and \"
going out, and é coming in.
at saves a lot of code in the server. A message is objects inside
objects, and
params.at(&["textDocument", "uri"])
walks down them and gives None if any step is missing.
The wire
Ahead of each message stands a header that says how long it is:
Content-Length: 44\r\n
\r\n
{"jsonrpc":"2.0","id":3,"method":"shutdown"}
read_message reads header lines up to the empty one, then exactly
that many bytes. write_message does the reverse. The length is in
bytes and not in characters, which matters as soon as a message holds
an é.
The server
pub struct Server {
open: BTreeMap<String, String>, // the text of each open file, by path
uris: HashMap<String, String>, // what the editor calls each file
shown: BTreeMap<String, Vec<Diagnostic>>, // the mistakes the editor was last told
workspace: Option<PathBuf>,
stopping: bool,
}
Server::handle takes one message and gives back the messages to
send. It reads no input and writes no output. That is why the tests
can hand it a message and look at what comes back, with no process
and no pipe:
let said = told(&open(&mut server, "main.zeno", "let count = 3\nprint(cuont + 1)\n"));
assert!(said[0].1[0].starts_with("1:6-11 "));
handle is one match on the method's name. The two that do the
work are didOpen and didChange: they put the new text into open
and call publish.
Checking the editor's text
publish checks every open file and compares with shown. Only a
file whose list of mistakes changed is sent, and a file that had
mistakes and has none now is sent an empty list, which is how an
editor learns to take the red lines away.
The check is the one zeno check does, with one difference in where
the text comes from:
fn read(&self, path: &str) -> Option<String> {
self.open.get(path).cloned().or_else(|| std::fs::read_to_string(path).ok())
}
The loader of modules.md never read files itself. It
was always handed a function that does, so that the tests could give
it files that exist only in memory. The server hands it this one: the
editor's text first, the disk second. Nothing in the loader changed
to make unsaved files work.
Which program a file belongs to
zeno check main.zeno knows the program's folder: it is the folder
of the file it was given. The editor may have drivers/uart.zeno
open, whose imports are found from the folder above. folder_of
walks up from the file to the nearest folder that has a
zeno.package or a main.zeno, and stops at the folder the editor
has open. The loader got a second door for this, modules::load_in,
which is told the folder and does not take it from the file.
What a name means
Put the cursor on a name and press "go to definition", and the editor
asks textDocument/definition with a line and a character. Rest the
mouse on it, and the editor asks textDocument/hover. Both come to
one question: what does the name at this place mean?
The checker has always known. To check count * 2 it has to find the
count that is meant, by the rules of scope. It used to forget the
answer as soon as it had the type. Now it writes each one down:
pub struct Use {
pub at: Span, // where the name is written
pub name: String,
pub ty: String, // its type at this place
pub declared: Option<Span>, // where it was declared
}
Two small pieces in src/checker.rs fill the record. declare, which
every let, parameter, function and class goes through, notes where
the declaration is. And the arm of the checker that reads a name
notes the use (the one for a name that is assigned to does the same):
ExprKind::Name(name) => match env.get(name) {
Some(variable) => {
self.used(name, &variable, span);
...
So Server::meaning has little to do:
- Find the token under the cursor, with the lexer. If it is not a name, there is no answer.
- Load and check the program, as
publishdoes. - Find the
Usethat stands at that token, and give back its type and where it was declared.
let found = [here, first].into_iter().find_map(|place| {
let at = written.tokens[place].span;
uses.iter().find(|used| used.at == at && shown_name(&used.name) == word)
})?;
Why two places, here and first? In shapes.area(3) the loader of
modules.md has made shapes.area into one name, and
the checker sees it at the place of shapes, where the line of dots
begins. So if nothing is used at the cursor's own token, the server
looks at the start of the dots, and takes what it finds only if it is
the word under the cursor. That is why area has an answer and
shapes has none: shapes alone is not a name of anything.
Nothing here reads the spelling of a declaration. In
let count = 3
function twice(count: int) -> int {
return count * 2
}
the count in the third line goes to the parameter, because that is
what the checker looked up.
Where a mistake is
The checker gives a line and a column, both from 1, counted in
characters. The protocol wants them from 0, and counts a character as
UTF-16 does: most are one unit and an emoji is two. diagnostic does
that sum, and stretches the mark to the end of the word, because the
checker gives only where a mistake starts.
What is left out
- Where a field or a method was declared. Resting the mouse on
the
xofp.xsaysx: int, and onqueue.putwhatputtakes and returns: the checker notes each read of a field with its type (Checker::field_used, andatinExprKind::Fieldis where the name is written). "Go to declaration" on a field still gets no answer, because a field does not yet keep where it was written (17.4). - Names in other places than code. A type after a colon
(
p: Point), a case in amatch, and a name inside anf"..."text get no answer either. - Completion and rename (9.11).
- Colours from the server. A grammar in the editor does that (9.13).
- Changes sent piece by piece. The editor sends the whole text each time. For files of the size Zeno has that is fast enough.
zeno.packageis not checked as you type.zeno checkshows its mistakes.- The settings for VS Code, Neovim and Helix have not been run in
those editors. The server is tested by its own tests and by
editors/talk.py, and the Vim colours were tried in Vim.
Things to try
- Run
python3 editors/talk.py target/debug/zeno examples/shapes.zenoand find the three kinds of message: a request, its answer, and a notification. How do you tell them apart? - In
editors/talk.py, change the typed line toprint(1 + "one"). Which characters does the server mark, and what is the message? - Change it to two mistakes on two lines. How many items does
diagnosticshave? Now make the second mistake a mistake of syntax, such aslet = 3. How many now, and why? (Which stage stops first?) - Start
target/debug/zeno lspin a terminal and type a message by hand:Content-Length: 44, Enter twice, then{"jsonrpc":"2.0","id":3,"method":"shutdown"}. You need the length right. What comes back? - In
src/lsp.rs, find the testa_program_in_several_files_is_checked_from_the_editor_s_text.shapes.zenoin it is on no disk. Which three lines of the server make that work? - Add a method to
handle: answertextDocument/referenceswith every place the name under the cursor is used.Types::uses()has all of them: the ones you want have the samedeclaredas the onemeaningfound. Add("referencesProvider", Json::Bool(true))to the capabilities, and try it in your editor or with a line more intalk.py. Json::Objectkeeps its members in aVecand not in aHashMap. Find the test insrc/json.rsthat would fail with aHashMap, and say why.- In the test
a_name_says_what_it_is_and_where_it_was_declared,countis asked about on two lines and goes to two places. Which line of the checker made the difference? Then say why asking aboutshapesgives nothing whilearearight after it gives a line of another file. - Set your editor up from
editors/README.md, openexamples/thermostat/main.zeno, and misspellfilters.lowpass. Then openexamples/filters/lowpass.zenoand break it. In which file does the mistake show?
This page is docs/lsp.md in the language repository.