kanso 簡素
chapter 13

a json library

the repository ships lib/json, a complete json decoder and encoder that handles escapes, \uXXXX, ints and floats, and failure positions. it is six files, and this chapter reads the core of each. every panel titled lib/json/… is cut from the real file, whole declarations unchanged, and the book's build fails if a panel stops matching. the other samples either run the library or replay one of its moves on a page. first, what using it looks like:

using/main.kso
import "./using"
import "std/testing"

ok = decode "\{\"name\": \"kanso\", \"tags\": [\"fast\", \"simple\"]}"
bad = decode! "\{\"name\": nope}"
ok .> (v -> print (describe v) .> (_ -> print "recoded: {encode v}"))
  .> (_ -> print (testing/when_failed bad told))
kanso run using
decoded: { "name":"kanso" "tags":["fast" "simple"] }
recoded: {"name":"kanso","tags":["fast","simple"]}
parse failed at byte 10: invalid literal

the module behind this panel is the library itself: the sample directory links the real lib/json files beside a small app.kso that defines describe and told. decode asks: it answers the value, or none for text that is not json. decode! insists: it answers a box holding the value or a failure that says where the parser stopped. here ok comes from decode, is opened with .>, printed and encoded back. bad comes from decode!, and its failure is read through std/testing's when_failed, because a rescue opens only a failure another module raised and this sample links json into its own. the failure reads byte 10, invalid literal: the parser saw n, committed to null, and reported where the input stopped matching.

six files, one namespace

the library is a module, so chapter 07 already described its shape: one namespace, split into files by topic.

  • json.kso—the public api: decode, encode, the failure types
  • value.kso—parsing values: objects, arrays, literals
  • text.kso—strings, escapes, hex digits
  • number.kso—ints, floats, and which bytes may start them
  • scan.kso—whitespace and low-level cursor work
  • json_test.kso—the suite

value.kso calls scan.kso's helpers with no import, and the test file sees only what the module marks pub, like any importer. the api file is where to start reading:

lib/json/json.kso
pub type json_null

pub type parse_failure
  position
  reason

type parsed
  pos
  value

# Text that is not JSON answers none, an outcome the caller handles. The
# decoder's failure is a box, and `plain` takes the box itself rather than
# opening it, so what went wrong is left to `decode!`.
pub fn decode s
  plain (document s)

# `decode!` insists the text is JSON. Like every `!` name it answers a box:
# the value, or a failure that bubbles carrying a `parse_failure`, which says
# where the decoder stopped and why.
pub fn decode! s
  effect (document s)

fn document s
  cs = text/bytes s
  finish cs (parse_value cs 1)

fn plain _:<some>effect
  none

fn plain v
  v

fn finish _ none
  err (parse_failure 0 "unexpected end of input")

fn finish cs (parsed p v)
  p2 = skip_ws cs p
  if (p2 > length cs) v (fail p2 "unexpected trailing characters")

decode and decode! share document, which turns the text into bytes and hands them to parse_value at position 1. every step of the parser answers a parsed, a record of the position it reached and the value it read, because the next step needs to know where to continue. finish makes the last two checks by dispatch: none means the input ended too early, and a parsed is accepted only if nothing but whitespace follows it, so decode "1 x" fails at byte 3. the two public forms differ only in how a failure comes back. decode! hands back the box as it is. decode passes it to plain, whose first arm takes any effect and answers none.

enso-neko - sleeps in a circle she never quite closes
ensō-neko

count what this library does not contain: no token type, no lexer, no visitor, and no code that checks whether the previous step failed. a failure passes every function it reaches, so the parser's happy path is the whole parser.

the next byte is the dispatch

most hand-written parsers begin with a lexer, a token type and a loop with a switch in it. this one has none of those. it dispatches on the next byte, which is already a value, using chapter 03's overloads:

lib/json/value.kso
fn parse_value cs p
  value_for cs[p] cs p

# value starts by byte: 34 quote, 91 bracket, 102 f, 110 n, 116 t, 123 brace
fn value_for 34 cs p
  parse_string cs (p + 1)

fn value_for 91 cs p
  parse_array cs (p + 1)

fn value_for 102 cs p
  matched = alse? cs p
  word matched p 5 false

fn value_for 110 cs p
  matched = ull? cs p
  word matched p 4 json_null

fn value_for 116 cs p
  matched = rue? cs p
  word matched p 4 true

fn value_for 123 cs p
  parse_object cs (p + 1)

fn value_for none _ p
  fail p "unexpected end of input"

fn value_for c cs p
  numeric = number_start? c
  if numeric (parse_number cs p) (value_blank c cs p)

fn value_blank c cs p
  blank = ws? c
  if blank (value_for cs[p + 1] cs (p + 1)) (bad_value_char c p)

# The first byte chose the arm, so what is left is the tail of one known
# keyword, compared where it sits. Slicing it out built a byte string to throw
# away, and comparing that to a list of codes walked both element by element
# with a tag check on each. Reading past the end answers none, which matches
# nothing, so a truncated literal needs no length test of its own.
fn alse? cs p
  cs[p + 1] == 97 and cs[p + 2] == 108 and cs[p + 3] == 115 and cs[p + 4] == 101

fn ull? cs p
  cs[p + 1] == 117 and cs[p + 2] == 108 and cs[p + 3] == 108

fn rue? cs p
  cs[p + 1] == 114 and cs[p + 2] == 117 and cs[p + 3] == 101

fn word true p n v
  parsed (p + n) v

fn word false p _ _
  fail p "invalid literal"

value_for has eight arms. six are the bytes that begin a string, an array, an object or one of the three keywords. one takes none: reading past the end of the bytes answers none, so running out of input is one more value to dispatch on. the last arm takes everything else. number_start? asks whether the byte can begin a number, and value_blank skips a blank byte and dispatches again, or names the unexpected character. each keyword has a checker for its own tail. the first byte already chose the arm, so rue? compares the three bytes after it where they sit, and word turns that verdict into a parsed or a failure. here is that path as a small project of its own: lit holds the decoder, report prints its answers, and the entry tries four inputs:

literal/lit/lit.kso
import "std/text"

pub type parse_failure
  position
  reason

# The first byte chose the arm, so what is left to check is the tail of one
# known keyword, compared where it sits. Reading past the end answers `none`,
# which equals no byte, so a truncated literal needs no length test.
fn alse? cs p
  cs[p + 1] == 97 and cs[p + 2] == 108 and cs[p + 3] == 115 and cs[p + 4] == 101

pub fn decode_literal s
  cs = text/bytes s
  value_for cs[1] cs 1

fn fail p reason
  parse_failure p reason

fn rue? cs p
  cs[p + 1] == 114 and cs[p + 2] == 117 and cs[p + 3] == 101

fn ull? cs p
  cs[p + 1] == 117 and cs[p + 2] == 108 and cs[p + 3] == 108

fn value_for 102 cs p
  matched = alse? cs p
  word matched p false

fn value_for 110 cs p
  matched = ull? cs p
  word matched p "null"

fn value_for 116 cs p
  matched = rue? cs p
  word matched p true

fn value_for none _ p
  fail p "unexpected end of input"

fn value_for c _ p
  fail p "unexpected character `{text/utf8 [c]}`"

fn word true _ v
  v

fn word false p _
  fail p "invalid literal"
literal/report/report.kso
import "../lit"

# The reporting lives here rather than beside the decoder, because the
# decoder refuses a document and says nothing about what a caller does with
# that. A foreign type is named, never destructured (membership crosses an
# import where structure does not), so its fields are read by name.
pub fn show r:lit/parse_failure
  "err at {r.position}: {r.reason}"

pub fn show v
  "ok: {v}"
literal/main.kso
import "./lit"
import "./report"

print (show (decode_literal "true"))
  .> (_ -> print (show (decode_literal "null")))
  .> (_ -> print (show (decode_literal "nope")))
  .> (_ -> print (show (decode_literal "?")))
kanso run literal
ok: true
ok: null
err at 1: invalid literal
err at 1: unexpected character `?`

the tail predicates compare bytes: 114, 117, 101 is rue in utf-8. nope starts with n, so the parser commits to null and word reports the mismatch. ? starts no value, so the catch-all arm names it. in the library a bad literal becomes an err deep inside and comes back out with its position, past every function in between, and none of them mentions it. the sample's fail answers a parse_failure record instead, the shape chapter 04 gives an outcome the caller handles, and report's show reads the position from it.

arrays and objects work the same way one level up. array_delim dispatches on the byte after an element: 44, a comma, means more items, 93 means the array is done, none means the input ended inside it, and anything else is whitespace to skip or a mistake to name. the object path adds a key: obj_key_start insists on a quote, obj_key_end demands the colon, and obj_value puts each pair into the map. every step passes its whole state, the bytes and a position, forward as arguments, so there is no parser state to restore. the comments in value.kso record why some of these states call themselves and others hand off to a twin: each choice was measured, in instructions, on the benchmark chapter 15 describes.

strings and the escape path

strings are where a json parser spends most of its time. parse_string reads a string in one runtime call when it can. builtin_plain_string answers the text in front of the closing quote when the string has no escape in it, or the closing quote's position when its escapes are the simple one-letter kind, which builtin_unescape then undoes. anything harder answers none, and string_at reads the string the long way:

lib/json/text.kso
# A string is read in one call where it can be. With no escape in it, the call
# answers the text in front of the closing quote. With escapes, it answers the
# closing quote's position, and `unescape` undoes the eight one-letter escapes
# over that range. A `\u` escape, any other escape, a string that runs out, or
# bytes that are not utf-8 answer none at either step, and the string is read
# the long way below, which is where each of those is undone or named.
fn parse_string cs p
  plain_at cs p (builtin_plain_string cs p)

fn plain_at cs p none
  string_at cs (text/span_to cs p 34 92)

fn plain_at cs p e:int
  undone cs p e (builtin_unescape cs p e)

fn plain_at _ p s
  parsed (p + 1 + length (text/bytes s)) s

fn undone cs p _ none
  string_at cs (text/span_to cs p 34 92)

fn undone _ _ e s
  parsed (e + 1) s

# The run up to the first quote or backslash is a span the scan proved
# inside cs, so slicing it cannot miss. A string ending in its last byte
# reaches one past the end, where span_to answers the empty run.
fn string_at cs none
  fail (length cs + 1) "unterminated string"

fn string_at cs run
  string_end cs cs[run.to + 1] run

fn string_end _ 34 run
  p = run.to + 1
  parsed (p + 1) (string_ok p (text/utf8 (text/slice run)))

fn string_end cs 92 run
  fresh = text/bytes ""
  str_chars cs (run.to + 1) (text/append fresh (text/slice run))

fn string_end _ none run
  fail (run.to + 1) "unterminated string"

text/span_to answers a span: the bytes it lies in and two positions, from and to, running up to the next quote (34) or backslash (92). string_end dispatches on the byte that stopped it. a quote ends the string, and its value is one slice of the input. a backslash moves the bytes so far into an accumulator, and the slow path appends decoded bytes one escape at a time. none means the input ended inside the string. slicing a span cannot fail, because only std/text builds one, and only after checking that its range lies inside the bytes. the slow path is another dispatch table, one arm per escape character:

lib/json/text.kso
fn str_escape cs 34 p acc
  str_chars cs (p + 2) (text/append acc 34)

fn str_escape cs 47 p acc
  str_chars cs (p + 2) (text/append acc 47)

fn str_escape cs 92 p acc
  str_chars cs (p + 2) (text/append acc 92)

fn str_escape cs 98 p acc
  str_chars cs (p + 2) (text/append acc 8)

fn str_escape cs 102 p acc
  str_chars cs (p + 2) (text/append acc 12)

fn str_escape cs 110 p acc
  str_chars cs (p + 2) (text/append acc 10)

fn str_escape cs 114 p acc
  str_chars cs (p + 2) (text/append acc 13)

fn str_escape cs 116 p acc
  str_chars cs (p + 2) (text/append acc 9)

fn str_escape cs 117 p acc
  str_unicode cs p acc

fn str_escape _ none p _
  fail p "unterminated string"

fn str_escape _ c p _
  fail p "invalid escape `\\{text/utf8 [c]}`"

# four hex digits: a bad digit is a failure, which passes `hex_on` by, so the
# first bad digit is the failure handed on
fn str_hex4 cs p
  hex_on (hex_three cs p) (hex_digit cs[p + 3])

fn hex_three cs p
  hex_on (hex_on (hex_digit cs[p]) (hex_digit cs[p + 1])) (hex_digit cs[p + 2])

fn hex_on n d
  n * 16 + d

# a surrogate is half a character: the high half joins the `\u` low half
# written beside it, and either half alone is a failure
fn str_surrogate cs p acc high
  if (56319 < high)
    fail p "a low surrogate arrived without its high half"
    str_pair cs p acc high

fn str_unicode cs p acc
  unicode_at cs p acc (str_hex4 cs (p + 2))

fn unicode_at cs p acc code
  if (code < 55296 or 57343 < code)
    str_chars cs (p + 6) (text/append acc (text/from_code code))
    str_surrogate cs p acc code

eight arms handle the one-letter escapes (110 is \n), one handles \u, and two name what went wrong: the input ended, or the escape is not one json has. for \uXXXX, str_hex4 reads four hex digits into a code point. a bad digit is a failure, and a failure passes hex_on by without running its body, so the first bad digit is the failure that comes out. text/from_code turns the code point into a character. a code point in the surrogate range is half a character, and str_surrogate looks for the other half written beside it. the digit functions are small enough to run on their own:

unicode.kso
import "std/text"

fn decode_u s
  cs = text/bytes s
  char_of (hex_on (hex_three cs 1) (hex_digit cs[4]))

fn char_of code
  text/from_code code

fn hex_three cs p
  hex_on (hex_on (hex_digit cs[p]) (hex_digit cs[p + 1])) (hex_digit cs[p + 2])

fn hex_on n d
  n * 16 + d

fn hex_alpha c
  if (96 < c and c < 103) (c - 87) (hex_upper c)

fn hex_digit none
  err "unexpected end of input"

fn hex_digit c
  if (47 < c and c < 58) (c - 48) (hex_alpha c)

fn hex_upper c
  if (64 < c and c < 71) (c - 55) (err "invalid hex digit")

print (decode_u "00e9")
  .> (_ -> print (decode_u "305f"))
  .> (_ -> print (decode_u "7b80"))
  .> (_ -> print (decode_u "7g80"))
kanso play unicode.kso
é
た
简
error[endpoint]: unhandled err reached the executor: "invalid hex digit"
  born in hex_upper at unicode.kso:26
  passed through char_of ← hex_on ← hex_on ← hex_on

hex_digit, hex_alpha and hex_upper are the library's, unchanged: they try digits, then lowercase letters, then uppercase, and a byte that is none of them is the failure. 7b80 is 简, the first character of 簡素. 7g80 has a g in it, and the report shows its failure passing through hex_on three times and char_of once on its way to the top, none of which mention it.

mugi the tanuki - runs the interpreter; pays for everything in boba pearls
mugi

the plain subscript starts at 1 and answers none past the end, and that is the parser's whole end-of-input story. there is no length check before a read and no eof token: read the byte, and if the input is over, none goes into the same dispatch as every other byte. count the none arms across the six files. each is a bounds check another parser would have had to remember.

numbers without a lexer

numbers take one runtime call. text/number_span starts at a number's first byte and answers the position just past its last byte, negated when a ., e or E went past. it answers zero for bytes rfc 8259 refuses as a number, such as a leading zero followed by another digit, or a minus or a point with no digit after it. the runtime classifies sixteen bytes at a time, where a walk written in kanso would dispatch on each one. so the library knows where the number ends and which converter to call before it converts anything:

lib/json/number.kso
import "std/text"

# Where the number's bytes end, and whether a `.`, `e` or `E` went past, in
# one call: the runtime classifies sixteen bytes a step where a walk here
# dispatched on every byte. A negative answer is the end of a float, and zero
# is bytes that are not a number as RFC 8259 writes one (#1999): the integer
# path then asks for a span that ends before it starts, gets none, and fails
# as `invalid number`.
fn scan cs start p
  spanned cs start (text/number_span cs p)

fn spanned cs start e
  if (e < 0) (number_done cs start (0 - e) true) (number_done cs start e false)

# The number's bytes are a span the scan proved inside cs.
fn number_done cs start p true
  parsed p (number_ok start (float_of (text/to_span cs start (p - 1))))

fn number_done cs start p false
  parsed p (number_ok start (int_of (text/to_span cs start (p - 1))))

fn float_of none
  none

fn float_of run
  text/to_float (text/slice run)

fn int_of none
  none

fn int_of run
  text/to_int (text/slice run)

fn number_ok p none
  fail p "invalid number"

fn number_ok _ n
  n

fn number_start? 45
  true

fn number_start? none
  false

fn number_start? c
  47 < c and c < 58

here is the same path on its own, with the library's spanned and number_done folded into one converted:

numbers.kso
import "std/list"
import "std/text"

# The library's number path on its own. One call says where the number's
# bytes end and whether a float mark went past, and that picks the converter.
fn read_number s
  cs = text/bytes s
  converted cs (text/number_span cs 1)

fn converted _ 0
  "refused"

fn converted cs e
  if (e < 0) (float_of (bytes_to cs (0 - e))) (int_of (bytes_to cs e))

# The span ends one byte before the position number_span answers.
fn bytes_to cs p
  text/to_span cs 1 (p - 1)

fn float_of none
  "refused"

fn float_of run
  "float64 {text/to_float (text/slice run)}"

fn int_of none
  "refused"

fn int_of run
  "int {text/to_int (text/slice run)}"

fn line s
  "{s} -> {read_number s}"

inputs = ["42" "-17" "3.25" "6e3" "007" "1."]
print (text/join (list/to_list (list/map inputs line)) "\n")
kanso play numbers.kso
42 -> int 42
-17 -> int -17
3.25 -> float64 3.25
6e3 -> float64 6000.0
007 -> refused
1. -> refused

an integer stays an integer: 42 decodes to the int 42, and anything with a point or an exponent becomes a float, so 6e3 is 6000.0. 007 and 1. are refused before any converter sees them. in the library, a converter that still says no is caught by number_ok, which turns the refusal into a parse_failure at the position where the number began.

failure carries its position

every failure in the library is made by one function, in the smallest file:

lib/json/scan.kso
fn fail p reason
  err (parse_failure p reason)

fn skip_ws cs p
  if (ws? cs[p]) (skip_ws cs (p + 1)) p

# json whitespace bytes: tab, newline, carriage return, space
fn ws? 9
  true

fn ws? 10
  true

fn ws? 13
  true

fn ws? 32
  true

fn ws? none
  false

fn ws? _
  false

fail wraps a parse_failure, the byte position and a reason, in an err. because the position is always an argument, it is always at hand when something goes wrong. there is no current location to reconstruct from a lexer's state. the project below is finish's trailing-characters check in miniature: skip whitespace, read an integer, skip whitespace again, and demand that the input be over. pos keeps the library's whitespace functions and answers the record itself, the shape for an outcome its caller handles:

positions/pos/pos.kso
import "std/text"

pub type parse_failure
  position
  reason

pub fn decode_int s
  cs = text/bytes s
  p = skip_ws cs 1
  q = digits_end cs p
  v = digits_value p (text/slice cs p (q - 1))
  trailing cs (skip_ws cs q) v

fn digits_end cs p
  if (digit? cs[p]) (digits_end cs (p + 1)) p

# the run stops where the digits do, inside `cs`, so it is never none; it is
# empty when no digit came first, and `to_int` answers none for that
fn digits_value _ none
  0

fn digits_value p run
  number_at p (text/to_int run)

fn number_at p none
  fail p "expected a digit"

fn number_at _ n
  n

fn fail p reason
  parse_failure p reason

fn digit? none
  false

fn digit? c
  47 < c and c < 58

fn ws? 9
  true

fn ws? 10
  true

fn ws? 13
  true

fn ws? 32
  true

fn ws? none
  false

fn ws? _
  false

fn skip_ws cs p
  if (ws? cs[p]) (skip_ws cs (p + 1)) p

fn trailing _ _ f:parse_failure
  f

fn trailing cs p v
  if (p > length cs) v (fail p "unexpected trailing characters")
positions/report/report.kso
import "../pos"

# The reporting lives here rather than beside the decoder, because the
# decoder refuses a document and says nothing about what a caller does with
# that. A foreign type is named, never destructured (membership crosses an
# import where structure does not), so its fields are read by name.
pub fn show r:pos/parse_failure
  "err at {r.position}: {r.reason}"

pub fn show v
  "ok: {v}"
positions/main.kso
import "./pos"
import "./report"

print (show (decode_int "  42  "))
  .> (_ -> print (show (decode_int "1 x")))
  .> (_ -> print (show (decode_int "  7 ;")))
  .> (_ -> print (show (decode_int "  ;")))
kanso run positions
ok: 42
err at 3: unexpected trailing characters
err at 5: unexpected trailing characters
err at 3: expected a digit

positions point at the first byte the parser could not accept: the x in 1 x sits at byte 3, the ; at byte 5, and a document with no digit in it is refused where the digit should have been.

chapter 04 drew the line: a failure a caller might reasonably handle is a value to dispatch on, and a failure nobody should handle rises to the endpoint. json offers both, and the caller picks one by the name it calls. decode answers none for text that is not json, because malformed input is often the caller's business: maybe it retries, maybe it asks a user again. decode! is for input you control, a config file baked into the program or a fixture, where a parse failure is a bug with your name on it, and its failure carries the position and the reason to the endpoint. json does not rescue its own failures. the library has nothing to say about a document it has already refused, and a rescue inside it would decide for every caller at once. the suite reads the position from outside, through std/testing.

the suite

a file of booleans pins the library: every kind of value, the escapes in both directions, unicode, whitespace, error positions, and a round trip.

lib/json/json_test.kso
import "std/testing"
import "std/text"

# `when_failed` hands back the reason out of the box, so this names a reason
# type and never an err — json cannot rescue what json raised.
test_decode_array = decodes? "[1, 2, 3]" [1 2 3]

test_decode_bool = decode "true"

test_decode_escapes = decodes? "\"a\\n\\\"b\\\"\"" "a\n\"b\""

test_decode_float = decodes? "3.25" 3.25

test_decode_exponent = decodes? "1e3" 1000.0

test_decode_exponent_upper = decodes? "2E2" 200.0

# the mark is a `.` here and an `e` further on: a scan that forgot the first
# one on reaching the second would hand these digits to the integer reader
test_decode_exponent_signed = decodes? "1.5e2" 150.0

test_decode_int = decodes? "42" 42

test_decode_negative = decodes? "-7" (0 - 7)

test_decode_nested =
  decoded = decode "\{\"a\": [1, \{\"b\": null}]}"
  same? decoded { "a":[1 { "b":json_null }] }

test_decode_object = decodes? "\{\"a\": 1, \"b\": 2}" { "a":1 "b":2 }

test_decode_string = decodes? "\"hello\"" "hello"

test_decode_surrogate_pair = decodes? "\"\\ud83c\\udf8c\"" "🎌"

test_decode_unicode = decodes? "\"\\u00e9\"" "é"

test_decode_whitespace = decodes? "  [ 1 , 2 ]  " [1 2]

test_encode_empty_list = encode [] == "[]"

test_encode_empty_map = encode {} == "\{}"

test_encode_escapes = encode "a\"b\nc" == "\"a\\\"b\\nc\""

test_encode_shorthands =
  raw = text/utf8 [120 8 121 12 122]
  encode raw == "\"x\\by\\fz\""

test_encode_nested =
  value = { "a":true "b":[1 2] }
  encode value == "\{\"a\":true,\"b\":[1,2]}"

test_error_position =
  decoded = decode! "[1, nope]"
  testing/when_failed decoded (r -> r.position == 5)

test_error_trailing =
  decoded = decode! "1 x"
  testing/when_failed decoded (r -> r.position == 3)

test_plain_decode_answers_none = decode "[1, nope]" == none

test_roundtrip =
  text = "\{\"a\":[1,2.5,\"x\"],\"b\":null}"
  encodes_back? (decode text) text

# `decode` answers none for bad input, so a test dispatches on what it
# decoded before comparing it.
fn decodes? text want
  same? (decode text) want

fn same? none _
  false

fn same? got want
  got == want

fn encodes_back? none _
  false

fn encodes_back? v text
  encode v == text
kanso test lib/json
test_decode_array ... ok
test_decode_bool ... ok
test_decode_escapes ... ok
test_decode_float ... ok
test_decode_exponent ... ok
test_decode_exponent_upper ... ok
test_decode_exponent_signed ... ok
test_decode_int ... ok
test_decode_negative ... ok
test_decode_nested ... ok
test_decode_object ... ok
test_decode_string ... ok
test_decode_surrogate_pair ... ok
test_decode_unicode ... ok
test_decode_whitespace ... ok
test_encode_empty_list ... ok
test_encode_empty_map ... ok
test_encode_escapes ... ok
test_encode_shorthands ... ok
test_encode_nested ... ok
test_error_position ... ok
test_error_trailing ... ok
test_plain_decode_answers_none ... ok
test_roundtrip ... ok
24 passed, 0 failed

each test is one comparison. test_error_position asserts that decoding [1, nope] fails at byte 5. it goes through std/testing, which hands the reason record to a function, because a rescue opens only a failure another module raised and the suite is inside json. test_plain_decode_answers_none pins the other form: decode answers none for the same text. test_roundtrip decodes a document, encodes the result, and demands the original text back. it holds because encode writes one canonical form, with no spaces and keys in map order, so canonically written input survives the round trip byte for byte.

encode and pretty-printing

the encoder has one arm per kind of value, and every arm appends to the same byte builder, so each byte of the output is written once however deep in the tree it was born. encode starts the builder and turns the finished bytes into a string:

lib/json/json.kso
# one byte accumulator threads the whole tree, so every emitted byte is
# copied once into the builder and once into the final string
fn elem_onto acc x
  encode_onto (text/append acc 44) x

# Everything the encoder appends is text or an ascii byte, so the bytes it
# hands back are text already and `built_text` takes them without checking.
pub fn encode v
  builtin_built_text (encode_onto (text/bytes "") v)

# the bound is written where the read is, so the checker proves the read in
# range and the plain form answers the element (ruled 2026-09-16)
fn encode_items acc xs i
  return acc if i < 1 or length xs < i
  encode_items (elem_onto acc xs[i]) xs (i + 1)

fn encode_list acc xs
  return text/append acc "[]" if length xs == 0
  opened = encode_onto (text/append acc 91) xs[1]
  text/append (encode_items opened xs 2) 93

fn encode_onto acc true
  text/append acc "true"

fn encode_onto acc false
  text/append acc "false"

fn encode_onto acc json_null
  text/append acc "null"

fn encode_onto acc n:int
  text/append acc "{n}"

fn encode_onto acc x:float64
  text/append acc "{x}"

fn encode_onto acc s:string
  builtin_append_string acc s

fn encode_onto acc xs:[]some
  encode_list acc xs

fn encode_onto acc m:map[string some]
  encode_map acc m (builtin_entry_count m)

the literal arms dispatch on a value, and the last five dispatch on a type: n:int, xs:[]some, m:map[string some], the other half of chapter 03. text/append is a builder that owns its buffer and extends it in place when it can, so the fold of appends costs about one pass over the output, while every intermediate stays an ordinary value.

the arm with real work is strings, and it is one call into the runtime: builtin_append_string writes the opening quote, the string with what json cannot hold escaped, and the closing quote. the runtime looks at eight bytes at a time for a quote, a backslash or a control byte, copies a clean run whole, and writes the escape for what it finds, so a string with nothing to escape costs about one copy. the sample spells the same escapes in kanso, a byte at a time, and quotes two awkward strings: one with a quote and a newline, one with a bell character (byte 7) in the middle:

escapes.kso
import "std/list"
import "std/text"

fn esc_byte acc 9
  text/append (text/append acc 92) 116

fn esc_byte acc 10
  text/append (text/append acc 92) 110

fn esc_byte acc 13
  text/append (text/append acc 92) 114

fn esc_byte acc 34
  text/append (text/append acc 92) 34

fn esc_byte acc 92
  text/append (text/append acc 92) 92

fn esc_byte acc b
  if (b < 32) (u_bytes acc b) (text/append acc b)

fn escape_able acc bs
  list/fold bs acc (a b -> esc_byte a b)

fn escape_clean acc s bs n
  if (length bs < n) (text/append acc s) (escape_able acc bs)

fn escape_onto acc s
  bs = text/bytes s
  escape_clean acc s bs (text/find2_below bs 1 34 92 32)

fn escape_str s
  text/utf8 (escape_onto (text/bytes "") s)

hex_byte_table = text/bytes "0123456789abcdef"

fn hex_code n
  hex_digit_of hex_byte_table[n + 1]

# `n` is a nibble, so the read is inside the table and a none is a bug
fn hex_digit_of none
  err "a nibble outside 0 to 15"

fn hex_digit_of b
  b

pub play =
  bell = text/from_code 7
  print (quote_str "a\"b\nc") .> (_ -> print (quote_str "ring{bell}ring"))

fn quote_str s
  "\"{escape_str s}\""

fn u_bytes acc b
  lo = hex_code (b % 16)
  hi = hex_code (b / 16)
  text/append (text/append (text/append acc "\\u00") hi) lo
kanso run escapes.kso
"a\"b\nc"
"ring\u0007ring"

encode writes no whitespace, which is what machine-to-machine json wants. a pretty-printer is the same fold with an indent depth threaded through, and it is small enough to write in full. it builds a string per node because that reads best on a page; threading the byte builder through it instead is a good exercise, with the library's encode as the worked answer:

pretty.kso
import "std/list"
import "std/text"

type json_null

fn entry_block (entry k v) d
  "{pad d}\"{k}\": {pretty v d}"

fn pad 0
  ""

fn pad n
  "  {pad (n - 1)}"

doc = { "name":"kanso" "port":8080 "tags":["fast" "simple"] }
print (pretty doc 0)

fn pretty true _
  "true"

fn pretty false _
  "false"

fn pretty json_null _
  "null"

fn pretty n:int _
  "{n}"

fn pretty x:float64 _
  "{x}"

fn pretty s:string _
  "\"{s}\""

fn pretty xs:[]some d
  d2 = d + 1
  lines = list/to_list (list/map xs (v -> "{pad d2}{pretty v d2}"))
  inner = text/join lines ",\n"
  "[\n{inner}\n{pad d}]"

fn pretty m:map[string some] d
  d2 = d + 1
  lines = list/to_list (list/map (entries m) (e -> entry_block e d2))
  inner = text/join lines ",\n"
  "\{\n{inner}\n{pad d}}"
kanso play pretty.kso
{
  "name": "kanso",
  "port": 8080,
  "tags": [
    "fast",
    "simple"
  ]
}

it has the shape of encode with one extra argument. the scalar arms ignore the depth with _, and the compound arms deepen it and put each element on its own indented line. the padding is a recursion whose base case is an arm for 0.

what you can do now

you have read the core of a real library, and nothing in it was new: byte dispatch is chapter 03, failure that passes through is chapter 04, the module is chapter 07, and the runtime calls are named where they appear. you can find your way around lib/json by name and extend it in its own style. you can also recognize its shapes elsewhere: a fast path that bets on the common case and a slow path that handles the rest, a position passed as an argument so every failure knows where it happened, and a caller choosing between a handled outcome and a failure with one character. the next chapter uses the same language to simulate elections.

exercises

  1. make pretty the module's seventh file. add pretty.kso to a copy of lib/json, give its string arm the escaping encode_onto gets from builtin_append_string, and add test_pretty_roundtrip: decoding a pretty-printed document must equal decoding the compact original.
  2. number_span refuses 007 before any converter sees it. predict the position and reason decode! "[1, 007]" reports, then pin them with a test beside test_error_position.
  3. str_surrogate refuses a low surrogate that arrives alone. write a test for decode! "\"\\udc00\"" that pins the reason, and read str_pair to find the other way a pair can be malformed.
  4. the suite pins positions for a bad literal and trailing characters, but not for an unterminated string. write test_error_unterminated asserting the position decode "\"abc" fails at, predicting the number before you run it.