measuring and shipping

chapter 13's wordfreq is correct and tested. this chapter asks what it costs, makes it cheaper, and keeps it that way. then it puts the binary on a server.

kanso gives two kinds of measurement. a clock tells you how long a run took, and it changes from run to run and machine to machine. a counter tells you how much work the program did: how many allocations, how many bytes. counters are deterministic. the same program on the same input gives the same numbers every time, on every engine and every machine, so a change in a counter is a change in the program. this chapter measures with counters first and reaches for the clock last.

counting

set KANSO_COUNTERS=1 and kanso prints its counters to standard error when the program ends. there are several dozen, and chapter 10 shows the full list. this chapter needs one: allocs, the number of allocations the program made. three copies of the tea text, at one, ten and a hundred times its length, show how the count grows:

allocs=366

allocs=1309

allocs=15864

the count grows in step with the input: about 160 allocations for each copy of a three-line text, or about 6 for each of its 26 words. growth in proportion to the input is the shape you want, since anything faster would mean work repeated for every word already seen. 6 allocations a word is still more than splitting and counting need.

finding the stage that costs

the program has three stages: split the text into words, count them, render the top five. the quickest way to see what one stage costs is to run it alone. this program is words from chapter 13 and nothing else:

import "std/io"
import "std/list"
import "std/regexp"
import "std/text"

io/stdin
  .> (t -> regexp/split "[^a-z']+" (text/lowercase t))
  .> (ws -> list/to_list (list/reject ws (w -> w == "")))
  .> (ws -> print "{length ws} words")

allocs=15685
2600 words

the split is almost all of it: 15,685 of the program's 15,864 allocations. counting and rendering cost under two hundred between them. inside the split, the regular-expression engine does the work, and it allocates as it walks.

the pattern [^a-z']+ says something simple: every character that is not a letter or an apostrophe separates words. that can be said without a regular expression. turn each such character into a space, and split on spaces:

import "std/io"
import "std/list"
import "std/text"

fn spaced c
  if (word_char? c) c " "

fn word_char? c
  ("a" <= c and c <= "z") or c == "'"

fn words text
  marked = list/map (text/chars (text/lowercase text)) spaced
  pieces = text/split (text/join (list/to_list marked) "") " "
  list/to_list (list/reject pieces (w -> w == ""))

io/stdin .> (t -> print "{length (words t)} words")

allocs=3128
2600 words

the same 2,600 words, from 3,128 allocations rather than 15,685. the regular expression was the right first version, because it said what a word is in eight characters. now there is a measurement that says what it costs, and a replacement that is still short enough to read.

making the change

wordfreq2 is a copy of the project with the new words in count.kso:

import "std/list"
import "std/text"

pub type ranked
  count
  word

# The words of a text, lowercased, in the order they appear. Every character
# that cannot be part of a word becomes a space, and the spaces split it.
pub fn words text
  marked = list/map (text/chars (text/lowercase text)) spaced
  pieces = text/split (text/join (list/to_list marked) "") " "
  list/to_list (list/reject pieces (w -> w == ""))

fn spaced c
  if (word_char? c) c " "

fn word_char? c
  ("a" <= c and c <= "z") or c == "'"

# The n most common words, the most common first. A map's entries come in
# key order and sort_by keeps ties in the order they arrived, so words with
# the same count stay alphabetical.
pub fn top n ws
  ranks = list/map (entries (list/tally ws)) (e -> ranked e.value e.key)
  list/to_list (list/take (list/sort_by ranks (r -> 0 - r.count)) n)

pub fn render rows
  text/join (list/to_list (list/map rows line)) "\n"

fn line (ranked count word)
  "{text/pad_start "{count}" 4 " "}  {word}"

words no longer answers an effect, because nothing in it can fail, so cli.kso calls it directly and the tests compare its answer with ==. the tests are the same five, and they still pass:

test_words_are_lowercased ... ok
test_top_two ... ok
test_defaults ... ok
test_top_and_path ... ok
test_top_needs_a_number ... ok
5 passed, 0 failed

the output on the largest text is what the old program printed, scaled by a hundred:

 500  the
 400  tea
 300  and

and the count, measured the same way as before:

allocs=3307

15,864 became 3,307, about a fifth. on a 1.27 MB text the release binaries made 1,040,272 and 310,218 allocations. kanso run and a release build count the same events, so the counters above describe the binary you ship. the regular expression used to cost more: #2214 made regexp/split stop allocating at every letter it skips, and before that the first program made 85,660 allocations on the largest text. the book's build reran these panels when it changed.

the clock, last

fewer allocations usually means less time, and the clock is what a user feels, so measure it too, with a release build and a large input, several times:

$ time ./wordfreq big.txt > /dev/null     # regexp/split, three runs
0.292s  0.275s  0.339s
$ time ./wordfreq2 big.txt > /dev/null    # chars and text/split, three runs
0.216s  0.205s  0.255s

about three quarters of the time, on a 1.27 MB text. these numbers belong to that machine on that day, which is why the panel says so, and the spread within each set of three runs is between ten and twenty percent. a clock is good for telling you which way a change went and roughly how far. it is a poor way to catch a regression of a few percent, and the counters are how you catch that.

keeping it cheap

a measurement taken once describes the program on that day. to keep the allocation count from creeping back, write it down and have ci compare it on every change. the kanso repository does this for its own benchmarks: it calls the files cost goldens. a project can keep one with a few lines of shell:

#!/bin/sh
# wordfreq's cost golden: the allocations it makes on three fixed texts.
# CI runs this and fails when the result differs from costs.txt.
set -e
for f in tea.txt tea10.txt tea100.txt; do
  printf '%s ' "$f"
  KANSO_COUNTERS=1 kanso run wordfreq2 -- "$f" 2>&1 >/dev/null | grep '^allocs='
done

tea.txt allocs=245
tea10.txt allocs=526
tea100.txt allocs=3307

in ci, the check is one line: sh costs.sh > costs.now && diff costs.txt costs.now. because the counts are exact, the comparison is equality, with no tolerance band to tune. a change that makes the program cheaper fails the diff too. regenerate costs.txt in the same change and say in the commit which way the number moved and why. a rise then has to be explained in review, and a drop is kept on record.

shipping it

kanso build . --release writes one executable named after the project directory. chapter 13 built it and looked inside: it links the c library and the maths library and nothing else. three facts decide where it can go:

• it runs on the operating system and processor that built it. kanso has no cross-compilation flag, so build on the kind of machine you deploy to.
• it links the c library dynamically, so the server's c library must be at least as new as the build machine's. building on the oldest system you deploy to avoids the problem.
• it needs nothing else: no runtime, no interpreter, no files beside it.

for a container, that means a slim image with the binary copied in:

FROM debian:stable-slim
COPY wordfreq /usr/local/bin/wordfreq
ENTRYPOINT ["wordfreq"]

build the binary on a debian stable machine (or in a debian stable container) so the two c libraries agree. the chapter 14 teashop can run as a systemd service the same way:

[Unit]
Description=the teashop order service
After=network.target

[Service]
ExecStart=/usr/local/bin/teashop
DynamicUser=yes
Restart=on-failure

[Install]
WantedBy=multi-user.target

Restart=on-failure restarts the shop if it crashes, and leaves it closed after a POST /close, because closing ends the program with status 0.

what you can do now

you can count what a program does with KANSO_COUNTERS=1, check that the count grows with the input the way it should, find the stage that costs by running it alone, and measure a replacement before you adopt it. you can keep the result in a cost golden that ci compares exactly. and you can ship the release binary to a container or a service, knowing the three things it depends on.

exercises

1. text/chars makes a string for every character, and chars.kso still allocates once for each. find a way to split with fewer, measure it the same way, and decide whether the code is still clear enough to keep.
2. add alloc_bytes to costs.sh. which of the two numbers would you rather pin, and why?
3. run KANSO_COUNTERS=1 kanso run day.kso for the chapter 14 teashop, then add a thousand orders to customer.sh. how do the counts grow with the number of orders, and what in server.kso explains it?
4. build wordfreq2 in a debian:stable container with the toolchain installed, copy the binary into the image above, and run it on tea.txt.