How to read this
Score is average negative log loss per predicted character over the
10,000 hidden (context, next char) pairs for that corpus. The
answers are not in the shared folder, so it measures the model, not the memory.
Units changes only how it is written: bits = nats / ln 2, and perplexity =
enats = 2bits, the number of equally likely characters the model is
effectively choosing between. Percentages and bars are ratios in whichever units are
shown, so nats and bits agree and perplexity does not: a model at 79% of the 0-gram log
loss can be at under half of the 0-gram perplexity.
0-gram entropy is the corpus's unigram character entropy — the number we computed in class, with one boundary token per item. It is what you pay knowing only how often each letter appears, so it is the 100% mark. Bigram is a context-1 model with add-one smoothing, fitted on the whole training corpus: the second standing baseline, and the one a real model has to beat.
context and reg were read back out of each submitted probability array, not taken from anyone's report. An n-gram of context C has to give identical probabilities to any two test rows whose last C characters agree, so C is the smallest width for which that holds. And within one context row the stored values are (count + reg) / (N + reg·V), so the smallest gap between two distinct values is 1/(N + reg·V) — and the smallest value divided by that gap is reg.
unseen counts hidden test rows where the model had no counts at all for that context and fell back to a uniform guess. It climbs fast with context length; compare it against the score. verbatim counts generated lines that already appear in the training corpus — memorized rather than invented — and is not computed for books corpora, where a line is not an item.