After “in”, the next word is “the” 35% of the time — because that is how often it was. Spin the wheel and a sentence writes itself.
after “in”, the next word is…
spin: a next word drawn in proportion
context:
–
What probability is here
Counting, then dividing. Of every time “in” appears in the Bible, how often is the next word “the”? That fraction is P(the | in). The bars are those fractions for the most frequent next words; the wheel gives each word a slice that size, and a spin draws one.
What it buys you
Spin after spin makes a sentence — the simplest language model there is, and exactly what a large one does at the last step: it turns a context into a distribution over next words and draws from it. Everything else (the transformer, the training) is about making that distribution sharper.
Evidence. Counts are over the 66-book World English Bible tokenised as in biblical_models.corpus.tokenize (– tokens); the table on this page holds the 400 most frequent words with their 12 most frequent successors, so the walk stops if it wanders off that table. Full per-context distributions and entropies are in foundations/data/probability.json.
Nine seconds elsewhere
The foundations in learning order, then the series they lead to. Each lands in about nine seconds.