What was built
A GPT-2-style decoder: byte-level BPE tokens (4,096, learned from the text) + learned positions → → LayerNorm → logits tied to the token embedding. Every piece — LayerNorm, causal multi-head attention, the GELU MLP, the residual stream, AdamW with warm-up and cosine decay — is written out in biblegpt/model.py and biblegpt/train.py; PyTorch supplies tensors and autograd, nothing else. The corpus is the 66-book World English Bible and nothing else.
What the numbers say
Trained on a two-core CPU in minutes. On the six books it never read it reaches nats per token = bits per word, against for a bigram model of the same books — a small model, honestly measured; the widening gap between the two curves is memorisation setting in. GPT-2 small on a rented GPU is the next page.