F. Validation log

A book that teaches “test against a trusted reference” should say what it tested itself. This appendix records how the code and the numbers in this book were checked, in which environment, and what could not be checked there.

Policy

• Every program output shown in the book was produced by running the command shown, unless the text explicitly calls it illustrative or quotes another source.
• Every engine milestone is checked by tests that the reference implementation (izh/) passes and that the generated skeleton (engine/) fails with a TODO(Chapter N) message.
• Architectures are checked against independent reference implementations (Hugging Face Transformers) with random weights in FP32 before any real checkpoint is involved.

Completion environment (2026-10-05)

The original book validation ran on an x86-64 CPU-only environment. This completion pass ran on a DGX Spark with an available CUDA GPU; the results below separate these records.

No pretrained checkpoint was downloaded during this pass. The original environment reported blocked Hugging Face Hub access; current correctness checks use locally generated checkpoints, fixtures and random weights, rather than claiming real-checkpoint validation.

Completion results

The benchmark smoke uses a tiny random CPU model and a character-level test tokenizer. Its timings prove that requests, SSE parsing, usage, timestamps and shutdown work end to end. They are not representative throughput measurements and are not entered into Chapter 44’s competitor table. The quality smoke fixtures likewise do not establish trained retrieval, vision or GSM8K accuracy.

Earlier CPU-only records

The previous log recorded 144 passing and 13 GPU-skipped tests before Part VIII, 17 Rust unit tests plus 2 fixtures, and 13 C++ demos. It also recorded a full pre-Part-VIII skeleton run (125 expected failures, 5 fixture errors, 14 provided-code passes). Those historical counts are not the current suite size. The completion run checks generated-file equality and the two new chapter TODO boundaries rather than claiming another full skeleton run.

Parity with reference implementations (FP32, random weights)

Bugs found by these checks

Recorded because each one is a lesson, and each is now covered by a test:

1. Chunked Gated DeltaNet kept the wrong triangle of its system matrix. Chunk size 1 passed; sizes 5, 8 and 64 against the recurrent form caught it (Chapter 28).
2. QSA top-k tie-breaking: masking invisible blocks with $-\infty$ in a longer vector broke ReLU-zero ties differently from the reference. Fixed by running topk over exactly the visible blocks (Chapter 29).
3. Qwen3-MoE config: Transformers 5 writes num_local_experts; the loader silently used its default of 8 experts and passed tests that happened to use 8. Fixed to accept both names and refuse missing fields; tests now use non-default sizes (Chapter 27).
4. safetensors: torch.frombuffer rejects zero-length tensors; empty tensors are now created directly (Chapter 9).
5. Full fine-tuning in BF16 with AdamW at learning rate 1e-5 loses most updates to rounding; finetune.py now keeps FP32 master weights with BF16 autocast for full fine-tuning (Chapters 13, 21).
6. Test collection: a test module that built models at import time turned one missing function into a collection error for the whole suite; models are now built inside tests (Chapter 16).

What was not validated here

Passing a teaching test establishes the tested contract and shapes, not every deployment or feature combination. The following boundaries remain explicit:

The original CUDA compile-only and CPU-interpreter records remain historical evidence; current GPU correctness tests add coverage without turning estimates into measurements.

Reproducing

From the code/ directory:

IZH_IMPL=izh OMP_NUM_THREADS=2 pytest -rs  # the reference, including available GPU paths
pytest                                     # your engine: red until you implement it
(cd rust && cargo test --release)
cmake -S cpp -B build/cpp && cmake --build build/cpp -j && build/cpp/izh all
CUDA_VISIBLE_DEVICES='' TRITON_INTERPRET=1 IZH_DEVICE=cpu IZH_IMPL=izh pytest \
  tests/test_ch38_gguf.py tests/test_ch39_quantized_checkpoints.py \
  tests/test_ch43_multimodal_lora_embeddings.py tests/test_ch44_benchmarking.py
python run.py features --device cpu
python run.py benchmark --device cpu