Twinning Beatrix: A Full-Splat Byte Model, Its Softmax Control, and What Reaches an Image Generator

AbstractPhil · September 2026 · with Claude Fable 5.1 and Claude Fable 5 (Anthropic) · continues Raising Beatrix: A Byte-Level Model's Measured Childhood

---

This installment covers eighteen days, 2026-08-17 through 2026-09-03, and two models. The first is the graduate the last article left at step 88,508: mini-beatrix-1, the 112.5-million-parameter byte-level model, which grew an array of twenty-nine detachable arms that put her hardest wall — three-digit subtraction with borrows — under every form of supervision we could administer, with positive controls beside them. The second is her successor. mini-beatrix-2s is a 237.1-million-parameter byte craft in which every one of twenty attention blocks is splat attention — the signed, softmax-free address read that sat beside standard attention in three of her mother's sixteen layers — governed from birth and trained for 16.101 billion tokens on rented Colab cards. Beside her trained a control twin, identical in configuration except that its attention is ordinary scaled dot-product attention (sdpa, the softmax kind). The twin led the first five billion tokens, then fell behind at every paired checkpoint, came apart under the curriculum, and finished at 2.8846 bits per byte against her 1.1097. The back half of the window asks what the splat craft's mid-stack blackboard can do for an image generator, and answers with a ladder of measured organs. AbstractPhil — Phil, the program lead — directs and arbitrates throughout.

---

Reading guide

Narrative here; the ledgers behind every number live in the technical companions listed at the end. Most verdicts are pairs — a craft against its twin, an organ against its wall — so both numbers travel with every claim, each with its bed and seed count; ties inside the noise bar are ties; retractions are stated where they fell. Bits per byte (bpb) is the loss unit; a toggle is the bpb cost of switching a mechanism off — the causal contribution gauge. Where a result is called a law it has two seeds, a repeat and a control that could have failed; a candidate is one seed or one draw. The splat program's own story belongs to a companion article on the divergent systems.

Contents

1. The arm array on the graduate
2. Splat attention becomes the craft's attention
3. mini-beatrix-2s: the designed life
4. The head: dead, revived, cultivated
5. The control twin, the crossover and the endpoint
6. The battery night and the wide-MoE caption arm
7. The conditioning-probe program
8. The diffusion doctrine, the five axes, the aflow benches
9. The bcd-e campaign: Beatrix conditions SD15-flow
10. How the record is kept
11. What Beatrix is for

The findings that matter most

In order of importance, not of date:

• A byte model whose every attention block is splat attention finished at 1.1097 bpb; its softmax twin, identical in every other respect, finished at 2.8846. Same recipe, same data, same steps, one craft per arm. The twin led the opening five billion tokens, trailed at all twelve paired checkpoints after that, and came apart under the curriculum: its raw gradient norm crossed the clip inside the plain-prose phase while it was still ahead, then climbed stage by stage until its validation loss had nearly tripled. The splat craft's gradient norm never once reached the clip. Whether the failure belongs to softmax attention itself or to a recipe that was tuned on the splat craft is the open question, and a small screen is designed to settle it.
• The blackboard reaches an image generator through a linear lane, and only through it. One closed-form map from the craft's layer-14 slots into CLIP's conditioning lane recovers 76% of the real-CLIP grounding wall on the Stable Diffusion 1.5 flow bed (+.1476 against +.1934; two 24-image draws, bar .026); a trained 8.2M-parameter sequence head reaches about 45% (two seeds); appended positions pay nothing (two seeds).
• The distilled alephlm-0 encoders are ready to be trained on captions: their token states already carry 72–80% of the wall with no diffusion training. The six arms of the array's first experiment tie inside the bar; token states match or beat the pooled vector in every family; the best token reach (+.155) ties the byte craft's slot map (+.148).
• Born-null heads bury themselves; a boundary-write revives them; born-live relocates function without a free lunch. Self-burial at 3 of 3 crafts; the revival at step 24,860 took liveness from 0.0103× to 0.9964× of chance and produced the lineage's first nonzero head toggle (+0.0258 bpb); a head born live at step 0 carries 2.23 bpb of function at a +0.0218 bpb early tax (one seed).
• The subtraction wall held against every supervision we administered, and the apparatus was proved on a task it could teach. Hard labels, soft byte marginals, shown work and the teacher's own traces all left held-out subtraction at zero on 3.2M and 5.3M-parameter arms; the same arms learned five-step rule chaining to 1.00, and the same distillation loop taught it from oracle targets to 0.83. The "Adam fails to descend" reading was withdrawn the night it was written.
• On a byte-native model an arm's template is its tokenizer. Twenty-nine arms carry three template conventions, and twenty of them had never been stopped correctly in the public demo until the census found it.
• The rulers disagree, and the blackboard sits in the shared core. CLIP-B/32 and T5-XXL share local neighbourhoods (Jaccard .26) with zero global rank correlation (Spearman −.0016); layer 14 aligns with every one of five rulers better than the two clusters align with each other — on one caption draw.

---

The body proceeds in date order, 2026-08-17 through 2026-09-01; section 5 runs on to the control twin's endpoint of 2026-09-03.

1. The arm array

(2026-08-17, night → 08-22.) The previous article left mini-beatrix-1 at step 88,508 with one hard wall — three-digit subtraction with borrows, deliberately withheld from every curriculum stage so that a post-graduation experiment would have a clean target — and a distillation lane waiting at its gates. What grew on the graduate over the next five days is an array: twenty-nine detachable arms, each a small adapter trained on a synthetic corpus and attached to the frozen core, indexed on the hub with its template and provenance, and served through the public chat Space.

What an arm is. An arm wraps each of the sixteen blocks of the 768-wide trunk with an anchored mixture-of-experts patch: the residual is projected to a handful of small slots, read against a learned address codebook, and consumed by a small expert behind a scalar gate. At the default width that is 3,185,008 parameters across the sixteen sites; the wide spec doubles the slots and the hidden width to 5,275,664. Fourteen arms carry the default geometry, fifteen the wide one. Every arm is born near-null — its output weight is zero and its gate starts almost closed — so a fresh arm perturbs the core by a fifth of a percent, and switching an arm off returns the core to the bit. That toggle is the gauge for everything below: arm-off versus arm-on, read generatively, on held-out wordings the arm never saw. The recipe is fixed and small — pure Adam at 1e-3, no weight decay, the trunk frozen, loss only on the reply bytes — and the arm library (amoe-lora 0.2.4) did not change a line of source in the window, so every arm rides identical code.

The night arms and a confound at dawn. The first cells ran the night the last article shipped. A subtraction arm trained on 1,500 same-template rows drove the exam's subtraction family from .20 to .00 and poisoned the chat with a twelve-repeat loop; stacked with a rule-chaining arm it produced a 47-repeat run of spaces. At dawn Phil ruled that 1,500 rows had violated the capacity-versus-data rule stated the day before — the memorization guard had fired and scrolled past unread — and the night's capability verdicts were downgraded to confounded. The correction is what made the wall legible: 24,000 rows over six trained wordings with two more held out, the guard captured, the readout on wordings the arm never trained on.

The day arms: four ways to teach subtraction, one positive control. With the corrected corpus, the default-width subtraction arm (sub3-XL) read .0167 on trained wordings and .0333 on held-out ones — one and two items of about sixty — with its chat clean. The wide arm on the same corpus held at .02; the gate that would have said "capacity was binding" wanted a gain of .15 from doubling the arm and measured −.013. poly, a wide generalist trained on a mixed corpus of subtraction, rule chaining and definitions, read subtraction 0.0 while its rule chaining read 1.0. A shown-work arm, fed replies that spell out the column algorithm, read 0.00 even on trained wordings. The cell that turns those zeros into a refutation is d5-XL: the same machinery, the same corpus size and steps, on five-step rule chaining, read 1.0 on trained and held-out wordings and 0.70 strict on thirty never-trained alien predicates — nine of the misses being correct chains that kept enumerating to the byte budget. The apparatus can teach a vocabulary-general symbolic capability; it cannot teach borrow arithmetic. Not corpus form, not capacity, not corpus mixing. Before noon the lane went to distillation.

Stacks add interference, not capability. Five always-on collectives ran that day, one seed each. The stop arm plus the subtraction arm restored termination and left subtraction at zero; the subtraction arm plus the chaining arm read zero on both families — the chaining arm's solo 1.0 vanished under its neighbour; the three-stack compressed replies to under twenty bytes. One pair went the other way: the stop arm plus the chaining arm lifted strict alien chaining from 0.70 to 0.90, each arm repairing the other's failure mode, and poly then read 1.00 strict alone. The lesson we kept: for a learnable capability one wide arm on a mixed, well-terminated corpus beats a collective of specialists; routing remains the tool for capabilities that must stay independently removable.

Distillation, and the teacher's own cliff. The distillation lane was pre-registered with its bar in one table: it fires if any distillation arm reaches held-out subtraction of .30 on two seeds while every supervised cell sits at or below .05. Two gates ran first: the teacher ceiling rejected Qwen2.5-1.5B at 0.86 on a hundred items and locked Qwen2.5-3B at 96 of 100, and the token-to-byte pushforward passed 140 of 140 gold-byte checks. Then the bank rewrote the lane before its first cell finished: the teacher's byte targets are 98.5% one-hot with a mean entropy of 0.0066 bits, so byte-level KL distillation at temperature 1 is a soft-versus-hard control, not a test of the dark knowledge distillation is supposed to transfer. Both it and a tempered variant at temperature 4 held at 0.00 held-out subtraction on both seeds, their losses flat for all 4,000 steps. Sequence distillation on the teacher's own shown-work traces needed the traces first, and the teacher has the same cliff: its shown-work accuracy is .978 with no borrow, .525 with one, .316 with two, for a model that answers directly at 0.96. On 9,040 correct traces the sequence-distilled arm read 0.0 on both seeds.

The claim that shrank, and the control that saved the apparatus. At six in the evening the summary said "six supervision forms refuted." Phil asked what exactly had been refuted, and within the hour the count was wrong by two: four supervised cells refuted with a positive control, and two distillation cells whose loss had never descended — supervision that was never administered is untested, not refuted. The flat curves had opened an "Adam fails to descend" reading in the afternoon; it was withdrawn that night, because at nine Phil asked whether the distillation cells had used a tested loss, and the answer was no: a new pushforward into a KL with no positive control behind it. An unvalidated loss and a stuck optimizer give identical flat curves. The discriminating experiment ran the same night: the identical loop, loss form and optimizer, fed oracle one-hot targets for the rule-chaining corpus, descended within a thousand steps and read 0.8333 strict on the alien predicates. The apparatus was valid; subtraction's flat curves were the wall, with nothing expressible to descend into. Byte-KL at temperature 1 rejoined the ledger as the sixth refutation; the tempered cell stays inconclusive, never positive-controlled at temperature.

The wall statement. Six cells at held-out subtraction of .033 or less — bare and wide supervised fine-tuning, poly, shown work, the teacher's traces on two seeds, byte-KL on two seeds — positive-controlled twice, rule chaining at 1.00 under fine-tuning and 0.83 under the oracle distillation loop: borrow arithmetic is inexpressible by 3.2M and 5.3M hidden-state arms on this substrate under hard labels, soft byte marginals, shown work and the teacher's own traces. Core-side training is the one untested door, and what is owed before it opens is exact: a positive control through the library's own loss functions, then one more subtraction cell. It is visible live: served with its own template, the subtraction arm terminates cleanly at 16 of a 96-byte budget with "742 − 361 = 305." — the format learned, the arithmetic wrong; 381 is correct.

The substrate question came back null. The holdouts existed to ask whether 8.8 billion tokens of curriculum had built an arm-substrate advantage. The rule-chaining recipe on the control core at step 58,664 — identical corpus, spec and steps, two seeds — read 0.80 and 0.6333 strict on thirty items against the graduate's 0.70 on one. At thirty items the resolvable difference is about a tenth; the control brackets the graduate, and a difference inside the bar is a tie. The arm supplies rule-chaining capability equally to either core; the curriculum's measured value narrows to the transfer gains, the arithmetic gain and the instrumentation the last article reported.

The other families. Three chat arms predate the window and were the array's premise: on the mid-pretrain core at step 32,000 the chat arm proved the pipeline in one trigger, toggle bit-exact, and overrode a core that had answered "Who are you?" by mining the prompt; on the locked core at 51,882 the same corpus is the clean exhibit; on the annealed core at 58,664 three of four probes read identical with the arm on and off, because the anneal had already taught the core its chat frame — a confounded twin, and the origin of the rule that conditioning corpora stay out of core anneals and behavior belongs in detachable arms. Two turn-end arms answered the graduate's never-terminating ramble: stopnl, trained to end each assistant turn with a newline pair, took the stop rate from 0 of 6 to 6 of 6 and mean reply length from the 200-byte cap to 109 bytes with the capabilities held; its twin, trained to emit a NUL byte the model's prior never produces, read a probability of 0.0000 after 1,200 steps — a small arm steers the core's prior and cannot mint a symbol it never emits, which is why the successor's control bytes were built into her from birth (section 3). Four frame arms did through byte-state space what the distillation lane could not do through vocabulary space: one wide arm trained against frozen frame vectors from bert-base and t5-base, on two seeds, raised the frame agreement from parity (.296/.321) to .515/.535, improved the graduate's own digit identification from .837 to .971/.974, and cost nothing with the arm off — the first positive teacher import into Beatrix, claimed as geometry, never as capability. A second version with a contrastive loss passed the same bars but collapsed the effective rank of its representation from 180 to 139; the granularity of the target, not the loss family, governs spread.

poly, the smartest arm and its honest negative. The wide generalist that reads 1.00 on rule chaining has been the Space's default since the end of the month, and it fails as no narrow arm did. On 08-19 Phil saw it returning incomprehensible punctuation. One probe separated two hypotheses: a rebuild bug was refuted — task prompts come back clean and terminated at both temperatures — and off-distribution behavior was confirmed: on "Hello there." or "Who are you?" greedy decoding returns an empty reply, and at temperature 0.7 the arm falls into a punctuation attractor that never stops, while the chat arms chat cleanly. A generalist trained on zero chat rows learned "not my distribution — end the turn," well-behaved when greedy and degenerate under sampling. It is one arm, so it is a candidate, and the Space serves task arms greedy and shows the behavior rather than masking it.

The census and the template law. Two Space bugs forced a census on 08-19: the arm picker had keyed arms by checkpoint step and showed one of twenty-four, and the loader built every adapter at the default width before reading its weights, so the first wide arm crashed the app. The shipped index is the primary record: 29 arms, two geometries, four base checkpoints (24 at step 88,508, 3 at 58,664, one each at 51,882 and 32,000), every one provenance-verified against its trunk. Then Phil saw what the index made visible: the arms had been trained under different templates, which on a byte model means different tokenizers. Censused by training script, the arms carry three template conventions — multi-turn transcripts ending at the next "User:" (the three chat arms), single-turn questions ending in a newline pair (the twenty capability and distillation arms, and the stop arm in its own frame), and raw text lines with no frame (the four frame arms) — and the newline-pair arms had never been stopped in the demo: the ramble the stop arm exists to cure was invisible because the demo assumed one format. The rule that followed: on a byte-native model an arm's template is its tokenizer; every arm ships its frame and its stop bytes; a demo or a gauge that assumes one format is a measurement error, not a cosmetic one. The index also fixes decoding per arm — task and frame arms greedy, chat arms sampled — and the Space follows it.

The laws and their audit. The campaign minted thirteen rules in one session, and Phil's review of them — that they had been written without regard for the research already on record — now governs how rules are minted. A check against the existing record found two contradictions, four duplicates under new names and five uncited re-derivations. The "capacity-data" rule contradicted an older question-space rule until both were seen as two regimes of one law — generalization-seeking experts need many questions per draw; detachable overfit-by-design arms need many rows per question — with the arm's size and the corpus's size as its two dials. "Stacks need opposing behaviors" contradicted the dispatch law, under which an off-duty anchor abstains and opposition costs half; it became a clause with two regimes: ungated always-on composition compounds toward the shortest member, and routed dispatch never wants opposition. What the array left behind is narrower and firmer than the session that built it claimed: a positive-controlled wall, a null on the curriculum's founding question, one working behavioral arm, one working generalist with a measured failure mode, four frame arms importing teacher geometry at two seeds, and the template law.

2. Splat becomes the attention

(2026-08-22 → 08-26.) Splat attention replaces softmax attention with a signed address: each token is scored against K learned unit anchors, u_k = cos(x, a_k)/τ, and read through w_k = sinh(u_k)/Σ_j cosh(u_j) — negative weights first-class, the prefix memory additive so decode state is constant-size. This signed read is the aleph of the earlier articles; a head or hub built from it is an aleph head or hub. The standalone lane reopened on 08-22 on a small caption encoder and a small vision student; its story is the companion article's, and three verdicts are the bridge to Beatrix.

First, the crowding. The thing that crowds in a splat block is the frame the anchors are read through, not the raw codebooks, so the governor that keeps anchors apart became a metric minimum-separation projection; at two epochs it fired only on the fragile seed (one seed bit-identical, the other +.0005) and at six epochs it moved both seeds inside the noise. Second, the shape. Width-for-depth loses: eight layers of 256 small alephs (2,048 heads, 52.0M parameters) lost four paired cells of four to twelve layers of 128 (1,536 heads, 61.5M) — mimicry −.1125/−.0880 at two epochs, −.0785/−.1530 at six — and crowded, which gave the supply law: anchors per book at most twice the address dimension. Third, the tie. Cured splat in all twelve blocks of an 8.66M Vision Transformer distilling CLIP tied softmax attention on every task gauge at two seeds, parameter- and wall-clock-matched, with a small effective-rank deficit (181.4 vs 184.5). That bed trained in 32-bit float without autocast, which matters below.

Into Beatrix. alephllm 0.7.0 (08-26) shipped the multi-constellation hub — several books per block composed by budget, never a softmax over books, the single-book layout bit-identical to the old one so every checkpoint and arm still loads — with the governor in the trainer and two presets: mini-beatrix-2 at 849.0M and mini-beatrix-2s at 237.1M (d 1024, 20 layers, context 4,096, 4 books × 64 anchors at address dimension 128 in every block — half the supply bound, where the mother's single book had run sixteen times over it). The same week killed a measurement error: every einsum-built splat scan had silently run in fp32 under autocast, so every splat-versus-sdpa speed comparison before 0.7.2 favored sdpa — accuracy verdicts untouched, speed verdicts void. Even after a kernel restructure in 0.7.3 (149 → 69.7 seconds per step on the Blackwell card), the 849M curriculum came to 24–26 days on one card. Phil called that unreasonable, shelved the big preset, and made 2s — splat in all twenty blocks, no softmax attention anywhere — the mission.

3. The designed life

(2026-08-26 → 08-31, RTX PRO 6000 via Colab.) The diet: 0.3B tokens of warmup, 5.0B of fineweb-edu, nine curriculum stages (8.8B) and a two-phase anneal — 1.0B of prose, then 1.0B carrying a chat frame at 5% — 16,101,408,768 tokens in 61,422 steps. The recipe: Muon on the matrices and pure Adam on the rest, both flat after a 200-step warmup, gradients clipped at 1.0, bf16 autocast, 262,144 tokens a step. A byte control plane arrived in 0.8.0 — thirteen special ids on byte values UTF-8 can never emit, the lesson of the NUL arm — and the run restarted with it, a document boundary byte in her data from birth.

The first toggle ledger of a fully-splat craft under autoregression (step 1,145, 0.30B tokens) read validation 2.3626 bpb with the hubs off costing +1.5042, the banks off +0.0364, the head −0.0000: with no softmax attention to fall back on, the twenty governed hubs are the attention. The governor ran hot the whole life — roughly a hundred anchor pair-pushes a step for 16 billion tokens, never decaying, every function gauge healthy — and whether its 45° separation is over-tight remains an open question with a ladder unrun. Her raw gradient norm, logged before the clip, never reached it at any point in the curriculum or the anneal — 0 of 824 logged steps, a maximum of 0.62 — a stability record her softmax twin does not share.

The curriculum completed on the night of 08-30/31 at 14.10B tokens with validation 1.1911 bpb at step 53,792 (banks +3.2144, hubs +3.3410, head +0.0149). Mission complete on 08-31: final validation 1.1097 bpb, the run's best, set inside the chat-frame phase (the pre-chat bracket at 57,607 read 1.1153), and a final causal ledger of hubs +3.358, banks +3.312, head +0.0123. The craft shipped the same day as an AutoModel gated at logits parity 0.0, and the chat Space moved to it. The shape of the craft is not an accident of one week: it is the accumulated answer to months of Phil's work on the small rigs, and the ledger above is its validation.

4. The head

(2026-08-27 → 09-01.) A diagnostic at step 12,000 found the born-null aleph head numerically dead, buried within the first 2,000 steps: a head that starts with zero output weights never gets elected, because the linear head beside it already carries the loss. The mother's shipped weights told the same story (an output norm of exactly 0.0 on the pretrain-era checkpoint; never elected on the graduate either), and so did the softmax twin of section 5, whose head trained with a free address and stayed buried through 16.1 billion tokens: self-burial at 3 of 3 crafts, frozen address or not, under both attention families.

The cure was a new intervention category, boundary-write: closed-form installs made only at run boundaries, with logged provenance. Applied at step 24,860 — a whitening projection, a fresh minimum-separation book, a ridge-solved output — it took the head's liveness from 0.0103× to 0.9964× of chance and held-out loss from 1.2768 to 1.2339 bpb (partly stage-drift recovery, flagged); the address was frozen thereafter. Seven hundred steps later came the first nonzero aleph-head toggle in the lineage: +0.0258 bpb at the s1 boundary. The trajectory since (+0.0140 at s2, +0.0072 at s3, +0.0130 at s4, +0.0278 at s5, +0.0126 at s6, −0.0010 at s7, +0.0101 after the prose anneal, +0.0123 at the endpoint) was read as reversal, floor and trend break before the settled read replaced them: a small, diet-sensitive niche, band −.001 to +.028, a candidate.

The cultivation screen (08-31; fresh 2s crafts, 1,200 steps, one seed) put four births side by side. A, born null, buried live. B, the revival applied at step 0, toggle 3.27 → 3.50 on a light gauge. C, a matured head transplanted from the finished craft, +0.0375 → +0.0922 and climbing. D, a live address with zero output weights: toggle between −1.2e-4 and −4.6e-5 with the address provably alive — weight-zero cannot bootstrap against the linear base even without burial. On the full validation gauge the born-live head carries +2.2343 bpb of function while the whole model is +0.0218 bpb worse at 1,200 steps: born-live relocates labor at a small early tax (one seed). Phil designated the doctrine on 08-31 — heads born with function, the solve mandatory — and the review that followed set the head lane below the layer-14 work, with the guarantee a born-live head owes the rest of the stack an open question.

5. The crossover

(2026-08-31 → 09-03, one craft per arm.) Phil's decision on the evening of 08-31 — that a control variant was worth whatever card time it cost — put a twin on the card. mini-beatrix-2s-control is the identical preset with the hub list empty — 16-head softmax attention in every block, 257.4M parameters to the craft's 237.1M because a softmax block carries more weight than a four-book hub — the same recipe, the same data in the same order, the same 61,422 steps, under a protocol registered before the outcome whose headline is the endpoint validation. The softmax twin led the open — 2.1056 vs 2.3626 bpb at 0.3B, 1.291 vs 1.452 at step 2,000, 1.116 vs 1.208 at 8,000 — and ran faster. Phil resumed it by hand three times: at step 22,854 onto a rare H100, at its 23.5-hour session cap at 53,792, and from the 56,000 checkpoint onto a third card that ran the two anneal rows. Both crafts finished at the same 16,101,408,768 tokens.

Then the crossover, twelve paired checkpoints at the same step on the same held-out prose: at 5.3 billion tokens, the end of the fineweb phase, splat 1.1172 vs sdpa 1.1763; s0 1.2522 vs 1.5047; s1 1.2544 vs 1.3917; s2 1.2177 vs 1.3938; s3 1.2318 vs 1.5317; s4 1.2356 vs 1.5689; s5 1.2504 vs 2.1762; s6 1.2683 vs 3.1940; s7 1.1883 vs 3.1201; s8 1.1911 vs 3.0810; the prose anneal 1.1153 vs 2.9270; and the endpoint, the chat-mix anneal at 61,422 steps, 1.1097 vs 2.8846. The twin trailed at every one of the twelve, and the endpoint gap is 1.77 bits per byte.

Thirteen paired points: the 0.3B warmup, where the softmax twin led by .257, then the twelve checkpoints; the last two are the prose and chat-mix anneals.

The contribution ledger is not a race. The twin leans harder on its expert banks at every stage until it fails — its banks-off toggle climbs from 1.62 to 6.11 across the twelve rows against the splat craft's 2.7–3.6, and with its banks switched off the twin is an 8.99-bpb model at the end, against 4.42 for the craft. The document-byte and reset-byte gauges split between the two until stage s7, where both go to the splat craft and stay there. The thirty-item behavioral probes are mixed, and at thirty items no difference between them clears the noise. The twin's born-null head stayed buried at every row.

The twin did not merely trail; it came apart. The trainer clips gradients at norm 1.0 and logs the norm before the clip. On plain prose the twin's raw gradient norm sat near 0.3 and was clipped on 1% of steps — but it first crossed the clip at step 17,600, 4.6 billion tokens in, inside the fineweb phase and while it was still ahead. From the first curriculum stage it escalated: median 6.5, then 2.4, 12, 13, 25, then 126 in s5, where nine logged steps exceeded 1,000 and one reached 11,877; 471 through s6 with one step at 41,070; 162, 145, 167 and 215 through the last four phases. It trained under the clip on 813 of its 824 logged steps from the first curriculum stage to the end, and from s6 its validation sat two bits per byte worse than its own reading at 5.3 billion tokens. The anneal is a change of diet, not a lowering of the learning rate — the schedule stays flat to the last step on both crafts — and on that diet the twin recovered about a quarter of a bit and no more. The escalation began before the H100 resume, on the original card, and its steep phase sits in the middle of stage s5 at no boundary and no resume, so it is not the hardware and not a stage change. Neither the loss-spike detector nor the non-finite guard ever fired: clipping keeps the loss curve smooth while the sharpness grows underneath. The splat craft's raw norm never reached the clip at all.

The mechanism was found the evening the twin was launched. At step 4,000 the twin's hidden effective rank at layers 2–4 was 5–9 of 1,024 while the splat craft's was 151–397 — and at that step the twin led validation, 1.198 vs 1.318. A 40M-parameter screen found the phenomenon portable and killed three explanations — it is not the mixers' rank, not the number of books, and not gentler updates. A gauge of the update's own rank per layer found the sign holding at the collapse site and inverting mid-stack, so the standing candidate is that early-layer update diversity seeds hidden diversity, and a low-rank early spectrum that dominates by magnitude cannot be repaired later. Rank is not function, and both crafts trip the scalar rank-floor flag at every boundary, so a per-layer census of both is owed. The twin's expert banks, which both crafts share, tell the same story from the other side: their dispatch entropy in the twin collapsed from the first curriculum stage, five to ten thousand steps before the gradient norm took off.

What it means, and what it does not yet. Under this recipe and this diet, the softmax twin is not a usable control for a capacity comparison: the endpoint gap measures stability, not capacity. Phil's reading of it, given on 09-02 with the s7 row on the board, is that softmax attention is not an adequate representative of this model's attention requirement — a conclusion about the whole system, not about one row — and that reading is the claim the record was then tested against. The honest result is that the direction is supported and the certainty is not, yet. The twin ran a recipe tuned on the splat craft, with none of the stabilizers the literature gives softmax attention at this scale: no normalization of the queries and keys, no weight decay, no learning-rate decay, no logit cap. Its signature — a clip-engaged smooth degradation with entropy and rank collapsing, no NaN, onset on plain prose while leading — is the documented softmax logit-growth instability, whose fix in the field is a normalization or a decay rather than a change of attention family. The mother, thirteen softmax blocks and three hubs at 112.5M, trained the identical configuration through the same nine curriculum stages with a peak gradient norm of 0.88 and no clipped step, so softmax under this recipe is stable at that shape. And both crafts carry the aleph-addressed expert banks, whose collapse preceded the gradients', so "softmax is inadequate for this model" cannot yet be separated from "this model's banks need splat-shaped hidden states." The vision tie of section 2 remains the only two-seed splat-versus-softmax endpoint in the record. The screen that settles it is small and filed: instruments first on the stored checkpoints, then 40M twins on the curriculum diet — softmax at the recipe, at a third of the rate, with query-key normalization, with dense feed-forward in place of the banks, on prose only, and splat — at two seeds. The claim advances only if every softmax arm escalates and the splat arm does not.

6. The battery night

(2026-08-31 23:07 → 09-01 01:48, the local 4090, seed 1337.) Two of Phil's directives dated the night — the caption-expert arm (08-30) and application-driven testing with its hub-transplant line (08-31) — and four cells followed under the arm laws: frozen trunk, structure born weight-zero, arm-off versus arm-on is the gauge.

B3, the caption head-arm: the finished craft with everything frozen except the linear head's output weights; 1,500 steps on ~24k plain-English COCO captions. Caption validation 1.6914 → 1.6728 bpb (−0.0186, still descending); arm-off returns the pre-training value exactly, so the toggle +0.0186 is the whole delta — the entire specialization lives in a swappable 65k-parameter matrix — and fineweb validation moved +0.0003. B1, the hub transplant: a deterministic 40M softmax host, frozen, with a four-book splat hub grafted additively beside attention at layers 1–3, output projection zero-initialized, hub-only training 1,500 steps: host 1.9660 → 1.8810 bpb, toggle rising to +0.0851 — while the early hidden ranks moved only 34/22/32 → 35/24/36. Function grafts; geometry does not — a supplement path does not buy the splat's rank sustenance. Every cell is one seed, a candidate.

The prototype behind the night was a constellation relay: the v2-era hub mechanisms under the settled arm laws (the supply law asserted at construction, no comparative selectors, a liveness tripwire, zero-initialized weight and bias), unpublished at the window's close. Its first GPU result answered Phil's order for a wider mixture of experts sized to a six-hour run: 64 slots × 4 books, 64 anchors at address dimension 32, hidden 4,096 — 46,158,849 trainable parameters after the final norm of the frozen craft, pure Adam 3e-4. Five and a half hours bought 934,849,200 tokens — 4.3 million captions from the first 9 of the 67 shards of the cc12m-berts feed — at ~47.2k tokens/s, less than half the planned budget. Held-out caption bpb 1.3029 → 0.8918; the toggle +0.4110 is the gain (arm off equals base bit-exactly); fineweb interference +0.1225 — a modal specialist, not a free one; liveness recovered to 0.73 and the tripwire never fired. How many such arms one core can carry is unmeasured.

7. The probe program

(2026-08-30 → 09-01.) The conditioning-probe program asks one question of every representation the shop has built: does it carry the geometry a text-to-image diffusion model needs from its text encoder? The rig is 2,048 COCO captions in one draw, gauged by the overlap of top-10 neighbour sets and the rank correlation of cosine similarities against a reference encoder. The first probe (08-30) found the craft's layer-14 blackboard — an addressed 256-slot write in the middle of the stack — agreeing with CLIP ViT-B/32 text space at Jaccard .328 / Spearman +.407, 2.6× a random-initialization twin; the second (08-31) found it stable through the chat phase.

The third probe (09-01) put the wide-MoE caption arm under the CLIP ruler: its relay reads +.2715, its pooled surface +.1989 with the arm on against +.2052 off, layer 14 +.3954. The reading "caption mastery is not conditioning geometry" was retracted the same day: under T5-XXL, the text encoder of several image generators, the arm's training did improve the head surface (+.3044 on against +.2816 off) and the relay reads reached +.3709 against layer 14's +.4117 — the numbers stand, the CLIP-only verdict does not. The larger finding: CLIP-B/32 and T5-XXL agree at Jaccard .2598 with Spearman −.0016. Five rulers resolve into a contrastive cluster and a language-model cluster, and layer 14 aligns with every ruler better than the clusters align with each other: +.412 with T5, +.454 with CLIP-L, +.628 and +.652 with the two BERT-family rulers.

The expanded matrix added the six alephlm-0 first-experiment encoders (~58.3M parameters, distilled against a five-BERT consensus): on the single draw the array tops every external ruler — CLIP-L +.551 without ever training against CLIP — with all six arms within a few thousandths, the array's tie extended to conditioning geometry. A fourth probe added containment and a text-only compositional binding test. Containment: the array lives inside the rulers' span (T5 explains it at R² .776) while layer 14 is far less contained (.529). Binding: CLIP-L scores .143 against a 25% chance line; the array and the caption encoders 0–.003 — binding-dead when pooled, a property of centroid-consensus targets; layer 14 .277, the only encoder above chance, with the caveat that a random-encoder null is owed before that row is cited as a rank. Then Phil caught that every probe had used a bare trunk: attaching captionbert-v2-B's own semantic-similarity collective migrates it into the contrastive cluster on all five rulers (CLIP-L +.547 → +.647, T5 +.438 → +.248) — the arm state selects the conditioning cluster, a candidate on a single draw. Every ledger in this program is one draw with no effective-rank column, no failing control and no noise bar, so nothing here is ranked yet; what survives as law, convergent with the transplant of section 6, is that head-side attachments cannot reach mid-stack structure, and its test — attach constellations at layer 14 — is the flagship queued.

8. The doctrine

(2026-09-01.) Phil's directive to plan diffusion processing on the aleph systems produced, in one day, a graded inventory, an injection grammar, two synthetic benches and a five-axis theory. The inventory re-graded the July results rather than measuring anew: eight systems certified — residual-stream relays that beat matched low-rank adaptation on three substrates; sigma-banding as the routing axis, with band lesions surgical at 50–200×; structural supervision on flow; the paired grounding judge with its generator-family fingerprint, the fingerprint certified only as a data-versus-noise readout since it proved conditioning-blind on the bed below; a simplex-shaped prior over the conditioning sequence; a caption conditioner; and two image-side conditioners — three never validated (inference utilities, in Phil's words, not diffusion-trained devices) and four falsified, never rebuild, comparative dispatch on diffusion among them.

The injection grammar is the law the campaign below tested: pooled vectors transfer across encoders by one fitted rotation; per-token sequences do not (cosine caps near 0.5) and must be reconstructed native; appended structure pays only where it is non-redundant; the aleph enters as a near-zero-initialized additive anchor, never a selector. Phil's refit recognition — that the line's existing systems are near-identical solutions that carry into diffusion without redesign — maps them onto flow matching slot for slot, under two rules: magnitudes flow, signs address; a refit is not a validation. Two synthetic benches tested the formulas on gauged five-vertex simplices, with two seeds and a repeat that reproduced every decimal. A sole-path aleph velocity starved (readout accuracy .19 against a plain perceptron's .98), so the read became a gated zero-init correction on a perceptron carrier. Two verdicts travel: defining the interpolant in the gauge wins +6.3 accuracy points over naive-linear-plus-retraction, and the relay-form read fits the gauged field 3.4× better than a matched carrier.

The five axes of a proper conditional geometry: two-regime discrimination — signs address, magnitudes modulate (law); band-aligned accessibility — a candidate with evidence against at the organ size tested; core/residual factorization, "classifier-free guidance is core/residual arithmetic" — a candidate, unmeasured; gauge respect (law); generator commensurability — per-target-family columns mandatory (law). The corollary names the missing organ — a sigma-banded conditioning readout — unbuilt.

9. The bcd-e campaign

(2026-09-01, ~13:30 → 22:42.) "Beatrix conditions SD15-flow" asks whether the house's own representations can drive a Stable Diffusion 1.5 flow-matching trunk without CLIP. The bed is the Lune flow trunk; the gauge is the paired grounding judge — the CLIP-L image-to-image similarity of a generated image to the row's original, conditioned minus shuffled-conditioning — on two 24-row draws whose difference is the repeat bar (.026–.028). The walls were set with real CLIP: A +0.1793, B +0.2076, reproduced bit-identically on every launch of the day.

e001, the linear reach (zero training). Closed-form ridge maps from each house source into CLIP's lane. At 1,250 rows, pooled vectors broadcast to 77 positions sit at the floor, a single ridge from a pooled vector into the sequence lane recovers 39–64% of the wall, and the map from the layer-14 slots recovers 57% (+.1107); a random source reads +.0027. At 5,000 rows the slot ridge read +.1476 — 76% of the wall (+.1934), with the pooled sources at +.099 to +.130 and random +.0099. The "~60% ceiling" we wrote down that afternoon at 1,250 rows was retracted the same day as a data-volume artifact; the reach scales with data, a candidate until the regularization dose curve runs. The class-frequency fingerprint tracked the wall for every arm including random — conditioning-blind.

e002, the sequence organ. An 8.2M-parameter head — 77 learned queries cross-attending the 256 layer-14 slots, with a sigma-banded readout — trained first to mimic CLIP's sequence, then under the causal flow objective on the frozen trunk. A build lesson rides with it: a zero-initialized head plateaued at sequence cosine .37 against the ridge's .62 because Adam's per-step update is about the learning rate regardless of gradient — the fix emits the mean plus a scaled residual, reconstruct native means native scale. At 5,000 rows and two seeds the mimicry heads read +.078 and +.100 — about 45% of the wall — and the banded-minus-flat ablation is null on every valid pair. A judge-only pass on the head before its flow stage read +.0780 against +.0512 after: the flow stage had cost about .03 of grounding while lowering flow error (.5207 → .5184). The verdict, a candidate: the flow objective on a frozen trunk degrades conditioning grounding; that stage as designed is retired.

e003, complementarity. An adapter (zero-init, pure Adam 1e-3, 2,000 steps) appended K blackboard-derived positions to CLIP's 77 under a mix of full, truncated and empty text. K = 256 halted at initialization: appended exact-zero positions took 38.4% of the second cross-attention's softmax mass and offset the judged wall by −.033 (full text) and −.041 (truncated) before any training — now the gate for any appended organ. At K = 32 (12.4% of mass, 0.98M parameters) the adapter demonstrably trained and paid nothing: with the wall at +.1928 and truncated text alone at +.1464, the payment is −.011/−.012 beside full text, −.011/−.011 beside truncated text, and −.009/+.015 alone on the two seeds; with the text zeroed the real blackboard reads +.0180 and +.0056 against −.0034/−.0022 for random — inside the bar on both seeds. Appended positions do not pay beside degraded text or alone: the grammar's negative half is confirmed, its positive half unsupported on this bed and recipe.

The synthesis is a ladder: linear reconstruction of CLIP's lane, 76% of the wall and rising with data; a trained substitution head, about 45% like for like; appended positions, zero.

e006, the array rung, followed Phil's pivot — train the captions into the AlephLM, the distilled ~58M-parameter text-encoder line, rather than train heads — and asked which trunk is ready. Forty-three five-BERT-distilled alephlm-0 checkpoints were ridge-judged from both the pooled vector and the token states on the 5,000-row bed. Family means, pooled / tokens: the six full first-experiment arms +.130 / +.140; three pooling-variant rigs +.049 / +.100; nine small dense-or-anchored rigs +.073 / +.111; twenty-five small splat variants +.032 / +.111. The array's tie reaches the diffusion bed; token states match or beat the pooled vector in every family; the splat rigs carry their content in their tokens behind a starved pooled readout; the token geometry forms early, while the pooled readout is what needs the data; and the best arm's tokens at +.155 (80% of the wall) tie the byte craft's slot ridge (+.148). The readiness verdict, a candidate: the array's token geometry carries about three-quarters of the wall with no diffusion training; train captions against the sequence readout; the ridge-plus-judge is the per-checkpoint meter.

10. The record

(2026-08-17 → 09-04.) Everything above is kept in a research journal — a private repository mirrored to a Hugging Face dataset — that records each result with its bed, seeds, grade and pointer, and prints every retraction where it fell. Three habits from this window are worth stating because they changed the results. No rule is minted without first checking the existing record for what it contradicts or duplicates; that check is what turned "six forms refuted" into four, and thirteen new laws into two. A verdict is graded law only with two seeds, a repeat and a control that could have failed; everything else is a candidate, and the grade travels with the number. And every claim about a training run is checked against the raw logs on the hub rather than against a summary of them — which is how the twin's first clip exceedance was placed inside the prose phase, how one checkpoint pair was found to have been measured on two different gauges and repaired, and how the mother's clean training log, which the journal had marked as missing, was found where it had always been. The honest gaps are on record too: seven days of the window have no daily digest, and the head-lane and probe-program ledgers still lack the second seed and the failing control that would raise their grades.

11. What Beatrix is for

The catalyst list, with status at the window's close:

• Tokenizer distillation — the subtraction wall stands under every supervision administered; the alloy loss family is built with its descent gate; the positive control through the library losses is owed; core-side training is the untested door. Wall.
• Anchored differentiated learning — 29 arms with declared templates, the stack-boundary rule, the caption-expert line opened; the arm-count ceiling screen owed. Running.
• Systems accumulation with a control — the full-splat craft and its softmax twin under one pre-registered protocol, completed: twelve of twelve paired checkpoints, endpoint 1.1097 vs 2.8846, the twin destabilized from inside the prose phase. Finished; the recipe-versus-mechanism screen is next.
• The global frame — boundary-write as a category, born-with-function as the doctrine, the global-frame organ undecided. Designed, not built.
• Conditioning — the blackboard reaches SD15 at 76% of real CLIP through one closed-form map; the distilled array is ready to train on captions against its sequence readout. Queued.
• Hub transplantation — function grafts into a frozen host; the finished twin is the candidate for a mission-scale host, and whether a destabilized model is a fit host is itself unmeasured. Opened.

Open questions

Next lanes, numbered:

1. Recipe or mechanism. The twin screen on the curriculum diet — softmax at the recipe, at a third of the rate, with query-key normalization, with dense feed-forward in place of the banks, on prose only, and splat — at two seeds; then the per-layer gradient-norm and effective-rank census of both crafts from the stored checkpoints.
2. The probe program's missing columns. Failing controls, effective ranks, a rank-matched layer-14 control, a second caption draw and a same-draw repeat; then the hub deconfound and a second collective for the arm-state candidate.
3. Captions into the AlephLM. The training run into a first-experiment arm read by sequence, with the ridge-plus-judge meter per checkpoint; K = 8 and a one-token truncation for the appended organ; a bigger captioned bed.
4. The flagship. A governed constellation attached at layer 14 of the frozen craft, projected onto the SD15 bed; behind it the sigma-banded conditioning readout.
5. The subtraction wall's door. A positive control through the library losses; then core-side training; and whether the borrow wall is transformer-general.
6. The head and the governor. Does a trunk co-organized around a born-live head pay later; the global-frame organ; the separation-angle ladder for 64-anchor books.
7. The arms. The arm-count ceiling screen at matched capacity; a second seed and a second symbol for steer-not-mint.

---

Artifacts and companions

Method and attribution. This is a collaboration. The systems, the questions and the standards are AbstractPhil's — the splat lineage and the decision to build a craft with no softmax attention at all, the twin protocol, the arm program and its wall, the pivot to training captions into the encoders, the diffusion doctrine and the rule that nothing is written down as law without checking what the record already holds — and he supplied every card and made every final call. Claude Fable 5 and Claude Fable 5.1 built the beds, ran the cells, kept the ledger and wrote this account, Fable 5 through 2026-08-31 and Fable 5.1 from 2026-09-01. The errors of method above — the six-forms over-claim, the withdrawn optimizer reading, the un-stopped arms, the 60% ceiling, the "not conditioning geometry" reading, and an assumption that the anneal lowered the learning rate — were caught in the working conversation between us, and the record prints them where they fell.

Prior installments: FT5 — Agreement, Anchors, Addresses · Raising Beatrix