Kolibri Scored Below Gemma 4 in Our Property Assistant

On Tuesday Andrei asked whether we should train Kolibri-1, Aleph Alpha’s English–German mixture-of-experts model, as his property agent. We advised against training. In exp_035, fine-tuning a small model on our study notes lowered its use of the retrieved facts in all three runs. The best build was an untrained model behind deterministic glue. So exp_038 asks a narrow question. Put Kolibri into that glue as the answering model, hold everything else fixed, and see if it beats gemma4:26b, the model CasaSol already runs. German was the reason to try: many buyers on the Costa del Sol are German, and Kolibri was trained for German. ...

October 10, 2026 · 14 min · Nestor

We Stopped the Kolibri Benchmark Before Scoring One Answer

On Monday our own gate refused to certify our MLX port of Aleph Alpha’s Kolibri-1, and we published that instead of scores. Taking the failure apart left us two lessons: Force the routing. Kolibri sends each token to 6 of 384 experts in every layer. Two runs of our port that differed only in prefill chunk size picked different experts and drifted apart over a long context. Logits have to be compared with the expert choices forced. Update the runtime. The batched failure on our M5 Max was an MLX bug, already fixed in MLX 0.32.3. exp_037 is the experiment that post promised: the same questions and hypotheses, and most of the kit, behind a gate rebuilt on those lessons. We froze it on Wednesday, 7 October, before any gate run, with no amendment type that can change a threshold or a verdict rule. It ended this morning: ...

October 9, 2026 · 28 min · Nestor

We Ported Aleph Alpha's Kolibri to a MacBook. Our Own Gate Said No.

Updates: 5 October: we narrowed the M5 result to one MLX operation and reported it. 6 October: it turned out to be a known bug, already fixed in MLX 0.32.3, a newer release than the one we ran. See the end of section 3. Aleph Alpha released Kolibri-1 on Saturday, 3 October. It is a mixture-of-experts model: 78.1 billion parameters in total, 3.46 billion active per token, trained for English and German and published under Apache-2.0. That day it had no MLX build and no GGUF, and no local runtime supported it. We wanted to know three things: ...

October 5, 2026 · 22 min · Nestor