Let another AI write your teacher data

The “gold data” that trains a specialist AI is something humans write — I threw that assumption out. I had a large AI write it, and a small AI copy it out. Japanese→English, the direction the small model was worst at, jumped 42% → 84% on straight imitation alone. Zero human translators.

The ceiling above that sat not with the teacher but with the student’s comprehension capacity (the teacher’s own translations passed almost across the board). Lump every ceiling together as “the teacher is bad” and you pick the wrong move.

And one pitfall. Hand the teacher surrounding context with the best of intentions, and the student learns to invent subjects. Build the teacher data under the same input conditions as production — the way kindness backfires only becomes visible once you measure.

Training a small AI into “a specialist at one job” takes a large volume of gold data. A stack of model answers saying “for this input, this translation is the right answer.” For translation, that’s a pair — the source text and its model translation.

And nearly everyone assumes the same thing — that those right answers are something humans produce. You hire professional translators, have them translate sentence by sentence, and use that as the model answers. The quality is solid, but it’s slow. It’s expensive. And for low-resource languages you can’t get hold of a professional in the first place. This “produce the gold data” step looks like the biggest bottleneck in training a specialist model.

I threw that assumption out. I had the gold data produced by another AI, not by people.

Have the large AI write the model answers

The method is simple. What I want to train is a small, lightweight model (4B = 4 billion parameters). What I have on hand is a far larger, far smarter general-purpose model (27B class).

So — have the large one write the right answers and the small one copy them out. Pour the source text into the large model, take the translations that come out as “model answers,” and have the small model imitate them. In machine learning this is called distillation. The side producing the right answers is the teacher model; the side learning by imitating it is the student model. The title of this article — “Let another AI write your teacher data” — means exactly what it says: have the teacher model write the gold data (= the teacher data), and have the student copy it out.

The effect showed up plainly. The un-fine-tuned small model was hopeless at Japanese→English, with accuracy of about 42%. It got more than half of them wrong. So I had the large teacher write about 14,000 translations (clean, a single generation at temp0.1, and — as described below — with no context), rolled those into about 5,000 training examples, and had the small model imitate them —

Accuracy: 84%. +42 points.

Worth adding: this 84% came out with no elaborate selection in between, just imitating the teacher’s plain translations as they came. Tricks like “generate N candidates and pick the good one” are the next lever, the one you pull after this. Straight imitation on its own gets you this far.

(The scoring is a reference-free binary judge. Rather than a human yardstick, an independent AI scores each translation “usable / not usable,” and I compared before and after training under identical conditions. These are point estimates from a single run, so the absolute values wobble by a few points.)

One training run took a model that was basically an amateur up to a level that draws level with the large general-purpose model. Not one human translator was hired. An AI mass-produced the gold data.

The pitfall — imitation has a ceiling

It would be a nice story if it ended there, but reality kept going. This approach has a clear ceiling. The 84% on Japanese→English plateaued right there. Look into the remaining 16% and most of the failures wore the same face — hallucination. Inventing a subject that isn’t in the source, supplying an object that was never there. As prose it’s fluent and looks plausible enough. But it says things the source doesn’t say.

This is where attribution goes wrong. “The teacher taught it lies, so the student lies too.” But for Japanese→English, that wasn’t it. The teacher (27B class) is strong at Japanese→English: score the teacher’s translations with an independent binary judge and they pass almost across the board. The one getting it wrong wasn’t the teacher — it was the student. Hand the 4B student a hard source sentence with a dropped subject and an idiom tangled into it, and it can’t read it all the way through; trying to make the pieces add up, it slips in a fluent lie. So this ceiling didn’t come from the teacher: it was hallucination coming out of the 4B student’s “comprehension capacity.” Which means no amount of cleaning up the model answers, and no amount of piling on data, moves the capacity wall itself (KTO and CPO below run into the same wall).

That said, “the teacher’s lies are inherited by the student” isn’t a thing that never happens. It happens clearly in two other situations. One is when the teacher itself is weak in the language — move into a language it’s weak in and the bigger the general-purpose model, the more abruptly it falls apart. There the teacher’s model answers become the rate limiter, and the more the student imitates, the more it gets contaminated. The other is “when you hand the teacher context,” covered in the next section. Both are genuine teacher-side contamination, confirmed by measurement. What the ceiling actually is differs from situation to situation — lump them together as “the teacher is bad” and you pick the wrong move.

The teacher’s first shot isn’t the ceiling — best-of-N

Even if the Japanese→English ceiling is on the student’s side, there’s still headroom on the teacher’s side. Because the teacher’s “first shot” isn’t necessarily the best the teacher can do.

Have the teacher translate the same source N different ways and take only the single best one. This is best-of-N. The yardstick for picking isn’t agreement with a reference translation (a human’s model answer) but QE (Quality Estimation — scoring the quality of a translation without looking at a right answer), which looks only at the quality of the translation itself. The payoff and the pitfalls of this selection mechanism get a separate article, so I’ll keep it to the essentials here.

In my comparison of levers, best-of-N selection is where I’ve placed the highest-ROI “next move” (projected +10 to 15 points). But to be honest, that isn’t a measured result from running the Japanese→English track end to end — it’s my read from the design. Where I have actually made best-of-N work and measured it is a different mode, Live: there, retraining on re-picked model answers gave +2pt (and +6pt, 90→96, together with the preference training stacked on top) (details). The value of “picking from candidates” is real, but I should be clear that the double-digit pt on Japanese→English is still a projection.

Selection is all about the “yardstick”

best-of-N is not a cure-all. If the yardstick you select on is off, you end up worse than not selecting at all. On a different dataset I had it select on “closeness to the official subtitles,” and the alignment-based selection (73–77%) lost to the plain single generation with no selection at all (82%). The official subtitles I used were a translation in a different direction, from a different source language, and every bit the candidates were optimized toward them was fidelity to the source given up. A QE judge, which never looks at the reference translation, does the opposite: it correctly fails those differently-directed official translations, and avoids the trap automatically.

The details are in the separate article, but the lesson is one line — it matters less how many candidates you roll than whether the yardstick you select on points at the kind of good you actually want.

KTO slipped; my bet is on CPO

After the student has copied out good model answers, there’s one more stage of polish. Show it a “good translation” and a “bad translation” and train the judgment itself — “this one is better.” That’s preference learning.

I tried KTO (Kahneman-Tversky Optimization) first. You attach nothing but a binary good/bad label to each individual translation — a method that’s light to implement. I drew up the design, ran it through the pipeline, and trained.

The result — almost nothing. 84% became 85%. That sits inside the judge’s margin of error. The content of the translation changed in 14 of 100 cases, with zero regressions among them, but not one instance of the main cause of failure (hallucination from the student’s capacity) went away. A signal as coarse as binary good/bad can’t fill in the 4B model’s comprehension ceiling — the controlled experiment put that in my face.

So I moved my bet. To CPO (Contrastive Preference Optimization). Instead of a bare binary, CPO ranks several candidates — {teacher translation, student translation, reference translation} — and trains on the difference itself, “A is better than B.” It’s a method with a track record on translation tasks (ALMA-R — the study that demonstrated CPO for translation), and it lets you etch in the “gap” between the good translation you picked with best-of-N and the ones you didn’t. But let me be honest about this: I have not yet run CPO end to end on the Japanese→English track — I’ve decided to adopt it on the strength of ALMA-R’s results, and that’s as far as it goes. If I have to point at measured evidence that preference learning actually worked, it’s in a different mode, Live, where Live-CPO contributed to the 90→96 push (details).

The lesson is that with preference learning, how you do it makes all the difference in the world. Same “teach it good from bad,” and the coarse binary (KTO) slips, while the ranked version that shows the difference (CPO) is where I’m putting my money. But to press the point: CPO doesn’t get over the comprehension ceiling itself either. It’s a region where a difference only appears once you pile up an order of magnitude more (preference data in the tens of thousands), and really breaking that ceiling means going deeper into how the distillation is done. Honestly, I haven’t climbed that one yet.

The backbone, in order: first build the foundation with straight imitation (42→84), then re-pick better model answers with best-of-N selection (the projected highest ROI), and on top of that etch in the differences with CPO.

An incidental finding — “give it context and it gets smarter” was backwards

Let me record one more moment where an assumption flipped.

If you want the teacher to write good right answers, surely handing it the surrounding context makes it translate more intelligently — that’s the natural thought. Human translators translate more accurately with context too.

But when I trained the student on model answers produced with context injected into the teacher’s prompt, the student learned to hallucinate subjects. Subjects the teacher had filled in from context, the student now filled in on its own — in production, where there is no context. At actual translation time the LYR app doesn’t hand the model that kind of conversation history the way the teacher got it. Training conditions and production conditions were mismatched.

The fix was to deliberately withhold context from the teacher as well and have it write the model answers with “no context,” exactly as in production. Kindness meant to make it smarter backfires in production. Build the teacher data under the same input conditions as production — an iron rule I only learned by measuring.

Thin and wide — 61 languages at once

One last property of this methodology, and the one that matters most in business terms.

Having an AI write the right answers makes adding a language essentially free. With human translators you’d re-hire a specialist per language; the teacher model spits out model answers for 61 languages at zero additional labor cost.

In practice, a single training run covered all 61 languages, thin and wide. The result was a clean gradient in resource volume.

(Same training run, held-out evaluation (scored on unseen data not used in training), binary judge. The point estimate for each language wobbles by a few points.)

The pass rate is a direct gradient of “how much parallel text exists for that language on the internet.” The easy languages come out high whether you do anything or not; the weaker the language, the thinner the data both teacher and student have to work with, and the lower it lands. And while hitting 84% on Japanese→English, it also held 80% on English→Japanese (it doesn’t get locked into one direction).

That gradient is also a map that shows at a glance where to invest next. The languages at the top are already fine. The bottleneck is always the weakest language. But — what the rate limiter is there can’t be known without diagnosing the content of the failures language by language. Is it the student’s capacity, the teacher’s quality, or does the base simply not have the language at all? You can’t lump it together as “the bottom of the list is always the teacher’s fault” (Swahili is a case where the base doesn’t have the language itself = the data floor). How to run that diagnosis, and the different moves each rate limiter calls for, are covered in a separate article.

Once you’ve laid down this “teacher-data production line,” the rest is knocking out the weak languages one at a time. The fuel for training a specialist — gold data — keeps getting supplied thin, wide, and cheap by an AI instead of by people.

Lessons

1. Gold data can be produced by an AI instead of by people. Have a large teacher model write the model answers and a small student copy them out (distillation). On Japanese→English, the direction it was worst at, straight imitation alone took accuracy from 42% to 84% (+42pt). Zero human translators. The step usually taken to be the biggest cost of training a specialist — producing the right answers — can be automated. (The manifesto’s second principle, Hunt the Constraint: get past the bottleneck by a different route.)

2. What the ceiling actually is differs from situation to situation. Stalling at 84% on Japanese→English was hallucination from the 4B student’s comprehension capacity, not from the teacher (the teacher’s own translations passed almost across the board). What does get inherited from the teacher’s lies are “languages the teacher is weak in” and “when you hand the teacher context” — and those I confirmed by measurement. Lump every ceiling together as “the teacher is bad” and you pick the wrong move.

3. Build the teacher data under the same input conditions as production. Hand the teacher context with the best of intentions and the student learns to invent subjects. A mismatch between training conditions and production conditions turns straight into a defect. And the next move for pushing quality up is selection, not generation (best-of-N) — but select on quality alone, because selecting on closeness to a reference translation pulls you off in a different direction.

---

Appendix: raw data

Shared conditions: student = my own lightweight model (distilled 4B). Teacher = a general-purpose large model (27B class). Evaluation compares before and after training under identical conditions with a reference-free binary judge (PASS/FAIL). Point estimates come from a single run = absolute values wobble by a few points.

Related: The details of quality selection are in Letting an AI pick a translation better than the “right answer”. The basics of training a small model on this teacher data are in Training a small AI into a specialist at one job. Diagnosing the weakest language, where the teacher itself becomes the rate limiter, is in Spotting the rate limiter in your weakest language.