Let the AI pick a translation better than the "right answer"

There is one right answer, and it is sacred — that article of faith in AI training cracked. Have the teacher produce N candidates and let a quality estimator pick among them, and what you get is a set of model answers that beats the teacher’s own first shot (+2 on Live; 90→96 together with the CPO that followed).

But the thing I’m sure of is the flip side. If the yardstick you select with is off target, you lose outright to a single clean answer. When I selected by closeness to a right answer that faced the wrong direction, 82% fell to 73–77%.

More volume doesn’t change that. Data inflated to 8,553 examples by raising the temperature lost to a clean 3,301. The question to ask wasn’t “what should N be” — it was “does this yardstick point at the kind of good I actually want?”

When you teach an AI something, the most straightforward approach is this: prepare one correct set of model answers and have it imitate them. For translation, you show it a great many pairs that say “the right answer for this sentence is this translation.” Straight out of the machine learning textbook, and it’s how I worked for a long time.

Behind it sits a quiet article of faith: the assumption that “there is one right answer (gold), and it is sacred.” Raise the quality of the model answers and you raise the quality of the student. So you carefully prepare one good right answer —

Then one experiment cracked that faith. Having the model produce N candidates and picking the “best one” gave better translations than having it imitate a single right answer. And another experiment from the same period taught me, just as clearly, that this way of picking is no cure-all — get the yardstick wrong and you lose outright to one clean set of model answers. This article is about both sides.

Where it started — one weak mode I just couldn’t close

LYR’s small model has three places it gets used (modes): Live, which translates video subtitles in real time; Page, which translates the whole screen; and Manga, which translates comics.

Of the three, Live was stubbornly the weak one (what I’m training here is the mid-size one among the small models — the 8B specialist). Same small model, and yet Page and Manga were roughly level with the large model above them, while Live alone was left well behind. Looking at the causes one case at a time, the shape of the failures came into view: idioms translated literally (reading “isn’t like you” as the opposite of 「あなたらしくない」, the idiom it should have produced), the wrong subject picked, words inserted that were never there.

The first move that comes to mind is “collect more gold data for Live and have it imitate more of it.” But who writes that gold data? The larger model above it — the teacher — writes it (the distillation from last time). You have the teacher translate once and use that as the model answer. Except the teacher’s own translations sometimes missed the idiom. If the model answers miss it, the student misses it too.

The realization — the teacher’s “first shot” isn’t necessarily its best

So I changed the framing. Stop having the teacher translate exactly once and then worshipping that as the one right answer. Instead —

Have the teacher translate the same sentence N different ways. Then mechanically pick the “best translation” out of that set.

That’s the idea behind best-of-N (take the single best of N candidates). The crux is how you decide which translation is “best.” What I used was a quality estimator (QE = quality estimation): a mechanism that scores “how good is this translation?” without ever looking at the right answer. Instead of matching a candidate against a right answer, it estimates how well the translation itself came out. Score the N candidates with QE and adopt the top scorer as the “new model answer.”

I retrained Live on those model answers. It worked — modestly. Re-imitating the model answers rebuilt with best-of-N moved Live’s pass rate +2. Stacking CPO on top of that (preference optimization that learns “the difference between a good and a bad translation” instead of imitating one right answer — a method with a track record in machine translation) added +4: +6 in total, 90→96 — level with the large model’s 96. register (how natural the speech style sounds) held at 98 against the 100 it had before specialization, still at large-model level.

Let me draw a line here, to be safe. Putting those three models side by side (small general-purpose / Live-specialized / large) is the subject of the sister article A small specialist drew level with the strongest model in service. What matters here is one point: of that +6, the part best-of-N moved is the +2 — not a magic strike, but a solid step.

What matters is the mechanism. The teacher’s “first shot” is not a ceiling on the teacher’s ability. Roll it N times and somewhere in there is a translation better than the first one. Pick that one up with QE and you have model answers that beat the teacher itself. The same mechanism is also the heart of the process of having an AI build the teacher data. The right answer wasn’t one thing — it was something you grow out of N candidates.

Another direction — and the “next move” I haven’t stacked yet

Live shouldn’t be the only place this selection idea works… or so I’d expect. But here I have to draw an honest line.

For Japanese → English — the direction LYR polishes so people can “read Japanese manga and anime in English” (this one is small too, but a lighter 4B airframe) — the un-fine-tuned accuracy was only around 42%. Run a chain of distillation training over it and it jumped to 84% — the same level as a large model, or as a powerful previous-generation model.

But that 84% is not a best-of-N result. What’s inside it is plain distillation: straightforward imitation of the teacher’s single most confident output (one low-temperature sample). best-of-N QE is the “next move” you stack on top of that — the lever my design notes rate as having the highest return on investment (a projected +10–15 points) — but I haven’t run it yet. The full picture and the numbers for that direction I’ll leave to the sister article Get a different AI to build your teacher AI.

So what I can say “from measurement” about best-of-N right here is only that +2 on Live. A jump like 42→84 is not yet best-of-N’s doing — it’s an expected value, not a track record.

One of my usual notes of caution here. These are LLM-as-judge verdicts, and the number of evaluated cases is limited (for some experiments I didn’t even record the count). Read the point estimates as a strong signal that something worked, not as numbers you can grip down to the decimal place and declare settled.

The pitfall — if the yardstick is off target, you lose to one clean answer

Up to here I could end on “best-of-N is universal magic.” But let me be honest about this: with this same method, I fell flat once.

On a different dataset, I used closeness to the official subtitles (gold) as best-of-N’s “selection yardstick.” “Pick the candidate closest to the real subtitle and it must be the best one” — intuitively that sounds right. The result: it lost to the straightforward single output with no selection at all.

The naturalness of the speech style (register) fell too, from 87% to 79%. I rolled candidates and selected among them, and ended up worse than not selecting at all.

Why? Because the “official subtitles” I used were a different thing entirely. Those subtitles were the Korean → Japanese translation of a Korean drama, and what I was translating was the English source text. The source language was different, and on top of that they carried the mismatch particular to subtitles (alignment noise between screen and dialogue). Select candidates by closeness to that off-target right answer, and the model abandons faithfulness to the English source and gets pulled toward content that came from the Korean. If the yardstick is skewed, the more candidates you add, the more carefully you optimize in the skewed direction.

I stepped in the same hole going for volume, too. When I padded the model answers (the SFT training data) out to 8,553 examples with many variants at a higher temperature, temperature noise came along with them and the result lost to a clean 3,301 examples built at a single temperature. This isn’t about increasing the candidate count N; it’s about how many examples I inflated the training data to — more was not better.

The lesson isn’t that best-of-N is wrong. It’s that the validity of the yardstick (the QE, the selection anchor) is everything. It worked on Live because QE is a yardstick pointed the right way — it estimates “how good this translation is” directly. It fell over on subtitles because I used a yardstick pointed the wrong way: closeness to a right answer facing a different direction. best-of-N is a device that amplifies the quality of your selection criterion as-is — a good criterion beats the teacher, a bad one leaves you worse off than not selecting.

This is also the same story as That “accuracy” is a definition you chose, which I wrote earlier. Picking “good” with a QE means picking one definition of “good.” If that definition misses the mark, then no matter how carefully you select among the candidates, what gets selected is off the mark.

Putting this in business terms

Carefully building one set of gold data is an easy investment to understand. But however good that one set is, it’s bound by the ceiling of the teacher’s first shot.

best-of-N × QE is a mechanism for pushing that ceiling up with no extra model and no data purchase. Roll the same teacher N times and pick the good ones — that’s all. On Live it did produce a real +2. And my design notes rank it as the highest-ROI lever I have (a projected +10–15 points) — though the version that stacks it up big is still a “next move,” not something I’ve run.

But this device’s performance is decided by how you measure “good.” With an accurate yardstick it beats the teacher; with an off-target one it loses even to a single clean answer. So the question to ask before adopting best-of-N isn’t “how many candidates should I generate?” It’s “does the yardstick I select with really point at the kind of good I want?”

Breaking the faith that “there is one right answer and it is sacred” is a good thing. But you mustn’t replace it with another faith: “just roll candidates and you win.” What belongs on the ground where the old faith stood is the discipline of continually doubting your definition of “good.”

Lessons

1. Don’t worship the one right answer. The teacher’s first shot is not a ceiling on the teacher’s ability. Roll N of them and select with a quality estimator (QE), and you can build model answers that beat the teacher itself. On Live this selection did produce a real +2 (90→96 together with the CPO that followed), and as a lever to stack up big I rate it the most promising one I have (not yet run).

2. best-of-N is no cure-all — the yardstick you select with is everything. Select by closeness to an off-target right answer (wrong direction, alignment noise) and you lose to a single clean answer with no selection at all (82% → 73–77%). Before you increase the candidate count, verify that the selection criterion is valid. A bad yardstick is worse than not selecting.

3. Direction over volume — picking “good” means picking a definition of “good.” Inflating to 8,553 examples with a higher temperature still lost to a clean 3,301. Piling on more, or rolling more, is not value in itself. The question before you adopt it isn’t “what should N be” but “does this yardstick point at the kind of good I want?”

---

Appendix: raw data

All LLM-as-judge (with sample spot-checks by hand). The number of evaluated cases is limited; read the point estimates as a strong signal, not as settled.

Related: the basics of distillation and SFT/CPO are in Raising a small AI into a specialist at one job. Using best-of-N to generate teacher data is covered in Get a different AI to build your teacher AI, and the full picture of Live specialization in A small specialist drew level with the strongest model in service.