A small specialist drew level with the strongest model in service

A model about a third the size (8B) drew level with the strongest class currently in service (27B). But only after narrowing to a single task — Live subtitles. 90 → 96, tied with its own teacher.

That draw came with a price tag. The price of taking Live up +6 was Page and Manga each breaking by −4. Specialization isn’t a free lunch; it’s a choice about allocation.

Which means the question to settle first isn’t “should I specialize” but “which mode is the main battleground, and what do I do with the others.” The ways out — remixing the data, per-mode routing, scaling capacity — each cost something different in shape.

There’s an unspoken pecking order in the world of machine translation. The more parameters, the smarter. So if you want the best quality, use the biggest model.

The teacher model LYR runs in production is a general-purpose model with 27 billion parameters (27B) — one of the strongest currently in service. The specialist model LYR raises in-house has about a third of that: 8 billion parameters (8B).

How far could I push that 8B on Live subtitles — the job of translating the one-line-at-a-time captions running along the bottom of a video, in real time? That’s what this article is about. The short version: the 8B drew level with the 27B. But that draw came with a price tag.

The assumption — “a small model can’t beat a big one”

Here’s where it started. I measured the rate at which Live subtitle translations actually pass as usable (overall = the share where meaning and naturalness are good enough for real use), side by side across models.

On Live, the 8B was at 90. The 27B at 96. A 6-point gap.

Read plainly, that looks like a difference in capacity. The 8B’s capacity is too small to handle Live’s hard cases — idioms, picking the wrong subject. If that’s true, there’s only one move: make the model bigger. It’s a tempting conclusion.

But is that 6 points really “the capacity wall”? Or is it a question of how it was sharpened? Running to a bigger model without checking is sprinting in the most expensive direction available. So first I looked at what the failures actually were.

Measured — the gap wasn’t capacity

Opening up the Live items the 8B was failing on, one by one, the culprits came in three kinds. Literal translation of idioms (rendering “isn’t like you” word for word), picking the wrong subject, and word hallucination (adding words that were never in the source).

Of those, idioms and subjects are not a question of capacity. They’re the kind of mistake that goes away if you show the model enough correct model answers. So maybe what the 8B lacked wasn’t capacity but high-quality model answers specialized for the Live job.

So I sharpened it in two stages.

Stage one: make the model answers better. Have the 27B teacher translate the same sentence N different ways, then pick the single best among them with quality estimation (QE = automatically scoring a translation by looking at the translation alone). The translations picked out by this “best-of-N QE” become the 8B’s model answers (→ Getting the AI to pick a translation better than the gold reference). That’s what makes the teacher data for idioms and subjects high quality.

Stage two: teach preference with pairs of good and bad translations. Build Live-specific pairs — “this translation is good, this one is bad” — and bake them in with CPO (training that corrects the model’s preferences directly from contrastive good/bad example pairs). Live’s pairs draw the sharpest line between good and bad, and the separation in training was where it worked hardest.

The result —

Live went 90 → 96. Tied with the 27B (96), and within a hair of Scout (97). The 6-point gap had gone to zero. And register held at 98 — still 27B-class (99). Split into its parts: the higher-quality model answers gave +2, the Live-specific CPO +4. What closed the gap wasn’t capacity, it was specializing on the single point that is Live.

A model about a third the size had drawn level with the strongest model in service. If the story ended here, it’d be a satisfying win. But there are columns in that table I can’t pretend not to see.

Specialization isn’t free — it’s a choice about allocation

Look at Page and Manga on the Live-specialized version. Page 93→89, Manga 92→88. Both broke by −4.

The training that took Live up +6 dragged Page and Manga down −4. This isn’t an accidental side effect. The 8B does double duty across three modes in a single model, and when you skew it toward one side (Live’s single-line, one-at-a-time subtitles), the distribution for Page/Manga — which translate a block at a time — gets pushed out. I observed the same shape of breakage — a regime shift (the training data’s distribution skews to one side and the mode on the other side falls apart) — over and over: with Manga-heavy data, and with training that included context. The mechanism of mode mixing itself, with all the numbers, is in Don’t stop at measuring a single mode. The −4 from Live specialization here is one instance of that general rule.

So Live 90→96 is not, properly speaking, “6 points out of nowhere.” It’s carved out of Page and Manga and handed to Live — a reallocation. And not even a clean, even trade at that (Live +6, against −8 combined across Page and Manga). Specialization is not a free lunch. It’s choosing where to put the weight inside a limited capacity. The right posture is to suspect that whenever something goes up, something you didn’t measure has gone down.

So what do you do with that −4?

Once the trade-off is visible, the move stops being a binary of “specialize or don’t.” It becomes a problem of designing the allocation.

1. Remix the Live data back into the general data and retrain — instead of adding single lines as single lines, blend them in a form that doesn’t wreck the block distribution, and dilute the interference.
2. Split the airframe per mode — Live gets the specialist airframe, Page/Manga the general-purpose airframe, switched by role. Don’t make them fight over the same capacity.
3. Scale up capacity — the only unconditional way to eliminate the interference at the root, but it’s also a return to the expensive direction of “bigger is smarter.” Reaching straight for capacity on problems that sharpening fixes, like idioms and subjects, is giving up on diagnosis. (Some jobs do genuinely demand capacity, though — deep reading comprehension in Japanese, for one. Where specialization can’t reach, I’ll leave to Spotting the rate limiter in the language it’s worst at.)

Whichever one you pick, the starting point was the same: don’t say “I won” while looking only at the targeted mode. Had I stared only at Live’s 96, I’d have gone around announcing “the 8B has caught up with the 27B” without ever noticing the −4 on Page and Manga. That’s half the truth at best.

Lessons

1. You don’t have to fight the giants head-on. Instead of challenging 27 billion parameters at general-purpose work, narrow 8 billion onto a single point — Live subtitles — and the success rate goes 90 → 96, drawing level with the strongest model in service. At about a third of the capacity, you can match it on the one trick you aimed at.
2. But specialization is a choice about allocation, not a free lunch. The same training that took Live up +6 broke Page/Manga by −4. As long as one model does double duty across several jobs, whatever you gained was carved out of somewhere else (and not necessarily as a clean, even trade).
3. So don’t decide win or lose from the metric you targeted alone. Whether specialization worked can only be judged by laying the target’s gain next to what the non-targets sacrificed. Measuring one side only fakes a success.

---

Appendix: raw data

Shared conditions: self-hosted 8B model (a distillation line taught by the 27-billion-parameter 27B general-purpose model). The three modes — Live / Page / Manga — measured side by side on the same evaluation set. Numbers are overall (= the share of translations good enough for real use, %); listed separately is register (how natural the voice sounds, %). Judged by Claude (Max plan) in a single judge round. The number of evaluation items (n) wasn’t recorded, so read each % as a point estimate — a signal, not a settled result (nothing here is established at a significance level. The direction of the −4, however, has been observed repeatedly when specializing other modes → Don’t stop at measuring a single mode).

Main results (overall / register, %, same set):

The general rule of mode mixing (the −4 regime shift, reproduced with three moves on 2026-07-14): beyond Live specialization, the same shape of degradation on the opposite side showed up with Manga-heavy data and with context-included training. All the numbers and the mechanism (drift in the learned distribution of entry counts / styles) are deferred to Don’t stop at measuring a single mode. The −4 from Live specialization here is one of those three moves.

Ways out: ① retrain with the Live data mixed into the general set without wrecking the distribution, ② per-mode routing (Live = the specialist airframe / Page-Manga = the general-purpose airframe), ③ capacity scaling (the only move that unconditionally eliminates the interference, but expensive).