I cut OCR's power budget by an order of magnitude — not by making it faster, but by calling it less

I made OCR faster on the device. One detection pass went 40→20ms, energy −58%. Battery life still didn’t stretch far enough to feel. Because halving one pass didn’t change how many times I was running it.

When I measured, OCR’s duty cycle was 10%. Nine tenths of the time it was reading a screen with nothing on it. So I dropped the question of how to make it faster and put in a layer that decides whether to call it at all — the Awake Layer, an always-on layer 70x cheaper. The power budget around detection came down by roughly an order of magnitude.

It isn’t a clean story, though. Making the gate itself 4.5x cheaper moved total power by only -22%. What dominated power wasn’t the gate’s cost but the number of calls the gate issued (6.9x the ideal).

Subtitle translation starts by reading the text on the screen. As long as OCR runs on the device, that power comes out of the user’s battery. Unlike a cloud bill, this one is an invisible bill. But it arrives without fail — as battery drain.

So the first thing I did was the obvious thing: make OCR faster.

It did get faster. The order of magnitude didn’t change

The move was a sound one. Stop reading the full screen; look only at the band where text is likely to be. Process fewer pixels and of course it gets faster.

Measured, one detection pass went 40ms → 20ms (-51%) and energy 146mJ → 61mJ (-58%). Not bad. It’s still in the product today.

But battery life didn’t stretch far enough to feel. The reason is simple: halving one pass doesn’t help when the number of passes stays the same. This work ran every cycle, five times a second, subtitles on screen or not. I was lifting something half as heavy, exactly as many times.

I measured, and nine calls in ten were wasted

So I measured the count instead.

Nine tenths of that capacity went into reading a screen with nothing on it. Subtitles aren’t up all the time. They vanish while nobody’s talking, and while they are up, the same characters just sit there. The moments actually worth reading are very short.

That changed the question. Not how do I make this work faster, but do I need this work at all.

Awake Layer — keep it asleep, wake it only at the moment you need it

The answer I put in was the Awake Layer. The idea is embarrassingly simple:

Put an always-on layer that is orders of magnitude cheaper in front of the expensive work, and let it decide one thing only: does this need to wake up right now?

This layer looks at exactly two things. Is there text on the screen worth reading right now (presence), and has it changed (change). It doesn’t read the text. It has no idea what the text says. It answers one question only: wake it, or leave it sleeping.

Why that works becomes obvious once you line up the orders of magnitude in cost.

About 70x. At that ratio, running the cheap one nonstop and keeping the expensive one off wins by a mile. If the gatekeeper’s salary is one seventieth of that of the craftsman working inside, it’s cheaper to keep the gatekeeper standing there 24 hours a day.

One more thing worked as a design: the breakdown of the changes. Decompose the subtitle change events and 45% appearances + 45% disappearances = 90% are there/not-there transitions, which even a cheap layer can tell apart. The hard part is the remaining 10% — the case where presence never breaks and only the content swaps out. Nine tenths of it, the cheap layer can catch.

The result, for detection’s continuous power:

• Running the expensive detection every cycle (5 times a second): 204mW
• Awake Layer always on + detection only at subtitle changes (0.5 times a second): about 32mW (-84%)

On top of that sits the “-58% per call” from earlier. Stack them up and the power budget around detection goes 204mW → somewhere near 20mW = roughly an order of magnitude. Live translation as a whole runs at about 2000mW, so that’s worth roughly +9% on battery life.

(Let me be honest about this. That order of magnitude is a projection stacked up from measured components — the measured energy of one detection pass, the measured power of the always-on layer, the measured firing rate — not one end-to-end ODPM measurement across the whole device.)

There was a side benefit too. This layer doesn’t “read” text. It only looks at whether there’s a text-like visual structure. As a result, what it learned on Japanese carried straight over to Latin script and to Hangul (confirmed across 4 titles and 4 scripts). All the per-language engineering simply stopped being necessary. Work you delete takes its multilingual cost with it.

Here’s the real story — I made the gate cheaper and power didn’t drop

It looks like a clean story, but this is exactly where I got it completely wrong once.

When I swapped the gatekeeper role onto a cheaper model, the gate alone got 4.5x more power-efficient. Obviously the total would drop a lot too, I thought. It didn’t. Total power: -22%.

I broke the cause down and had to look twice.

For 33 subtitles, it was calling the expensive work about 228 times. 6.9x the ideal (once per subtitle).

Make the gatekeeper cheap all you like: if that gatekeeper keeps banging on the craftsman’s door for no reason, the craftsman is the one eating the electricity. What dominated power wasn’t the gatekeeper’s cost — it was the number of calls the gatekeeper issued.

In other words — “make it cheaper” isn’t enough; power only moves once you get all the way to “call it fewer times.” If I’d been satisfied the moment the gate was in, I’d never have noticed this 6.9x.

Cut too far and now the experience breaks

Fine, then narrow the calls — so I tightened the firing condition to “once on the rising edge only.” Calls went 274 → 69 (-75%), the power proxy -48%. It dropped exactly as intended.

In exchange, subtitle capture rate fell from 82% to 61%. On landscape screens it was worse — 71% → 32% — two thirds of the subtitles missed. Power halved, but as a translation app that’s disqualified.

What I learned here is that “reduce” doesn’t mean “reduce sloppily.” Narrowing down to a single call presupposes that the layer can hit the moment worth waking on precisely. Cut the count before that’s true and everything you cut turns straight into misses. The quality of the decision and the number of calls cannot be separated. That’s where I am now — investing in getting the decision side right.

Lessons

1. Look at the count, not the cost of one call. Halving one call does nothing to power if the wasted calls outnumber it 10 to 1. The first thing to measure wasn’t processing time; it was duty cycle (measured: 10%).
2. A layer an order of magnitude cheaper pays off even running nonstop. But adding the gate isn’t where it ends. If the gate is 70x cheaper, keeping it awake 24 hours a day is the cheaper option. But making the gate cheaper (4.5x) and cutting the wasted calls (6.9x over-firing) are separate problems, and the one that worked was the second.
3. Work you delete takes its incidental costs with it. But the right to cut is bought with accuracy. The per-language engineering, the evaluation, the maintenance — all of it went away along with the work; speeding things up never does that. On the other side, narrowing the calls presupposes never missing the moment worth waking on, and if you cut while that’s still weak, you pay for the power win with a loss in the experience.

This is the manifesto’s fourth principle — Eliminate, Don’t Accelerate — in its plainest form. Making it faster bought -51%. Deciding not to call it bought an order of magnitude.

Appendix: raw data

Note: part of the power proxy (a relative value of firings × cost) is an accumulation inside the harness, not an ODPM measurement. The “order of magnitude” above is likewise a projection stacked up from measured components.