The Traffic Cop Inside Your Phone
You take a birthday photo, tap reframe, and in under a second the shot is cropped, the lighting lifted, the faces sharpened. The main processor barely noticed. Something else did that work so fast the hand-off was invisible by the time your thumb left the screen.
That something else is a Neural Processing Unit, an NPU. Every flagship phone sold in the last several years ships with one, tucked onto the same piece of silicon as the main CPU cores and the GPU. The question worth asking isn't what it does. It's how the phone decides, in real time, which jobs get routed there versus everywhere else.
The short answer: the operating system keeps a running catalogue of task types, and anything that looks like a matrix multiplication chain, a pattern-matching loop, or a repetitive inference job gets flagged for the NPU automatically.
The long answer is more interesting.
Why Dedicated Silicon Exists in the First Place
Think of the CPU as a Swiss Army knife: flexible, capable of almost anything, optimised for none of it. The NPU is a bread knife. Completely useless for opening a can, but it slices through neural network layers the way a CPU never could, at a fraction of the energy cost.
That energy gap is the whole argument. A CPU running a real-time speech recognition model continuously would drain a typical 4,500 mAh battery in a few hours. The same workload on an NPU can run all day, because the NPU's architecture does one thing well: multiply large arrays of numbers together in parallel and apply activation functions. That's it. Neural networks are, at their core, enormous piles of exactly that operation.
Qualcomm calls its implementation the Hexagon NPU. Apple calls its version the Neural Engine. MediaTek uses the term APU. The branding differs; the underlying logic is identical.
The Routing Decision, Step by Step
Here's how it actually plays out. You open the camera on a Snapdragon-powered Android phone, point it at a crowded street, and the phone starts segmenting pedestrians from the background for a portrait-mode blur in real time.
The camera app calls a framework layer, Android's Neural Networks API or its successor TFLite delegates, which receives the pre-trained segmentation model and immediately checks a compatibility table baked into the device driver. Three questions get answered fast: Can the NPU execute each layer type in this model? Is the model small enough to fit in the NPU's dedicated SRAM without constant main-memory fetches? Is the latency requirement tight enough that the overhead of offloading is actually worth it?
All three yes: the framework compiles the model into NPU-native instructions and hands it off. The CPU supervises, waiting for results.
If a particular layer type isn't supported, some exotic attention mechanism the NPU firmware doesn't recognise, that specific layer falls back to the GPU or CPU while the rest still runs on the NPU. Partial offloading is common, not a failure state.
The whole negotiation takes milliseconds on first run. After that, the compiled model is cached, and next time you open the camera it loads instantly. The system is more elegant than most people give it credit for, and it runs this way thousands of times a day without a single prompt from you.
Two Phones, One Difference
Priya and Jae bought the same phone model eighteen months apart. Priya on launch day, Jae a refurbished unit on an older software stack.
Priya's phone has had two major OS updates, both shipping improved NPU compiler passes. More layer types now route to the Neural Engine cleanly, the fallback rate dropped, and her live translation feature runs at roughly 15 frames per second with almost no CPU spike.
Jae's unit is still on original shipping software. The same translation model hits a layer the older compiler doesn't support on-device, falls back to the CPU for that segment, and his phone runs noticeably hotter during the same task.
Same chip. Same model. Different outcome.
Software updates to NPU compilers and driver tables are often more consequential than anything on the spec sheet, and almost nobody talks about them.
What People Consistently Misread
Here's the one that frustrates me: the popular assumption that more NPU TOPS (Tera Operations Per Second) means a faster phone. It mostly doesn't, for everyday use. TOPS measures raw throughput on a specific benchmark, not how intelligently the workload gets routed, and not whether the models your apps actually use are even compiled to take advantage of it.
A phone with 35 TOPS and a mature, well-optimised driver stack will beat a phone with 45 TOPS and a half-baked compiler on real camera and voice tasks. The spec is not the story.
So can we stop letting marketing departments win this one?
Also worth knowing: not every AI-labelled feature actually uses the NPU. Some manufacturers run lightweight filters on the GPU because the model is simple enough and the GPU is already warm, like a chef using the oven that's already on rather than firing up a second one. The marketing says AI; the silicon says GPU shader. Neither is technically wrong, but they are not the same thing.
And the NPU isn't always faster. For a single, one-off inference on a tiny model, the overhead of compiling and offloading can cost more time than just letting the CPU handle it. The routing layer knows this and uses threshold heuristics: below a certain model complexity, the job stays on the CPU. That's not a flaw. That's the system working correctly.
The Bit That Actually Matters
The NPU in your phone isn't a separate brain making autonomous decisions. It's a specialised workshop the operating system sends specific jobs to, based on a contract negotiated between the app framework, the driver, and the hardware's known capabilities.
Every time that negotiation runs well, you get longer battery life, cooler thermals, and a camera that doesn't stutter. Every time it runs poorly, you get a phone that benchmarks brilliantly and still feels sluggish on the features you actually use.
The chip matters. The software that talks to the chip matters more, and that's the part nobody puts on the box.