Heterogeneous Computing in Mobile SoCs: CPU, GPU, NPU and DSP Orchestration

Modern mobile SoCs no longer rely on the CPU alone. The Snapdragon 8 Gen 3, Dimensity 9300 and Tensor G4 all integrate four distinct compute domains — latency-optimized CPUs, throughput-optimized GPUs, tensor-optimized NPUs and signal-optimized DSPs — tied together by a unified fabric and a sophisticated orchestration stack. Understanding how workloads are partitioned across them is essential to performance, power, and user experience.

1. Why Heterogeneous? The End of CPU-Centric Scaling

A decade ago, mobile performance meant faster Cortex-A cores. That model collapsed against two walls: the power wall (sub-5W thermal envelope) and the utilization wall (single-threaded performance scaling ~8% per year). Meanwhile, workloads became fundamentally diverse. A 4K60 video capture, a live-translation call, and a generative AI wallpaper require completely different compute patterns — scalar branching, data-parallel shading, matrix multiplication, and fixed-point filtering.

Heterogeneous computing solves this by embracing specialization. Each domain is architected for a different efficiency point on the power-performance curve:

• Dimension 1: Parallelism granularity. CPUs handle 1-8 threads with deep branch prediction; GPUs handle 1024+ threads with SIMD; NPUs handle 2D systolic arrays.
• Dimension 2: Precision. CPUs favor FP64/FP32; GPUs FP32/FP16; NPUs INT8/INT4 and FP16; DSPs fixed-point 16/24-bit with saturation.
• Dimension 3: Data locality. CPUs optimize for latency with large caches; accelerators optimize for bandwidth with scratchpads and DMA.

The result is a 4-10x performance-per-watt gain when a workload runs on its affinity engine versus a general-purpose core. The challenge shifts from raw horsepower to orchestration: deciding where and when to run each kernel.

2. Anatomy of a Modern Heterogeneous SoC

Take a current flagship SoC as a reference design. The fabric is typically a coherent mesh or NoC (Network-on-Chip) with a system-level cache (SLC) of 8-12MB shared across all agents. Each compute island controls its own voltage/frequency domain and can be power-gated in microseconds.

What matters is not just the blocks but the interconnect. Legacy SoCs used a two-level bus fabric with separate coherent and non-coherent domains. Modern designs expose a single physical address space via an IOMMU/SMMU per agent, enabling zero-copy buffer sharing via Android’s AHardwareBuffer or iOS’s IOSurface. This unified memory architecture (UMA) cuts the cost of handing off a 12MP image from ISP to NPU from ~6ms (copy) to ~0.2ms (cache-coherent handle pass).

For a deeper look at how the radio subsystem fits into this shared fabric without starving accelerators, see our analysis of mobile RF front-end architecture.

3. The Four Compute Pillars Compared

Each domain excels at a different computational idiom. Selecting the wrong one doesn’t just cost power — it can miss real-time deadlines entirely.

* 7B-parameter LLM token generation on NPU with 4-bit weight quantization; vision models <4ms at 30W SoC power envelope.

CPU: Still the orchestrator. Even with offload, the big Cortex-X core owns scheduling, security, and Amdahl’s serial bottleneck. Its new trick is data source hinting — using MPAM and QoS extensions to prioritize NPU traffic during camera bursts.

GPU: No longer just for games. Compute pipelines handle bokeh rendering, video super-resolution, and increasingly, LLM prefill where its high occupancy beats the NPU’s systolic constraints. The companion role in mobile display calibration also runs almost entirely on GPU 3D LUTs and compositor shaders.

NPU: The star of on-device AI. Architected as a 2D array of MAC units (e.g., 4096 INT8 MACs/cycle) with private SRAM (4-8MB) to keep weights resident. Key innovation in 2025-2026: native transformer acceleration — dedicated softmax, layer-norm, and KV-cache management units that cut large language model decode energy by ~60%.

DSP: The unsung efficiency king. A Hexagon-style VLIW DSP can run always-on keyword spotting at 2mW and handle 108MP demosaicing at 4K60 without waking the CPU. Its tightly coupled memory (TCM) and hardware loops make it ideal for 1D/2D filtering where GPUs would waste thread scheduling overhead.

4. The Orchestration Stack: From HAL to Scheduler

Hardware heterogeneity means nothing without software that can see it. The Android and iOS stacks now expose three layers of orchestration:

Layer 1 — Framework APIs & HALs

Developers rarely target engines directly. They call high-level APIs: NNAPI / Core ML for inference, Vulkan / Metal for graphics/compute, Camera HAL3 for imaging. Each framework queries capabilities via getCapabilities() and delegates to the vendor HAL. For example, NNAPI’s ANEuralNetworksCompilation_setPreference(LOW_POWER) steers a model to the DSP/NPU, while FAST_SINGLE_ANSWER favors the low-latency NPU path.

Layer 2 — Vendor Runtime & Graph Compiler

Qualcomm’s QNN, MediaTek’s NeuroPilot, and Google’s LiteRT (formerly TFLite+XNNPACK) perform graph-level optimization: operator fusion (Conv+BN+ReLU → single NPU op), quantization (FP32 → INT8 with 0.5% accuracy drop), and partitioning — splitting a single model graph across NPU (convolutions), DSP (pre-processing), and CPU (softmax fallback). The compiler emits a binary with per-subgraph affinity tags.

Layer 3 — Kernel Scheduler & Power HAL

At the bottom, Linux’s Energy Aware Scheduling (EAS), the GPU’s firmware scheduler, and the NPU’s micro-sequencer coordinate. Modern SoCs use a Global Task Scheduler (GTS) in the always-on microcontroller that arbitrates based on:

• Thermal headroom (via skin and junction sensors)
• Deadline awareness (vsync at 8.3ms for 120Hz, audio at 5ms)
• Memory bandwidth quota (e.g., limiting GPU to 60% during NPU burst)

This is critical for AR workloads where a missed deadline means motion sickness. Our breakdown of mobile AR glasses display technology shows how photon-to-motion latency budgets force co-scheduling of SLAM on DSP, rendering on GPU, and reprojection on NPU simultaneously.

5. Workload Distribution in Practice

Consider a real user story: Night-mode portrait video at 30fps with live bokeh and speech enhancement.

1. Frame 0ms: Spectra ISP ingests RAW 12MP, does 3A + demosaic on fixed-function hardware, outputs YUV to DSP TCM.
2. Frame 4ms: DSP (HVX) runs temporal denoise + skin-tone segmentation using 16-bit vector ops, 0.8W.
3. Frame 9ms: NPU runs 4MP portrait segmentation (DeepLabv3, INT8, 3.2 TOPs) + depth estimation in parallel tiles — 7ms wall.
4. Frame 16ms: GPU composites blurred background via compute shader, applies tone mapping 3D LUT.
5. Frame 22ms: CPU encodes H.265 at 20Mbps and mixes 48kHz noise-suppressed audio (DSP AEC).

Each handoff is a zero-copy fence, not a memcpy. The scheduler ensures NPU and GPU never contend on the SLC simultaneously by time-slicing at the warp level.

 // Query driver affinity and allocate coherent buffer
 ANeuralNetworksDevice* npu;
 ANeuralNetworks_getDevice(0, &npu);
 const char* name; ANeuralNetworksDevice_getName(npu, &name); // "qti-npu"

 AHardwareBuffer* buf;
 AHardwareBuffer_Desc desc = { .width=1920, .height=1080,
   .layers=1, .format=AHARDWAREBUFFER_FORMAT_Y8Cb8Cr8_420,
   .usage=AHARDWAREBUFFER_USAGE_CPU_READ_OFTEN |
          AHARDWAREBUFFER_USAGE_GPU_SAMPLED_IMAGE |
          AHARDWAREBUFFER_USAGE_NNAPI_ACCELERATOR };
 AHardwareBuffer_allocate(&desc, &buf);

 // Compile with preference -> NPU, fallback to GPU if quantized ops unsupported
 ANeuralNetworksCompilation* comp;
 ANeuralNetworksCompilation_createForDevices(model, &npu, 1, &comp);
 ANeuralNetworksCompilation_setPreference(comp, ANEURALNETWORKS_PREFER_LOW_POWER);
 ANeuralNetworksCompilation_setPriority(comp, ANEURALNETWORKS_PRIORITY_HIGH);
 ANeuralNetworksCompilation_finish(comp);

 // Execute with fence sync — GPU bokeh stage waits on NPU fence
 ANeuralNetworksExecution* exec;
 ANeuralNetworksExecution_create(comp, &exec);
 ANeuralNetworksExecution_setAHardwareBuffer(exec, 0, buf);
 int syncFd; ANeuralNetworksExecution_startComputeWithDependencies(exec,
   nullptr, 0, nullptr, 0, &syncFd); // syncFd signaled when segmentation mask ready

On iOS, the equivalent uses MLProgram + computeUnits = .all (CPU+GPU+ANE), while vendors exposing 5G Advanced Release 18 features are now moving modem LDPC decode and AI-based CSI feedback onto the same DSP fabric to save die area.

6. Power, Thermal, and Memory — The Real Constraints

Performance specs lie if thermals throttle in 90 seconds. Heterogeneous scheduling is fundamentally a power management problem. Each domain sits in its own AVS rail:

• DVFS granularity: CPU per-core, GPU per-slice, NPU per-tile array, DSP per-thread.
• Power gating: The NPU can cold-boot in ~120µs, making it viable to power-gate between camera frames (33ms period) and save 40-60mW idle.
• BW shaming: LPDDR5X at 68 GB/s is shared. A naïve NPU burst that saturates 80% of BW will starve the display pipe and cause jank. QoS regulators assign priorities: Display > ISP > NPU > GPU > CPU background.

Thermal mitigation now uses per-IP throttling, not just CPU frequency capping. If skin temp hits 42°C, the scheduler first reduces GPU clock and migrates LLM decode from NPU to a more efficient INT4 NPU slice before touching CPU performance — preserving UI responsiveness.

7. What’s Next: Unified Compute

The trajectory is toward less heterogeneity at the ISA level and more at the microarchitecture level. Apple’s AMX, ARM’s SME2, and Qualcomm’s HTA all expose matrix extensions to the CPU, blurring the CPU-NPU line for small-batch inference. Meanwhile, SoCs are adopting chiplet-style accelerators with shared virtual memory (SVM) so a single pointer is valid on CPU, GPU and NPU without explicit map/unmap.

For developers, the lesson is clear: write to the framework, not the engine. Models exported as ONNX or ML Package with dynamic shapes and quantization metadata will automatically exploit tomorrow’s hardware. Hard-coding to “run on DSP” guarantees suboptimal performance in two product cycles.

The SoC has become a distributed system on a 120mm² die. Mastering its orchestration — not just its specs — is what separates a flagship that stutters from one that feels effortless.