HDR Capture and Tone Mapping: How Smartphones Handle High Dynamic Range

Your smartphone sees about 12–14 stops of dynamic range in a single exposure. A sunlit window and a shadowed room can span 18 stops. The gap between what the sensor captures and what the scene contains is where HDR lives — and where modern mobile imaging earns its keep.

High Dynamic Range on phones is not a filter. It is a full computational photography pipeline that begins before the shutter clicks and ends at the display driver. Understanding it requires tracing photons from the photodiode to the OLED subpixel, with tone mapping as the critical translation layer.

1. Dynamic Range Limits of Mobile Sensors

Dynamic range (DR) is 20·log10(FWC / read noise) where FWC is full-well capacity. A flagship 1/1.3" 50 MP sensor with 1.2 µm binned pixels might achieve ~71 dB (~11.8 stops) in a single shot at base ISO. Drop to 0.61 µm quad-bayer without binning and DR falls to ~65 dB.

By comparison, a backlit interior easily exceeds 100 dB (16.6 stops) between 100,000 lux outdoors and 50 lux in shadow. This is why even advanced neuromorphic computing in mobile chips still relies on multi-frame HDR rather than single-shot inference to recover scene luminance.

Two capture strategies expand DR at the hardware level:

• Staggered HDR (sHDR / DOL-HDR): Alternate rows expose long/short within one frame readout. Common in video (3-exposure DOL up to 120 dB). Trades spatial resolution and can cause motion zipper artifacts.
• Temporal bracketing: Sequential frames at –2, 0, +2 EV. Preserves full resolution but requires alignment. This is the dominant photo mode and the focus of this article.

Modern stacks combine both: Qualcomm Spectra, MediaTek Imagiq and Google Tensor ISPs support 3-frame-temporal plus 2-exposure staggered for 14-bit effective pipelines.

2. The Exposure Bracketing Pipeline

When you tap the shutter, the ISP has already been metering. Most phones run continuous 30 fps preview in 10-bit or 12-bit RAW and maintain a histogram and highlight clipping map.

How exposure triplets are chosen

Auto-exposure bracketing (AEB) is not fixed at ±2 EV. The algorithm adapts:

• Highlight anchor: Short exposure set so that 99.5th percentile of luminance is just below sensor saturation (RAW ~ 0.95·FWC). This recovers sky and speculars.
• Shadow anchor: Long exposure set to lift shadows to target SNR > 6 dB after merging, limited by motion blur threshold (typically 1/15s handheld with OIS).
• Mid anchor: 0 EV for tonal reference and deghosting mask; often shot at ISO 100–400 for clean midtones.

On devices with large aperture (f/1.4–f/1.7), ND-like electronic dimming via reduced analog gain is preferred over ultra-short shutter <1/8000s, which would cause banding with LED flicker.

Capturing three 12 MP 12-bit RAW frames takes ~36 ms on a modern MIPI CSI-2 interface at 2.5 Gbps/lane. The frames are held in the ISP's line buffer and DRAM — about 70–90 MB before demosaic — so zero-shutter-lag (ZSL) buffering is essential.

3. Alignment and Ghost Suppression

Sequential bracketing introduces parallax and motion. Hand tremor of 2–5 pixels between frames is typical, but subject motion can be 30+ pixels. The alignment stage works at two levels:

Global alignment uses gyroscope data (200–400 Hz) and feature matching to estimate a homography. This corrects camera rotation and translation in < 5 ms using the ISP's warp engine.

Dense alignment computes optical flow on downscaled luma (approx. 1/4 resolution) to handle local motion like leaves or pedestrians. Modern ASICs do this with pyramid Lucas-Kanade at ~15 ms for 12 MP.

Even with perfect alignment, moving objects cause ghosts if naively averaged. Ghost detection generates a per-pixel inconsistency map:

• Exposure-compensated difference: each frame normalized to mid exposure via L = (RAW / exposure) ^ gamma and compared.
• Pixels where variance exceeds threshold are marked and excluded from fusion, or replaced by the single reference (mid) frame.
• AI-based weighting (CNN on device NPU) now outperforms handcrafted heuristics for hair and foliage, as seen in recent DXOMARK methodology analysis of ghosting artifacts.

4. Exposure Fusion: From Brackets to Radiance Map

Fusion reconstructs scene radiance in linear light before any tone curve is applied. Two families dominate phones:

Weighted averaging (Mertens-style)

Each pixel gets weights for contrast, saturation, and well-exposedness. The under-exposed frame dominates highlights, the over-exposed frame fills shadows. Implemented in YUV with pyramid blending to avoid halos. This is fast and runs on the ISP.

Radiometric reconstruction (Debevec & Malik)

Recovers camera response function (CRF) and builds a true HDR radiance map in 32-bit float: E = (f⁻¹(Z) / exposure time) where Z is pixel value. More accurate for extreme DR but heavier; often reserved for ProRAW or night mode.

Most flagships hybridize: quick YUV fusion for viewfinder and a heavier linear float path for the saved image. The resulting linear HDR buffer holds 18–20 stops with values spanning 0.001 to 1000+ relative exposure.

5. Tone Mapping: Compressing Reality to Screen

Tone mapping is where engineering meets perception. The goal is to squeeze 100,000:1 scene contrast onto an 800–1,600 nit OLED that can display ~1000:1 simultaneously, while keeping it looking natural.

Global operators

A single S-curve is applied to all pixels based on luminance. The classic Reinhard operator L_d = L / (1 + L) compresses smoothly, but flattens midtones. Phones use tuned filmic curves derived from ACES:

Global mapping is cheap and temporal-stable (no flicker in video), but it cannot boost shadow detail without washing out highlights.

Local operators

Local tone mapping adjusts each pixel based on its neighborhood, similar to dodging and burning. Techniques include bilateral filtering, guided filters, and exposure-aware 2D LUTs. Apple Photonic Engine, Samsung Shadow HDR, and Xiaomi's Local Tone use edge-aware decomposition: base layer gets compressed, detail layer is boosted and re-added.

Result: shadows lift by 10–15 dB with halos < 2% if tuned well. The cost is higher compute (30–40 ms on GPU) and risk of "HDR look" — glowing halos and flat contrast if overdone.

Current flagships run a hybrid: global curve for base toning, then a local pass masked to shadows and highlights only, blended via luminance-dependent alpha. This preserves temporal coherence while gaining local punch.

6. From Tone Map to Photons: Display Rendering

After tone mapping, the image is still not display-ready. Color is mapped through a 3D LUT from sensor-native gamut to Display P3 or Rec.2020, gamma-corrected (OOTF), and then sent to the display pipeline.

On HDR displays, there is a fork:

• SDR output (most shared JPEGs): Tone-mapped to 8-bit sRGB with peak 100–200 nits. Highlights are compressed to appear bright but not clipped.
• HDR output (AVIF, Ultra HDR, Dolby Vision): Stores gain map or PQ-encoded luminance up to 10-bit. Tone mapping is inverted on HDR screens — the display re-expands highlights to 1,200–2,600 nits using the embedded gain map, achieving true highlight pop without affecting SDR viewers.

Google's Ultra HDR (Android 14+) and Apple's Adaptive HDR exemplify this dual path: an SDR base image plus a 0.5–1 MB gain map that encodes gain = HDR_luminance / SDR_luminance per pixel. It costs little bandwidth but doubles perceived dynamic range on compatible OLEDs.

Display-side tone mapping also matters. OLED drivers apply their own APL-dependent curve to avoid burn-in and manage power, which interacts with content tone mapping. This is why peak brightness specs alone mislead — sustained window brightness and the EOTF tracking error (ΔE < 3 is reference) determine actual HDR fidelity. For sustained data rates to stream HDR, the improvements in 5G-Advanced Release 18 help maintain 4K HDR uplink without throttling the ISP power budget.

7. Engineering Trade-offs: Power, Latency, and Trust

Every HDR frame burns 0.4–0.7 J. At 60 fps HDR video, the ISP + NPU can draw 3.5 W continuously, triggering thermal throttling after ~4 minutes in a 25 °C ambient. Designers therefore cap local tone mapping to every nth frame in video and fall back to global curves under thermal stress.

There is also a security dimension. The HDR gain map and auxiliary frames live in the media pipeline, which shares DRAM with the baseband and GPU. Hardening this path — including attestation of ISP firmware — is part of broader mobile supply chain security for camera integrity, preventing malicious tone-map injection that could hide forensic details.

Looking ahead, single-exposure HDR via dual-gain readout (2-in-1 HDR with simultaneous high/low conversion gain per pixel, as in Sony LYTIA 900) will reduce reliance on bracketing. Paired with learned tone mapping on NPUs exceeding 45 TOPS, we expect ghost-free 14-stop single-shot capture within two generations. The curve will remain, but the need to bracket will fade.

For now, the best HDR is invisible. If you notice halos, waxy shadows, or neon skies, tone mapping has failed its perceptual test. When done right, you simply feel you can see into both the shadows and the light — as your eyes do.