Mobile ML Frameworks: Core ML vs TensorFlow Lite vs NNAPI

On-device inference is no longer optional. With privacy constraints, offline requirements, and latency budgets under 16ms, choosing the right mobile ML stack determines whether your model ships or stalls. Core ML, TensorFlow Lite, and NNAPI look similar on a slide — they diverge sharply in hardware abstraction, quantization, and deployment reality.

This analysis cuts through marketing abstractions. We examine how each framework maps neural graphs to silicon — Apple's Neural Engine, Android's heterogeneous DSP/GPU/NPU fabric, and the fragmented reality of vendor drivers — and what that means for model conversion, performance, and maintainability in production apps.

1. Core ML: Vertical Integration as a Feature

Core ML is not just an inference runtime — it's Apple's contract between app developers and its silicon roadmap. Since Core ML 3, models are compiled ahead-of-time into an optimized graph for the Apple Neural Engine (ANE), GPU via Metal Performance Shaders, and CPU via BNNS/Accelerate. The compiler decides placement. You don't.

That opacity is intentional. A .mlpackage contains MIL (Model Intermediate Language) and weight files. At app install or first load, MLModel.compileModel(at:) lowers MIL to ANE's private ISA. This enables aggressive operation fusion, static memory planning, and use of ANE's 16-core architecture without exposing driver details. On A17 Pro and M3, ANE delivers ~35-38 TOPS for INT8, enough for Stable Diffusion inference at ~0.6s on-device.

Where Core ML wins

Zero-config acceleration. No delegate selection. If an op is ANE-compatible (convolution, matmul, attention variants as of Core ML 7), it runs on ANE. Unsupported ops fall back to GPU/CPU with negligible developer intervention. For teams shipping to a controlled hardware matrix — 6 iPhone generations and 4 iPad chips — this is ideal.

Quantization maturity. Core ML Tools supports palettization, linear 8/6/4-bit, and mixed-precision with minimal accuracy loss. A 7B-parameter LLM quantized to 4-bit via Core ML's coremltools.optimize.coreml.linear_quantize_weights fits within 4GB and runs via MLModelConfiguration.computeUnits = .all. The toolchain also validates numerics against the ANE simulator before deployment.

System services. Core ML integrates with Vision, Natural Language, and Sound Analysis frameworks, sharing the same memory pool as the smartphone sensor hub and IMU pipeline. This matters for always-on features where sensor data never leaves the Secure Enclave boundary.

Limitations to plan for

Core ML is Apple-only and opaque. Profiling requires Instruments + ML Performance trace; you cannot dump the ANE schedule. Custom ops are limited to MLCustomLayer on CPU/GPU. And model updates require app update or on-demand MLModelCollection — no dynamic code patching, which interacts heavily with mobile kernel exploit mitigation policies that block JIT.

2. TensorFlow Lite: The Portable Workhorse

TensorFlow Lite (TFLite) trades vertical control for portability. It runs on Android, iOS, embedded Linux, and microcontrollers from the same .tflite FlatBuffer. The runtime is ~400KB compressed and offers a stable C API, making it the default for cross-platform SDKs and OEM integrations.

The delegate architecture is TFLite's core abstraction. The base interpreter executes ops on CPU via XNNPACK (NEON-accelerated). Delegates intercept subgraphs and dispatch them to accelerators:

• GPU Delegate — OpenCL/Vulkan on Android, Metal on iOS. Excellent for CNNs, poor for RNNs/transformers due to shader compilation cost.
• NNAPI Delegate — wraps NNAPI for vendor NPUs on Android 8.1+. Increasingly bypassed by vendor-specific delegates (Samsung ENN, MediaTek NeuroPilot, Qualcomm QNN).
• Hexagon Delegate / QNN — direct path to Hexagon DSP/HTP on Snapdragon, avoiding NNAPI overhead. Required for INT8 efficiency.
• Core ML Delegate — allows TFLite models on iOS to benefit from ANE without full conversion.

For many teams, mobile app runtime comparisons miss that TFLite's real competitor is not Core ML but server inference. Its ability to share pre/post-processing code with TensorFlow and to validate parity via tflite_diff reduces ML engineering drag.

Quantization and tooling

TFLite's quantization story is the most mature cross-platform: dynamic range, full integer (INT8), INT16x8, and experimental FP16. Full integer quantization with representative datasets yields models that run entirely on fixed-point hardware — critical for DSPs that lack FPUs. Google's Model Optimization Toolkit integrates pruning + quantization-aware training (QAT), often recovering 1-2% accuracy over post-training quantization.

However, delegate fragmentation hurts. An op that accelerates on Pixel 8's Tensor G3 may fall back to CPU on a Galaxy A54's Exynos. You must test delegate compatibility per SoC tier, and handle Delegate creation failed gracefully. The new LiteRT rebrand (2024) aims to fix this with a unified CompiledModel API, but adoption is still early.

3. NNAPI: The Abstraction That Vendors Outgrew

Android Neural Networks API (NNAPI) was introduced in Android 8.1 to solve a distribution problem: expose a single API for the NPU/DSP/GPU accelerators shipping in Snapdragon, Exynos, Dimensity, and Tensor SoCs. Apps supply a computation graph; NNAPI drivers (HAL 1.0 → AIDL) partition and accelerate it.

In theory, NNAPI is elegant. A single call to ANeuralNetworksExecution_burstCompute can leverage burst mode and prepared model caching. In practice, from Android 10 onward, adoption diverged:

Qualcomm, MediaTek, and Samsung now ship superior direct SDKs (QNN, NeuroPilot, ENN) with better op coverage, lower dispatch latency (2-4ms vs NNAPI's 8-12ms), and support for transformer ops that NNAPI 1.4 never standardized. Google itself now recommends using TFLite's vendor delegates directly rather than NNAPI, and NNAPI is in maintenance mode for compatibility.

NNAPI remains relevant only for two cases: legacy apps targeting Android 10-12 without bundling vendor SDKs, and privacy-sensitive system services where Google's Private Compute Core uses NNAPI for isolated inference. For new development in 2026, target TFLite + vendor delegates or Google's new LiteRT + Google AI Edge path rather than raw NNAPI.

Head-to-Head

Choosing Your Stack: Decision Framework

If you ship iOS-only or iOS-first: Commit to Core ML. Convert PyTorch/TensorFlow via coremltools early and validate with ANE performance reports. Use FP16 activations + INT4 weights for LLMs; rely on .mlpackage weighting for fast app thinning. The computeUnits API lets you pin sustained workloads to ANE to avoid GPU contention with rendering.

If you ship Android + iOS with one model: Use TFLite (LiteRT) as the single source of truth. Maintain a single .tflite artifact, test delegate coverage per tier (flagship vs mid-range), and ship with XNNPACK baseline. On iOS, enable the Core ML delegate — you get 80% of native Core ML performance without maintaining two pipelines.

If you target peak Android performance: Bypass NNAPI. Integrate QNN (Qualcomm), NeuroPilot (MediaTek) or Samsung ENN directly via TFLite delegates. Add a CI matrix that benchmarks on physical devices; emulator numbers are meaningless for NPU dispatch. Cache compiled delegate artifacts to avoid first-launch compilation jank.

Future Trajectory

Apple is doubling down on ANE — Core ML 7 adds stateful models and native KV-cache for generative AI, and rumors point to a unified memory API for diffusion models. Google's path is consolidation around LiteRT and the new GenAI Edge APIs that expose Gemini Nano via AICore on Pixel — essentially a curated TFLite runtime with vendor HALs hidden. NNAPI will remain as a compatibility shim, much like RenderScript.

For engineers, the implication is clear: invest in delegate-aware architecture today. Abstract your inference behind a thin interface — predict(tokens) -> tokens — so you can swap Core ML, TFLite/XNNPACK, or a future LiteRT compiled model without rewriting pipelines. The silicon will keep changing; your app abstraction shouldn't.