NPU Manager

Android 17 and higher supports the Neural Processing Unit (NPU) Manager (com.android.npumanager), a system component in the Android Open Source Project (AOSP) that coordinates NPU hardware resources across system services and app workloads. By moving resource arbitration from custom vendor daemons to the Android platform, the NPU Manager increases predictability, prevents resource starvation, manages memory and power boundaries, and improves overall device performance.

Background and motivation

Before the NPU Manager, apps and system services submitted machine learning (ML) workloads directly to vendor drivers or proprietary runtimes. That direct-access model introduced several systemic problems:

  • Inefficient resource competition: Heavy ML workloads, such as large language model (LLM) inference engines and on-device vision pipelines, competed directly for finite NPU static RAM (SRAM), model weight memory, and hardware execution queues.
  • System instability: Uncoordinated model allocations pinned gigabytes of physical DRAM, triggering thermal throttling, page faults, or background app terminations by the low memory killer daemon (lmkd) when demands exceeded hardware capacity.
  • Missing context awareness: The system server couldn't adjust NPU execution priority or reclaim idle model weights during user context shifts, such as when a background task held a multi-gigabyte model while a latency-sensitive camera pipeline or assistant entered the foreground.

The NPU Manager addresses these challenges by acting as a central system arbiter that coordinates scheduling priority, model buffer lifecycles, hardware execution sessions, model compilation, and power accounting based on real-time device health and app states.

System architecture

The NPU Manager runs as a system service named npu inside system_server, packaged in the com.android.npumanager system APEX. The service separates platform policy and app lifecycle tracking from low-level system-on-chip (SoC) driver implementations.

The following diagram illustrates the NPU Manager environment layers:

NPU Manager environment layers

Figure 2. NPU Manager environment layers.

Subsystem pillars

The NPU Manager documentation is organized into the following functional areas:

Subsystem Primary interfaces Responsibilities
Workload scheduling android.npumanager.NpuManager,
android.hardware.npu.IScheduling
Gates model load admission and relays app importance and effective job priorities to the NPU driver.

Device readiness and diagnostics

Devices that integrate an NPU and support NPU Manager declare the android.hardware.npu hardware feature (PackageManager.FEATURE_NEURAL_PROCESSING_UNIT). Apps can declare optional or required NPU usage in AndroidManifest.xml as shown in the following example:

<uses-feature android:name="android.hardware.npu" android:required="false" />

Inspect service state

During bringup and validation, you can inspect active NPU devices, client sessions, priority tables, and registered buffers from adb shell using the following command:

adb shell dumpsys npu

For compatibility test configuration on Android 17 and higher, see Run NPU Manager Mobly tests and Workload scheduling.