Reduce OTA size

This page describes changes added to AOSP to reduce unnecessary file changes between builds. Device implementers who maintain their own build systems can use this information as a guide for reducing the size of their over-the-air (OTA) updates.

Android OTA updates occasionally contain changed files that don't correspond to code changes. They're actually build system artifacts. This can occur when the same code, built at different times, from different directories, or on different machines produces a large number of changed files. Such excess files increase the size of an OTA patch, and make it difficult to determine which code changed.

To make the contents of an OTA more transparent, AOSP includes build system changes designed to reduce the size of OTA patches. Unnecessary file changes between builds have been eliminated, and only patch-related files are contained in OTA updates. AOSP also includes a build diff tool, which filters out common build-related file changes to provide a cleaner build file diff, and a block mapping tool, which helps you keep block allocation consistent.

A build system can create unnecessarily large patches in several ways. To mitigate this, in Android 8.0 and higher, new features were implemented to reduce the patch size for each file diff. Improvements that reduced OTA-update package sizes include the following:

  • Usage of ZSTD, a generic-purpose, lossless-compression algorithm for full images on non-A/B device updates. ZSTD can be customized for higher compression ratios by increasing compression level. Compression level is set during OTA generation time and can be set by passing the flag --vabc_compression_param=zstd,$COMPRESSION_LEVEL
  • Increasing the compression window size used during OTA. The maximum compression window size can be set by customizing the build parameter in a device's .mk file. This variable is set as PRODUCT_VIRTUAL_AB_COMPRESSION_FACTOR := 262144
  • Usage of Puffin recompression, a deterministic patching tool for deflate streams, that handles the compression and diff functions for A/B OTA update generation.
  • Changes to the delta-generation tool usage, such as how the bsdiff library is used for compressing patches. In Android 9 and higher, the bsdiff tool selects the compression algorithm that would give the best compression results for a patch.
  • Improvements to the update_engine resulted in less memory consumed when patches are applied for A/B device updates.

The following sections discuss various issues that affect OTA-update sizes, their solutions, and examples of implementation in AOSP.

File order

Problem: Filesystems don't guarantee a file order when asked for a list of files in a directory, though it's commonly the same for the same checkout. Tools such as ls sort the results by default, but the wildcard function used by commands such as find and make don't sort. Before using these tools, you must sort the outputs.

Solution: When you use tools such as find and make with the wildcard function, sort the output of these commands before using them. When using $(wildcard) or $(shell find) in Android.mk files, sort them as well. Some tools, such as Java, do sort inputs, so before you sort the files, verify that the tool you're using hasn't already done so.

Examples: Many instances were fixed in the core build system using the builtin all-*-files-under macro, which includes all-cpp-files-under (as several definitions were spread out in other makefiles). For details, refer to the following:

Build directory

Problem: Changing the directory in which things are built can cause the binaries to be different. Most paths in the Android build are relative paths so __FILE__ in C/C++ isn't a problem. However, the debug symbols encode the full pathname by default, and the .note.gnu.build-id is generated from hashing the pre-stripped binary, so it will change if the debug symbols change.

Solution: AOSP now makes debug paths relative. For details, refer to CL: https://android.googlesource.com/platform/build/+/6a66a887baadc9eb3d0d60e26f748b8453e27a02.

Timestamps

Problem: Timestamps in the build output result in unnecessary file changes. This is likely to happen in the following locations:

  • __DATE__/__TIME__/__TIMESTAMP__ macros in C or C++ code.
  • Timestamps embedded in zip-based archives.

Solutions/Examples: To remove timestamps from the build output, use the instructions given below in __DATE__/__TIME__/__TIMESTAMP__ in C/C++. and Embedded timestamps in archives.

__DATE__/__TIME__/__TIMESTAMP__ in C/C++

These macros always produce different outputs for different builds, so don't use them. Here are a few options for eliminating these macros:

Embedded timestamps in archives (zip, jar)

Android 7.0 fixed the problem of embedded timestamps in zip archives by adding -X to all uses of the zip command. This removed the UID/GID of the builder and the extended Unix timestamp from the zip file.

A new tool, ziptime (located in /platform/build/+/main/tools/ziptime/) resets the normal timestamps in the zip headers. For details, refer to the README file.

The signapk tool sets timestamps for the APK files that may vary depending on the server timezone. For details, refer to the CL https://android.googlesource.com/platform/build/+/6c41036bcf35fe39162b50d27533f0f3bfab3028.

The signapk tool sets timestamps for the APK files that may vary depending on the server timezone. For details, refer to the CL https://android.googlesource.com/platform/build/+/6c41036bcf35fe39162b50d27533f0f3bfab3028.

Version strings

Problem: APK version strings often had the BUILD_NUMBER appended to their hardcoded versions. Even if nothing else changed in an APK, as a result, the APK would still be different.

Solution: Remove the build number from the APK version string.

Examples:

Enable on-device verity computation

If dm-verity is enabled on your device, then OTA tools automatically pick up your verity configuration, and enable on-device verity computation. This allows verity blocks to be computed on android devices, instead of being stored as raw bytes in your OTA package. Verity blocks can use approximately 16MB for a 2GB partition.

However, computing verity on-device can take a long time. Specifically, the Forward Error-correction code can take a long time. On pixel devices, it tends to take up to 10 minutes. On low-end devices it could take longer. If you want to disable on-device verity computation, but still enable dm-verity, you can do so by passing --disable_fec_computation to the ota_from_target_files tool when generating an OTA update. This flag disables on-device verity computation during OTA updates. It decreases OTA installation time, but increases OTA package size. If your device doesn't have dm-verity enabled, passing this flag has no effect.

Consistent build tools

Problem: Tools that generate installed files must be consistent (a given input should always produce the same output).

Solutions/Examples: Changes were required in the following build tools:

Use the build diff tool

For cases where it isn't possible to eliminate build-related file changes, AOSP includes a build diff tool, target_files_diff.py for use in comparing two file packages. This tool performs a recursive diff between two builds, excluding common build-related file changes, such as

  • Expected changes in the build output (for example, due to a build number change).
  • Changes due to known issues in the current build system.

To use the build diff tool, run the following command:

target_files_diff.py dir1 dir2

dir1 and dir2 are base directories that contain the extracted target files for each build.

Keep block allocation consistent

For a given file, although its contents remain the same between two builds, the actual blocks that hold the data might have changed. As a result, the updater must perform unnecessary I/O to move the blocks around for an OTA update.

In a Virtual A/B OTA update, unnecessary I/O can greatly increase the storage space required to store the copy-on-write snapshot. In a non-A/B OTA update, moving the blocks around for an OTA update contributes to update time as there is more I/O due to block moves.

To address this issue, in Android 7.0 Google extended the make_ext4fs tool for keeping block allocation consistent across builds. The make_ext4fs tool accepts an optional -d base_fs flag that attempts to allocate files to the same blocks when generating an ext4 image. You can extract the block mapping files (such as the base_fs map files) from a previous build's target files' zip file. For each ext4 partition, there is a .map file in the IMAGES directory (for example, IMAGES/system.map corresponds to the system partition). These base_fs files can then be checked-in and specified via PRODUCT_<partition>_BASE_FS_PATH, as in this example: 

  PRODUCT_SYSTEM_BASE_FS_PATH := path/to/base_fs_files/base_system.map
  PRODUCT_SYSTEM_EXT_BASE_FS_PATH := path/to/base_fs_files/base_system_ext.map
  PRODUCT_VENDOR_BASE_FS_PATH := path/to/base_fs_files/base_vendor.map
  PRODUCT_PRODUCT_BASE_FS_PATH := path/to/base_fs_files/base_product.map
  PRODUCT_ODM_BASE_FS_PATH := path/to/base_fs_files/base_odm.map

While this doesn't help reduce the overall OTA package size, it does improve OTA update performance by reducing the amount of I/O. For Virtual A/B updates, it drastically reduces the amount of storage space needed to apply the OTA.

Avoid updating apps

In addition to minimizing build diffs, you can reduce OTA update sizes by excluding updates for apps that get updates through app stores. APKs often comprise a significant part of various partitions on a device. Including the latest versions of apps that get updated by app stores in an OTA update may have a large size impact on OTA packages, and provide little user benefit. By the time users receive an OTA package, they may already have the updated app, or an even newer version, received directly from app stores.