Android 17 Camera Image Test Suite release notes

The Android 17 release includes a number of changes to Camera ITS. This page summarizes the Camera ITS changes for Android 17.

Python and package versions

For Android 17 and higher, we strongly recommend using package management software to bundle the correct versions of packages to create a virtual environment for Android release development. For an example package management tool that handles most of the packages required, see the Python 3 documentation for venv.

To create the virtual environment with venv, follow these steps:

  1. Install Python 3.12 and python3-venv

    sudo apt install python3.12
    sudo apt install python3.12-venv python3.12-dev
    
  2. Verify python installation

    python3.12 --version
    
  3. Update apt and install pip

    sudo apt-get update
    sudo apt install python3-pip
    
  4. Create a virtual environment

    python3.12 -m venv its_android17_env
    
  5. Activate the virtual environment

    source its_android17_env/bin/activate
    
  6. Create its_android17_env.txt with the list of required packages

    contourpy==1.3.1
    cycler==0.12.1
    fonttools==4.56.0
    imagecodecs==2026.3.6
    ImageIO==2.37.3
    kiwisolver==1.4.8
    lazy-loader==0.5
    matplotlib==3.10.0
    mobly==1.12.3
    mobly-android-partner-tools==1.5.2
    networkx==3.6.1
    numpy==2.2.2
    opencv-python==4.11.0.86
    packaging==24.2
    pillow==11.1.0
    portpicker==1.6.0
    protobuf==6.33.6
    psutil==6.1.1
    pyparsing==3.2.1
    pyserial==3.5
    python-dateutil==2.9.0.post0
    pyudev==0.24.4
    PyYAML==6.0.2
    scikit-image==0.26.0
    scipy==1.15.1
    six==1.17.0
    snippet-uiautomator==1.1.2
    tifffile==2026.3.3
    
  7. Install packages listed its_android17_env.txt

    pip install -r its_android17_env.txt
    
  8. Install FFmpeg version 7.0.2 with ffmpeg downloader

    pip install ffmpeg-downloader
    ffdl install 7.0.2
    

    (optional) If ffmpeg downloader doesn't work, download 7.0.2 from an online source.

  9. Verify FFmpeg installation of version 7.0.2

    ffmpeg -version
    

    If the FFmpeg version still isn't 7.0.2, symlink the binaries (adjust PATH based on where you save its_android17_env).

    ln -s ~/.local/share/ffmpeg-downloader/ffmpeg/ffmpeg ~/PATH/its_android17_env/bin/ffmpeg
    ln -s ~/.local/share/ffmpeg-downloader/ffmpeg/ffprobe ~/PATH/its_android17_env/bin/ffprobe
    
  10. Verify which packages are installed

    pip freeze
    
  11. (Optional) Command to deactivate the environment:

    deactivate
    

New or updated scenes

Android 17 introduces the following scenes:

Scene Description
gen2_chart New scene that uses the test_chart_gen2 paper chart instead of tablet scenes to improve test consistency and reliability . The test_chart_gen2 is also used in scene_ip.
wide_gamut New scene for tablets with wide gamut support.
3 Uses ArUco markers for chart detection to improve speed and reliability. This change accommodates the wider range of FoV and distances seen in tele camera testing. Using markers bypasses the current sizing logic and optimizer issues. For more information, see scene3.
flash Tests can be run using the Gen2 rig.

New tests

Android 17 introduces the following tests:

Scene Test name Description
gen2_chart test_tonemap_sequence Verifies the android.tonemap.mode parameter is applied correctly for uniform and non-uniform distribution.
ip test_jca_jpegr_ip Verifies that the white balance difference between JPEG_R JCA preview snapshot and captured images is within the chosen threshold.
wide_gamut test_display_p3 Verifies P3 JPEG output has an appropriate icc profile and contains more than 1 percent of colors outside of the sRGB gamut.

Refactored tests

In Android 17, the following tests are refactored to increase test coverage and group tests more logically:

Scene Test name Description
1_2 tests_param_tonemap_mode The scene1_2 version of the tests_param_tonemap_mode test is deprecated and is used only on devices running Android 16 or lower.
1_2 test_tonemap_sequence The scene1_2 version of the test_tonemap_sequence test is used only on devices running Android 16 or lower.
2_b test_yuv_jpeg_capture_sameness The RMS difference threshold is decreased to help ensure that hue-related color discrepancies are flagged as failures whenever they result in a visible difference for the user.
3 test_edge_enhancement Uses ArUco markers to validate both presence and orientation. This method provides a stable detection for cameras from Ultra-Wide (UW) to tele applications.
3 test_flip_mirror Uses ArUco markers to validate both presence and orientation. A PASS status is only granted if all markers are detected and correctly aligned. The test fails for displaced or rotated charts.
3 test_landscape_to_portrait Loads updated scene3 chart with ArUco markers.
3 test_lens_movement_report Uses ArUco markers to validate both presence and orientation. This method provides a stable detection for cameras from Ultra-Wide (UW) to tele applications.
4 test_multi_camera_alignment Skips the test if the devica has only one wide and one super TELE camera.
flash test_auto_flash Updates the test to verify that the flash works properly at various zoom levels.
ip test_default_jca_ip Mandates the existing checks: brightness difference, white balance difference and FoV mismatch. Adds a not_yet_mandated check for color rendering difference between default camera app and JCA captures.

Separated test activities

In Android 17, for parallel testing on separate devices, the camera ITS tests are separated into two CTS Verifier activities, Camera ITS Test and Camera ITS Sensor Fusion Rig Test. The Camera ITS Sensor Fusion Rig Test activity contains the feature_combination and sensor_fusion scenes, and can be run separately in a sensor fusion test rig. The following table describes the scenes included in each activity.

Activity Scenes Description
Camera ITS Test gen2_chart, scene0, scene1_1, scene1_2, scene1_3, scene2_a, scene2_b, scene2_c, scene2_d, scene2_e, scene2_f, scene2_g, scene3, scene4, scene5, scene6, scene8, scene9, scene_flash, scene_hdr, scene_ip, scene_low_light, scene_video, wide_gamut Scenes that run in the camera ITS-in-a-box test rig and Gen2 test rig.
Camera ITS Sensor Fusion Rig Test feature_combination, sensor_fusion Scenes that run in the sensor fusion test rig.

Deprecated tests

In Android 17, the following tests are deprecated due to test function being covered by other tests:

Scene Test name Description
sensor_fusion test_video_stabilization This test is deprecated. Use test_video_stabilization_jca instead.

Gen2 rig updates

Android 17 the following tests have been migrated to the Gen2 rig:

Scene Test name Description
7 test_multi_camera_switch Migrated the test to gen2 rig setup to use the Gen2 chart for AE and AWB checks. This transition replaces tablet-based targets with physical charts to eliminate Moiré effects and provide a more stable, reproducible testing environment.
flash all tests Migrated scene_flash tests in gen2 rig. Users can execute these tests in either a sensor fusion rig or Gen2 rig environment.
sensor_fusion all tests Migrated sensor_fusion tests to gen2 rig. Users can execute these tests in either a sensor fusion rig or Gen2 rig environment.

Before running the migrated tests, install the Gen2 rig chart introduced in Android 17. The new chart lets you run all sensor fusion tests in the Gen2 rig. For more information, see Set up the chart for testing.

New test status: PASS*

17 introduces the test status PASS* to detect marginally passing tests. For more information, see Marginally passing tests (PASS* test status).

Chart scaling improvements

Android 17 adds the chart_scaling parameter in config.yml for TEST_BED_TABLET_SCENES to handle the chart scaling issues with wider field of view (FoV) tele camera devices. For more information, see Chart scaling improvements: chart_scaling param.

New tablet for wide-gamut testing support

For Android 17 and higher, the Samsung Galaxy Tab S10 FE (both Wi-Fi and 5G models) is on the tablet allowlist. The Samsung Galaxy Tab S10 FE is approved for displaying wide-gamut test charts, as well as all other tablet-based scenes. To verify that your device is on the allowlist, use the command adb shell getprop ro.product.device to confirm the product ID matches the value listed in the following list.

    'gts10fewifi',  # Samsung Galaxy Tab S10 FE
    'gts10fe',  # Samsung Galaxy Tab S10 FE 5G

Aggregated result submission for build approvals

Android 17 introduces a process for submitting aggregated Camera ITS test results for build approval using the updated CTS Verifier APK.

The new aggregation system provides the following improvements:

  • Time savings: Eliminates the need to repeatedly rerun passing scenes on devices that share the same build fingerprint.
  • Parallel execution: Allows concurrent testing of multiple scenes across several devices, provided they share the same build fingerprint.
  • Simplified reporting: Enables aggregation of test results from multiple CTS Verifier reports (collected from different test runs or devices) into a single, unified submission.

For more information, see Submit aggregate test results.