Effective in 2026, to align with our trunk stable development model and ensure platform stability for the ecosystem, we will publish source code to AOSP in Q2 and Q4. For building and contributing to AOSP, we recommend utilizing android-latest-release instead of aosp-main. The android-latest-release manifest branch will always reference the most recent release pushed to AOSP. For more information, see Changes to AOSP.

SDV Test Framework

This document provides an overview of the SDV test framework and guides you through setting up, configuring, and executing tests. Use the SDV test framework for end-to-end testing. The tool builds on Mobly; you write tests in Python.

Set up the environment

This document assumes you have already set up your workstation for SDV. To run Framework tests, install virtualenv.

Python dependencies

The recommended approach for managing dependencies in Android is to package tests, Python, and their dependencies as an executable using the build configuration. You must follow this approach in the SDV testing repository.

The SDV Test Framework library provides these dependencies by default:

mobly
absl-py Required to use parameterization in tests.
pexpect Required for interactive commands functionality.
ptyprocess Required for interactive commands functionality.

If you need test-specific dependencies, include them in the test's Android.bp file.

python_test_host {
    …
    libs: [
        "<DEPENDENCY1",
        "<DEPENDENCY2",
        …
    ],
    …
}

System test types

You create system tests using the SDV Test Framework. The framework distinguishes between the following types. Categorize the tests correctly:

Sample tests: Execute SDV sample services and verify that the result matches the expected output.
E2E tests: Do not use SDV sample services.
Long-running tests: Designed for extended execution periods.
Performance tests: Use the perfetto library.
Hardware tests: Designed to be executed exclusively on a physical device, not in a Cuttlefish (CF).

Create a test

All tests written using the framework must be located in the testing repository. The exact location of the test depends on its type.

Configure a test

Configure all tests to run using both atest and CATBox.

Configure atest

The framework provides templates for one, two, and three virtual machine (VM) tests. Add the following to your Android.bp file:

One VM

python_test_host {
  ...
  test_config_template: ":sdv_one_device",
}

Two VMs

python_test_host {
  ...
  test_config_template: ":sdv_two_devices",
}

Three VMs

python_test_host {
  ...
  test_config_template: ":sdv_three_devices",
}

The default template is sufficient for most tests. Avoid creating custom configurations unless there is a compelling reason.

Update the default timeout

The default timeout should be sufficient to execute the tests locally.

If your test fails locally due to a timeout, consider splitting it into different tests. Very long tests with many test cases can cause a bottleneck in execution.

If you must set a higher timeout than the default, continue reading.

Override the default timeout

The framework doesn't support extending the template for atest execution. Therefore, use the provided templates that offer an extended timeout.

One VM long test

python_test_host {
  ...
  test_config_template: ":sdv_one_device_long_test",
}

Two VMs long test

python_test_host {
  ...
  test_config_template: ":sdv_two_devices_long_test",
}

Add a comment that clearly indicates the purpose of using this template and explains why the extended time is necessary (for example, if the test requires downloading an artifact that takes a specific amount of time).

Push files to a device

The following steps describe how to push files to a device for atest configurations.

Push files to a device

The framework doesn't support extending the template for atest execution. Therefore, you must create an AndroidTest.xml file that fully replaces the default template and includes the corresponding target_preparer options for the devices.

Copy the contents of the template:
- sdv-one-device.xml
- sdv-two-devices.xml
- sdv-three-devices.xml
Update {MODULE} with the module name of your test.
Add com.android.tradefed.targetprep.PushFilePreparer to each device.

<device name="device1">
  <target_preparer class="com.android.tradefed.targetprep.PushFilePreparer">
    <option name="cleanup" value="true" />
    <option name="push-file" key="<FILE_NAME>" value="<REMOTE_FILE_PATH>" />
  </target_preparer>
</device>

Configure CATBox

To run the test as a suite, a CATBox configuration is required.

Create a new configuration file in libraries/sdv/sdv_catbox/res/config/ and follow these naming conventions:
- Sample test: sdv-sample-<NAME>-test.xml
- E2E test: sdv-e2e-<NAME>-test.xml
- Long-running test: sdv-long-running-<NAME>-test.xml
- Performance test: sdv-performance-<NAME>-test.xml
- Hardware test: sdv-hw-<NAME>-test.xml
Add the corresponding information based on the test type:

One VM test config

<configuration description="SDV <NAME> Test">
    <!-- Template for Device Configuration -->
    <template-include name="device-config" default="sdv-device-config/one-device" />

    <!-- Plan. Use the same name as the file name -->
    <option name="plan" value="<CONFIG_FILE_NAME>"/>

    <!-- Test: mobly-par-file-name should match the module name -->
    <option name="mobly-host:mobly-par-file-name" value="<ModuleName>" />
</configuration>

Two VMs test config

<configuration description="SDV <NAME> Test">
    <!-- Template for Device Configuration -->
    <template-include name="device-config" default="sdv-device-config/two-devices" />

    <!-- Plan. Use the same name as the file name -->
    <option name="plan" value="<CONFIG_FILE_NAME>"/>

    <!-- Test: mobly-par-file-name should match the module name -->
    <option name="mobly-host:mobly-par-file-name" value="<ModuleName>" />
</configuration>

Three VMs test config

<!-- Copyright 2025 Google LLC -->
<configuration description="SDV <NAME> Test">
    <!-- Template for Device Configuration -->
    <template-include name="device-config" default="sdv-device-config/three-devices" />

    <!-- Plan. Use the same name as the file name -->
    <option name="plan" value="<CONFIG_FILE_NAME>"/>

    <!-- Test: mobly-par-file-name should match the module name -->
    <option name="mobly-host:mobly-par-file-name" value="<ModuleName>" />
</configuration>

The default template is sufficient for most tests. Avoid creating custom configurations unless there is a compelling reason.

Update the default timeout

If the default timeout is not sufficient for your test, you can override it.

Override the default timeout

To override the default timeout, add the following to the end of your configuration file. Long tests should have a maximum timeout of 10 minutes.

  <!-- Long Test - 10 minutes timeout: <REASON> -->
  <option name="mobly-host:mobly-test-timeout" value="600000" />

Push files to a device

The following section describes how to push files to a device when configuring CATBox.

Push files to a device

To push files to the corresponding device, add the following configuration after the plan option:

<!-- Automatically pushes config file to device-->
<device name="device1">
  <target_preparer class="com.android.tradefed.targetprep.PushFilePreparer">
    <option name="cleanup" value="true" />
    <option name="push-file"
      key="<FILE_NAME>"
      value="<REMOTE_FILE_PATH>"
    />
  </target_preparer>
</device>

If you require the files on each device, add the configuration for each device (for example, name=device1, name=device2).

Test execution

These instructions assume you have completed setting up a working SDV environment.

Set up the environment

The framework does not start Cloud Virtual Device (CVD) instances, so you must start them before running the tests.

Set up the environment and launch the SDV target:
```
. build/envsetup.sh
lunch <TARGET>
```
Start the first Cloud Virtual Device (CVD) instance:
```
cvd create --config=sdv_core_instance1
```
Start the second Cloud Virtual Device (CVD) instance (for multi-VM tests):
```
cvd create --config=sdv_core_instance2
```
Start the third Cloud Virtual Device (CVD) instance (for multi-VM tests):
```
cvd create --config=sdv_core_instance3
```

Run atest

atest <TestModuleName>

Run CATBox

These commands assume you are in the root directory of the repository.

Build CATBox: m catbox
(Optional) View all available SDV test plans in CATBox:

out/host/linux-x86/catbox/android-catbox/tools/catbox-tradefed l p | grep sdv-

Run a test:

Run one VM test

NOTIFY_AS_NATIVE=0.0.0.0:6520 out/host/linux-x86/catbox/android-catbox/tools/catbox-tradefed run commandAndExit TEST_PLAN_NAME --{device1}serial 0.0.0.0:6520

Run two VMs test

NOTIFY_AS_NATIVE=0.0.0.0:6520,0.0.0.0:6521 out/host/linux-x86/catbox/android-catbox/tools/catbox-tradefed run commandAndExit TEST_PLAN_NAME --{device1}serial 0.0.0.0:6520 --{device2}serial 0.0.0.0:6521

Run three VMs test

NOTIFY_AS_NATIVE=0.0.0.0:6520,0.0.0.0:6521,0.0.0.0:6522 out/host/linux-x86/catbox/android-catbox/tools/catbox-tradefed run commandAndExit TEST_PLAN_NAME --{device1}serial 0.0.0.0:6520 --{device2}serial 0.0.0.0:6521 --{device3}serial 0.0.0.0:6522

SDV Test Framework Stay organized with collections Save and categorize content based on your preferences.

Set up the environment

Python dependencies

System test types

Create a test

Configure a test

Configure atest

One VM

Two VMs

Three VMs

Update the default timeout

Override the default timeout

One VM long test

Two VMs long test

Push files to a device

Push files to a device

Configure CATBox

One VM test config

Two VMs test config

Three VMs test config

Update the default timeout

Override the default timeout

Push files to a device

Push files to a device

Test execution

Set up the environment

Run atest

Run CATBox

Run one VM test

Run two VMs test

Run three VMs test

SDV Test Framework