ITestInvocationListener

public interface ITestInvocationListener
implements ITestLifeCycleReceiver, ITestLogger

com.android.tradefed.result.ITestInvocationListener
AggregatePostProcessor A metric aggregator that gives the min, max, mean, variance, standard deviation, total, count and optionally percentiles for numeric metrics collected during multiple-iteration test runs, treating them as doubles. 
AtraceCollector A IMetricCollector that runs atrace during a test and collects the result and log them to the invocation. 
AtraceRunMetricCollector Collects the traces from all the test directory under the given directory from the test device, log the test directory and post process the trace files under the test directory and aggregate the metrics. 
AveragePostProcessor Implementation of post processor that calculate the average of the list of metrics. 
BaseDeviceMetricCollector Base implementation of IMetricCollector that allows to start and stop collection on onTestRunStart(com.android.tradefed.device.metric.DeviceMetricData) and ERROR(/#onTestRunEnd(com.android.tradefed.device.metric.DeviceMetricData,Map)). 
BasePostProcessor The base IPostProcessor that every implementation should extend. 
BluetoothConnectionLatencyCollector The collector will push a pre-defined statsd duration metric config to devices and collect Bluetooth connection duration for each profile. 
BluetoothConnectionStateCollector This collector will collect BluetoothConnectionStateChanged metrics and record connection state number for each profile. 
BluetoothConnectionSuccessRatePostProcessor Implementation of postprocessor which calculates success rate for a bluetooth profile

Use "metric-key-match" to specify metrics which contain bluetooth connection states in numeric array Example [0, 1, 2, 3, 0, 1, 2, 3]. 

BluetoothHciSnoopLogCollector Collector to enable Bluetooth HCI snoop logging on the DUT and to collect the log for each test. 
BugreportCollector A pass-through ITestInvocationListener that collects bugreports when configurable events occur and then calls ITestInvocationListener.testLog on its children after each bugreport is collected. 
BugreportzOnFailureCollector Collect a bugreportz when a test case fails. 
BugreportzOnTestCaseFailureCollector Collect a bugreportz when a test case in a run fails. 
ClangCodeCoverageCollector A BaseDeviceMetricCollector that will pull Clang coverage measurements off of the device and log them as test artifacts. 
CodeCoverageCollector A BaseDeviceMetricCollector that will pull Java and native coverage measurements off of the device and log them as test artifacts. 
CollectingTestListener A ITestInvocationListener that will collect all test results. 
CommandStatusHandler Handler helping to monitor and update the status of an invocation. 
ConditionFailureMonitor Monitor for any failures from the invocation. 
ConsoleResultReporter Result reporter to print the test results to the console. 
CountTestCasesCollector Count and report the number of test cases for a given IRemoteTest. 
DebugHostLogOnFailureCollector Collector that will gather and log the host-side logs when a test case failure occurs. 
DeviceReleaseReporter Device release reporter that handles communicating with the parent process the device(s) to be released. 
DeviceTraceCollector Collector that will start perfetto trace when a test run starts and log trace file at the end. 
DeviceUnavailableMonitor This listener attempts to capture a test case level DNAE only. 
EventsLoggerListener Listener that logs all the events it receives into a file 
FileProtoResultReporter Proto reporter that dumps the TestRecord into a file. 
FilePullerDeviceMetricCollector A BaseDeviceMetricCollector that listen for metrics key coming from the device and pull them as a file from the device. 
FilePullerLogCollector Logger of the file reported by the device-side. 
FilteredResultForwarder Variant of ResultForwarder that only allows an allowlist of TestDescription to be reported. 
FormattedGeneratorReporter Reporter that allows to generate reports in a particular format. 
GcovCodeCoverageCollector A BaseDeviceMetricCollector that will pull gcov coverage measurements off of the device and log them as test artifacts. 
GcovKernelCodeCoverageCollector A BaseDeviceMetricCollector that will pull gcov kernel coverage measurements out of debugfs and off of the device and then finally logs them as test artifacts. 
GranularRetriableTestWrapper.StartEndCollector Class helper to catch missing run start and end. 
HostStatsdMetricCollector A IMetricCollector that collects statsd metrics from host side using statsd utility commands. 
ICommandScheduler.IScheduledInvocationListener Listener for invocation events when invocation completes. 
ILogSaverListener Allows for ITestInvocationListeners to listen for when log files are saved. 
IMetricCollector This interface will be added as a decorator when reporting tests results in order to collect matching metrics. 
IPostProcessor Post processors is a Trade Federation object meant to allow the processing of metrics and logs AFTER the tests and BEFORE result reporting. 
IShardableListener A ITestInvocationListener that can be sharded. 
ISupportGranularResults Interface specifying whether a ITestInvocationListener supports receiving the granular results or not. 
ITestSummaryListener Interface that allows ITestInvocationListener to exchange some limited information via summaries. 
InvocationProtoResultReporter Reporters to store invocation level caching information and its list of modules 
InvocationToJUnitResultForwarder A class that listens to ITestInvocationListener events and forwards them to a TestListener. 
JarHostTest.HostTestListener Wrapper listener that forwards all events except testRunStarted() and testRunEnded() to the embedded listener. 
JavaCodeCoverageCollector A BaseDeviceMetricCollector that will pull Java coverage measurements off of the device and log them as test artifacts. 
JsonHttpTestResultReporter A result reporter that encode test metrics results and branch, device info into JSON and POST into an HTTP service endpoint 
LUCIResultReporter A result reporter that saves test results needed by ResultDB and LUCI into JSON format (go/result-sink) and logs the file location in the console. 
LastShardDetector When running local sharding, sometimes we only want to execute some actions when the last shard reaches invocationEnded(long). 
LegacySubprocessResultsReporter A frozen implementation of the subprocess results reporter which should remain compatible with earlier versions of TF/CTS (e.g. 8+), despite changes in its superclass. 
LogSaverResultForwarder A ResultForwarder for saving logs with the global file saver. 
LogcatCrashResultForwarder Special listener: on failures (instrumentation process crashing) it will attempt to extract from the logcat the crash and adds it to the failure message associated with the test. 
LogcatOnFailureCollector Collector that will capture and log a logcat when a test case fails. 
LogcatTimingMetricCollector A metric collector that collects timing information (e.g. user switch time) from logcat during one or multiple repeated tests by using given regex patterns to parse start and end signals of an event from logcat lines. 
MetricFilePostProcessor Used for uploading the metrics log file collected during the test and run level. 
MetricsXMLResultReporter MetricsXMLResultReporter writes test metrics and run metrics to an XML file in a folder specified by metrics-folder parameter at the invocationEnded phase of the test. 
ModuleListener Listener attached to each IRemoteTest of each module in order to collect the list of results. 
ModuleLogcatCollector Version of logcat collector but for module. 
ModuleProtoResultReporter A result reporter meant to report only the module level results. 
NameMangleListener A proxy listener to translate test method, class, and package names as results are reported. 
PassingTestFileReporter A ITestInvocationListener that saves the list of passing test cases to a test file 
PerfettoGenericPostProcessor A post processor that processes text/binary metric perfetto proto file into key-value pairs by recursively expanding the proto messages and fields with string values until the field with numeric value is encountered. 
PerfettoPullerMetricCollector Base implementation of FilePullerDeviceMetricCollector that allows pulling the perfetto files from the device and collect the metrics from it. 
PrettyTestEventLogger Logger matching the events and logging them in order to make it easier to debug. 
ProtoResultReporter Result reporter build a TestRecord protobuf with all the results inside. 
PythonBinaryHostTest.PythonForwarder Result forwarder to replace the run name by the binary name. 
RebootReasonCollector Collector that collects device reboot during the test run and report them by reason and counts. 
RemoteTestTimeOutEnforcer Listeners that allows to check the execution time of a given test config and fail it if it goes over a given timeout. 
ReportPassedTests Report in a file possible filters to exclude passed test. 
ResultAggregator Special forwarder that aggregates the results when needed, based on the retry strategy that was taken. 
ResultAndLogForwarder Forwarder for results and logs events. 
ResultDBReporter Result reporter that uploads test results to ResultDB. 
ResultForwarder A ITestInvocationListener that forwards invocation results to a list of other listeners. 
RetryLogSaverResultForwarder Forwarder that also handles passing the current attempt we are at. 
RetryResultForwarder An Extension of ResultForwarder that always push to a given attempt number. 
RuntimeRestartCollector Collector that collects timestamps of runtime restarts (system server crashes) during the test run, if any. 
ScreenshotOnFailureCollector Collector that will capture and log a screenshot when a test case fails. 
ShardListener A ITestInvocationListener that collects results from a invocation shard (aka an invocation split to run on multiple resources in parallel), and forwards them to another listener. 
ShardMainResultForwarder A ResultForwarder that combines the results of a sharded test invocations. 
ShowmapPullerMetricCollector Base implementation of FilePullerDeviceMetricCollector that allows pulling the showmap files from the device and collect the metrics from it. 
StatsdBeforeAfterGaugeMetricPostProcessor A post processor that processes gauge metrics collected in a "before/after" approach, i.e. 
StatsdEventMetricPostProcessor A post processor that processes event metrics in statsd reports into key-value pairs, using the formatters specified on the processor. 
StatsdGenericPostProcessor A post processor that processes binary proto statsd reports into key-value pairs by expanding the report as a tree structure. 
StreamProtoResultReporter An implementation of ProtoResultReporter 
SubprocessResultsReporter Implements ITestInvocationListener to be specified as a result_reporter and forward from the subprocess the results of tests, test runs, test invocations. 
SuiteResultReporter Collect test results for an entire suite invocation and output the final results. 
TestRunResultListener Listener that allows to read the final test run status. 
TestTimeoutEnforcer Listeners that allows to check the execution time of a given test case and fail it if it goes over a given timeout. 
TextResultReporter A test result reporter that forwards results to the JUnit text result printer. 
TraceCmdCollector A IMetricCollector that collects traces during the test using trace-cmd, and logs them to the invocation. 
TradefedSandboxRunner.StubScheduledInvocationListener A stub IScheduledInvocationListener that does nothing. 
XmlFormattedGeneratorReporter Implementation of the FormattedGeneratorReporter which format the suite results in an xml format. 
XmlResultReporter Writes JUnit results to an XML files in a format consistent with Ant's XMLJUnitResultFormatter. 

Listener for test results from the test invocation.

A test invocation can itself include multiple test runs, so the sequence of calls will be

  • invocationStarted(BuildInfo)
  • testRunStarted
  • testStarted
  • [testFailed]
  • testEnded
  • ...
  • testRunEnded
  • ...
  • testRunStarted
  • ...
  • testRunEnded
  • [invocationFailed]
  • [testLog+]
  • invocationEnded
  • getSummary

Summary

Public methods

default TestSummary getSummary()

Allows the InvocationListener to return a summary.

default void invocationEnded(long elapsedTime)

Reports that the invocation has terminated, whether successfully or due to some error condition.

default void invocationFailed(FailureDescription failure)

Reports an incomplete invocation due to some error condition.

default void invocationFailed(Throwable cause)

Reports an incomplete invocation due to some error condition.

default void invocationInterrupted()

Called on scheduler shutdown, gives the invocation the opportunity to do something before terminating.

default void invocationSkipped(SkipReason reason)

Reports an invocation as skipped

default void invocationStarted(IInvocationContext context)

Reports the start of the test invocation.

default void testModuleEnded()

Reports the end of a module run.

default void testModuleStarted(IInvocationContext moduleContext)

Reports the beginning of a module running.

Public methods

getSummary

public TestSummary getSummary ()

Allows the InvocationListener to return a summary.

Returns
TestSummary A TestSummary summarizing the run, or null

invocationEnded

public void invocationEnded (long elapsedTime)

Reports that the invocation has terminated, whether successfully or due to some error condition.

Will be automatically called by the TradeFederation framework.

Parameters
elapsedTime long: the elapsed time of the invocation in ms

invocationFailed

public void invocationFailed (FailureDescription failure)

Reports an incomplete invocation due to some error condition.

Will be automatically called by the TradeFederation framework.

Parameters
failure FailureDescription: the FailureDescription describing the cause of the failure

invocationFailed

public void invocationFailed (Throwable cause)

Reports an incomplete invocation due to some error condition.

Will be automatically called by the TradeFederation framework.

Parameters
cause Throwable: the Throwable cause of the failure

invocationInterrupted

public void invocationInterrupted ()

Called on scheduler shutdown, gives the invocation the opportunity to do something before terminating.

invocationSkipped

public void invocationSkipped (SkipReason reason)

Reports an invocation as skipped

Parameters
reason SkipReason

invocationStarted

public void invocationStarted (IInvocationContext context)

Reports the start of the test invocation.

Will be automatically called by the TradeFederation framework. Reporters need to override this method to support multiple devices reporting.

Parameters
context IInvocationContext: information about the invocation

testModuleEnded

public void testModuleEnded ()

Reports the end of a module run.

testModuleStarted

public void testModuleStarted (IInvocationContext moduleContext)

Reports the beginning of a module running. This callback is associated with testModuleEnded() and is optional in the sequence. It is only used during a run that uses modules: suite based runners.

Parameters
moduleContext IInvocationContext: the IInvocationContext of the module.