Test filtering

Marathon Cloud filters your test suite using a YAML file passed with --filter-file. Use it to run subsets such as smoke tests, a single feature’s tests, or everything except a quarantined package. The file uses the same filteringConfiguration schema as the OSS Marathon runner, so existing filter files work unchanged.

Quick start

marathon-cloud run android \
  --application app-debug.apk \
  --test-application app-debug-androidTest.apk \
  --filter-file filters.yaml

filters.yaml

filteringConfiguration:
  allowlist:
    - type: "annotation"
      values: ["com.example.annotations.SmokeTest"]
  blocklist:
    - type: "package"
      values: ["com.example.legacy"]

File structure

The top-level key is filteringConfiguration. It contains an optional allowlist and an optional blocklist; at least one must be present.

• allowlist: a test runs only if it matches at least one allowlist filter.
• blocklist: a test is skipped if it matches any blocklist filter.

Each filter has a type and exactly one selector: values, regex, or file. The exception is composition, which takes op and a nested filters list instead of a selector.

Selector	Description
values	List of exact values to match.
regex	A single regular expression.
file	Path to a text file with one value per line, relative to the filter file. Lines starting with # are ignored.

Filter types

Type	Matches against
fully-qualified-class-name	com.example.LoginTest
simple-class-name	LoginTest
fully-qualified-test-name	com.example.LoginTest#testLogin
simple-test-name	LoginTest#testLogin
package	com.example
method	testLogin
annotation	Fully qualified annotation name, e.g. com.example.annotations.SmokeTest
composition	Combines nested filters with an op (UNION, INTERSECTION, SUBTRACT)

The allure, fragmentation, and annotationData filter types from OSS Marathon are not supported and cause a validation error.

Maestro flows

For Maestro runs the directory path becomes the package (flows/auth/login.yaml → auth), the class is always MaestroTest, the method is the file name without extension (login), and the flow’s tags: list is exposed to the annotation filter. A single flow is therefore auth.MaestroTest#login for fully-qualified-test-name, or login for method. See Maestro setup for a worked example.

Examples

Run one test class

filteringConfiguration:
  allowlist:
    - type: "fully-qualified-class-name"
      values: ["com.example.LoginTest"]

Run tests matching a regex

filteringConfiguration:
  allowlist:
    - type: "fully-qualified-test-name"
      regex: "com\\.example\\.checkout\\..*"

Load test names from a file

filteringConfiguration:
  allowlist:
    - type: "fully-qualified-test-name"
      file: "smoke-tests.txt"

smoke-tests.txt

com.example.LoginTest#testLogin
com.example.CheckoutTest#testHappyPath

Combine filters

Run every test in com.example.ui that is either a @Smoke test or has a method name starting with testCritical, excluding anything tagged @Flaky.

filteringConfiguration:
  allowlist:
    - type: "package"
      values: ["com.example.ui"]
    - type: "composition"
      op: "UNION"
      filters:
        - type: "annotation"
          values: ["com.example.annotations.Smoke"]
        - type: "method"
          regex: "testCritical.*"
  blocklist:
    - type: "annotation"
      values: ["com.example.annotations.Flaky"]

Apply the filter

1. Save the filter definition as a YAML file, for example smoke-tests.yaml.

2. Pass it with --filter-file. The flag works with run android, run ios, and run maestro.

   marathon-cloud run android \
     --api-key $MARATHON_CLOUD_API_KEY \
     --application app.apk \
     --test-application tests.apk \
     --filter-file smoke-tests.yaml

The CLI validates the file before uploading and reports the failing filter if the schema is wrong.

Best practices

• Store filter files in version control next to your CI configuration.
• Name files by intent: smoke-tests.yaml, nightly-regression.yaml.
• Prefer annotation filters over hand-maintained class lists; they stay correct as tests move between classes.

Note

The schema is shared with the OSS runner. See the Marathon runner filtering docs for the full reference, including composition semantics.