Migrate from Firebase Test Lab

Marathon Cloud replaces Firebase Test Lab (FTL) for Android instrumentation and iOS XCTest suites with a single CLI, automatic sharding, and retries that re-run individual tests instead of the whole suite.

Command mapping

Firebase Test Lab	Marathon Cloud
gcloud firebase test android run	marathon-cloud run android
gcloud firebase test ios run	marathon-cloud run ios
--app app.apk	--application app.apk
--test test.apk	--test-application test.apk
--device model=Pixel2,version=28	--os-version 9 --system-image google_apis (API 28 is Android 9; versions below 10 need google_apis; the emulator profile is chosen by the platform)
--device model=...,version=... for Wear / TV	--device watch / --device tv
--environment-variables KEY=VALUE	--instrumentation-arg KEY=VALUE
--test-targets "class com.example.LoginTest"	--filter-file with a fully-qualified-class-name filter
--num-flaky-test-attempts 3	--retry-quota-test-reactive 3 (or leave the platform default)
--num-uniform-shards N	Not needed; sharding is automatic
--timeout 30m	Not needed; per-test timeouts are set with --test-timeout-default on iOS
--results-bucket BUCKET	--output ./results, or marathon-cloud download --id RUN_ID
--directories-to-pull /sdcard/screenshots	--pull-files EXTERNAL_STORAGE:/sdcard/screenshots
--use-orchestrator	No direct equivalent. Tests run in batches; --isolated makes each batch a single test. See batching

Concept mapping

Device selection

FTL asks for a specific model and API level. Marathon Cloud asks for an OS version and optionally a form factor; the platform selects an emulator profile for it. Run marathon-cloud devices android to see the catalog.

Sharding

FTL shards only when you configure --num-uniform-shards or --test-targets-for-shard. Marathon Cloud shards every run automatically, sizing the device pool from your suite’s historical duration to target a 15-minute wall clock. See batching.

Flaky tests

FTL’s --num-flaky-test-attempts re-runs the suite. Marathon Cloud retries only the failing test, on a different device, and reports tests that pass on retry as flaky rather than failed. See retries.

Results

Instead of a Cloud Storage bucket, results are attached to the run in the console and downloadable with --output or marathon-cloud download. JUnit XML, Allure, per-test video, and logs are included. See artifacts.

GitHub Actions migration

Firebase Test Lab

- uses: google-github-actions/auth@v2
  with:
    credentials_json: ${{ secrets.GCP_SA_KEY }}
- name: Test with FTL
  run: |
    gcloud firebase test android run \
      --type instrumentation \
      --app app-debug.apk \
      --test app-debug-androidTest.apk \
      --device model=Pixel2,version=28 \
      --num-flaky-test-attempts 2

Marathon Cloud

- name: Test with Marathon Cloud
uses: MarathonLabs/[email protected]
with:
  version: 1.0.65
  apiKey: ${{ secrets.MARATHON_CLOUD_API_KEY }}
  platform: Android
  application: app-debug.apk
  testApplication: app-debug-androidTest.apk
  osVersion: 9
  systemImage: google_apis
  output: marathon

No service account or project setup is needed; an API key is the only credential.

First migrated pull request

1. Add MARATHON_CLOUD_API_KEY to your CI secrets.
2. Keep the FTL step in place and add the Marathon Cloud step next to it with ignoreTestFailures: true, so the first runs do not block merges while you compare results.
3. Set output and point your JUnit publisher at <output>/tests/omni/marathon_junit_report.xml.
4. Check that anything the tests call is reachable from the internet; FTL and Marathon Cloud both run outside your network, so this is usually already true.
5. Once results match, drop ignoreTestFailures and remove the FTL step.

Platform differences

iOS execution

FTL runs iOS tests on a limited pool of physical devices. Marathon Cloud runs iOS on ARM64 simulators, which provision on demand. If your suite depends on physical-device-only behaviour (camera hardware, real push notifications, device-specific performance), verify it on a simulator before migrating.

Android devices

Marathon Cloud provisions Android emulators, not physical devices. Tests that rely on OEM-specific behaviour should be reviewed.

Concurrency

FTL applies per-project device quotas. Marathon Cloud scales the pool to the suite. If your backend cannot take that load, cap it with --concurrency-limit; see limiting concurrency.

What you gain

• Per-test retries instead of whole-suite reruns, with flaky tests labelled in the report.
• Automatic sharding targeting a 15-minute run, without shard configuration.
• One credential and one binary for Android, iOS, and Maestro.
• Second-level billing of device time; see pricing.