Skip to main content
Version: 0.9.1

Sorting

Sorting strategy

Prioritising performance of test execution requires tests to be sorted. Sorting is possible only when analytics backend is available.

Sorting can be done based on test duration and success/failure rate.

No sorting

No sorting of tests is done at all. This is the default behaviour.

sortingStrategy:
type: "no-sorting"

Success rate sorting

For each test analytics storage is providing the success rate for a time window specified by time timeLimit parameter. All the tests are then sorted by the success rate in an increasing order, that is failing tests go first and successful tests go last. If you want to reverse the order set the ascending to true.

sortingStrategy:
type: "success-rate"
timeLimit: "2015-03-14T09:26:53.590Z"
ascending: false

Execution time sorting

For each test analytics storage is providing the X percentile duration for a time window specified by time timeLimit parameter. Apart from absolute date/time it can be also be an ISO 8601 formatted duration.

Percentile is configurable via the percentile parameter.

All the tests are sorted so that long tests go first and short tests are executed last. This allows marathon to minimise the error of balancing the execution of tests at the end of execution.

sortingStrategy:
type: "execution-time"
percentile: 80.0
timeLimit: "-PT1H"

Random order sorting

Sort tests in random order. This strategy may be useful to detect dependencies between tests.

sortingStrategy:
type: "random-order"