83fc06a8fe
A complete overhaul of the sanitycheck script and how we build and run tests. This new version of sanitycheck uses python for job distribution and drop use of Make. In addition to the move to python threading library, the following has been changed: - All handlers now run in parallel, meaning that any simulator will run in parallel and when testing on multiple devices (using --device-testing) the tests are run in parallel. - Lexicial filtering (using the filter keyword in yaml files) is now evaluated at runtime and is no long being pre-processed. This will allow us to immediately start executing tests and skip the wait time that was needed for filtering. - Device testing now supports multiple devices connected at the same time and is managed using a hardware map that needs to be generated and maintained for every test environment. (using --generate-hardware-map option). - Reports are not long stored in the Zephyr tree and instead stored in the output directory where all build artifacts are generated. - Each tested target now has a junit report in the output directory. - Recording option for performance data and other metrics is now available. This will allow us to record the output from the console and store the data for later processing. For example benchmark data can be captured and uploaded to a tracking server. - Test configurations (or instances) are no longer being sorted, this will help with balancing the load when we run sanitycheck on multiple hosts (as we do in CI). And many other cleanups and improvements... Signed-off-by: Anas Nashif <anas.nashif@intel.com> |
||
|---|---|---|
| .. | ||
| boards/unit/unit_testing | ||
| .gitignore | ||
| expr_parser.py | ||
| harness.py | ||
| platform-schema.yaml | ||
| sanity_compare.args | ||
| sanity_daily.args | ||
| sanity_last_release.csv | ||
| sanity_verify.args | ||
| scl.py | ||
| testcase-schema.yaml | ||