zephyr/tests/bsim

This folder contains tests meant to be run with BabbleSim's physical layer simulation, and therefore cannot be run directly from twister.

The compile.sh and run_parallel.sh scripts are used by the CI system to build the needed images and execute these tests in batch.

You can also run them manually if desired, but be sure to call them setting the variables they expect. For example, from Zephyr's root folder, you can run:

WORK_DIR=${ZEPHYR_BASE}/bsim_out tests/bsim/compile.sh
RESULTS_FILE=${ZEPHYR_BASE}/myresults.xml SEARCH_PATH=tests/bsim tests/bsim/run_parallel.sh

Or to run only a specific subset, e.g. host advertising tests:

WORK_DIR=${ZEPHYR_BASE}/bsim_out tests/bsim/bluetooth/host/compile.sh
RESULTS_FILE=${ZEPHYR_BASE}/myresults.xml \
SEARCH_PATH=tests/bsim/bluetooth/host/adv tests/bsim/run_parallel.sh

Check the run_parallel.sh help for more options and examples on how to use this script to run these tests in batch.

After building the tests' required binaries you can run a test directly with its individual test scripts.

For example you can build the required binaries for the networking tests with

WORK_DIR=${ZEPHYR_BASE}/bsim_out tests/bsim/net/compile.sh

and then directly run one of the tests:

tests/bsim/net/sockets/echo_test/tests_scripts/echo_test_802154.sh

Generating coverage reports

You can use tests/bsim/generate_coverage_report.sh to generate a coverage report of the BabbleSim tests which have been run. Check its help for more info.

Conventions about these tests scripts

Each test is defined by a shell script with the extension .sh. Scripts starting with an underscore (_) are ignored.

Each of these test scripts expects the binaries their require already built, and will execute the required simulated devices and physical layer simulation with the required command line options.

Tests must return 0 to the invoking shell if the test passes, and not 0 if the test fails.

It is recommended to have a single test for each test script.

Each test must have a unique simulation id, to enable running different tests in parallel.

These scripts should not compile the images on their own.

Neither the scripts nor the images should modify the workstation filesystem content beyond the ${BSIM_OUT_PATH}/results/<simulation_id>/ or /tmp/ folders. That is, they should not leave straneous files behind.

If these scripts or the test binaries create temporary files, they should preferably do so by placing them in the ${BSIM_OUT_PATH}/results/<simulation_id>/ folder. Otherwise they should be named as to avoid conflicts with other test scripts which may be running in parallel.

When running tests that require several consecutive simulations, for ex. if simulating a device pairing, powering off, and powering up after as a new simulation, they should strive for using separate simulation ids for each simulation part, in that way ensuring that the simulation radio activity of each segment can be inspected a posteriori.

Conventions about the build scripts

The build scripts (compile.sh) simply build all the required test and sample applications for the tests' scripts placed in the subfolders below.

This build scripts use the common compile.source which provide a function (compile) which calls cmake and ninja with the provided application, configuration and overlay files.

To speed up compilation for users interested only in a subset of tests, several compile scripts exist in several subfolders, where the upper ones call into the lower ones.

Note that cmake and ninja are used directly instead of the west build wrapper as west is not required, and some Zephyr users do not use or have west, but still use this build and tests scripts.