tests: posix: test_realtime: improve test reliability
Previously posix_apis.test_realtime was failing (very)
frequently in CI, and in particular, when running on Qemu,
POSIX, or SMP targets.
We are using CLOCK_REALTIME for this test, which incurs an
additional syscall overhead above CLOCK_MONOTONIC. The act
of sleeping itself also incurs a syscall overhead.
The latency from one iteration to the next of the internal
loop is a bit of a random process due to scheduler or clock
noise (although the noise itself is still bounded).
In order to make this test robust against such noise, assert
only on the average time from one iteration to the next,
rather than the instantaneous time.
Rather than calculating a sample mean, use a running average
(aka Cumulative Moving Average) to save some bytes.
Report results, including low and high watermarks before
asserting that the average iteration time within expected
range.
==============================================================
START - test_realtime
I: n: 20, sleep: 100, margin: 10, lo: 110, avg: 110, hi: 110
PASS - test_realtime in 2.198 seconds
==============================================================
Expect to see the low and high watermarks change more on
Qemu and POSIX platforms when running several jobs in parallel
with twister (such as in CI).
Signed-off-by: Christopher Friedt <cfriedt@meta.com>
2023-12-28 06:56:27 +08:00
|
|
|
# Copyright (c) 2023, Meta
|
|
|
|
|
# SPDX-License-Identifier: Apache-2.0
|
|
|
|
|
|
|
|
|
|
# Options specific to clock.c / test_realtime
|
|
|
|
|
|
|
|
|
|
config TEST_CLOCK_RT_ITERATIONS
|
|
|
|
|
int "Number of iterations to check clock_gettime() reliability"
|
|
|
|
|
range 10 100
|
|
|
|
|
default 20
|
|
|
|
|
help
|
|
|
|
|
This option is specific to posix_apis.test_realtime in clock.c
|
|
|
|
|
|
|
|
|
|
config TEST_CLOCK_RT_SLEEP_MS
|
|
|
|
|
int "Time to sleep between iterations in milliseconds"
|
|
|
|
|
range 50 1000
|
|
|
|
|
default 100
|
|
|
|
|
help
|
|
|
|
|
This option is specific to posix_apis.test_realtime in clock.c
|
|
|
|
|
|
|
|
|
|
config TEST_CLOCK_RT_ERROR_MS
|
|
|
|
|
int "Maximum overshoot (error) in milliseconds"
|
|
|
|
|
range 10 500
|
|
|
|
|
default 10
|
|
|
|
|
help
|
|
|
|
|
This option is specific to posix_apis.test_realtime in clock.c
|
|
|
|
|
|
2024-01-20 23:25:52 +08:00
|
|
|
config TEST_SEM_N_LOOPS
|
|
|
|
|
int "Number of loops in semaphore test"
|
|
|
|
|
range 16 1024
|
|
|
|
|
default 32
|
|
|
|
|
help
|
|
|
|
|
This option is specific to semaphore.test_named_semaphore in semaphore.c
|
|
|
|
|
|
tests: posix: test_realtime: improve test reliability
Previously posix_apis.test_realtime was failing (very)
frequently in CI, and in particular, when running on Qemu,
POSIX, or SMP targets.
We are using CLOCK_REALTIME for this test, which incurs an
additional syscall overhead above CLOCK_MONOTONIC. The act
of sleeping itself also incurs a syscall overhead.
The latency from one iteration to the next of the internal
loop is a bit of a random process due to scheduler or clock
noise (although the noise itself is still bounded).
In order to make this test robust against such noise, assert
only on the average time from one iteration to the next,
rather than the instantaneous time.
Rather than calculating a sample mean, use a running average
(aka Cumulative Moving Average) to save some bytes.
Report results, including low and high watermarks before
asserting that the average iteration time within expected
range.
==============================================================
START - test_realtime
I: n: 20, sleep: 100, margin: 10, lo: 110, avg: 110, hi: 110
PASS - test_realtime in 2.198 seconds
==============================================================
Expect to see the low and high watermarks change more on
Qemu and POSIX platforms when running several jobs in parallel
with twister (such as in CI).
Signed-off-by: Christopher Friedt <cfriedt@meta.com>
2023-12-28 06:56:27 +08:00
|
|
|
source "Kconfig.zephyr"
|