The final executable output from the native_sim
(or native_posix) build is zephyr.exe
(in native_posix zephyr.elf happened to be just a copy of
the exe, but in native_sim it is an intermediate build step)
So we need to use the exe instead.
Note: As the exe is generated with a custom target, we need
to install it using install(FILES), which also requires the
destination and permissions to be listed manually. The
permissions set are just the cmake default permissions
for install(PROGRAMS).
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
Many functions return non-zero return codes on errors. I added an assert
for the case, when a function is expected to fail. It is just a
shorthand for `zassert_not_equal(0, ret);`, analogous to
`zassert_ok`, introduced in c5d85e175f.
I also added the corresponding `zassume_nok` and `zexpect_nok`.
Signed-off-by: Greter Raffael <rgreter@baumer.com>
Move the syscall_handler.h header, used internally only to a dedicated
internal folder that should not be used outside of Zephyr.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
This test cannot be run in this architecture as it does not
support userspace.
Today it is filtered by kconfig, which works but spends
time running cmake.
As native_posix is a default test platform it is better
to filter it alltogether by arch, which saves quite a lot
of time.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
Skip `test_catch_fatal_error` test case for Armv8-R because
divide by zero trapping is not supported on this architecture.
Fixes#63268
Signed-off-by: Manuel Argüelles <manuel.arguelles@nxp.com>
The main.cpp was introduced as a symbolic link of main.c
but this makes the test build failure on Windows, so just
remove the symlink and include directly the source
Signed-off-by: Dat Nguyen Duy <dat.nguyenduy@nxp.com>
- Add integration_platforms to avoid excessive filtering
- Make sure integration platforms are actually part of the filter
- Fix some tags and test meta data
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
The SW div implementation from libgcc doesn't handle division by
zero case, so the HW exception isn't triggered.
So, skip this test case (as we are already doing for some other
ARC targets).
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
Twister now supports using YAML lists for all fields that were written
as space-separated lists. Used twister_to_list.py script. Some artifacts
on string length are due to how ruamel dumps content.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
Its not needed for this to build and run. Additionally we've
deprecated the CONFIG_LIB_CPLUSPLUS symbol.
Signed-off-by: Kumar Gala <kumar.gala@intel.com>
Modifying counter drivers (rtc and timer) to rely completely on
device tree and not on Kconfig of MDK flags.
Adapting dtsi for all SoCs and adapting test configuration.
Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
The zexpect API tests included a 15 sec timeout. This is an unnecessarily
short timeout and not consistent with the rest of the ztest API tests,
which don't include timeouts themselves. Particularly this timeout seems to
go off on slower hardware.
Remove unnecessary timeout. Fixes
https://github.com/zephyrproject-rtos/zephyr/issues/55335.
Signed-off-by: Aaron Massey <aaronmassey@google.com>
Add the zexpect API, inspired by GoogleTest's EXPECT API. This API reports
test failures while allowing test execution to continue. This enables test
reports to show more than a singule failing property on a failing test.
Signed-off-by: Aaron Massey <aaronmassey@google.com>
This commit updates all deprecated `CONFIG_LIB_CPLUSPLUS` usages to:
* check if the Zephyr minimal C++ library is enabled using
`CONFIG_MINIMAL_LIBCPP` instead of relying on the
`CONFIG_LIB_CPLUSPLUS`-based inference.
* select `CONFIG_REQUIRES_FULL_LIBCPP` when there exists a component-
level C++ standard library dependency. This allows a component to
declare C++ standard library dependency without designating a
specific libray implementation.
* select the correct type of C++ standard library implementation to use
through one of the `CONFIG_LIBCPP_IMPLEMENTATION` choices.
Signed-off-by: Stephanos Ioannidis <stephanos.ioannidis@nordicsemi.no>
This commit updates all in-tree code to use `CONFIG_CPP` instead of
`CONFIG_CPLUSPLUS`, which is now deprecated.
Signed-off-by: Stephanos Ioannidis <stephanos.ioannidis@nordicsemi.no>
The old random timer test was not random-looking
enough on some platforms.
Replace with new test which is psuedo-xoshiro.
The generator is still deterministic
and does not depend on entropy at all,
but should look more random for testing.
Change name of generator tree-wide also.
Signed-off-by: Declan Snyder <declan.snyder@nxp.com>
Add a test to make sure that when a zassume fails, we mark the test
as failed and print an "Assumption failed" error for the test
Signed-off-by: Yuval Peress <peress@google.com>
Adding the new Kconfig (enabled by default) to make a failed assumption
mark the final result as failed. This change has the following benefits
which have been asked for by the Zephyr community:
1. A failed assumption does not go silent. In this example, the failed
assumption will still mark the test as skipped, but the final result
will be to mark the full test run as failed. This would allow
blocking the CI when an assume fails.
2. Normal test skipping via the ztest_test_skip() is unaffected by this
change. Those tests will be marked as skipped, but the binary will
still pass.
Signed-off-by: Yuval Peress <peress@google.com>
I've update FFF to run tests in the native CI for GitHub, so we no
longer need to run these extra tests. Remove the fff module from
west.yml since the module was only used in the CI and the header
was directly included in the Zephyr tree.
See https://github.com/zephyrproject-rtos/fff/pull/2
Signed-off-by: Yuval Peress <peress@google.com>
When CONFIG_ZTEST_SHUFFLE is enabled and ASSERTS are disabled
`start_pos` becomes an unused variable leading to a compile error.
Cleaned-up shuffling algorithm to not need a `start_pos` check.
Signed-off-by: Al Semjonovs <asemjonovs@google.com>
This commit disables the `testing.ztest.ztress` test on
`qemu_cortex_r5` because it fails at run-time when compiled with
GCC 12.
Revert this commit when the GitHub issue #49494, which tracks this bug,
is fixed.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
Add specialized tests that execute another test indirectly and ensure
that the results report errors correctly. See the README.rst file
provided with this commit for details.
Signed-off-by: Yuval Peress <peress@google.com>
With the support of Kconfig in unittest CMake module then prj.conf is
now supported.
Add a prj.conf with ZTEST config settings and remove hardcoded values
from CMakeLists.txt file.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Add test summary after all test suites finish running.
The summary can be one-line or verbose, which is configured
with CONFIG_ZTEST_VERBOSE_SUMMARY. The one-line summary covers
overall suite stats. The verbose summary covers each test
function within the suite besides the one-line summary.
The new ztest output ultimately go through the printk. If
printk go through the logging subsystem, there may be log
messages dropped. And if log_panic is invoked, log messages
can be flushed in a mess. So several explicit log flush
are used when printing summary to ensure no content is lost
and content is in good shape.
Some macros are shared between old and new ztests. Such as
TC_START_PRINT and TC_END_PRINT. The are defined accordingly.
Signed-off-by: Ming Shao <ming.shao@intel.com>
This commit enables the assert test mode (`CONFIG_ASSERT_TEST`) for the
ztest error hook test because it implements a custom post assert fail
hook (`ztest_post_assert_fail_hook`) that returns without aborting to
faciliate the testing of the assert functions.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
Updates the ztest_test_fail() function to allow failures in setup.
When executed, a failed assert will fail every test in the suite owning
the setup function. This was verified by adding a suite which asserts
in the setup function and has a test that should pass. During
exeuction, ztest marks the test as failing.
In order to verify exection I also added 2 new APIs:
- ZTEST_EXPECT_FAIL(suite_name, test_name)
- ZTEST_EXPECT_SKIP(suite_name, test_name)
Signed-off-by: Yuval Peress <peress@google.com>
Add a bunch of missing "zephyr/" prefixes to #include statements in
various test and test framework files.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
Add an assume API which works like JUnit's. Assumptions can be made
at any point before your test returns (setup, before, and during the
test). If an assumption fails, the test will be marked as skipped.
This helps avoid a cascading affect of failed tests when a base
feature breaks. The feature is expected to have tests and the tests
which depend on it should be skipped (not failed) if that feature
is broken.
Issue #42472
Signed-off-by: Yuval Peress <peress@google.com>
Alberto Escolar Piedras