Commit Graph

5100 Commits

Author SHA1 Message Date
Pieter De Gendt 2ee5a24417 scripts: ci: check_compliance: Simplify line number in loop
Use the builtin enumerate function rather than a manual variable.

Signed-off-by: Pieter De Gendt <pieter.degendt@basalte.be>
2024-11-16 14:06:12 -05:00
Mark Inderhees 1993ea019b build: support newlines in syscall decls
Some auto formatters will wrap long lines of code and insert newlines
that are part of function decls outside of arguments. This change strips
out all newlines so syscall typename regex function as expected.

Signed-off-by: Mark Inderhees <markind@meta.com>
2024-11-16 13:43:51 -05:00
Maximilian Deubel 9bed2b7a98 scripts: logging: dictionary: Add support for size_t %z format specifier
This patch adds support for the size_t %z format specifier to the
dictionary parser.
It's the correct format to use for size_t types in modern C,
but it's not supported in python directly.

Signed-off-by: Maximilian Deubel <maximilian.deubel@nordicsemi.no>
2024-11-16 13:32:22 -05:00
Anas Nashif cfc232fa32 twister: stats: fix platform stats
Fix wrong reporting about where testcases were executed.

We have:

INFO    - 1130 of 1130 executed test cases passed (100.00%) on 0 out of
total 860 platforms (0.00%).

which is obviously wrong.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-11-16 13:29:23 -05:00
Anas Nashif 049b243b51 twister: stats: use anytree to create summary
Do not create the tree structure manually, use anytree instead.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-11-12 18:05:16 -06:00
Anas Nashif a43a67e3a3 twister: stats: fix suite statistics
suite stats were not correct, a mixup between skipped and filtered
suites was leading to inconsistent numbers. This is now fixed.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-11-12 18:05:16 -06:00
Anas Nashif b578fa5b2a twister: stats: more renames skipped -> filtered
Additional variable renames leading to confusion between skipped and
filtered.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-11-12 18:05:16 -06:00
Anas Nashif d69d14aac6 twister: stats: skipped_configs -> filtered_configs
Rename variables leading to confusion between skipped and filtered
suites.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-11-12 18:05:16 -06:00
Anas Nashif 61aefee3ad twister: stats: skipped testcases are part of selected group
Count skipped cases as part of selected group in stats.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-11-12 18:05:16 -06:00
Anas Nashif 1647b46a07 twister: stats: remove double counting of statuses
We have been double counting some statuses, remove to get the stats
right.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-11-12 18:05:16 -06:00
Anas Nashif eddb0b77a3 twisters: tests: adapt twister testsuite
Adapt tests for new changes in twister.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-11-12 18:05:16 -06:00
Anas Nashif 7e23b35c33 twister: stats: 'Executed test cases' -> 'Completed test cases'
Built tests are not executed, change column title.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-11-12 18:05:16 -06:00
Anas Nashif 950dccc5bb twister: status inconsistencies are now warnings
Do not report status issues as errors, very confusing and developer end
up looking at the wrong thing, instead, treat those as warnings and
count them and report them at the end.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-11-12 18:05:16 -06:00
Anas Nashif 5ad5b95fde twister: remove verbose debug message about adding platforms
This is very verbose and very long sometimes getting the way when trying
to debug a problem.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-11-12 18:05:16 -06:00
Anas Nashif 11447e4463 twister: stats: not run tests are also part not executed summary
Add notrun tests to the count of non-executed tests.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-11-12 18:05:16 -06:00
Anas Nashif 64f950420a twister: do not report filtered cases in summary
It is confusing to report filtered testcases as  testcases that were
selected but not exexuted. If they are filtered, then there should not
be considered as selected.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-11-12 18:05:16 -06:00
Anas Nashif 81af36202b twister: use consistent language for test configurations
In the summary, use "configurations" instead of instances.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-11-12 18:05:16 -06:00
Anas Nashif c394a27d77 twister: count filtered testcases as such, not as skipped
When a suite is filtered, its testcases are also filtered and not
skipped.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-11-12 18:05:16 -06:00
Anas Nashif 52f9a129b5 twister: custom simulator needs an exec defined
custom simulator needs an executable defined, or else we will have tests
marked runnable with nothing to run them which would result in an
error/warning.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-11-07 11:06:10 -08:00
Grzegorz Swiderski 8fc161f820 west: runners: nrf: Check for missing UICR
On nRF54H and nRF92, booting certain cores requires programming a UICR,
which is normally generated using nrf-regtool. This should be considered
an optional dependency, because we do not wish to force non-Nordic users
to install it just to work with Zephyr, or just for build-only tests.

When nrf-regtool is not installed, a CMake warning will be displayed,
but people ignore warnings all the time. As the last line of defense,
check for missing UICR in the nrfutil flash runner, to prevent our users
from unintentionally programming unbootable firmware. Show a fatal error
specifically if CONFIG_NRF_REGTOOL_GENERATE_UICR=y, yet no UICR exists.

Signed-off-by: Grzegorz Swiderski <grzegorz.swiderski@nordicsemi.no>
2024-11-06 16:39:09 -06:00
Christophe Dufaza b0b278503f Revert "edtlib: fix "last modified" semantic for included ... specs"
[1] was introduced to get more valuable answers from
the PropertySpec.path API, which is supposed to tell
in which file the property's specification was "last modfied".

Further work on related issues [2] showed that the
approach chosen in [1] is dead end: we need to first rethink
how bindings (and especially child-bindings) are initialized.

[1] edtlib: fix last modified semantic in included property specs
[2] edtlib: Preserve paths of properties from included child bindings

See also: #65221, #78095

This reverts commit b3b5ad8156.

Signed-off-by: Christophe Dufaza <chris@openmarl.org>
2024-11-06 14:43:19 -06:00
Christophe Dufaza c58d6761bc edtlib: tests: cover basics of filtering inherited properties
Use-case "B includes I includes X":
- X is a base binding file, specifying common properties
- I is an intermediary binding file, which includes X
  without modification nor filter
- B includes I, filtering the properties it chooses
  to inherit with an allowlist or a blocklist

Check that the properties inherited from X via I
are actually filtered as B intends to,
up to the grandchild-binding level.

Signed-off-by: Christophe Dufaza <chris@openmarl.org>
2024-11-06 14:43:19 -06:00
Christophe Dufaza 0b946dfc01 Revert "edtlib: test "last modified" semantic for ... specs"
This unit test was added to cover the change introduced by [1].

Further work on related issues [2] showed that the chosen approach
is dead end.
We're reverting all changes made in [1].

[1] edtlib: fix last modified semantic in included property specs
[2] edtlib: Preserve paths of properties from included child bindings

See also: #65221, #78095

This reverts commit 70eaa61cb0.

Signed-off-by: Christophe Dufaza <chris@openmarl.org>
2024-11-06 14:43:19 -06:00
Christophe Dufaza 308b568219 Revert "edtlib: test filters set by including bindings"
This unit test was added specifically to cover a regression
reported by the CI while working on [1].

Further work on related issues [2] showed that:
- [1] and [2] are dead end: we need to first rethink
  how bindings (and especially child-bindings) are initialized
- the inclusion mechanism supported by Zephyr deserves more systematic
  testing in edtlib if we want to work with confidence

The approach we choose is to:
- revert all changes made in [1]
- from there, systematically add unit tests as we address
  the issues we identified (or the additional features we need)
  one after the other

[1] edtlib: fix last modified semantic in included property specs
[2] edtlib: Preserve paths of properties from included child bindings

See also: #65221, #78095

This reverts commit 33bb3b60d9.

Signed-off-by: Christophe Dufaza <chris@openmarl.org>
2024-11-06 14:43:19 -06:00
Luca Burelli ba58b066d4 west build: fix a leftover "log" reference in build.py
A recent commit changed all references to the global 'west.log' instance
(now deprecated) to the new WestCommand logging API, but another PR
merged in the same period added an extra instance that is now causing CI
to fail. Convert this last reference to the new API.

Signed-off-by: Luca Burelli <l.burelli@arduino.cc>
2024-11-06 10:07:13 -08:00
Luca Burelli 6c2bc2ff37 west build: warn about conditional flags in 'extra_configs'
The 'west build' command does not know about conditional flags (in the
format 'type:value:CONFIG_FOO=bar') in the 'extra_configs' argument of
Twister testcase.yaml files, and currently converts them to malformed
arguments that are silently ignored by cmake.

This change adds a check to 'west build' to clearly warn the user if the
'extra_configs' list contains conditional flags and provide a hint on
how to add them to the CMake command line.

Signed-off-by: Luca Burelli <l.burelli@arduino.cc>
2024-11-06 10:07:13 -08:00
Henrik Brix Andersen 97d6cd335c scripts: check_maintainers: add scripts for checking GitHub accounts
Add script for checking if maintainer and collaborator GitHub accounts
exist.

Signed-off-by: Henrik Brix Andersen <hebad@vestas.com>
2024-11-06 09:53:52 -08:00
Dmitrii Golovanov 7e1cd18be4 twister: Fix NOTRUN status
Fix Twister TestCase statuses left not assigned ('NONE') in these 'NOTRUN'
situations:

 * a test suite has `integration platform` which is not available to run.

 * `--cmake-only` execution.

Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
2024-11-06 09:52:25 -08:00
Lukasz Mrugala 4bb945300b scripts: twister: Python Version Guard
Twister shall now verify that the user does not
use an obsolete Python version. If user's Python
is deemed too old, it will raise a relevant error.
This check will also happen when running Twister
via west.

Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
2024-11-04 17:24:24 -05:00
Gerard Marull-Paretas 5e8905bb94 scripts: checkpatch: add NRF_GPIO_Type to typedefs file
So that we do not get false warnings about consistent spacing around
'*'.

Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
2024-11-01 12:10:12 -05:00
Joel Hirsbrunner 8d07197d6d devicetree: Remove deprecated enum macro
Remove deprecated _ENUM_TOKEN and _ENUM_UPPER_TOKEN. These are
deprecated for over three years by now.

Signed-off-by: Joel Hirsbrunner <jhirsbrunner@baumer.com>
2024-11-01 09:55:11 -05:00
Lukasz Mrugala 9dc0af55e2 scripts: twister: Fix NOTRUN in test_only
When using the --build-only into --test-only
Twister setup, NOTRUN statuses were not properly rerun.

Now they are properly run again if runnable.

Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
2024-10-31 14:18:17 -05:00
Armin Kessler 78f688a3e7 scripts: twister: Fix serial_py referenced before assignment
This fixes `serial_py referenced before assignment` if `flash_before`
is set.

Signed-off-by: Armin Kessler <ake@espros.com>
2024-10-31 14:16:36 -05:00
Armin Kessler e31bf8bf50 scripts: twister: Add Espressif as manufacturer
Adds Espressif as manufacturer so that generation of HW-Map file is
possible

Signed-off-by: Armin Kessler <ake@espros.com>
2024-10-31 14:16:25 -05:00
Lukasz Mrugala ff1857af55 scripts: twister: Enchance TestCase/Instance info and presentation
ExecutionCounter has been expanded and now hold i.a.
more information on the statuses of TestCases.
This information is now incorporated in relevant summaries
- runner.py and reports.py.
Layout of those was changed to present that
and previous information in a clear and concise way.

TestInstance execution counter now is more intuitive.
Instances filtered out before running are no longer included there.
Retries now properly reset the counter.

TestCases with None and other incorrect final statuses
are logged as errors, but do not
exit Twister with a nonzero exit code.
This is because None statuses, although incorrect,
are currently common.

Inconsistent spacing in ERROR and FAILED fixed.

Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>

scripts: Dmitri fix

Fix of a problem noticed by Dmitri

Removed unnecessary additional spaces when
printing FAILED and ERROR status.
Now TwisterStatus.get_color is used more.

Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
2024-10-31 09:24:18 -05:00
Reto Schneider 743761d7d1 scripts: Fix CMake spelling
As per its creators, CMake is written with a capital "M".

The initial reason for this change is that I want Twister to print
"ERROR : CMake build failure" instead of "ERROR : Cmake build failure".

Signed-off-by: Reto Schneider <reto.schneider@husqvarnagroup.com>
2024-10-30 16:32:24 -05:00
Sa Sasu 5dc5fa5ee2 scripts: print the file name when decode syscall
the script shows an incorrect file name, I check the wrong file first, then
I find the script did not print the current reading file name.

Fix this to prevent others from wasting their time on this.

The new error message:

```
[1/179] Generating syscalls.json, struct_tags.json
Error decoding zmk/.../altera_msgdma.c (included in zephyr/.../ethernet.c)
```

Signed-off-by: Sa Sasu <i@sasa.su>
2024-10-30 14:30:55 -05:00
Anas Nashif 2c43d7fe76 twister: oot soc: set soc_root using Path
Set soc_root using Path to avoid wrong generated path in the list of
soc_roots and other roots read from module.yml file

Fixes #80531

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-10-30 08:54:49 -05:00
Anas Nashif b417524b66 twister: cleanup notrun tests with -M
Cleanup notrun tests with the -M option. This is used in the weekly
build and fails because devices will run out of space.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-10-28 15:58:23 -07:00
Grzegorz Swiderski 276c1fc4a0 scripts: Undo Board.{dir -> directories[0]} renames
`Board.dir` can be used in place of `Board.directories[0]` for brevity.

Signed-off-by: Grzegorz Swiderski <grzegorz.swiderski@nordicsemi.no>
2024-10-26 17:14:06 +02:00
Grzegorz Swiderski a1f3f882fd scripts: list_boards: Reintroduce Board.dir as @property
In the `Board` class, the `dir` member was renamed to `directories`,
both to indicate that it is now a list (in HWMv2 with board extensions)
and to reflect the addition of the `BOARD_DIRECTORIES` CMake variable.
Considering that the build system also keeps the `BOARD_DIR` variable,
and for the sake of backwards compatibility and brevity, it should be
useful to retain `Board.dir` in Python as well, symmetrically.

Signed-off-by: Grzegorz Swiderski <grzegorz.swiderski@nordicsemi.no>
2024-10-26 17:14:06 +02:00
Jamie McCrae 33e70b32dc scripts: list_hardware: Do not check qualifiers for run once config
Removes validating the qualifiers for flash run once configuration
as files may be present that contain information for qualifiers
that are not present in a single repository but are spaced out in
other repositories, or might be optional

Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
2024-10-26 17:14:06 +02:00
Jamie McCrae a04dfc4ece scripts: west_commands: run: Add priority to run once config
Adds an optional priority parameter to the flash runner run once
configuration which allows for deciding upon which file should
ultimately be used

Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
2024-10-26 17:14:06 +02:00
Torsten Rasmussen 98b186c110 cmake: scripts: support SoC extension
Fixes: #72374

Support extending an existing SoC with new CPU clusters.

This commit introduces the following changes to allow an SoC to be
extended out-of-tree.

The SoC yaml schema is extended to support an extend field which
will be used to identify the SoC to be extended with extra CPU clusters.

A SoC 'a_soc' can be extended like this:
> socs:
>   extend: a_soc
>   cpuclusters:
>     - name: extra_core

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2024-10-26 17:14:06 +02:00
Torsten Rasmussen 536d34fa7a cmake: scripts: support board extension
Fixes: #69548

Support extending an existing board with new board variants.

This commit introduces the following changes to allow a board to be
extended out-of-tree.

The board yaml schema is extended to support an extend field which
will be used to identify the board to be extended.

A board 'plank' can be extended like this:
> board:
>   extend: plank
>   variants:
>     - name: ext
>       qualifier: soc1

For the rest of the build system this means that there is no longer a
single board directory.
The existing CMake variable BOARD_DIR is kept and reference the
directory which defines the board.
A new CMake variable BOARD_DIRECTORIES provides a list of all
directories which defines board targets for the board.
This means the directory which defines the board as well as all
directories that extends the board.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2024-10-26 17:14:06 +02:00
Anas Nashif 7066c40afc twister: also convert platform names from the hardwaremap
Convert platform names from the hardware map to full target names.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-10-26 13:41:55 +02:00
Anas Nashif e90559f0da twister: fix platform filter when loading plan from file
if alias or shorthand name is provided on the command line, we need
convert this to complete target name for the filters to work.

Fixes #80332

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-10-26 13:41:55 +02:00
Anas Nashif 625246f241 twister: --device-testing always implies --filter runnable
--device-testing implies runnable tests, this was the case before the
rework and was missed in one spot, so instead do that directly in the
argument parser.

Fixes #80428

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-10-26 13:41:43 +02:00
Pieter De Gendt c2b6bcc660 scripts: west_commands: build: Remove deprecated west.log
The global state west.log is deprecated, replace with WestCommand
logging.

Signed-off-by: Pieter De Gendt <pieter.degendt@basalte.be>
2024-10-26 11:30:59 +02:00
Pieter De Gendt 8b0edb2b22 scripts: west_commands: sign: Remove deprecated west.log
The global state west.log is deprecated, replace with WestCommand
logging.

Signed-off-by: Pieter De Gendt <pieter.degendt@basalte.be>
2024-10-26 11:30:49 +02:00