Commit Graph

3012 Commits

Author SHA1 Message Date
Nicolas Pitre f97d12936e kernel: add an architecture specific structs header
Add the ability to define architecture specific structures, notably
the ability to extend struct _cpu with per-CPU arch-specific stuff that
can be accessed with _current_cpu->arch.* similarly to _current->arch.*
for per-thead architecture data.

This is opt-in for architectures that want to benefit from this,
otherwise empty defaults are provided. A placeholder for ARM64 is
included to show the pattern.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2021-04-21 09:03:47 -04:00
Torsten Rasmussen 3dd65a7663 runners: remove dependencies from runners
Today, there is a build target is added for each runner: flash, debug,
debugserver, attach.

And those runners will have a dependency to Zephyr logical target that
is built before invoking `west <runner>`.

This design has some flaws, mainly that additional dependencies directly
on the target will not be built when running `west <runner>` directly.
That generator expressions cannot be used for the DEPENDS argument.

Instead, the build target `<runner>` will not have any dependencies, and
will raise a build error if a dependency is added to the target.
Due to how `add_dependencies()` work, this must be done as a build time
check, and not configure time check.

`west <runner>` will invoke a build before executing the runner, and
this way ensure the build target is up-to-date, which again removes the
need for a dedicated `west_<runner>_target`.

It also minimizes the risk of developer errors, as developers no longer
need to consider the need for adding additional dependencies.
If a custom target is part of the default `all` build, then it's ensured
to be up-to-date.

Fixes: Issue reported on slack.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2021-04-19 15:35:30 +02:00
Maciej Perkowski 484ef6710d twister: Fix memory footprint reporting logic
With the previous logic memory footprint can only be saved in a report
if a given test/sample was fully executed and passed (built and run),
hence build-only tests were not providing these metrics. This commit
modifies the logic so that it is enough to have the build successful
to be able to get the memory footprint.

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2021-04-18 21:49:21 -04:00
Maciej Perkowski 73c5594573 twister: Fix ram/rom reporting
The info of rom/ram usage by an application was lost along the
way of data processing in twister. The commit add a line which
pass further these metrices as well.

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2021-04-18 21:49:21 -04:00
Jordan Yates c6cc927506 scripts: gen_relocate_app.py: fix undefined _ADDR
With the addition of #34185, it is not longer gauranteed that for a
memory region `NAME` there exists a symbol `NAME_ADDR`. Use the linker
builtin function `ORIGIN` to instead directly get the start address of
the selected memory region.

Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
2021-04-16 14:44:10 -05:00
Anas Nashif bc747e7167 tracing: remove cpu_stats in favor of thread runtime stats
Removing CONFIG_TRACING_CPU_STATS in favor of
CONFIG_THREAD_RUNTIME_STATS which provides per thread stats. The same
functionality is also available when Thread analyzer is enabled with the
runtime stats enabled.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-04-16 15:31:24 -04:00
Jorgen Kvalvaag 36d96d2cd9 twister: Allow deep folder scan
Upon discovery of a sample/testcase.yaml file, twister will disregard
further subfolders. Given the folder structure below, the testcase.yaml
will currently not be discovered:

samples/sample1/
samples/sample1/sample.yaml
samples/sample1/tests/
samples/sample1/tests/testcase.yaml

This is desirable to allow placing test code closer to sample code.

Signed-off-by: Jorgen Kvalvaag <jorgen.kvalvaag@nordicsemi.no>
2021-04-16 15:46:09 +02:00
Yasushi SHOJI 2b32e47a9a scripts: coccinelle: Fix parse error
The commit coccinelle/coccinelle@47bd4cae52 changed a behavior of
parsing right before coccinelle v1.1.0.  With the commit, the current
scripts under scripts/coccinelle/ errors out with:

   minus: parse error:
     File ".../zephyr/scripts/coccinelle/deref_null.cocci",
     line 25, column 42, charpos = 666
     around = '...',
     whole content =  (E != NULL && ...) ? <+...E->f@p1...+> : ...

I've already raised an issue upstream coccinelle/coccinelle#257.  But
Debian is already shipping v1.1.0 and we need a fix.

The proposed fix doesn't change the semantics, it just explicitly
states that the rule is an expression.

Signed-off-by: Yasushi SHOJI <yashi@spacecubics.com>
2021-04-14 14:18:02 -04:00
Detlev Zundel e5f33bd740 scripts: size_report: Add handling of depth argument
The script already accepts a depth parameter to configure the output,
but that parameter was not used at all.  This commit adds the correct
handling by passing it to anytrees RenderTree iterator.

Signed-off-by: Detlev Zundel <dzu@member.fsf.org>
2021-04-14 07:05:58 -04:00
Anas Nashif 70dc6b40ff ci: pylint: relax similar lines rule
We are getting hits on similar imports in different files, relax that.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-04-13 16:33:18 -05:00
Jaxson Han 8af11d40d0 cmake: emu: Add arm fvp emulator
Add arm fvp emulator in order to use ninja run or west build -t run.
Add armfvp in order to run twister.

Set env ARMFVP_BIN_PATH before using it,
e.g. export ARMFVP_BIN_PATH=<path/to/fvp/dir>
NOTE: ARMFVP_BIN_PATH is the dir path.

Signed-off-by: Jaxson Han <jaxson.han@arm.com>
2021-04-13 07:47:44 -04:00
Håkon Øye Amundsen c5c55ee47e scripts: size_report: fix bug where key is used as a dict
When printing the unassigned values the 'sym' variable is
used as a dict from which we try to get the 'name' value.
However, 'symbols['unassigned']' gives a list of keys, so
we get an 'TypeError' when trying to access ['name'] of
a string (the key).

Fix this issue by iterating over the values from
the 'symbols['unassigned']' dict instead.

Signed-off-by: Håkon Øye Amundsen <haakon.amundsen@nordicsemi.no>
2021-04-13 07:16:15 -04:00
Maciej Perkowski 9c6dfce0c0 twister: Add quarantine feature
Adds feature allowing to use yaml file with dictionaries defining
tests to be quarantined (extra arg "--quarantine-list FILENAME").
The dictionaries are validated according to the proper schema
and loaded.

A flat list is created containing quarantined configurations
(configuration = platform + scenario). Configurations under quarantine
are skipped and get "Quarantine" as a reason in the results reports.

A "comment" can be added to a quarantine entry in the quarantine yaml
with more details (e.g. issue #) and it will be also added to
the report.

The status of tests under quarantine can be verify if
`--quarantine-verify` is used in addition to
"--quarantine-list FILENAME". Using these args will make twister skip
all tests which are not on the quarantine list.

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2021-04-12 08:42:29 -04:00
Gerard Marull-Paretas 61b85b60d6 doc: allow up to breathe 4.28.x
It looks like latest release (4.29.0) breaks docs build.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2021-04-10 07:27:58 -04:00
Kumar Gala d590e0d18d twister: Adjust platform selection logic for platform_allow
In CI we typically build for "default" platforms.  However some
testcases that set "platform_allow" may have no overlap with the default
platforms and thus thoses testcases will not get built.

Change the platform selection logic in these cases to set the platforms
to the list of allowed platforms ("platform_allow") in the testcase.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2021-04-09 14:54:34 -04:00
Dominik Ermel 96bfaffc02 runners: nrfjprog: Fix nrfjprog after changes to BuildConfiguration
The commit fixes problem with nrfjprog runner for west flash,
that has been introduced with latest changes to BuildConfiguration
class.

Signed-off-by: Dominik Ermel <dominik.ermel@nordicsemi.no>
2021-04-09 16:34:01 +02:00
Dominik Ermel e12e377e72 scripts: runners: Modify BuildConfiguration._parse to iterate over file
The method has been changed to iterate directly over file instead of
preloading it.

Signed-off-by: Dominik Ermel <dominik.ermel@nordicsemi.no>
2021-04-08 16:38:28 +02:00
Dominik Ermel 0dfd191e04 scripts: west_commands: Remove no longer needed load_dot_config(path)
The functionality of the function is now covered by BuildConfiguration
class.

Signed-off-by: Dominik Ermel <dominik.ermel@nordicsemi.no>
2021-04-08 16:38:28 +02:00
Dominik Ermel 09980d6a50 scripts/west sign: Use BuildConfiguration instead of load_dot_config
The ImgtoolSigner.sign() is passed a BuildConfiguration object instance
that holds all of Kconfig options that have been defined for a build;
yet it has been reading the .config file once again, with the
load_dot_config, to check some of these options.
The commit replaces the code that has been using the load_dot_config
obtained data with the code that uses the BuildConfiguration object.

Signed-off-by: Dominik Ermel <dominik.ermel@nordicsemi.no>
2021-04-08 16:38:28 +02:00
Dominik Ermel cabb7d69ee scripts: runners: Copy load_dot_config to BuildConfiguration.get()
The body of load_dot_config method has been reimplemented in
BuildConfiguration.get(), replacing the previous code.

Signed-off-by: Dominik Ermel <dominik.ermel@nordicsemi.no>
2021-04-08 16:38:28 +02:00
Dominik Ermel 185d695986 scripts: runners: Add BuildConfiguration.path attribute to class
The BuildConfiguration.path attribute has been added that stores path
to .config file which has been used as source for options collected to
BuildCOnfiguration object.

Signed-off-by: Dominik Ermel <dominik.ermel@nordicsemi.no>
2021-04-08 16:38:28 +02:00
Gerard Marull-Paretas 274f44d272 doc: remove docutils dependency
docutils is a core dependency of Sphinx. It is only used by some of our
custom Sphinx extensions.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2021-04-08 15:21:40 +02:00
Pieter De Gendt b6979736ca modules: nanopb: introduce new module
Add the nanopb library and generator tools as a module.
Nanopb is a small code-size Protocol Buffers implementation in ansi C.
It is especially suitable for use in microcontrollers,
but fits any memory restricted system.

Nanopb home: https://jpa.kapsi.fi/nanopb/

Signed-off-by: Pieter De Gendt <pieter.degendt@basalte.be>
2021-04-08 14:57:55 +02:00
Flavio Ceolin a632c52f50 ci: guideline: Run script to check rule 14.4
Run script to check violations to rule 14.4

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2021-04-08 07:56:28 -04:00
Flavio Ceolin a025fb2747 coccinelle: Add script to check rule 14.4
Controlling expression shall have essentially boolean
type.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2021-04-08 07:56:28 -04:00
Flavio Ceolin 0bbe5e4fda ci: coccinelle: Suppress pylint false positive
Suppress pylint false positive on 'sh' library.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2021-04-08 07:56:28 -04:00
Flavio Ceolin 58fdc01ca0 ci: coccinelle: Add support to write to a file
Add parameter to write the script output to a file.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2021-04-08 07:56:28 -04:00
Flavio Ceolin a869da55c6 scripts: coccinelle: Add guideline number on a script output
Add information about the guideline number in the script checking a
identifier length.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2021-04-08 07:56:28 -04:00
Flavio Ceolin 8552542f6e ci: Apply coccinelle scripts in git diffs
This scripts receives the same parameter of what_changed.py. And run
coccinelle scripts for code guideline compliance in the given git
commits. e.g: ./guideline_check.py --commits origin/master..HEAD

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2021-04-08 07:56:28 -04:00
Flavio Ceolin e8fdac3be0 scripts: coccinelle: Check rule 21.2
This coccinelle script can check some violations for rule 21.2.
Currently it is checking the follow reserved names:

"remove", "rewind", "malloc", "free", "exp", "signal"

It can easily be extended in the future though.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>

Rebnase
2021-04-08 07:56:28 -04:00
Flavio Ceolin 0058c02252 scripts: coccinelle: Check rule 5.7
This coccinelle script can check some violations for rule 5.7.
It can identify things like:

struct device *device

But it is not capable to identify:

struct test { ... }

...

int test;

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2021-04-08 07:56:28 -04:00
Gerard Marull-Paretas e14b44d720 scripts: ci: twister_ignore: remove old reference to .known-issues
Remove old reference to the .known-issues folder.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2021-04-08 09:55:16 +02:00
Gerard Marull-Paretas 5f42a3d40b doc: flexibilize sphinx-rtd-theme allowed versions
Allow versions starting from 0.5.2 up to < 1.0, be the next major
releases. Major releases tend to include breaking changes, so better
restrict such releases until they are manually tested for potential
regressions.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2021-04-07 17:39:43 -04:00
Gerard Marull-Paretas 2acc54c92e doc: pin sphinx_rtd_theme version
We have recently been hit by an issue between docutils and
sphinx_rtd_theme. The problem is fixed since release 0.5.2. In order to
avoid similar problems in the future, pin the theme version.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2021-04-07 12:12:07 -04:00
Kumar Gala 6e01c6abb6 scripts: edt: Add support for include property filtering
Add the ability to filter which properties get imported when we do an
include.  We add a new YAML form for this:

include:
  - name: other.yaml
    property-blocklist:
      - prop-to-block

or

include:
  - name: other.yaml
    property-allowlist:
      - prop-to-allow

These lists can intermix simple file names with maps, like:

include:
  - foo.yaml
  - name: bar.yaml
    property-allowlist:
      - prop-to-allow

And you can filter from child bindings like this:

include:
  - name: bar.yaml
    child-binding:
      property-allowlist:
        - child-prop-to-allow

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-04-07 14:16:33 +02:00
Gerard Marull-Paretas 96c7d10a15 doc: use warnings_filter
Replace current filter setup with the warnings_filter Sphinx extension.
Note that current regexes have been simplified to make them more
readable while keeping the most relevant information.

Sphinx warnings are now treated as errors (-W).

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2021-04-07 14:15:29 +02:00
Torsten Rasmussen d33fc38e09 scripts: menuconfig: proper handling of NULL character as input
Fixes: #33212

Upstream PR: https://github.com/ulfalizer/Kconfiglib/pull/103

Ignoring when user inputs NULL in a text field.
menuconfig exits with a python stack trace if NULL is provided as input
character, therefore ignore NULL as an input character to prevent this
behaviour.

A NULL character may be given accidentally by the user through the
following ways:
- Pressing `Win` key on keyboard (Windows only)
- Pressing `<CTRL>-@` / `<CTRL>-2`.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2021-04-07 10:19:14 +02:00
Mikkel Jakobsen 3fe785d4a7 scripts: west sign: fix devicetree module include
PR #33746 introduced changes to the devicetree python file
that requires changes in the python code that imports the
devicetree module.

This was omitted in the west sign command implementation.

Signed-off-by: Mikkel Jakobsen <mikkel.aunsbjerg@prevas.dk>
2021-04-06 14:54:02 -05:00
Martí Bolívar 5332847644 dts: separate DT libraries from gen_defines.py
We are now in the process of extracting edtlib and dtlib into a
standalone source code library that we intend to share with other
projects.

Links related to the work making this standalone:

    https://pypi.org/project/devicetree/
    https://python-devicetree.readthedocs.io/en/latest/
    https://github.com/zephyrproject-rtos/python-devicetree

This standalone repo includes the same features as what we have in
Zephyr, but in its own 'devicetree' python package with PyPI
integration, etc.

To avoid making this a hard fork, move the code that's being made
standalone around in Zephyr into a new scripts/dts/python-devicetree
subdirectory, and handle the package and sys.path changes in the
various places in the tree that use it.

From now on, it will be possible to update the standalone repository
by just recursively copying scripts/dts/python-devicetree's contents
into it and committing the results.

This is an interim step; do NOT 'pip install devicetree' yet.
The code in the zephyr repository is still the canonical location.

(In the long term, people will get the devicetree package from PyPI
just like they do the 'yaml' package today, but that won't happen for
the foreseeable future.)

This commit is purely intended to avoid a hard fork for the standalone
code, and no functional changes besides the package structure and
location of the code itself are expected.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-04-02 08:28:12 -05:00
Maciej Perkowski c2df1aaec6 twister: Fix removing dead code
Remove dead code from twisterlib.py which is a copy of code
from twister script. This redundancy causes pylint errors in
compliance checks.

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2021-04-01 11:02:39 -04:00
Pieter De Gendt f282c93b9a scripts: west_commands: runners: jlink check for WSL
The jlink runner performs a version check which is skipped on
Windows. If running inside WSL we also need to skip.

Signed-off-by: Pieter De Gendt <pieter.degendt@basalte.be>
2021-03-30 18:45:25 -05:00
Pieter De Gendt ff5bb8d916 scripts: west_commands: runners: Custom jlink gdb host
Allow a custom (remote) gdb host passed as a debug argument
which tries to connect to an existing one instead of creating one.

This also allows to run the gdb server outside of a WSL
in Windows and the debugger inside of the WSL environment.

Signed-off-by: Pieter De Gendt <pieter.degendt@basalte.be>
2021-03-30 18:45:25 -05:00
Gerard Marull-Paretas 681262d1d9 scripts: checkpatch: do not require top-level Makefile
checkpatch requires a Makefile to be present at the top-level directory.
Remove this requirement.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2021-03-29 10:59:28 -04:00
Martí Bolívar 2dca9f4ade dts: edtlib: fix child bindings which have compatibles
Whenever a child-binding: dict has a compatible, we ought to make
that available in EDT._compat2binding. If we don't, we can't look it
up later.

This has adverse effects like missing child bindings which describe
buses.

Fixes: #32071

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-03-25 09:37:42 -05:00
Carlo Caione d804ee3bf0 gen_kobject_list.py: Add support for 64-bit addresses
Currently the python script is assuming we only have objects on 32-bit
addresses. This is obviously wrong for 64-bit platforms. Fix this.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2021-03-25 14:07:50 +01:00
Martí Bolívar 524853ecfa runners: nrfjprog: workaround for multi-core hex files
The nrfjprog utility is not capable of flashing a hex file which
affects the flash memories of both coprocessors of the nRF53 family of
SoCs.

However, the user is capable of creating such a hex file using the
HEX_FILES_TO_MERGE build system variable.

An example use case is to build a bluetooth controller application for
the network core, then use the zephyr.hex file in that build directory
as the HEX_FILES_TO_MERGE argument for a separate Bluetooth
application build targeting the app core.

Work around this by detecting the situation and doing the right thing
by splitting the hex file back up again, even if thats a bit awkward.
Splitting the hex into app and network core components allows them to
be flashed separately. This is the only way we can get the job done
with nrfjprog.

This is arguably nicer since there's just one 'west flash' invocation.
At least in the use case named above, you wouldn't need to rebuild the
controller application very often, so this is a simpler user workflow.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-03-19 09:52:13 -04:00
Carles Cufi d2465d6268 debug: Move away from openocd and make it generic
The exported structures that were originally introduced for OpenOCD have
since then been reused for other debugger plugins, including PyOCD and
Segger J-Link.

Rename the Kconfig option and the implementation from openocd to debug
thread info, so that it reflects the fact that this is no longer
specifically tied to OpenOCD.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2021-03-17 14:43:01 -05:00
Maciej Perkowski e37d12e631 Twister: Refactor Test(Harness).handle()
The function `handle()` from class `Test(Harness)` had most of its
code being the direct copy of the code from
`Harness.process_test(line)`. This patch replaces the copied code
with the call for `process_test(line)` and removes repetition of
this function call.

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2021-03-17 15:18:06 -04:00
Martí Bolívar 847ba43df9 scripts: run_common: improve error handling for missing runner
Catch the ValueError returned by get_runner_cls() and turn it in to a
log.die().

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-03-17 11:30:45 +01:00
Martí Bolívar d61545b513 scripts: runners: handle failure to import individual runners
We need all available runners to be defined as subclasses of
runners.core.ZephyrBinaryRunner in order to be able to look them up by
name at runtime. We do this by importing them from runners.__init__.

This process periodically fails when some runner or other cannot be
imported, usually because it is trying to import something outside of
stdlib and not handling ImportError.

Rather than letting this bring down the entire Python process, catch
and log the error. Sort the list again while we're here.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-03-17 11:30:45 +01:00