Commit Graph

412 Commits

Author SHA1 Message Date
Stephanos Ioannidis 0d9fea87bb ci: Switch to CI image v0.24.11
This commit updates the CI workflows to use the CI image v0.24.11 that
includes BabbleSim v1.1.2, which improves simulation stability and
fixes the intermittent failures currently reported by the Bluetooth
test workflow.

Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
2023-02-11 22:20:03 +09:00
Stephanos Ioannidis d56ed39e20 ci: Switch to CI image 0.24.10
This commit updates the CI workflows to use the CI image 0.24.10, in
order to pull in a more recent version of BabbleSim (v1.1.1) and an
additional thrift-compiler dependency for the Apache Thrift module.

Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
2023-02-09 10:21:05 +09:00
Anas Nashif 66fa169623 ci: do not hardcode es server in script
Use environment variable instead of hardcoding server in script. This
will allow reuse.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-02-03 20:53:20 +09:00
Anas Nashif 2ff466b225 ci: use elasticsearch instead of opensource
Set the right key for elasticsearch.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-01-28 01:48:27 +09:00
Anas Nashif fd702ebb34 ci: push test results to elasticsearch instance
Move away from opensearch and switch to elasticsearch for reporting.
The results will be available at https://kibana.zephyrproject.io/

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-01-27 22:57:26 +09:00
Anas Nashif a4b2d58d3f Revert "twister: Update path structure for tests"
This reverts commit 21eb27c5c0.

This change has been a source of much trouble and ends up preventing us
from tracking test results across tree. It uses the repo name as the
namespace, and that is not always the same and does not have to be
called 'zephyr', depending on where you are running and in which
environment.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-01-24 20:21:57 -05:00
Gerard Marull-Paretas a003b725bb ci: doc-build: change Doxygen download URL
It looks like versions >= 1.9.2 have disappeared from the Sourceforge
website. Since they use GH releases since 1.9.6, the version we use,
let's just update download URL.

Ref. https://github.com/doxygen/doxygen/issues/9801

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2023-01-19 17:13:36 +01:00
Fabio Baltieri cf39a737d9 ci: west_cmds: add pyserial
Add the pyserial Python module so that it can be used for unit testing.

Signed-off-by: Fabio Baltieri <fabio.baltieri@gmail.com>
2023-01-19 10:16:00 +01:00
Al Semjonovs 90a86688ef ci: west_cmds: Add ply for twister support
Install ply for twister support

Signed-off-by: Al Semjonovs <asemjonovs@google.com>
2023-01-13 15:39:04 -08:00
Gerard Marull-Paretas 036e4f2902 ci: doc-build: use Doxygen 1.9.6
Use the latest release of Doxygen, 1.9.6.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2023-01-10 10:27:54 +01:00
Fabio Baltieri 9d3681c10c scripts: compliance: add support for YAMLLint
Add a YAMLLint compliance check that uses the yamllint package to report
linting error on YAML files.

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2023-01-04 17:29:23 +01:00
Fabio Baltieri dd462028b3 yamllint: indentation: MAINTAINERS and workflows
Fix the indentation on the remaining YAML files.

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2023-01-04 14:23:53 +01:00
Fabio Baltieri 7db1d17ee3 yamllint: fix all yamllint line-length errors
Fix all line-length errors detected by yamllint:

yamllint -f parsable -c .yamllint $( find -regex '.*\.y[a]*ml' ) | \
  grep '(line-length)'

Using a limit is set to 100 columns, not touching the commandlines in
GitHub workflows (at least for now).

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2023-01-04 01:16:45 +09:00
Fabio Baltieri 7dd902d035 yamllint: fix all yamllint comments-indentation errors
Fix all comments-indentation errors detected by yamllint:

yamllint -f parsable -c .yamllint $( find -regex '.*\.y[a]*ml' ) | \
  grep '(comments-indentation)'

This checks that the comment is aligned with the content.

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2023-01-04 01:16:45 +09:00
Fabio Baltieri bd4cdde4b0 yamllint: fix all yamllint colons, commas and empty-lines errors
Fix all colons and commas errors detected by yamllint:

yamllint -f parsable -c .yamllint $( find -regex '.*\.y[a]*ml' ) | \
  grep '(brackets)'

yamllint -f parsable -c .yamllint $( find -regex '.*\.y[a]*ml' ) | \
  grep '(commas)'

yamllint -f parsable -c .yamllint $( find -regex '.*\.y[a]*ml' ) | \
  grep '(empty-lines)'

Default config is no space before, one space after, max 2 empty lines.

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2023-01-04 01:16:45 +09:00
Fabio Baltieri f39f04f232 yamllint: fix all yamllint brackets errors
Fix all brackets errors detected by yamllint:

yamllint -f parsable -c .yamllint $( find -regex '.*\.y[a]*ml' ) | \
  grep '(brackets)'

Default config is to have no spaces inside brackets, changed few
documentation strings as well that refered to lists even though the
linter does not care about those.

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2023-01-04 01:16:45 +09:00
Fabio Baltieri 5c32300861 yamllint: fix all yamllint truthy errors
Fix all thruthy errors detected by yamllint:

yamllint -f parsable -c .yamllint $( find -regex '.*\.y[a]*ml' ) | \
  grep '(truthy)'

This only accepts true/false for boolean properties. Seems like python
takes all sort of formats:

https://github.com/yaml/pyyaml/blob/master/lib/yaml/constructor.py#L224-L235

But the current specs only mention "true" or "false"

https://yaml.org/spec/1.2.2/#10212-boolean

Which is the standard yamllint config.

Excluding codeconv and workflow files, as some are using yes/no instead
in the respective documentation.

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2023-01-04 01:16:45 +09:00
Fabio Baltieri e01d8dbbd7 actions: manifest: update action-manifest
Update action-manifest to the latest commit, pick up:

a6d0c6e action: match revisions with a refs/ prefix

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2022-12-27 15:25:47 +01:00
Stephanos Ioannidis e11d82ba31 ci: doc-build: Disable PDF documentation build for pull requests
This commit disables the PDF documentation build for pull request CI
runs because the HTML documentation build already validates the
documentation changes and the PDF build takes significantly longer than
the HTML build.

Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
2022-12-14 01:13:45 +09:00
Stephanos Ioannidis 264afa8ae7 ci: doc-build: Increase PDF documentation build timeout
This commit increases the timeout for PDF documentation build job from
45 minutes to 60 minutes because it can occasionally take more than 45
minutes on the GitHub Actions hosted runners.

Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
2022-12-14 01:13:45 +09:00
Carles Cufi f70eddce5f github: workflows: compliance: Ensure no merge commits
Add a simple one-liner that checks for "0" in the count of merge
commits. If a merge commit is present the output will be "1"
or higher, failing the job.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-30 12:06:03 -05:00
Stephanos Ioannidis e872ef57db ci: bluetooth-tests: Run with GNU parallel
This commit updates the bluetooth-tests workflow to install the GNU
parallel utility to enable parallel execution of the Bluetooth tests.

Note that the Bluetooth test script automatically parallelises the
tests when `parallel` is available in the PATH; otherwise, it
sequentially executes the tests.

Signed-off-by: Stephanos Ioannidis <stephanos.ioannidis@nordicsemi.no>
2022-11-27 03:41:11 +09:00
Stephanos Ioannidis 22d043dad3 ci: bluetooth-tests: Use zephyr-runner-linux-x64-4xlarge
This commit updates the bluetooth-tests workflow to use the
zephyr-runner-linux-x64-4xlarge runner, which has 16 vCPU and 32 GiB
RAM, and the embedded repository cache.

Signed-off-by: Stephanos Ioannidis <stephanos.ioannidis@nordicsemi.no>
2022-11-27 03:41:11 +09:00
Stephanos Ioannidis 5155907d7f ci: Update to Zephyr SDK 0.15.2
This commit updates the CI workflows to use the Zephyr SDK 0.15.2 for
building and testing Zephyr in the CI.

Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
2022-11-24 07:35:13 -05:00
Stephanos Ioannidis 7e008d2b18 ci: Switch to CI image 0.24.6
This commit updates the CI workflows to use the CI image 0.24.6, in
order to pull in the Zephyr SDK 0.15.2 release.

Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
2022-11-24 07:35:13 -05:00
Carles Cufi 090572c801 github: compliance: Remove maintainers check
The assigner.yml workflow already performs the same checks.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-21 11:02:39 -05:00
Carles Cufi 5ed037023b github: workflows: Avoid listing tests manually
Instead of listing each test manually, use the script itself to list
them:

- When invoking it just exclude the one we don't want by default
- When parsing the output files, use the -l output as a list

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-21 11:02:39 -05:00
Carles Cufi 95053c8358 scripts: compliance: Fix DT Bindings check name
Check names must not contain spaces, since they are passed via `-m` in
the command-line.

Also update .gitignore and compliance.yml.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-21 11:02:39 -05:00
Anas Nashif 23f0578042 ci: twister: checkout tree when merging results
Needed for the script used to upload results to opensearch.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-11-21 14:40:53 +01:00
Gerard Marull-Paretas 8c1aa8aa61 ci: doc: use Ubuntu 22.04
Use the latest Ubuntu image. There's no special reason for this change
other than staying up-to-date.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2022-11-19 14:17:26 +09:00
Gerard Marull-Paretas 61f252fe33 ci: doc-build: remove unnecessary libclang dependency
Doxygen 1.9.4 no longer needs libclang.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2022-11-19 14:17:26 +09:00
Stephanos Ioannidis 3d8e651992 ci: Clone cached Zephyr repository with shared objects
In the new ephemeral Zephyr runners, the cached repository files are
located in a foreign file system and Git clone operation cannot create
hard-links to the cached repository objects, which forces the Git clone
operation to copy the objects from the cache file system to the runner
container file system.

This commit updates the CI workflows to instead perform a "shared
clone" of the cached repository, which allows the cloned repository to
utilise the object database of the cached repository.

While "shared clone" can be often dangerous because the source
repository objects can be deleted, in this case, the source repository
(i.e. cached repository) is mounted as read-only and immutable.

Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
2022-11-17 15:42:27 +09:00
Anas Nashif 1eb50d5ea3 ci: twister: fix typo
Fix typo in conditional allowing upload of test results on scheduled
events.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-11-14 18:08:15 +01:00
Maciej Perkowski 21eb27c5c0 twister: Update path structure for tests
With the patch a relative path between a test and west's project top
directory is always calculated instead of being relative to zephyr
base. The test names and results structur change by addition of one
level. This way the names/structure conventon for tests from zephyr and
modules is alligned.
For external tests ".." parts are replaced with "external".
This allows to maintain unified folder structure also for results of
external tests.
Without this patch the folder/name allignment is lost for
external and module test becaming depandant on a way how
twister was called.
.

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2022-11-13 17:45:09 -05:00
Carles Cufi 1951071f14 requirements: Add lxml as a requirement
junitparser has a bug that prevents it from working correctly when lxml
is not installed.

See this GH issue for more information:
https://github.com/weiwei/junitparser/issues/99

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-13 06:25:59 -05:00
Carles Cufi 8275d21a6c scripts: compliance: Add GitHub-compatible annotation support
Add support for GitHub workflow annotations, so that errors are
displayed inline with the Pull Request diff.

More info here:
https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#about-workflow-commands

Enable the new option in the compliance GitHub action.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-13 06:25:59 -05:00
Anas Nashif f98a240537 ci: twister: capture footprint data on push/schedule
Capture footprint data on push/schedule so we visualize footprint in
dashboards.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-11-10 13:45:00 -06:00
Stephanos Ioannidis 08316429f5 ci: Add workflow for cleaning up stale queued workflow runs
This commit adds a workflow to periodically clean-up the stale workflow
runs that are stuck in 'queued' state.

These stale workflow runs can result from occasional GitHub webhook
delivery mishaps and may affect the CI autoscaler decision making
process.

Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
2022-11-09 11:12:48 +09:00
Anas Nashif e2cd634185 ci: also upload results of twiser a week full builds
Upload results of full runs we do twice a week to opensearch.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-11-07 14:20:02 -06:00
Stephanos Ioannidis 0aded739d3 ci: twister: Do not run on third-party repositories
This commit updates the twister workflow to only run on the
repositories in the zephyrproject-rtos organisation because this
workflow requires a custom runner type only available within the
zephyrproject-rtos organisation.

Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
2022-11-05 19:12:21 +09:00
Stephanos Ioannidis 863a886e63 ci: clang: Do not run on third-party repositories
This commit updates the clang workflow to only run on the repositories
in the zephyrproject-rtos organisation because this workflow requires
a custom runner type only available within the zephyrproject-rtos
organisation.

Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
2022-11-05 19:12:21 +09:00
Stephanos Ioannidis dfabedf262 ci: codecov: Do not run on forks
This commit updates the codecov workflow to not run on the fork
repositories because this workflow requires a custom runner type only
available within the zephyrproject-rtos organisation and there is not
much point in running this workflow on the repositories other than the
main Zephyr repository.

Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
2022-11-05 19:12:21 +09:00
Stephanos Ioannidis afdf012a63 ci: manifest: Drop west label for manifest updates
This commit updates the manifest workflow to not label the pull
requests updating the west manifest (`west.yml`) with the `west` label
because the `manifest` label alone is enough for this purpose and the
`west` label can be easily confused with the `area: West` label.

Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
2022-11-04 15:51:17 +01:00
Anas Nashif 662aed81d2 actions: Upload test results to opensearch service
On push, upload test results to opensearch for analysis and reporting.
Goal is to use this data to understand test coverage better and use this
services for all test reporting, also for results coming from testing on
hardware.
Opensearch is currenly being used for evaluation, we are considering
the switch to elasticsearch later.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-11-02 10:26:43 -04:00
Stephanos Ioannidis 7272159c11 ci: Limit workflow scope branches
This commit updates the CI workflows that trigger on both push and pull
request events to limit their event trigger scope to the main and the
release branches.

This prevents these workflows from simultaneously triggering on both
push and pull request events when a pull request is created from an
upstream branch to another upstream branch (e.g. pull requests from
the backport branches to the release branches).

Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
2022-11-02 22:10:19 +09:00
Stephanos Ioannidis 69ef22b717 ci: codecov: Clone cached Zephyr repository
This commit updates the codecov workflow to pre-clone the Zephyr
repository from the runner repository cache.

Note that the `origin` remote URL is reconfigured to that of the GitHub
Zephyr repository because the checkout action attempts to delete
everything and re-clone otherwise.

Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
2022-10-31 08:08:35 +09:00
Stephanos Ioannidis 1bfd46bc71 ci: codecov: Use zephyr-runner
This commit updates the codecov workflow to use the new Kubernetes-
based zephyr-runner.

Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
2022-10-31 08:08:35 +09:00
Stephanos Ioannidis 408473f85d ci: clang: Remove obsolete clean-up steps
The repository clean-up steps are no longer necessary because the new
zephyr-runner is ephemeral and does not contain any files from the
previous runs.

Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
2022-10-31 08:08:35 +09:00
Stephanos Ioannidis 6c676827e7 ci: clang: Clone cached Zephyr repository
This commit updates the clang workflow to pre-clone the Zephyr
repository from the runner repository cache.

Note that the `origin` remote URL is reconfigured to that of the GitHub
Zephyr repository because the checkout action attempts to delete
everything and re-clone otherwise.

Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
2022-10-31 08:08:35 +09:00
Stephanos Ioannidis e81964bb90 ci: clang: Use zephyr-runner
This commit updates the clang workflow to use the new Kubernetes-based
zephyr-runner.

Note that the repository cache directory path has been changed for the
new runner.

Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
2022-10-31 08:08:35 +09:00