This PR changes the Labeling of PRs so that the Architecture (arm, risc-v, xtensa, ...) is specified in the Board Label. Previously the CI Workflow would set Generic Board Labels like "Area: Board support". Now it will set the Arch-Specific Board Label like "Board: arm".
The Board Label will be used by the upcoming CI Build Rules to skip unnecessary builds. So "Board: arm" will enable only the builds for `arm-01` to `arm-14`. This is explained here: https://github.com/apache/nuttx/issues/13775
CI Build Job `arm-05` (runtime 2 hours) has become the Performance Bottleneck for CI Workflow. That's because `arm-05` builds too many targets for nRF, RP2040, SAM 3, SAM A and SAM D. This PR splits `arm-05` into multiple smaller jobs, to reduce the CI Build Duration.
Before the PR: `arm-05` is overloaded, build requires 2 hours
- `arm-05` (2 hours): nRF, RP2040, SAM 3, SAM A, SAM D
- `arm-06` (56 mins): STM32 [a-m]*
After the PR: `arm-05` is offloaded (to `arm-06` and `arm-07`), completes within 1 hour
- `arm-05` (47 mins): nRF
- `arm-06` (1 hour): Reserve for RP2040 exclusively
- `arm-07` (1 hour 15 mins): SAM 3, SAM A, SAM D, STM32 [a-m]*
Build Jobs are sorted by Target Name. So we cascade the changes and rename the Build Jobs: `arm-07` becomes `arm-08`, `arm-08` becomes `arm-09` etc. Then `arm-13` becomes a new job `arm-14`. (Which we added to `build.yml`)
Performance of `arm-05` is discussed in https://github.com/apache/nuttx/issues/13775 and https://github.com/apache/nuttx/issues/12773
This PR continues to enhance the CI Workflow, to skip the unnecessary NuttX Builds. The PR will update the CI Build Workflow `build.yml`, to call the Refactored Build Rules in `arch.yml` (which is a Reusable Workflow).
The original rules were migrated from `build.yml` to `arch.yml`:
- We target only the Simple PRs: One Arch Label + One Size Label (e.g. "Arch: risc-v, Size: XS")
- For "Arch: risc-v": Build `risc-v-01`, `risc-v-02`
- For "Arch: xtensa": Build `xtensa-01`, `xtensa-02`
- The above rules apply when the PR is Created or Modified
- When the PR is Merged: All targets shall be built
This PR applies the New and Updated Rules defined in `arch.yml`:
- For "Arch: arm": Build `arm-01`, `arm-02`, ...
- For "Arch: arm64": Build `other`
- For "Arch: simulator": Build `sim-01`, `sim-02`
- For "Arch: x86_64": Build `other`
- For Simple PRs (One Arch Label + One Size Label): Skip the macOS and Windows builds (`macos`, `macos/sim-*`, `msys2`) since these builds are costly and slow
- Except for "Arch: Simulator", which will enable the macOS Builds for `sim-01` and `sim-02`
- If GitHub CLI Fails: Build all targets
The code is explained here: https://github.com/apache/nuttx/issues/13775
This PR continues to enhance the CI Workflow, to skip the unnecessary NuttX Builds. The changes in this PR will not take effect until the next PR, which will switch `build.yml` to use the rules in this PR.
In this PR, we refactor the CI Build Rules into a separate Reusable Workflow `arch.yml`. The original rules were migrated to `arch.yml`:
- We target only the Simple PRs: One Arch Label + One Size Label (e.g. "Arch: risc-v, Size: XS")
- For "Arch: risc-v": Build `risc-v-01`, `risc-v-02`
- For "Arch: xtensa": Build `xtensa-01`, `xtensa-02`
- The above rules apply when the PR is Created or Modified
- When the PR is Merged: All targets shall be built
New and Updated Rules:
- For "Arch: arm": Build `arm-01`, `arm-02`, ...
- For "Arch: arm64": Build `other`
- For "Arch: simulator": Build `sim-01`, `sim-02`
- For "Arch: x86_64": Build `other`
- For Simple PRs (One Arch Label + One Size Label): Skip the macOS and Windows builds (`macos`, `macos/sim-*`, `msys2`) since these builds are costly and slow
- Except for "Arch: Simulator", which will enable the macOS Builds for `sim-01` and `sim-02`
- If GitHub CLI Fails: Build all targets
The code is explained here: https://github.com/apache/nuttx/issues/13775
When building a branch like `releases/12.7`, the CI Workflow incorrectly checks out the `master` branch of `nuttx-apps`, instead of `releases/12.7`. This PR fixes a typo in `apps_ref`, to check out the correct branch.
This PR proposes to enhance the CI Workflow, to skip the unnecessary NuttX Builds. Currently, NuttX Devs wait for the CI Builds to complete across All Architectures (Arm32, Arm64, RISC-V, Xtensa), even though they modified a Single Architecture. With this PR, the CI Workflow will build only the Modified Architecture.
The solution uses the Arch Labels for PRs. We target only the Simple PRs: One Arch Label + One Size Label (e.g. "Arch: risc-v, Size: XS")
- If "Arch: arm / arm64" is the only non-size label, then we build `other`, `arm-01`, `arm-02`, ...
- If "Arch: risc-v" is the only non-size label, then build `risc-v-01`, `risc-v-02`
- If "Arch: xtensa" is the only non-size label, then build `xtensa-01`, `xtensa-02`
- The above rules apply when the PR is Created or Modified
- When the PR is Merged: All targets shall be built
The code is explained here: https://github.com/apache/nuttx/issues/13775
This change adds a specifier to github lables created with "new issue":
- for labels related to architecture specifier is "Arch: xxx"
- for labels related to issue type specifier is "Type: xxx"
- for labels related to host OS type specifier is "OS: xxx"
- for labels related to issue area type specifier is "Area: xxx"
The simple improvement is designed to speed up compilation and reduce download errors on github and local.
Added a folder nxtmpdir for storing third-party packages
nuttxworkspace
|
|- nuttx
|- apps
|- nxtmpdir
tools/Unix.mk:
added export NXTMPDIR := $(WSDIR)/nxtmpdir
tools/configure.sh:
added option -S creates the nxtmpdir folder for third-party packages.
tools/Config.mk:
added macro
CLONE - Git clone repository.
CHECK_COMMITSHA - Check if the branch contains the commit SHA-1.
tools/testbuild.sh:
added option -S
For now I added in the folder this package
ESP_HAL_3RDPARTY_URL = https://github.com/espressif/esp-hal-3rdparty.git
ARCH
arch/xtensa/src/esp32/Make.defs
arch/xtensa/src/esp32s2/Make.defs
arch/xtensa/src/esp32s3/Make.defs
arch/risc-v/src/common/espressif/Make.defs
arch/risc-v/src/esp32c3-legacy/Make.defs
but you can also add other packages (maybe also of apps)
This PR modifies NuttX CI and GitHub Actions, to comply with ASF Policy. Right now, every NuttX Pull Request will trigger 24 Concurrent Jobs (GitHub Runners), executing them in parallel: https://lupyuen.github.io/articles/ci
According to ASF Policy: We should run at most 15 Concurrent Jobs: https://infra.apache.org/github-actions-policy.html
Thus we'll cut down the Concurrent Jobs from 24 down to 15. That's 12 Linux Jobs, 2 macOS, 1 Windows. (Each job takes 30 mins to 2 hours)
(1) Right now our "Linux > Strategy" is a flat list of 20 Linux Jobs, all executed in parallel
(2) We change "Linux > Strategy" to prioritise by Target Architecture, and limit to 12 concurrent jobs
(3) So NuttX CI will initially execute 12 Build Jobs across Arm32, Arm64, RISC-V, Simulator and Xtensa. As they complete, NuttX CI will execute the remaining 8 Build Jobs (for Arm32).
(4) This will extend the Overall Build Duration from [2 hours](https://github.com/apache/nuttx/actions/runs/10817443237) to [2.25 hours](https://github.com/lupyuen4/ci-nuttx/actions/runs/10828246630)
(5) We'll also limit macOS Jobs to 2, Windows Jobs to 1
Currently concerns only arm.
tools/ci/testlist/msys2.dat:
At the moment I only added the board nucleo-l152re:nsh
.github/workflows/build.yml:
Enabled cmake for msys2
Template
Bug report
Report a bug to improve NuttX stability
Feature request
Request an enhancement for NuttX
General Help
Get general support regarding NuttX
Action
An action for automatically labelling issues
tools/testbuild.sh
added option -N Use CMake with Ninja as the backend.
updated help.
added date to print startup board
added HOST=Msys
github/workflows/build.yml
added option -N job docker
function bloaty() Add -D CMAKE_INSTALL_PREFIX="${NUTTXTOOLS}"/bloaty.
Improves workflow execution time because it is now cached.
Changed reference file to calculate the hash for key of actions/cache@v4. Now it is darwin.sh.
the check isn't really enabled and enforce before
due to a mass of false alarm, but recently it break
ci frequently, so it's better to remove it now.
Signed-off-by: Xiang Xiao <xiaoxiang@xiaomi.com>
This implements the changes required to genreate the pdf version
of the documentation. The pdf will be generated as a build artifact
attached to the build. This is the first patch to enable this
a follow on patch to the website repo will publish the pdf to the
NuttX website.
Lup Yuen Lee