Commit Graph

598 Commits

Author SHA1 Message Date
Marc Herbert 665817fe84 xtensa-build-zephyr.py: add flexibility to generate `tools/` only
Installing most of the tools/ directory does not technically require
building any platform, so add that possibility.

This is useful because the Jenkins-based CI builds the (userspace) tools
separately. So far it has been picking hardcoded source paths but of
course the HWMv2 transition just broke that:
https://github.com/thesofproject/sof/pull/8913

While it's too late this time (we want to keep CI able to test stable
branches for some time), this commit prepares a future where all CIs can
stop hardcoding Zephyr source paths and pick the output of the
xtensa-build-zephyr.py installer and indirection layer instead.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2024-04-22 18:26:09 +03:00
Guennadi Liakhovetski ed33af4d33 llext: add support for building relocatable modules
Add support for relocatable objects to the llext module build system.
In such builds no ELF segments are created, so we need to process all
sections individually.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
2024-04-16 15:30:02 +01:00
Marc Herbert 9d7c33adc3 xtensa-build-zephyr.py: make --deployable-build the default
Every Linux developer should use deployable builds by default.

Until Peter Ujfalusi's very recent work in this script, we had a
complete `/lib/firmware/` structure disconnect between the IPC4 output
of this script and the IPC4 expectations of the Linux kernel. To
workaround this disconnect, every CI and Linux developer used to
implement duplicate and inconsistent firmware deployment hacks.

People crafting sof-bin releases also had to organize IPC4 releases
manually, which was extremely error-prone and with limited test
coverage (Thanks Kai and Mengdong!)

Now that Peter gracefully fixed the layout, documented it in sof-docs
and implemented it in this script, the time for all Linux developers to
drop their inconsistent deployment hacks is overdue. All these hacks
must be replaced with a simple, one-line recursive copy which makes sure
the layout committed in version control is constantly tested by
everyone.

So, make deployable builds the new default.

The new default will also help with sof-bin releases, making sure they
use a well tested /lib/firmware/ layout.

The --no-deployable-build was recently introduced to help minimize
disruption and migration effort for people and automation who do NOT use
Linux. The `/lib/firmware/` directory structure is irrelevant outside
Linux (but everyone is of course free to choose it)

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2024-04-02 20:00:13 +03:00
Laurentiu Mihalcea f0740ed103 west.yml: update Zephyr to 1f55be8b42df
Contains the following squashed SOF commits:

nxp: imx8ulp: change SOC name to MIMX8UD7
zephyr: CMakeLists.txt use new `CONFIG_SOC_C` for 8ULP
cmake: update configs for NXP ADSP

and the following Zephyr patches affecting SOF:

951763939034 nxp: imx8ulp: change SOC name to MIMX8UD7
b8214b673970 dts: xtensa: nxp_imx8: add SAI1 node
a0e32f07ef76 dts: intel_adsp: ace: update host dma copy alignment
3fde2c50c6ef tracing: add intel ADSP memory window backend
6b9d01f995c7 intel_adsp/ace: power: No pending transaction before power gate
6ea749de5283 arch: rename arch_start_cpu() to arch_cpu_start()
b69d2486fee6 kernel: rename Z_KERNEL_STACK_BUFFER to K_KERNEL_STACK_BUFFER
1f55be8b42df nxp: imx8: change CONFIG_SOC_<name> to match the value
688fbb53aeb2 intel_adsp: ace: Fix sparse error

Signed-off-by: Laurentiu Mihalcea <laurentiu.mihalcea@nxp.com>
Signed-off-by: Iuliana Prodan <iuliana.prodan@nxp.com>
2024-04-02 13:13:07 +03:00
Marc Herbert 12ad82a756 xtensa-build-zephyr.py: checksum autoconf.h instead of configs.c
autoconf.h has strings, configs.c has not.

We can do this now since this $BOARD_REVISION fix:

https://github.com/zephyrproject-rtos/zephyr/commit/811a74c0199f

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2024-03-18 14:45:27 +02:00
Laurentiu Mihalcea 7cb15d722a scripts: xtensa-build-all: remove imx8ulp
XTOS is going to be deprecated on i.MX8ULP so prevent
building it by removing it from the list of supported
platforms from `xtensa-build-all.sh`. This is needed for
switching to Zephyr native drivers.

Signed-off-by: Laurentiu Mihalcea <laurentiu.mihalcea@nxp.com>
2024-03-15 13:14:15 +02:00
Marc Herbert 469c0a7e63 xtensa-build-zephyr.py: checksum newly generated rimage_config.toml file
Add new file generated since commit cf9a444bfc ("toml: modularise TOML
configuration")

Optional because it's not generated for all platforms.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2024-03-11 20:12:01 +02:00
Guennadi Liakhovetski 980b1c64f2 smart-amp-test: make a loadable module
Convert the smart-amp-test in its IPC4 version to a loadable LLEXT
module. Use an overlay configuration to select between monolithic and
modular builds.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
2024-03-08 16:25:28 +02:00
Kai Vehmanen 22b3518601 west.yml: update Zephyr to f9f44b6dcdd
Switch back to main Zephyr repository and commit f9f44b6dcdd.

This includes following squashed SOF commits that are
needed to adapt to HWMv2 changes in Zephyr:

zephyr: app: scripts: intel_adsp: change board names to HWMv2
zephyr: sof: update board name for HWMv2
zephyr: intel_adsp: Change ACE SoC name to HWMv2
app: boards: imx93: updates for zephyr hwmv2

Signed-off-by: Iuliana Prodan <iuliana.prodan@nxp.com>
Signed-off-by: Laurentiu Mihalcea <laurentiu.mihalcea@nxp.com>
Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
2024-03-08 13:16:58 +02:00
Marc Herbert 1b0c084fb9 xtensa-build-zephyr.py: clarify XTENSA_SYSTEM and XTENSA_CORE in --help
This has always been mysterious, try to shed a bit of light.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2024-03-01 16:45:59 +00:00
Marc Herbert a90ed21be3 xtensa-build-zephyr.py: use a triple-quoted string for the help epilog
Pure formatting simplification, zero logical change.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2024-03-01 16:45:59 +00:00
Marc Herbert eeb0f2ab36 xtensa-build-zephyr.py: simplify and clarify Cadence env vars
Remove unnecessary complexities in the Python code.

Document all variables in the comments.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2024-03-01 16:45:59 +00:00
Marc Herbert cb4c3ee74d xtensa-build-zephyr.py: fix message when XTENSA_TOOLS_ROOT is wrong
Like most error handling, the exception was never tested and the error
message was wrong.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2024-03-01 16:45:59 +00:00
Marc Herbert 582c81fab8 fuzz.sh: add -jobs parameter. Switch to built-in -max_total_time.
There was no need to use the external "timeout" command + some
complicated exit code logic, libFuzzer has that feature built-in. Switch
to it and delete a few good line.

Add an option to use the built-in, concurrent -jobs feature which saves
a massive amount of time. It has some quirks so do not enable it by
default.

Decrease verbosity by default.

Add in the help message the magic replay command that saves a ton of time.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2024-02-29 13:26:02 +00:00
Marc Herbert 0fee9d9630 scripts/fuzz.sh: add a couple debug -DCONFIG_ options
They shouldn't do much but they can't hurt.

See https://github.com/thesofproject/sof/pull/8632 for more details.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2024-02-28 15:39:56 +00:00
Laurentiu Mihalcea ac4cf6175f scripts: xtensa-build-all: remove imx8/imx8x
To prepare for XTOS deprecation and switch to Zephyr native
drivers on imx8/imx8x, remove the ability to build SOF
with XTOS on them.

Signed-off-by: Laurentiu Mihalcea <laurentiu.mihalcea@nxp.com>
2024-02-16 21:48:10 +02:00
Laurentiu Mihalcea a3701fea01 script: test-repro-build: change imx8 with AMD's rmb
XTOS is going to be deprecated on all NXP's platforms so
replace imx8 with AMD's rmb so that the CI test can be kept.

Signed-off-by: Laurentiu Mihalcea <laurentiu.mihalcea@nxp.com>
2024-02-16 21:48:10 +02:00
Marc Herbert e04289d6a9 scripts/fuzz.sh: add a -p "pristine" flag that deletes build-fuzz/
It's slower but safer when changing some parameters.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2024-02-07 12:50:08 +00:00
Marc Herbert dd8b9328a9 app/prj.conf: stop forcing CONFIG_OUTPUT_DISASSEMBLY=y
This reverts 8f6018057b ("app: enable OUTPUT_DISASSEMBLY for Zephyr
builds")

The disassembly is useful in some contexts but can't be parallelized and
takes an enormous amount of time. On my system OUTPUT_DISASSEMBLY
almost doubles the compilation time!

Quoting Zephyr commit cc5763344709:

> This on-by-default option takes the majority of "link" time. If people
> need the disassembly, they can turn it on.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2024-02-02 17:19:38 +02:00
Marc Herbert d324af52bd xtensa-build-zephyr.py: add new --no-deployable-build
Everyone should use deployable builds by default.  Don't switch the
default behavior yet but add a --no-deployable-build option in case
anyone is stuck.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2024-01-23 14:16:28 +02:00
Marc Herbert f4ce8b537b xtensa-build-zephyr.py: fail on --deployable-build + older options
From experience, no one scrolls up and looks at warnings that are not in
the current screen.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2024-01-15 15:33:25 +02:00
Marc Herbert e24ccd2c19 xtensa-build-zephyr.py: fix --help text for --fw-naming=AVS
Better late than never... zero functional change.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2024-01-15 15:33:25 +02:00
Marc Herbert c8eb21621a xtensa-build-zephyr.py: show symbolic links at the end
Not sure why I thought it would be complicated; I should have done this
much sooner.

Before:

```
build-sof-staging
|-- sof
|   +-- intel
|       +-- sof-ipc4
|           |-- adl
|           |   +-- community
|           |       +-- sof-adl.ri
|           |-- adl-n
|           |   +-- community
|           |       +-- sof-adl-n.ri
|           |-- arl
|           |   +-- community
|           |       +-- sof-arl.ri

```

After:

```
build-sof-staging
|-- sof
|   +-- intel
|       +-- sof-ipc4
|           |-- adl
|           |   +-- community
|           |       +-- sof-adl.ri  -> ../../tgl/community/sof-tgl.ri
|           |-- adl-n
|           |   +-- community
|           |       +-- sof-adl-n.ri  -> ../../tgl/community/sof-tgl.ri
|           |-- arl
|           |   +-- community
|           |       +-- sof-arl.ri  -> ../../mtl/community/sof-mtl.ri

```

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2024-01-15 13:27:56 +02:00
Peter Ujfalusi 98cc1d3da3 xtensa-build-zephyr.py: Add arl-s alias to mtl platform
ARL-S is using the same debugkey/community key binary as MTL.

Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
2024-01-02 12:35:14 +02:00
Peter Ujfalusi 1adb1e45d3 xtensa-build-zephyr.py: Add option to create deployable build
The default firmware file path is standardized among vendors as follows:
IPC3
    /lib/firmware/VENDOR/sof/
    ├── community
    │   └── sof-PLAT.ri
    ├── dbgkey
    │   └── sof-PLAT.ri
    └── sof-PLAT.ri
IPC4
    /lib/firmware/VENDOR/sof-ipc4/
    └── PLAT
        ├── community
        │   └── sof-PLAT.ri
        ├── dbgkey
        │   └── sof-PLAT.ri
        └── sof-PLAT.ri\n

Currently the binaries created by the build can only be used for direct
deployment on IPC3 platforms but if one builds different vendor firmwares
the files have to be manually picked and sorted out.
We have two flags: --fw-naming and --use-platform-subdir which can be
played with but still not going to produce deployable build.

Introduce a new flag: --deployable-build
With the flag specified all other modificators are going to be ignored and
the build will do the 'right thing' to create a directory structure which
can be deployed as it is to the target's firmware directory.

To achieve this several changes needed:
PlatformConfig:
- drop the name member and replace it with a vendor string
- add a flag to indicate IPC4 platforms
  Later a new option can be added if needed for platforms which can be
  IPC3 or IPC4

Ignore fw-naming and use-platform-subdir in case of deployable build. The
options will be reset to their default in case they are changed.

symlink_or_copy extended to handle relative symlinks when the target and
link is not under the same directory.

The --deployable-build is disabled by default, it has to be enabled to
create deployable build for now.

Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
2023-12-19 17:02:38 +00:00
Peter Ujfalusi 7e42c43c21 xtensa-build-zephyr.py: Print out the platform aliases in help
Print platform alias information to reduce the need for guessing when
trying to figure out what platform supports what platform.

Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
2023-12-19 17:02:38 +00:00
Peter Ujfalusi 38d447f3f9 xtensa-build-zephyr.py: Fill in missing platform aliases
The following aliases are missing:

tgl: adl-n, rpl
tgl-h: rpl-s
mtl: arl

Update the aliases list accordingly.

Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
2023-12-19 17:02:38 +00:00
Peter Ujfalusi 4d0da9896b xtensa-build-zephyr.py: Remove IPC4_CONFIG_OVERLAY from PlatformConfig
The IPC4_CONFIG_OVERLAY is no longer used, remove it.

Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
2023-12-19 17:02:38 +00:00
Marc Herbert 38a730b9c9 rebuild-testbench.sh: add incremental build command in help message
Add interactive menuconfig and incremental build commands in help
message of rebuild-testbench.sh

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2023-12-19 16:54:45 +00:00
Marc Herbert 27e5f40b9a rebuild-testbench.sh: error when unknown arguments are passed
Silently discarding user input is really bad.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2023-12-19 16:54:45 +00:00
Marc Herbert acc4f672db rebuild-testbench.sh: use cmake -B to simplify rebuild_testbench()
Cosmetic change.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2023-12-19 16:54:45 +00:00
Marc Herbert caf28abbcd rebuild-testbench.sh: rename BUILD_TESTBENCH_DIR to TESTBENCH_DIR
It's a source directory, not a build directory.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2023-12-19 16:54:45 +00:00
Marc Herbert 954b862017 rebuild-testbench.sh: move all constants at the top
Cosmetic change.

Also exit 0 when using -h; it's not a crime!

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2023-12-19 16:54:45 +00:00
Marc Herbert de28dd5fe5 Give fuzz.sh a `proper -h` help text
Long overdue.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2023-12-19 16:52:33 +00:00
Marc Herbert 03fe9d3149 fuzz.sh: use set -x to make the `west build ...` command visible
Users want to know what exactly failed.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2023-12-19 16:52:33 +00:00
Daniel Baluta af3fa413cb xtensa-build-zephyr.py: Re-add imx8ulp to -all
This re-adds imx8ulp to --all platform list. This was removed
in commit 7737efadf4 ("xtensa-build-zephyr.py: remove imx8ulp from --all")

Signed-off-by: Daniel Baluta <daniel.baluta@nxp.com>
2023-12-14 11:42:21 +02:00
Marc Herbert 3e09f233a4 set_xtensa_params.sh: add ZEPHYR_TOOLCHAIN_VARIANT
This helps with Zephyr and does not hurt anyone else.

Adjust rebuild-testbench.sh accordingly.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2023-12-13 22:17:35 +02:00
Marc Herbert 7b8068963c scripts: rename XTENSA_TOOLS_VERSION to TOOLCHAIN_VER as in Zephyr
Rename the variable name used by SOF script with the one expected by the
Zephyr build system for simplicity and consistency.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2023-12-13 22:17:35 +02:00
Marc Herbert d661aadd7c xtensa-build-zephyr.py: don't pass -c $plat.toml to rimage
Let west/sign.py find the appropriate .toml file.

This prepares for .toml modularization #8490

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2023-12-12 23:50:40 +02:00
Marc Herbert ded019bd46 xtensa-build-zephyr.py: remove IPC3/IPC4 "cavs" switch
Rename:
- tgl-cavs.toml   to tgl.toml
- tgl-h-cavs.toml to tgl-h.toml

Remove the IPC3/IPC4 switch added by commit 6f71808e3e
("xtensa-build-zephyr.py: add ipc4 build support for tgl")

This brings back consistency which is required for the .toml
split (#8490)

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2023-12-12 23:50:40 +02:00
Marc Herbert 3b8d872619 xtensa-build-zephyr.py: untabify XCC installation in --help message
This was broken from the start when converting the script from shell
script to Python.

A good reference was left in the non-Zephyr script which is still
active: `./scripts/xtensa-build-all.sh -h`

Fixes initial commit 1de3ef3675 ("Rewritten xtensa-build-zephyr.sh to
python")

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2023-12-11 12:06:40 +02:00
Marc Herbert 8fa93b1c86 xtensa-build-all.sh: stop copy/paste/diverge set_xtensa_param.sh
... source it instead.

We still have duplication with xtensa-build-zephyr.py for now but this
gets at least rid of bash duplication.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2023-12-05 17:29:37 +02:00
Marc Herbert 44d9acb72d set_xtensa_params.sh: add missing LNL, ACP_6_3 and vangogh
This adds support for:

   ./scripts/rebuild-testbench.sh -p lnl

Also clean-up, re-order and add comments to set_xtensa_params.sh

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2023-12-05 17:29:37 +02:00
Marc Herbert 330d73eec1 cmake: a few new add_local_sources[_ifdef]() compatibility macros
For reducing CMake duplication and centralization, see #8260

Not used yet.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2023-11-29 14:52:36 +00:00
Marc Herbert 7737efadf4 xtensa-build-zephyr.py: remove imx8ulp from --all
As of today's Zephyr commit 92fb8b223825, imx8ulp does not compile.

Do not remove any of its data from `xtensa-build-zephyr.py` but exclude
it from --all

Fixes:
```
scripts/xtensa-build-zephyr.py --all

-- Board: nxp_adsp_imx8ulp
No board named 'nxp_adsp_imx8ulp' found.

CMake Error at zephyr/cmake/modules/boards.cmake:167 (message):
  Invalid BOARD; see above.
Call Stack (most recent call first):
  cmake/modules/zephyr_default.cmake:129 (include)
  share/zephyr-package/cmake/ZephyrConfig.cmake:66 (include)
  share/zephyr-package/cmake/ZephyrConfig.cmake:92 (include_boilerplate)
  CMakeLists.txt:5 (find_package)
```

Fixes 61ccb4c8da ("scripts: zephyr: Add support to build sof for
imx8ulp")

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2023-11-20 10:57:34 +02:00
Marc Herbert ef563ebfe3 cmake/zephyr: decentralize src/init/
Adding all source files in a single, giant zephyr/CMakeLists.txt is
inconvenient and does not scale.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2023-11-08 16:53:28 +00:00
Seppo Ingalsuo 2006297ee2 Scripts: Testbench: Add xt-testbench build for MTL with xt-clang
Build for MTL needs other compiler xt-clang, so the COMPILER is
set by default to xt-xcc and changed for MTL in set_xtensa_params.sh
where also core and tools version is defined.

Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
2023-11-08 16:39:29 +00:00
Zhang Peng 61ccb4c8da scripts: zephyr: Add support to build sof for imx8ulp
Add support to build sof for imx8ulp

Signed-off-by: Zhang Peng <peng.zhang_8@nxp.com>
2023-11-06 09:34:37 +01:00
Marc Herbert 6458f48471 scripts/fuzz.sh: fix multiple issues with "stub" overlays
Fixes commit deed9a8808 ("scripts: fuzz: add support for build and
overlays")

The main issue was the way fuzz.sh was trying to parse the overlay
file. Drop that and just pass it as is to `west` and `cmake` instead,
they know what to do with it.

Also:

- Fix invalid syntax in stub_build_all_ipc4.conf
- Make fuzz.sh shellcheck-clean again. Always use shellcheck.
- Temporarily disable `CONFIG_COMP_SMART_AMP` in
  stub_build_all_ipc3.conf because `smart_amp.c` does not compile (in
  any configuration)

```
sof/src/audio/smart_amp/smart_amp.c:748:9: error:
   no member named 'in_channels' in 'struct smart_amp_data'
   sad->in_channels = audio_stream_get_channels(&source_buffer->stream);
```

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2023-10-31 14:23:26 +00:00
Marc Herbert 59a5bc7448 rebuild-testbench.sh: explain BUILD_TYPE=native in usage()
Explain what happens when running the script without any argument.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2023-10-25 16:09:17 +03:00