Add support to specify an overlay file to use with the fuzz script and
another flag to build without fuzzing
Signed-off-by: Curtis Malainey <cujomalainey@chromium.org>
This restores parity with the installer/GNUmakefile in the stable-v2.2
branch and simplifies CI deployment.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Don't rely on some semi-random default value.
The final `zephyr/.config` and binaries are strictly identical.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
We used to provide the `default_rimage_key` in two places:
1) as a default `RIMAGE_KEY` attribute (newer `otc_private_key_3k`)
2) as an explicit `else` clause that was never run because of 1) (older
`otc_private_key`)
Remove the redundant `else` clause in 2); it became dead code. This
simplifies the code and makes the script ready for the day when the key
argument will be optional for rimage (more about this in
https://github.com/zephyrproject-rtos/zephyr/pull/58356)
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Fixes commit a769d3941d ("xtensa-build-zephyr.py: stop calling west
sign, rely on west build")
Before that commit, `xtensa-build-zephyr.py` used to invoke `west sign`
directly and in the same current directory. This allowed relative key
paths to just work. Now that `west sign` is indirectly invoked by `west
build`, there is no possible way to make relative key paths work. So we
must resolve them before using them.
Fixes#7718
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Stop invoking west sign manually. Configure rimage using `west config`
and a platform-specific WEST_CONFIG_LOCAL file instead, then rely on
west build to invoke rimage.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Do not make the switch yet. Make the switch in a smaller commit that can
more easily be reverted in case of a bug or some missed dependency
somewhere.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
We used to use `-- -c rimage/config/platf.toml` for the IPC4 overlay and
the older `--tool-data rimage/config/` for the rest. This was confusing,
inconsistent and used to print this warning:
```
WARNING: --tool-data /var/home/mherber2/SOF/sof/rimage/config ignored,
overridden by: -- -c ...
```
`--tool-data` was always a bad idea. It looks generic but it's not; it
allows changing only a config directory that is always in the same
location.
Copy what `west sign` does with `--tool-data`, drop it and use `-- -c`
always.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
This makes no difference now because it has no side-effect yet but it
will make a difference once it starts using `west config`.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
In some cases it was difficult to separate the command's output from the
extra environment it ran in.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
For rimage parameters, parse new, static versions.json file instead of
generated sof_versions.h file. This will make possible to configure
rimage before west build.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
This replaces and simplifies commit a823958a8d ("xtensa-build-zephyr:
pass sof build version to rimage") with a constant 1.
That commit was submitted to avoid a test failure in a broken, internal
test looking for an SOF_BUILD value... hardcoded to 1! (internal FW issue
257, test TestLoadFwExtended::()::test_00_01_load_fw_and_check_version")
The final goal is for this script to stop extracting anything from
`generated/sof_versions.h` so rimage can be configured _before_ the build
has started. Other commits will deal with other parts of sof_versions.h
SOF_BUILD can be replaced by a constant because it has always been one
when building with Zephyr. The optional BLD_COUNTERS feature - disabled
by default in XTOS since commit 9f8cce1522 ("Disable __TIME__ and the
non-reproducible build counter by default") for build reproducibility
reasons - has never worked with Zephyr for the following reasons:
- The sof_add_build_counter_rule() invocation is located in the
top-level sof/CMakeLists.txt file which has never been used by the
Zephyr build.
- sof_add_build_counter_rule() registers a POST_BUILD command for the
top-level "sof" CMake target but there is no "sof" build target in the
Zephyr build.
- Variables like VERSION_BUILD_COUNTER_CMAKE_PATH are not defined in the
Zephyr build for some unknown reason.
- Probably others.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
This is required to investigate build failures. Because of the way the
build is split, a failure in one component can become totally drown in
the long build logs of another component.
Example: with -j2 the `parser_ep` build failure below is completely
drown by the long (and successful!) build logs of `sof_ep`:
```
./scripts/rebuild-testbench.sh -j 1 -p tgl
sof/tools/testbench/build_xt_testbench/sof_parser/build/include/alsa/
sound/asoc.h:196: error: expected specifier-qualifier-list before ‘__le32’
```
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Building a Docker image takes several hours. To reduce costs and save
time we don't want to maintain a different Docker image for the
stable-v2.2 branch. Restore toolchains that were accidentally removed
from the Dockerfile because this will unnecessarily paint red every PR
submitted to the stable-v2.2 branch, example:
https://github.com/thesofproject/sof/actions/runs/4867162042/jobs/8679468096
If we want to stop supporting something then let's keep things simple
and remove it only where it actually gets in the way and where it
actually becomes a problem. No need to search and scrub every reference
to it from the face of the planet and accidentally make our life harder
on release branches, create pointless git conflicts and _increase_
maintenance costs when trying to reduce them!
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Remove one intermediate and unnecessary .ri file. Simplify the code and
the build directory. Example below with `sof-mtl.ri` for better
readability but this applies the same to `sof-imx8.ri` and every other
`sof-$platf.ri` file.
There were 3 .ri copies before this commit:
```
build-mtl/zephyr/zephyr.ri
copied to -> build-mtl/zephyr/sof-mtl.ri
copied to -> build-sof-staging/___/sof-mtl.ri
```
Only 2 left after dropping the second and pointless copy:
```
build-mtl/zephyr/zephyr.ri
copied to -> build-sof-staging/___/sof-mtl.ri
```
Fewer copies means less wondering about what is what, two identical
files in the same directory is at best pointless and at worst misleading.
The `build-mtl/` directory belongs to the zephyr build system
exclusively, this wrapper script has no business interfering with
`build-mtl`. `build-mtl/zephyr/sof-mtl.ri can even be dangerous because
it does not get cleaned. Demonstration:
```
# Compile successfully
$ ./scripts/xtensa-build-zephyr.py mtl
# Write some code and make a mistake.
# Fail to compile.
$ echo BROKEN >> ../zephyr/kernel/sched.c
$ ./scripts/xtensa-build-zephyr.py mtl
# Obsolete .ri files are still there
$ find ../build-mtl/ -name *.ri
../build-mtl/zephyr/zephyr.ri
../build-mtl/zephyr/sof-mtl.ri
$ ninja -C ../build-mtl clean
# Obsolete sof-mtl.ri is still there!
$ find ../build-mtl/ -name *.ri
../build-mtl/zephyr/sof-mtl.ri
```
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
This patch adds to rebuild-testbench option -x <platform> that
can be used to build testbench for xt-run execution. The enhanced
script reuses native testbench build but with CC, LD, LDFLAGS,
etc. defines to use the xt-xcc compiler for build.
Currently TGL (HiFi3) is the only supported platform. More will
be added later.
Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
As discussed in #6952, relying on `git describe` for defining SOF_MAJOR
and friends is very brittle and unreliable. Switch to a static
versions.json file instead.
Note the full `git describe --tags` output is _still_ present in the
binary, this is useful and left unchanged. It's just not used any more
to guess SOF_MAJOR, SOF_MINOR and SOF_MICRO.
Use JSON because CMake and pretty much every other piece of software has
a JSON parser out of the box.
This aligns with linux/Makefile, Zephyr/VERSIONS and probably many
others. - except we use JSON because we're in 2023 so we don't spend
time having fun re-implementing parsers any more.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
There is a UID mistmatch and file permission problem. sudo-cwd.sh will
switch id every docker run command.
80e9c3454a was reverted due to
missing a toolchain.
Signed-off-by: Fred Oh <fred.oh@linux.intel.com>
TGL has always been using the `xtensa-cnl-elf` toolchain. That
toolchain was removed by accident when cavs 1.8 support (including
CNL) was removed by massive search/replace commit dc9ba281d7
("platform: remove support for cAVS 1.8 platforms")
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
There is a UID mistmatch and file permission problem. sudo-cwd.sh will
switch id every docker run command.
This can be done like this,
./scripts/docker-run.sh ./scripts/sudo-cwd.sh CMD
But not to make build script, just include sudo-cwd.sh in docker-run.sh.
Signed-off-by: Fred Oh <fred.oh@linux.intel.com>
Upgrade Ubuntu to latest LTS version, 22.04.
Permission issue has started when UID didn't match with local UID.
As an solution, created high value UID to unmatch local UID.
sof home doesn't have permission for others, so add o+rx permission to
avoid any permission issue when id does not match.
Signed-off-by: Fred Oh <fred.oh@linux.intel.com>
As recommended by Andrey in #7364, reduce the noise and log the
environment only for the `west build` command - unless the log level is
increased with `-v`, then log for every command.
Also fix a very unlikely corner case where the caller would invoke
execute_command() with `check=None` and get the opposite effect.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Catch the additional two warnings below:
warning: too many warnings
warning: cast removes address space
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Make sure users do not misinterpret our single `debug_overlay.conf` file
as some kind of complex and elaborate "debug build" concept. Also save
users who want to see the file a lot of time by naming it directly in
the --help.
Empty layers of indirection are just spurious obfuscation.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
--cmake-args is actually safe, just a bit slower. See explanation in
https://github.com/zephyrproject-rtos/zephyr/pull/56671
Fixes unnecessary restriction added by commit 9fb7a607eb
("xtensa-build-zephyr.py: fix -C option so it can support whitespace")
Also warn for overlays because they act the same.
Log the `west config` due to the suggestion to use build.cmake-args and
the recent rimage changes in Zephyr PR 54700
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
This is a typical example of why environment variables are (a sometimes
necessary) evil: imagine you're trying to reproduce exactly the `west
build` command run by xtensa-build-zephyr.py when building with the
`xt-xcc` toolchain. So you would typically look at the logs, feel lucky
that it shows the extra environment variables used and copy them:
```
XTENSA_TOOLCHAIN_PATH=/srv/home/jenkins/xcc/install/tools
TOOLCHAIN_VER=RG-2017.8-linux
XTENSA_SYSTEM=/home/jenkins/xcc/install/builds/RG-2017.8-linux/cavs...
In dir: workspace/sof; running: west build ...
```
Except this won't work because there's one variable currently missing:
`ZEPHYR_TOOLCHAIN_VARIANT`!
Of course there could be more in the future.
Fix this by leveraging the recent os.environ.copy() added by
commit 8aab18351f ("xtensa-build-zephyr: fix DEFAULT_TOOLCHAIN_VARIANT
spill on next platf") compare it to the current os.environ and show the
difference in a totally generic, non-hardcoded way.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
- Create new setup() function
- Separate export for set -e compatibility
- Don't use the generic `build` directory but a more specific
`build-fuzz` instead.
- De-hardcode zephyr path thanks to west
- shellcheck clean
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
The original fuzzer has been deprecated for some time now. Remove it.
All fuzzing now done by OSS-fuzz
Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Fixes recent commit 8aab18351f ("xtensa-build-zephyr: fix
DEFAULT_TOOLCHAIN_VARIANT spill on next platf")
Some XtensaTools installation are missing this `default-params` symbolic
link:
```
XtensaTools/config/
|-- X4H3I16w2D48w3a_2017_8-params
|-- X6H3CNL_2017_8-params
|-- cavs2x_LX6HiFi3_2017_8-params
`-- default-params -> cavs2x_LX6HiFi3_2017_8-params
```
Maybe it's missing when installing with the graphical interface?
This symbolic link is surprisingly enough to make `xt-objcopy` work
_without_ the XTENSA_ variables. But when the variables _and_ the link
are both missing, then `xt-objcopy` fails with the usual error:
```
in current dir: work/current/sof; running command:
XtDevTools/install/tools/RG-2017.8-linux/XtensaTools/bin/xt-objcopy
--remove-section .comment sof/build-tgl/zephyr/zephyr.strip
build-sof-staging/sof-info/tgl/stripped-zephyr.elf
Error: there is no Xtensa core registered as the default.
You need to either specify the name of a registered Xtensa core (with
the --xtensa-core option or the XTENSA_CORE environment variable) or
specify a different registry of Xtensa cores (with the --xtensa-system
option or the XTENSA_SYSTEM environment variable).
The following Xtensa cores are available:
hifiep_bd5
cavs2x_LX6HiFi3_2017_8
sample_config
sample_flix
...
```
Fix this failure by simply passing the XTENSA_ variables to xt-objcopy.
Kudos to Seppo Ingalsuo for the interactive debugging session that
allowed root-causing this problem extremely quickly.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
xtensa-build-zephyr.py has always defined XTENSA_ and other environment
variables in the current environment shared by all platforms. This was
always bad but apparently never a problem until the addition of the new
DEFAULT_TOOLCHAIN_VARIANT (xcc or clang) variable.
Before DEFAULT_TOOLCHAIN_VARIANT, each platform's environment would
simply override the previous one. However with the new
DEFAULT_TOOLCHAIN_VARIANT, the current environment has precedence for
more flexibility. This makes each platform "spill" onto the next one and
`xtensa-build-zephyr.py -p tgl mtl` fail like this:
```
-- Board: intel_adsp_ace15_mtpm
-- Found toolchain: xcc (/home/XCC/install/tools)
CMake Error at zephyr/cmake/compiler/xcc/generic.cmake:9 (message):
Zephyr was unable to find the toolchain. Is the environment
misconfigured?
User-configuration:
ZEPHYR_TOOLCHAIN_VARIANT: xcc
Internal variables:
CROSS_COMPILE:
/home/XCC/install/tools/RI-2022.10-linux/XtensaTools/bin/xt-
```
To fix this, stop modifying the current os.environ and use a new, fresh
os.environ.copy() for each platform instead.
Fixes commit 309fa264e2 ("xtensa-build-zephyr.py: upgraded Xtensa
toolchain for MTL")
History repeats itself: commit 6bedd8e742 ("xtensa-build-zephyr: fix
RIMAGE_KEY when building multiple platforms") fixed the same logical
error months ago in a different script.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Remove all support for cAVS 2.0 platformsm including Ice Lake and
Jasper Lake, they aren't supported any more.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
Curtis Malainey