- Create new setup() function
- Separate export for set -e compatibility
- Don't use the generic `build` directory but a more specific
`build-fuzz` instead.
- De-hardcode zephyr path thanks to west
- shellcheck clean
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
The original fuzzer has been deprecated for some time now. Remove it.
All fuzzing now done by OSS-fuzz
Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Fixes recent commit 8aab18351f ("xtensa-build-zephyr: fix
DEFAULT_TOOLCHAIN_VARIANT spill on next platf")
Some XtensaTools installation are missing this `default-params` symbolic
link:
```
XtensaTools/config/
|-- X4H3I16w2D48w3a_2017_8-params
|-- X6H3CNL_2017_8-params
|-- cavs2x_LX6HiFi3_2017_8-params
`-- default-params -> cavs2x_LX6HiFi3_2017_8-params
```
Maybe it's missing when installing with the graphical interface?
This symbolic link is surprisingly enough to make `xt-objcopy` work
_without_ the XTENSA_ variables. But when the variables _and_ the link
are both missing, then `xt-objcopy` fails with the usual error:
```
in current dir: work/current/sof; running command:
XtDevTools/install/tools/RG-2017.8-linux/XtensaTools/bin/xt-objcopy
--remove-section .comment sof/build-tgl/zephyr/zephyr.strip
build-sof-staging/sof-info/tgl/stripped-zephyr.elf
Error: there is no Xtensa core registered as the default.
You need to either specify the name of a registered Xtensa core (with
the --xtensa-core option or the XTENSA_CORE environment variable) or
specify a different registry of Xtensa cores (with the --xtensa-system
option or the XTENSA_SYSTEM environment variable).
The following Xtensa cores are available:
hifiep_bd5
cavs2x_LX6HiFi3_2017_8
sample_config
sample_flix
...
```
Fix this failure by simply passing the XTENSA_ variables to xt-objcopy.
Kudos to Seppo Ingalsuo for the interactive debugging session that
allowed root-causing this problem extremely quickly.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
xtensa-build-zephyr.py has always defined XTENSA_ and other environment
variables in the current environment shared by all platforms. This was
always bad but apparently never a problem until the addition of the new
DEFAULT_TOOLCHAIN_VARIANT (xcc or clang) variable.
Before DEFAULT_TOOLCHAIN_VARIANT, each platform's environment would
simply override the previous one. However with the new
DEFAULT_TOOLCHAIN_VARIANT, the current environment has precedence for
more flexibility. This makes each platform "spill" onto the next one and
`xtensa-build-zephyr.py -p tgl mtl` fail like this:
```
-- Board: intel_adsp_ace15_mtpm
-- Found toolchain: xcc (/home/XCC/install/tools)
CMake Error at zephyr/cmake/compiler/xcc/generic.cmake:9 (message):
Zephyr was unable to find the toolchain. Is the environment
misconfigured?
User-configuration:
ZEPHYR_TOOLCHAIN_VARIANT: xcc
Internal variables:
CROSS_COMPILE:
/home/XCC/install/tools/RI-2022.10-linux/XtensaTools/bin/xt-
```
To fix this, stop modifying the current os.environ and use a new, fresh
os.environ.copy() for each platform instead.
Fixes commit 309fa264e2 ("xtensa-build-zephyr.py: upgraded Xtensa
toolchain for MTL")
History repeats itself: commit 6bedd8e742 ("xtensa-build-zephyr: fix
RIMAGE_KEY when building multiple platforms") fixed the same logical
error months ago in a different script.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Remove all support for cAVS 2.0 platformsm including Ice Lake and
Jasper Lake, they aren't supported any more.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
Remove all support for cAVS 1.8 platformsm including Cannon Lake,
Comet Lake, Whiskey Lake and Coffee Lake, they aren't supported
any more.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
Remove all support for cAVS 1.5 platformsm including Apollo Lake,
Sky Lake, Kaby Lake, Broxton and Gemini Lake, they aren't supported
any more.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
Zero functional change. Next commit will build rimage _before_ building
the firmware which will be required when `west build` starts signing by
default (https://github.com/zephyrproject-rtos/zephyr/pull/54700)
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Intel Meteorlake release toolchain is Xtensa RI-2022.10 version.
New toolchain does not support xt-xcc driver anymore so we are
forced to switch to new xt-clang driver.
To distinguish between default driver for the platform built,
(Xtensa GCC or Clang) added new field to script platform list
containing a toolchain name "xcc" or "xcc-clang" expected by
Zephyr project.
Signed-off-by: Andrey Borisovich <andrey.borisovich@intel.com>
Remove all support for Broadwell and Haswell platforms, they
aren't supported any more.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
Remove all support for Baytrail and Cherrytrail platforms, they
aren't supported on the "main" branch any more. To build SOF for them
use the "table-v2.2" branch.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
When using --fw-naming AVS the .ri file is renamed to `dsp_basefw.bin`.
Also explain why it's still useful to identify non-deterministic images.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
`git describe` differences happen, especially when trying to optimize
cloning and this is for instance what happened in
https://github.com/thesofproject/sof/pull/6950#issuecomment-1396150533
where zephyr tags were fetched on Linux but not on Windows
This also makes plain git version differences more obvious; no need to
scroll all the way up and scan the build logs.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Make CONFIG_BUILD_OUTPUT_STRIPPED mandatory so we can always compare
builds. It makes practically zero build space and time difference and
has huge reproductibility value, see discussion in
https://github.com/zephyrproject-rtos/zephyr/pull/51954
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Commit 3315681c69 ("cmake: make sure sha1 is computed without
filters") added the `--no-filters` option to the `git hash-object`
invocation used to produce a `src/` checksum embedded in all firmware
binaries and .ldc files.
This fixed a small mismatch for the (very few) people using the
_asymmetric_ `autocrlf = input`. However it created a huge mismatch for
the numerous people using the _symetric_ `autocrlf = true`.
So revert that commit, remove `--no-filters` and replace it with a build
time warning.
`--no-filters` produces a checksum of the files as they are on the
filesystem, as opposed to how they would be input into git after
optional CRLF -> LF conversion.
When `--no-filters` was added, no one was compiling in Windows. Most
developers were not performing any conversion because there is no
`autocrlf` conversion by default on Linux. Files were identical inside
and outside of git and life was simple.
Simple life except for at least one indomitable Linux developer who had
the _asymmetric_ `autocrlf = input` setting. Also, the SOF git repo
always had two exceptional (and generated) files with CRLF end of lines;
see issue #5917 for details (all other files have LF end of lines). So
for that "asymmetric" developer only and these two files only, `autocrlf
= input` converted these 2 files to LF before hashing and this caused a
different checksum. `--no-filters` stopped that conversion and "solved"
that mismatch.
But now Windows has entered the stage. On Windows, the default is
usually (and unfortunately) the symmetric `autocrlf = true`. It is the
default for Github's Windows runner. This symmetric default converts ALL
other files to CRLF when extracting them out of git. This causes `git
hash-object --no-filters` to produce a _different hash for all but 2
files on Windows_!
The only solution is to:
1. Drop the `--no-filters` to align everything on what is _stored in
git_. What is stored in git is the only thing we are sure all users have
in common.
2. Request users to use only _symmetric_ `autocrlf` settings so any
optional input+output conversion cancels itself out.
3. Convert odd files and make sure they all have LF end of lines.
4. In the future, drop this entire git hash-object technique which is
flawed in multiple ways and has been causing multiple issues. Much,
much more work than this very small revert though.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
To test whether CMake configuration can be skipped, look for either a
ninja.build or Makefile.
An existing build directory is NOT proof that CMake configuration
succeeded. When CMake configuration failed, trying to build again fails
with a super confusing "Makefile not found" error message.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Building on Windows generates CRLF text files. Standardize checksums to
LF files.
As usual, files in the build directory are left untouched.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Fixes pylint warning "E1135 (unsupported-membership-test)". Not sure
this warning is valid because the code worked anyway but it does not
hurt.
Fixes commit bc394916d0 ("xtensa-build-zephyr.py: switch type of
InstFile() names to pathlib")
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
During testing of Windows CI workflow on windows-latest turned out
that github action console has issues rendering python
anytree.RenderTree graph with default style of
anytree.render.ContStyle (continuous vertical and horizontal lines).
Tests on local machines in cmd, pwershell and msys shells
proves, that all shells can render the graph (while powershell
does not do this correctly and graph is malformed).
Changing graph style to ASCII resolves the issue with Github actions.
Signed-off-by: Andrey Borisovich <andrey.borisovich@intel.com>
On my system this brings the install_platform() duration down from about
3 seconds to less than 1 second.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
In theory a failure from this script should immediately stop the show
and no one should then use the staging directory.
In practice we cannot make sure, especially not with shell scripts. So
make sure no previous build of a requested platform is left behind.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
.config is not deterministic because it has absolute paths in comments
and its order seems hard to predict. configs.c has the same information
generated in a determistic way.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Make sure any forward slashes in strings are immediately converted to
OS-dependent directory separators.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Make stripped ELF files compiled by XCC identical across different
machines.
Some Xtensa compilers (ab?)use the .ident / .comment
section and append the typically absolute and not
reproducible /path/to/the.c file after the usual
compiler ID.
https://sourceware.org/binutils/docs/as/Ident.html
strip --strip-all does not remove the .comment section.
Remove ourselves like some gcc test scripts do:
https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=c7046906c3ae
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
This reverts commit ce28e09bd3.
This fixes Zephyr's git describe command and build reproducibility.
I tried fairly hard various git fetch options like --shallow-exclude
and --shallow-since but they did not save that much download (200MB at
best), required some hardcoding and most importantly they make complete
clones shallow again when invoked unconditionally. Not worth the
effort, build reproducibility is more important.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
This saves a couple seconds when building from scratch on Linux.
On Linux the default CMake generator is "Makefiles" which is _not_
parallel by default.
Thanks to the previous commit it's still possible to manually switch to
"Makefiles" if desired.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
It's pointless and hardcodes the CMake generator.
Also remove wrong comment added in commit
6cba64d2cb ("xtensa-build-zephyr.py: fix a few minor pylint warnings")
The rimage part of the comment was flat out wrong.
The smex part of the comment is correct but in the wrong place.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Update READY_IPC value based on changes regarding MU reset.
READY_IPC value comes from:
- clear GP pending interrupt #0 and #1 from MU's xSR register;
- enable GP #0 and #1 for Host -> DSP and DSP -> Host
message notification from MU's xCR register;
- now interrupt host to tell it we are done booting
by setting GIRn bit in MU's xCR register.
So, "00 00 00 c0 00 00 04 c0" is the MU's xSR and xCR registers:
xSR: c0000000 and xCR: c0040000
Signed-off-by: Iuliana Prodan <iuliana.prodan@nxp.com>
Zero functional change.
Lowers the score of the following pylint warnings:
- Too many local variables (38/15) (too-many-locals) 41->38
- Too many branches (26/12) (too-many-branches) 29->26
- Too many statements (116/50)(too-many-statements) 127->116
Keep existing indentation for now to help git blame -Mnn. Will fix in next
commit.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Copy the entire list of UNSIGNED_RI platforms from
src/arch/xtensa/CMakeLists.txt, adding the ones that don't use Zephyr
yet.
Also rename NO_RIMAGE_PLATFORMS to RI_INFO_UNSUPPORTED not to give the
wrong impression that imx8* don't use rimage.
Fixes commit 3a9413eebd ("xtensa-build-zephyr.py: restore lost
reproducible .ri checksum")
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
When cloning sof under a different directory name like
sof-experiment1 ((which is bad but we can't stop it unfortunately and it
keeps happening), it's common to end up with two rimage git checkouts:
sof/rimage and sof-experiment1/rimage. Detect this and fail immediately.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
These hardcoded 511 permissions came with the initial commit
1de3ef3675 ("Rewritten xtensa-build-zephyr.sh to python") without any
particular explanation.
On Unix, permissions of new files and directories are a user preference
set with the `umask`, different applications are not supposed to create
directories differently.
On Windows/NTFS it's not clear what 511 maps to.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
At the end of the build, parse the .ri file and print a reproducible
checksum of the its content.
This feature was in xtensa-build-zephyr.sh but never made it to the .py
conversion. It was lost when the .sh file was deleted in commit
ef028f634a ("Removed obsolete xtensa-build-zephyr.sh")
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Marc Herbert