Commit Graph

473 Commits

Author SHA1 Message Date
Andrey Borisovich b34cb8bfbb xtensa-build-zephyr.py: change install tree print to ASCII style
During testing of Windows CI workflow on windows-latest turned out
that github action console has issues rendering python
anytree.RenderTree graph with default style of
anytree.render.ContStyle (continuous vertical and horizontal lines).
Tests on local machines in cmd, pwershell and msys shells
proves, that all shells can render the graph (while powershell
does not do this correctly and graph is malformed).
Changing graph style to ASCII resolves the issue with Github actions.

Signed-off-by: Andrey Borisovich <andrey.borisovich@intel.com>
2022-12-21 11:46:28 +00:00
Marc Herbert 9b2453e236 xtensa-build-zephyr.py: compress sof-info files in parallel
On my system this brings the install_platform() duration down from about
3 seconds to less than 1 second.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-12-20 15:44:20 +00:00
Marc Herbert 4ac3e3ff44 xtensa-build-zephyr.py: don't leave the install of an old build behind
In theory a failure from this script should immediately stop the show
and no one should then use the staging directory.

In practice we cannot make sure, especially not with shell scripts. So
make sure no previous build of a requested platform is left behind.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-12-19 17:25:28 +00:00
Marc Herbert 424da2cd9d zephyr/docker: pass http[s]_proxy variables to the container
Support downloads from within the container.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-12-16 15:19:44 -08:00
Marc Herbert 906508bd79 xtensa-build-zephyr.py: checksum deterministic configs.c
.config is not deterministic because it has absolute paths in comments
and its order seems hard to predict. configs.c has the same information
generated in a determistic way.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-12-16 13:42:08 +02:00
Marc Herbert bc394916d0 xtensa-build-zephyr.py: switch type of InstFile() names to pathlib
Make sure any forward slashes in strings are immediately converted to
OS-dependent directory separators.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-12-16 13:42:08 +02:00
Tinghan Shen f28dc601cb platform: Add mt8188 to Kconfig
Add mt8188 to Kconfig.

Signed-off-by: Tinghan Shen <tinghan.shen@mediatek.com>
2022-12-15 16:11:23 +00:00
Marc Herbert 501204c684 xtensa-build-zephyr.py: objcopy --remove-section .comment zephyr.strip
Make stripped ELF files compiled by XCC identical across different
machines.

Some Xtensa compilers (ab?)use the .ident / .comment
section and append the typically absolute and not
reproducible /path/to/the.c file after the usual
compiler ID.

https://sourceware.org/binutils/docs/as/Ident.html

strip --strip-all does not remove the .comment section.
Remove ourselves like some gcc test scripts do:

 https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=c7046906c3ae

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-12-15 13:55:07 +02:00
Marc Herbert 6516f907aa sparse: pass platform argument to error filter script
So we can make adjust the warnings based on the platform.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-12-12 12:05:26 +00:00
Marc Herbert e39205cc11 parse_sparse_output.sh: new sparse_errors array
Zero functional change, pure preparation for the next commit.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-12-12 12:05:26 +00:00
Marc Herbert 09386bc0d9 Revert "xtensa-build-zephyr.py: west update --depth 5 --narrow"
This reverts commit ce28e09bd3.

This fixes Zephyr's git describe command and build reproducibility.

I tried fairly hard various git fetch options like --shallow-exclude
and --shallow-since but they did not save that much download (200MB at
best), required some hardcoding and most importantly they make complete
clones shallow again when invoked unconditionally. Not worth the
effort, build reproducibility is more important.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-12-09 15:49:28 +00:00
Marc Herbert 280dca92b5 xtensa-build-zephyr.py: build rimage with -GNinja by default
This saves a couple seconds when building from scratch on Linux.

On Linux the default CMake generator is "Makefiles" which is _not_
parallel by default.

Thanks to the previous commit it's still possible to manually switch to
"Makefiles" if desired.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-12-07 17:21:43 +00:00
Marc Herbert 2fcab330b9 xtensa-build-zephyr.py: do not CMake-reconfigure rimage every time
It's pointless and hardcodes the CMake generator.

Also remove wrong comment added in commit
6cba64d2cb ("xtensa-build-zephyr.py: fix a few minor pylint warnings")
The rimage part of the comment was flat out wrong.
The smex part of the comment is correct but in the wrong place.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-12-07 17:21:43 +00:00
Iuliana Prodan 6131901c46 scripts: qemu-check.sh: update READY_IPC for imx8
Update READY_IPC value based on changes regarding MU reset.
READY_IPC value comes from:
- clear GP pending interrupt #0 and #1 from MU's xSR register;
- enable GP #0 and #1 for Host -> DSP and DSP -> Host
message notification from MU's xCR register;
- now interrupt host to tell it we are done booting
by setting GIRn bit in MU's xCR register.

So, "00 00 00 c0 00 00 04 c0" is the MU's xSR and xCR registers:
xSR: c0000000 and xCR: c0040000

Signed-off-by: Iuliana Prodan <iuliana.prodan@nxp.com>
2022-12-05 14:33:46 +00:00
Marc Herbert 9fa1a99d8b xtensa-build-zephyr.py: install and checksum .elf, .lst and other files
Sample new output below. The sha256 is computed on the _uncompressed_
files and _not_ affected by gzip metadata.

```
build-sof-staging
├── sof
│   ├── community
│   │   ├── sof-imx8.ri         sha256=50c5423d2355ef3ed91...
│   │   └── sof-tgl.ri          sha256=6b1d26c12a63de5dfc8...
│   ├── sof-imx8.ldc    sha256=520d365d188d5ef4e907d3dd8c9...
│   └── sof-tgl.ldc     sha256=cdce6ad340d4dc047e71e738eb0...
├── sof-info
│   ├── imx8
│   │   ├── config.gz
│   │   ├── zephyr.elf.gz
│   │   ├── zephyr.lst.gz       sha256=8c83c3fc92df0a871dd...
│   │   ├── zephyr.map.gz
│   │   └── zephyr.strip.gz     sha256=d3ce3d3450c67bb3580...
│   └── tgl
│       ├── boot.mod.gz         sha256=d9c9e82e75fa6d061bf...
│       ├── config.gz
│       ├── main.mod.gz
│       ├── stripped-main.mod.gz        sha256=c367dccca6d...
│       ├── zephyr.elf.gz
│       ├── zephyr.lst.gz       sha256=5474bc5e58a5d000109...
│       ├── zephyr.map.gz
│       └── zephyr.strip.gz     sha256=6285f41c0682b33b7e0...
└── tools
    ├── cavstool.py
    ├── cavstool_client.py
    ├── cavstwist.sh
    ├── mtrace-reader.py
    ├── remote-fw-service.py
    └── sof-logger
```

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-11-30 11:59:33 +00:00
Marc Herbert 94f93c21dc xtensa-build-zephyr.py: indent new install_platform() function
Pure whitespace change.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-11-30 11:59:33 +00:00
Marc Herbert 4f98441d3c xtensa-build-zephyr.py: extract new function install_platform()
Zero functional change.

Lowers the score of the following pylint warnings:

- Too many local variables (38/15) (too-many-locals)  41->38
- Too many branches (26/12) (too-many-branches)       29->26
- Too many statements (116/50)(too-many-statements)  127->116

Keep existing indentation for now to help git blame -Mnn. Will fix in next
commit.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-11-30 11:59:33 +00:00
Marc Herbert 61fe7da105 xtensa-build-zephyr.py: fix misleading -i IPC4 help string
-i IPC4 is _not just_ an overlay change, there's an rimage change too.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-11-30 11:59:33 +00:00
Marc Herbert d70ac67032 parse_sparse_output.sh: ignore skipped topology2 error
Off-topic here.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-11-22 09:19:45 +00:00
Marc Herbert 842c39fb0f xtensa-build-zephyr.py: add missing RI_INFO_UNSUPPORTED platforms.
Copy the entire list of UNSIGNED_RI platforms from
src/arch/xtensa/CMakeLists.txt, adding the ones that don't use Zephyr
yet.

Also rename NO_RIMAGE_PLATFORMS to RI_INFO_UNSUPPORTED not to give the
wrong impression that imx8* don't use rimage.

Fixes commit 3a9413eebd ("xtensa-build-zephyr.py: restore lost
reproducible .ri checksum")

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-11-17 13:54:54 +00:00
Marc Herbert c97493b37a xtensa-build-zephyr.py: detect multiple rimage checkouts and abort
When cloning sof under a different directory name like
sof-experiment1 ((which is bad but we can't stop it unfortunately and it
keeps happening), it's common to end up with two rimage git checkouts:
sof/rimage and sof-experiment1/rimage. Detect this and fail immediately.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-11-09 14:15:07 +00:00
Marc Herbert d95004a62e xtensa-build-zephyr.py: trivial re-use of rimage_source_dir variable
Removes some minor duplication.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-11-09 14:15:07 +00:00
Marc Herbert b7708182fb .github: fail on errors and address space warnings in sparse output
sparse does not have a useful exit status so let's use 'grep' instead.

Fixes: #6317

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-11-01 10:55:58 +00:00
Marc Herbert e0722f493a xtensa-build-zephyr.py: de-hardcode mode=511 in mkdir()
These hardcoded 511 permissions came with the initial commit
1de3ef3675 ("Rewritten xtensa-build-zephyr.sh to python") without any
particular explanation.

On Unix, permissions of new files and directories are a user preference
set with the `umask`, different applications are not supposed to create
directories differently.

On Windows/NTFS it's not clear what 511 maps to.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-10-27 21:38:56 +01:00
Marc Herbert 3a9413eebd xtensa-build-zephyr.py: restore lost reproducible .ri checksum
At the end of the build, parse the .ri file and print a reproducible
checksum of the its content.

This feature was in xtensa-build-zephyr.sh but never made it to the .py
conversion. It was lost when the .sh file was deleted in commit
ef028f634a ("Removed obsolete xtensa-build-zephyr.sh")

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-10-24 16:15:12 +01:00
Marc Herbert ce28e09bd3 xtensa-build-zephyr.py: west update --depth 5 --narrow
Restore previous optimization. Can easily be undone with
git -C zephyr/ fetch --unshallow.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-10-24 12:10:45 +01:00
Marc Herbert 9a3a518644 xtensa-build-zephyr.py: pass -j option to west -> ninja
This is especially useful when compiling each C file requires a
network round-trip to a license server

Also stop overriding the ninja default when building rimage.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-10-24 12:10:45 +01:00
Marc Herbert adce544450 xtensa-build-zephyr.py: simplify installation, prepare for more files
Simplify the final installation code to prepare for installing more
files. Zero functional change.

Notably perform the cavs naming trick inside the build directory first
which allows using a destination directory instead of a destination file.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-10-21 20:02:11 +03:00
Marc Herbert ab7c1a2c7b xtensa-build-zephyr.py: de-interleave signing and installation code
Very minor, zero-change re-ordering to make the next commit easier to
review. Fixes commit 762f63bf8b ("xtensa-build-zephyr: add option to
build FW in cavs naming style")

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-10-21 20:02:11 +03:00
Marc Herbert a42592eda1 xtensa-build-zephyr.py: don't make "." a special case in tree listing
Small simplification.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-10-21 20:02:11 +03:00
Marc Herbert 0497212dca xtensa-build-zephyr.py: sort glob(**/*) list used for tree display
`pathlib.glob()` (and filesystems in general) provide no ordering
guarantee, which is likely why every glob() example in
https://docs.python.org/3/library/pathlib.html#pathlib.Path.glob
is sorted(). Follow these examples and use `sorted()`.

Fix error handling to run _before_ the `list index out of range`
exception hits. Also make it fatal now because there's no more any good
reason for this to fail.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-10-21 20:02:11 +03:00
Marc Herbert 6cba64d2cb xtensa-build-zephyr.py: fix a few minor pylint warnings
Declare globals at the top level.
Stop shadowing globals.
Replace `== or ==` with `in [ ]`.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-10-21 20:02:11 +03:00
Marc Herbert b5abf6c8f3 xtensa-build-zephyr: disable pylint bad-whitespace, module-doc, global
Also re-organize the header.

Absolutely zero functional change.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-10-21 20:02:11 +03:00
Marc Herbert d155062195 scripts/build-tools.sh: switch to incremental build by default
Faster build and focused and readable logs at last.

Add a warning not to surprise anyone.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-10-10 11:34:11 +01:00
Marc Herbert d56d8ab01c scripts/build-tools.sh: switch to Ninja because make is too verbose
Make prints a line for every topology that is already up to date and not
rebuilt. It's impossible to see what was done.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-10-10 11:34:11 +01:00
Marc Herbert c28400baaa scripts/docker-run.sh: fix wrong uid in warning 1001 -> 1000
Fixes commit d09844ab98 ("zephyr/docker-build.sh: match UID with
'adduser' instead of 'chgrp -R'")

Also clarify comment and add reference to new sudo-cwd.sh script.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-10-10 11:29:05 +01:00
Marc Herbert 0a4b1d62d5 sudo-cwd.sh: drop the all 'sof_' prefixes and references
This script is now generic. This was not done earlier to be gentle on
git blame.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-10-10 11:29:05 +01:00
Marc Herbert 027be98371 zephyr/docker: move exec_as_sof_uid() to new sudo-cwd.sh
Besides making things more obvious, the important functional change is
that the user switch is now performed for _every_ invoked, command, not
just for the build command.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-10-10 11:29:05 +01:00
Marc Herbert 5694cfa012 scripts: add link to CMake envvars FAQ next to XTENSA_SYSTEM definition
Environment variables like XTENSA_SYSTEM are an absolute CMake
pain. Add a link to the CMake FAQ and a one-line description why.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-09-26 12:33:56 +01:00
Marc Herbert b19cd3f080 xtensa-build-zephyr.py: don't complain about cmake_args when pristine
Don't ask the user to delete the build directory when they are using the
option that deletes the build directory.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-09-26 12:33:56 +01:00
Marc Herbert 9462baf8d4 xtensa-build-zephyr.py: remove misleading -i IPC3 option
`./xtensa-build-zephyr.py -i IPC3 mtl` builds the MTL default: IPC4.
This is wrong, remove the misleading "IPC3" option which never did
anything at all.

Fix the --help string to describe what actually happens. This is a build
script, no need for fancy abstractions and indirections that don't even
match reality. Just tell it like it is.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-09-23 16:29:48 +01:00
Adrian Warecki 784bce763c mtl: Added meteorlake platform to the build system
Added new mtl platform to xtensa-build-zephyr.py.
Added ace directories to cmake files.
Added ACE to kconfig.
Add Meteorlake platform to be built with Zephyr under
CONFIG_ACE_VERSION_1_5 flag.

Signed-off-by: Andrey Borisovich <andrey.borisovich@intel.com>
Signed-off-by: Marcin Szkudlinski <marcin.szkudlinski@intel.com>
Signed-off-by: Adrian Warecki <adrian.warecki@intel.com>
Signed-off-by: Konrad Leszczynski <konrad.leszczynski@intel.com>
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Signed-off-by: Tomasz Leman <tomasz.m.leman@intel.com>
Signed-off-by: Rafal Redzimski <rafal.f.redzimski@intel.com>
Signed-off-by: Adrian Bonislawski <adrian.bonislawski@intel.com>
2022-09-21 15:15:26 +02:00
Damian Nikodem a823958a8d xtensa-build-zephyr: pass sof build version to rimage
Rimage expects to receive fw_ver_build to prepare the manifest.

Signed-off-by: Damian Nikodem <damian.nikodem@intel.com>
2022-09-15 09:19:16 +02:00
Fred Oh 8136e514bf host-testbench.sh: add FullTest variable and set 0 by default
FullTest variable is by default 0. When fulltest is required,
set the variable and run the script like,

$ FullTest=1 ./scripts/host-testbench.sh

Signed-off-by: Fred Oh <fred.oh@linux.intel.com>
2022-09-07 22:09:33 +01:00
Fred Oh 6b744bcc63 host-testbench.sh: use process_test.m for 8 components
Expand to more components with process_test. Components for test are
volume, eq-iir, eq-fir, dcblock, drc, multiband-drc, src and tdfb.

Signed-off-by: Fred Oh <fred.oh@linux.intel.com>
2022-09-07 22:09:33 +01:00
Keqiao Zhang 0e4208d85b xtensa-build-zephyr.py: copy mtrace-reader.py to tools folder
we will switch to using this new logging tool, copy it to the tools
folder, then it will be uploaded to CI storage after building.

Signed-off-by: Keqiao Zhang <keqiao.zhang@intel.com>
2022-09-07 13:17:54 +01:00
Marc Herbert 9e546b4b84 scripts/test-repro-build.sh: leave only TGL
Other platforms are not supported anymore in the main branch.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-09-02 18:30:02 +01:00
Marc Herbert dfc6b46e1d scripts/docker-run.sh: fix wrong "id -n" command -> "id -u"
Fixes commit d09844ab98 ("zephyr/docker-build.sh: match UID with
'adduser' instead of 'chgrp -R'")

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-09-02 18:28:30 +01:00
Marc Herbert 8fee9e1306 xtensa-build-all.sh: remove byt..jsl platforms from '-a'
These platforms are not supported in the main branch anymore.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-08-31 21:03:16 +01:00
Marc Herbert 8328cc50a9 xtensa-build-all.sh: add rmb to default platforms built with `-a`
The rmb toolchain was added in commit 4e85dfc95f ("docker: add
AMD/Rembrandt gcc toolchain") and before.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-08-31 21:03:16 +01:00