These attributes were added to Zephyr's include/zephyr/debug/sparse.h by
commit 17eb313a1b15 ("sparse: add an address space and a __sparse_force
annotation")
This gets rid of dozens of warnings in
https://github.com/thesofproject/sof/runs/6387526478
```
ERROR: need consistent spacing around '*' (ctx:WxV)
+ struct comp_buffer __sparse_cache *buffer_c, *sink_c;
^
ERROR: need consistent spacing around '*' (ctx:WxV)
+ struct comp_buffer __sparse_cache *source_c;
^
```
etc.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Add a new -d option to support SOF debug build. This
option will add CONFIG_DEBUG=y to SOF configuration,
which is aligned with xtensa-build-all.sh script.
The -d option is a shortcut for '-o debug_overlay_path'.
Signed-off-by: Chao Song <chao.song@linux.intel.com>
This can fail for various reasons listed in a comment.
Remove now redundant `west topdir` check.
Without this check, a not found SOF fails with very cryptic Kconfig
errors:
zephyr/samples/subsys/audio/sof/prj.conf:5:
warning: attempt to assign the value 'n' to undefined symbol HAVE_AGENT
zephyr/samples/subsys/audio/sof/prj.conf:8:
warning: attempt to assign the value 'n' to undefined symbol
DEBUG_MEMORY_USAGE_SCAN
zephyr/samples/subsys/audio/sof/boards/intel_adsp_cavs15.conf:1:
warning: attempt to assign the value 'y' to undefined symbol APOLLOLAKE
etc.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
This fixes SOF version.cmake which was just broken by a recent git
security update and started to fail like this:
```
-- SOF version.cmake starting at 2022-04-25T18:14:56Z UTC
-- /workdir/zephyr/.. is at git commit with parent(s):
fatal: unsafe repository ('/workdir' is owned by someone else)
To add an exception for this directory, call:
git config --global --add safe.directory /workdir
```
(example at https://github.com/thesofproject/sof/runs/6162885265)
chgrp -R was always an ugly hack because it was messing with
(persistent) file permissions on the host, outside the container. This
new adduser solution is unfortunately much more code but it does not
leak any side effect outside the container.
Do not fix scripts/docker-run.sh yet because there is still no UID
mismatch between Github Actions and the SOF container (they're both
1001) but add a warning + TODO.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
There was no actual "IPC3" value, it was ignored. The platform default
was used instead.
Stop ignoring invalid arguments, fail and report them.
Allow empty string as a no-op for scripting convenience.
Don't restrict IPC4 to tigerlake, it's not the job of
xtensa-build-all.sh to filter what's supported versus not.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Commit 475e09f17d ("xtensa-build-zephyr.py: default to
subprocess.run(check=True)") switched the default value of the
execute_command() function to "True". Remove all the "check=True"
arguments that are now redundant.
Zero functional change.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
There are warnings about bison and flex not available.
Interestingly these were not fatal errors but time to fix them.
Signed-off-by: Fred Oh <fred.oh@linux.intel.com>
Add ipc4 build support for tgl with IPC4_CONFIG_OVERLAY.
If IPC_VERSION is chosen as IPC4, then overlay configuration file
for different platforms will be used automatically without assigning
the configuration file with options.
Different configuration files will be used according to what platform
is building firmware for.
Signed-off-by: Xiaoyun Wu(Iris) <xiaoyun.wu@intel.com>
This reduces the number of warnings from about 600 to a more manageable
350.
Disable W0312 because the entire file uses tabs.
Disable C0103 because we have higher priority than naming conventions.
Disable C0116 because the script is small enough, shouldn't require
pydoc everywhere.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Building in parallel is much faster but it makes logs unreadable and
build failures impossible to understand. This is especially true when
building with recent ALSA that produces of deprecation warnings, see
examples below. For test topologies the problem is even worse: its
XARGS parallel build provides no log at all.
To find what actually fails, it is required to fall back on a single
threaded and verbose build and this is achieved with the variables
USE_XARGS, NO_PROCESSORS and VERBOSE. Pass these through docker-run.sh
and CMake.
Examples from #5608https://github.com/thesofproject/sof/runs/5717553555?check_suite_focus=truehttps://sof-ci.01.org/sofpr/PR5608/build12556/build/tools.txt
ALSA pcm.c:1523:(parse_hw_config) deprecated fsync value 'codec_slave',...
ALSA pcm.c:1471:(parse_hw_config) deprecated bclk value 'codec_slave',...
ALSA pcm.c:1523:(parse_hw_config) deprecated fsync value 'codec_slave',...
ALSA pcm.c:1471:(parse_hw_config) deprecated bclk value 'codec_slave',...
ALSA pcm.c:1523:(parse_hw_config) deprecated fsync value 'codec_slave',...
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
This removes two layers of indirection when trying to find what fails.
The lack of this proved especially painful when working on #5632 and
similar backports.
Also move outputs to a subdirectory by default. No change when invoked
from CMake which does not use the default output directory.
Also show the exact m4 and alsatplg commands run when not using XARGS
parallelization. Again this is critical when something fails.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Move software configuration from Zephyr samples directory
to SOF platforms directory using Zephyr overlay mechanism.
How it works:
1. Gets zephyr board defconfig values
2. Applies Zephyr samples prj.conf values on top of it
3. Then applies our new overlay.conf on top of it
4. Final "summary" of Kconfig settings is written to autoconf.h header
Signed-off-by: Andrey Borisovich <andrey.borisovich@intel.com>
This restores the performance of the original shell script and makes a
big difference in CI. CI clones over and over again whereas developers
clone rarely and can run a simple git fetch --unshallow.
Don't use --depth 0 to provide more context in the logs and also make it
more obvious that this is a shallow clone and not just git log -n 1.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Running on a mac crashed with "undefined xtensa_tools_version_postfix",
that should not be fatal (there are other toolchains).
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Duplicating all parameters of subprocess.run() is not just tedious and
error-prone, it makes the script compatible only with a range of
specific Python versions.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
In scripts/xtensa-build-zephyr.sh line 199:
if test $platform = tgl-h ; then
^-------^ SC2086: Double quote to prevent globbing
and word splitting.
Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Use intel_adsp_cavs20_jsl Zephyr board when building for
Intel Jasper Lake.
BugLink: https://github.com/thesofproject/sof/issues/4539
Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Same -u option as the new Python script.
We need this for the new sof/stable-v2.1 branch on
https://github.com/thesofproject/zephyr while our CI has not entirely
switched to the python script yet.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
This script is run from an existing SOF repo that is assumed to be fully
initialized. Updating existing submodules is a potentially destructive
operation.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
The older shell script had a deprecated convenience hack to clone
submodules before building rimage but _only if submodules were missing_!
While trying to preserve that, the new script changed that to an
unconditional git submodule update that can be destructive when
submodules are already present.
Generally speaking, using git and building must always be two very
distinct activities. No one wants the source code to change quietly from
one build to the next.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
A second git_submodules_update() immediately after the first one does
not make sense. This looks like something leftover from a past
experiment.
Fixes initial commit 1de3ef3675 ("Rewritten xtensa-build-zephyr.sh to
python")
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Recent combination of tgl and tgl-h building resulted in use of an invalid
defconfig file for tgl-h platform. This commit reverts that change.
Fixes: f1e6e1fdd3
Signed-off-by: Adrian Warecki <adrianx.warecki@intel.com>
This is especially important considering some sof-bin releases are now
"hybrid": with a mix of XTOS and Zephyr.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Fixes copy/paste of commit de41202f8f ("zephyr: build: Add initial
build support for SOF application.")
This fixes the dictionary hash when using Zephyr; no more fallback on
the git SHA1.
When using Zephyr, SOF_ROOT_SOURCE_DIR (and SOF_ROOT_BINARY_DIR) are used
only by version.cmake
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
When building Zephyr, CMAKE_CURRENT_SOURCE_DIR does not point at
SOF. Use SOF_ROOT_SOURCE_DIRECTORY instead.
No impact besides the logs.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Some git servers like Github allow fetching by full length SHA so this
useful information to share.
(Others forbid this entirely, see last page of `git help fetch-pack`)
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
version.cmake has a not very intuitive logic to make sure sof_versions.h
is always up to date without triggering a full rebuild. Add comments and
rephrase some logs to make it less hard to follow.
Absolutely zero functional change.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Combined case label tgl and tgl-h similarly as in xtensa-build-zephyr.sh
Corrected code formatting.
Signed-off-by: Adrian Warecki <adrianx.warecki@intel.com>
If the user doesn't set the path to own private key, the default path is
placed in the variable containing the user's key. This causes a leak of the
default key of one platform to the others platforms, which will treat it as
a key supplied from the user.
Signed-off-by: Adrian Warecki <adrianx.warecki@intel.com>
PosixPath doesn't convert to string automatically, and
it is not acceptable in sign_cmd, convert it to string
explicitly.
Signed-off-by: Chao Song <chao.song@linux.intel.com>
Zephyr recently added support for separate configs for 2-core and 4-core
variants of cAVS2.5. Use the new "tgph" configuration for when building
SOF for TigerLake-H variant.
BugLink: https://github.com/thesofproject/sof/issues/5018
Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
The most important fix is the addition of the "flush" argument which
means are commands are now printed when they are run and not all at the
end when everything is done.
Commands run by this script are very few so they are always shown. This
makes it possible to debug just this script without being completely
drown in build noise. All '-v's are passed to west.
Pass -v to `west sign`, not just to `west build`.
Drop the manual "-DCMAKE_VERBOSE_MAKEFILE=ON", west knows how to do that.
Print directories first so commands are aligned, more visible and can be
copied. Use shlex to quote whitespace.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Make it possible to invoke -C multiple times which is required to
support whitespace as in:
-C=--warn-uninitialized -C '-DEXTRA_FLAGS=-Werror -g0'
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Existing xtensa-build-zephyr.py failed to copy sof logger
executable to staging directory due to missing '.exe' file extension.
This fixes the script.
Signed-off-by: Andrey Borisovich <andrey.borisovich@intel.com>
Created new script with similar functionality to allow SOF + Zephyr
builds on both Windows and Linux operating systems.
Signed-off-by: Andrey Borisovich <andrey.borisovich@intel.com>
Fixes incremental builds of topology2 that were missing ALSA_CONFIG_DIR.
This was making it impossible to troubleshoot alsatplg issues like #5249.
Now incremental builds fail or pass the same as builds from scratch.
Fixes commit 308a24a92b ("topology2: Add build support")
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
The -a option was recently broken by the addition of platform(s) that
don't have a toolchain available in the Docker image. Every platform can
be built by someone but no one can built -a(ll) platforms right now.
Add a new platform array to make the distinction between default
platforms built by the -a option versus work in progress.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
No one else has it.
Gets rid of the only shellcheck warning.
Fixes commit 7453e3d571 ("scripts: add renoir support")
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
scripts/ has kconfig defaults and CMake code that can affect the
dictionary. Note this does not fix#3890 because .config (and maybe
others) are still not hashed but it helps a bit.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
execute_process() runs in the current directory of the process invoking
cmake. This can be completely outside the git repo.
Fixes commit a5899812b7 ("version.cmake: don't trust CI to record time
and versions and log ourselves")
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Include DEFAULT_PLATFORMS in SUPPORTED_PLATFORMS instead of the other
way round. SUPPORTED_PLATFORMS will always be bigger than
DEFAULT_PLATFORMS and it's easier to add a platform than remove one.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Passing spurious --board and source directory arguments to an already
build directory does not help but it does not hurt either and it
simplifies the code. It also provides a more consistent west command in
the set -x "logs", one that can be re-used in any circumstance.
The only restriction is to make sure CMAKE_ARGS is empty on all but the
first invocation but that's not new and unchanged here.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Zephyr also a version.h too so the situation was a buggy #include mess
that (among others) dropped the SOF git version from the SOF banner when
using Zephyr. It's absolutely impossible to fix this mess while keeping
two .h files both named exactly the same and even included with the
exact same '#include version.h' line, I mean no sof/version.h or other
prefix.
As a bonus, this rename also reduces the XTOS-only confusion between
"version.h" and cavs/version.h".
Generating a file in the source tree is a serious bug (to be fixed
later), it means this file is never cleaned. Do not hide this bug with a
.gitignore rule.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
We should gradually switch to ninja as a default.
ninja-build is apparently missing from build-essential
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
It's super annoying and time-consuming to update the image and wonder
why the previous one is still used. Shortcuts are nice in interactive
use but pointless in scripts.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
There's frequent confusion between image ID and image digest: display
both.
Show both sof and thesofproject/sof.
Use set -x to display the full command that is run.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Add build support for topology2. Topology2 will be built only if the
alsatplg version if 1.2.6 or greater.
Signed-off-by: Ranjani Sridharan <ranjani.sridharan@linux.intel.com>
Align with xtensa-build-all.sh and add a "-v" option to toggle
verbose build log ("west -v build").
Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Fixes new shellcheck 0.8.0 warning:
SC2268 (style): Avoid x-prefix in comparisons as it no longer serves a
purpose.
This script requires bash anyway.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Fixes old (Aug 2020) and untested commit b2a325a3b7 ("cmake: Handle
empty SOF_SRC_HASH_LONG") which tried to use GIT_LOG_HASH as a fallback
for the SOF_SRC_HASH .ldc checksum but always fell back on "0" instead
because of a misunderstanding of cmake's surprisingly complex "if"
operator.
Before Zephyr this was not an issue in practice because GIT_LOG_HASH was
empty anyway when SOURCE_DIRECTORY/.git was missing, so there was
nothing to lose.
For the Zephyr builds this will now use the SOF SHA1 as the .ldc checksum
for now. Probably not ideal yet but far better than the current constant
"0" which means no check at all.
Also log the SOF_SRC_HASH fallback value now and change the second
fallback (when GIT_LOG_HASH is also missing) from "0" to the
searchable hexspeak "baadf00d".
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
This is a logical revert of eb45907
Based on recently merged #4941, C99 comments are now OK. I never found
any rationale or even written down coding style for excluding them in
the first place.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Recent commit b9fcc4156e ("xtensa-build-zephyr: fix incremental build,
pass CMAKE_ARGS only once") fixed incremental builds. For this it first
needs to guess whether the most recent CMake configuration step failed
or succeeded so it knows what is the current "checkpoint" and what is
the next west command to run. Merely testing whether the build directory
already exists is a very bad guess because that directory could have
been created by some other person or script, or - much worse - it could
have been created by a failed CMake configuration step!
The latter situation makes debugging CMake failures very tedious because
it requires manually deleting the build directory after each CMake
configuration failure.
Fix this by not merely looking for a build directory but by looking
whether there is either a build.ninja or a Makefile is found in that
directory. This is a much better indication that the CMake configuration
succeeded.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
The standard location of dictionary.txt is under codespell's package, on
my machine atm (codespell 2.1, Artix Linux):
/usr/lib/python3.9/site-packages/codespell_lib/data/dictionary.txt
Since we enable the codespell by default for SOF I have constant:
No codespell typos will be found - file '/usr/share/codespell/dictionary.txt': No such file or directory
The patch proposes to try to fix up the path following the
recommendation found here:
https://github.com/codespell-project/codespell/issues/1540
Mainline commit:
0ee3e7b8893e ("checkpatch: get default codespell dictionary path from package location")
Link: https://lkml.kernel.org/r/29e25d1364c8ad7f7657cc0660f60c568074d438.camel@perches.com
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
Acked-by: Joe Perches <joe@perches.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Default behavior is still to clone the default Zephyr branch (main) but
this could be changed with just one line.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Zero functional change.
Rename clone() to west_init_update() and build_all() to
build_platforms(). Also expand description of the west_init_update()
feature in the usage message and add a direct SHA1 fetch example.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
When compiling with Zephyr, SOF_ROOT_SOURCE_DIRECTORY is changed to
sof/zephyr/ and sof/zephyr/.git does not exist. Change error message to
show SOF_ROOT_SOURCE_DIRECTORY/.git
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Update the scripts/test-repro-build.sh test accordingly.
Finally, different people and CI systems can make sure they're building
the same firmware.
reproducible.ri is sof.ri with variable parts removed.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Also convert 'differ' to all caps 'DIFFER' to highlight different files.
No other functional change.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Zephyr XCC builds for Apollolake can be re-enabled now after the RAM
footprint has been reduced.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
... because there's no reason to make one depend on the other.
Same for the opportunistic creation of the sof symbolic link.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Do not define a default WEST_TOP at the very start because we may not
have one. Define WEST_TOP only when the user gives one or when we
actually find one. And search harder.
Do not trust optional WEST_TOP user input blindly either. This was
leading mkdir -p to create a bogus modules/audio/ directory.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
In order to support AMD as part of the docker image and CI we need to
include it in the build list as the overlays are already added to the
overlay repo.
Signed-off-by: Curtis Malainey <cujomalainey@chromium.org>
"tree" is very useful but it's not critical, not worth failing the build
for.
Also remove a single quote from the help message as it breaks parsing
in one simple editor I use (jed) and possibly others.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Without CMP0079 we cannot conditionally include libraries against SOF in
sub directories without seriously restructuring the project. This is
because the old policy requires the link target must be created in the
same folder. This does not work well from a configuration standpoint for
3P audio libraries trying to keep their config in src/audio/*. Rather
than enable the policy, lets simply upgrade since 3.13 is widely
available.
With this upgrade we can also remove the two version dependent checks at
the top of our scripts.
Signed-off-by: Curtis Malainey <cujomalainey@chromium.org>
This is needed by CI to clone only once and then take control of the
iteration over platform builds. CI already does all that but by
copy/paste/diverge of this clone() function.
Once cloning Zephyr in CI code is de-duplicated and uses this new
clone-only feature instead, it will be possible to submit any Zephyr
commit to SOF testing. In other words it will make failed attempt
https://github.com/thesofproject/sof/pull/4728 possible
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
main() was growing too big. Zero functional change.
Also rename the too generic "build()" to build_all()
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Recently merged rework of multicore support in Zephyr has
broken builds with XCC. Disable XCC based builds in CI until
issue is resolved in Zephyr mainline. Issue tracked as:
https://github.com/zephyrproject-rtos/zephyr/issues/38349
Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
We care about other build errors more.
This also makes it easier to test locally that "west build" does not
depend on rimage to create zephyr.elf
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
This is motivated by the zephyr.elf build that does not need rimage at
all.
Also build smex later, only when it's needed.
After building and deleting the rimage/ clone, a plain SOF clean +
RE-build now fails much later: it now fails AFTER successfully building
the sof binary, only when it tries to build either bootloader,
boot_module or base_module that actually need rimage:
[ 87%] Performing configure step for 'rimage_ep'
CMake Error: The source directory "/home/SOF/sof/rimage" does not appear
to contain CMakeLists.txt.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
The BSS usage exceeds available memory when building for
APL/cavs15 hardware with XCC, so disable it for now. Issue
tracked as https://github.com/thesofproject/sof/issues/4645
Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Remove the unnecessary assumption that Xtensa tools are installed under
'XtDevTools'.
Align indentation to use tabs as is used elsewhere in this file.
Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Add support to build with XCC compiler by setting XTENSA_TOOLS_ROOT
env variable. The interface follows xtensa-build-all.sh, so existing
CI scripts for SOF builds can easily update to build with XCC.
Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Add imx8 platform for local testing.
TODO: Update xtensa-build-zephyr.sh when Zephyr repo
is updated with imx8 support.
Signed-off-by: Iuliana Prodan <iuliana.prodan@nxp.com>
Adds --error-exitcode=1 to valgrind options (otherwise what's the point
of using valgrind?)
Skip alloc test that does not pass on HOST (passes with xt-run)
Add help message.
Runnable from anywhere.
Use shell functions
https://github.com/thesofproject/sof-test/issues/740
Fix all quoting issues and other shellcheck warnings.
Add comments.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Link to the CMake section in sof-docs which documents a lot of what this
script does and is probably out of date now. It would be less out of
date if this link had been there and more people had been aware that
this documentation even existed.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Thanks Ranjani for finding most of these magic values.
Also move READY_MSG closer to the only place where it's used so it does
not look like a "global" anymore.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
-p is required when the real (not symbolic) directories are not nested
in one another.
The documentation gap was clearly demonstrated by me getting confused
even though _I_ implemented this logic some time back.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Ready to be transferred to the remote system. Sample output:
build-sof-staging
├── sof
│ ├── community
│ │ ├── sof-apl.ri
│ │ └── sof-tgl-h.ri
│ ├── sof-apl.ldc
│ └── sof-tgl-h.ldc
└── tools
└── sof-logger
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
This enables support for running the allocator mocks with valgrind by
building the allocator for the host library target using a similar
heap map to Intel CAVS targets (memory.c is almost identical copy of CAVS
version).
Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Zephyr's incremental build works and saves time. I have been using it
with SOF for a long time. Slowly rebuilding from scratch is as simple as
deleting the build-$platform/ build directory. Quickly building not from
scratch is not possible without this commit.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Zero functional change.
Makes the code and the build logs clearer, especially for SOF people new
to west. Short options are for saving repetitive typing; no reason to
use them in a script.
Also show the real /bin/pwd which can help when west gets lost in
complex symbolic link situations.
https://github.com/zephyrproject-rtos/west/issues/419
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Catching up latest LTS Ubuntu image, 20.04. It is supposed to be minor
upgrade but alsa-lib need to be set default library path,
/usr/lib/x86_64-linux-gnu/. Otherwise topology library loading error
is found for SOF tool build. This is typical error message,
alsatplg: error while loading shared libraries: libatopology.so.2:
cannot open shared object file: No such file or directory.
Signed-off-by: Fred Oh <fred.oh@linux.intel.com>
There was a assumption /etc/apt/apt.conf has apt proxy settings for
the system. This workaround is not required.
Signed-off-by: Fred Oh <fred.oh@linux.intel.com>
More robust support for "manual" installations.
Also rephrase usage string to remove a single quote breaking
the shell parser in one editor I use (sorry).
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
By default, the Zephyr build does not log its version. The fix has
already been submitted at
https://github.com/zephyrproject-rtos/zephyr/pull/34842 however:
- 34842 may take some time. Zephyr reviewers are even busier than the
SOF reviewer.
- 34842 could even be rejected
- 34842 will at best log the SHA1 and no context. This is more verbose
and future-proof because it supports more complex situations like
testing a pull/12345/merge
As seen in (unrelated) PR #4123, we need Zephyr git versions NOW.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Fixes 33c5f5d38e ("xtensa-build-zephyr: do not clone a second version of
sof.git")
The very first version of this test was checking whether the rimage/
directory exists, which proves nothing. So I switched to
rimage/CMakelists.txt but forgot change the `test` flag.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
This script lives in a sof.git/ clone yet it was systematically cloning
a second sof.git/. Besides the obvious confusion and risk of editing the
wrong files, this meant it was not possible to build code that has not
been merged yet! This was a problem for both CI and developers. Fixed by
using symbolic links to ourselves instead.
Note it is _still_ possible to build from another sof.git clone if
desired, however this script will never git re-clone a second sof.git
itself, that second clone has to be created (e.g.: by west) before this
script runs.
When cloning a brand new zephyrproject, use a shallow zephyr clone and
download only the two zephyr modules we actually use. This speeds up
automation considerably and makes it much faster for non-Zephyr
developers to reproduce Zephyr issues. Developers can always git
unshallow and west update once if they want to.
Rename the default west top to "zephyrproject" to not just match the
zephyr documentation but to also avoid creating a double zephyr/zephyr/
directory.
See the new print_usage() for a few more implementation details.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
It still looks smart but the \n addition compared to the original
version in xtensa-build-all.sh broke it for more advanced cases:
die '%s %d' str 5
-bash ERROR: -bash: printf: 5\n: invalid number
As reported by shellcheck. shellcheck saves lives.
In scripts/xtensa-build-zephyr.sh line 22:
>&2 printf "$@\n"
^-- SC2145: Argument mixes string and array.
Use * or separate argument.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
... because the latter does not print on stderr when not found; so IDEs
and CIs don't display this as a warning.
Also remove the "... and ignore set -e" comment that made sense at the
time of commit cad86dc340 ("scripts: xtensa-build: fix alias detection
for bxt/apl gcc") but not anymore.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Allows the mocks to be quickly run on the host with full access to host
debug and development tooling.
Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
In an ideal world, every CI engine records and shares the most important
CI information:
- current date and time in a well identified timezone
- git version of the pull request
- git version of the moving branch the PR is being merged with
In the real world we have multiple CI solutions and they unfortunately
cannot not all be trusted to perform their most basic job
correctly. Fortunately, they all make at least build logs available so
these very few lines of code adding very few lines of output cost near
zero extra build time and solve the problem once for all. I feel stupid
I didn't do this sooner, this would have saved me hours and hours in
vain requests and discussions and in trying to puzzle that information
together.
Sample output:
-- Preparing Xtensa toolchain
version.cmake starting SOF build at 2021-03-31T18:09:46Z UTC
Building git commit with parent(s):
150fd1e4c968 4249bdb1b305 [other parent if merge] (HEAD -> main) cmake: ...
-- GIT_TAG / GIT_LOG_HASH : v1.7-rc1-174-g150fd1e4c968-dirty / 150fd1e4c968
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Required by #3459 / #3975 "elfsize" proof of concept and probably by
other things too in the future - we use both ELF and Python
everywhere; it's surprising they haven't met each other yet.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
For at least two reasons:
- exposes sneaky change(s) performed by automation if/when any
- solves the mystery of the Source content hash (printed on the next
line) changing while the git version does not.
Example, at https://sof-ci.01.org/sofpr/PR3941/build8429/build/bdw_gcc.txt
-- Found Git: /usr/bin/git (found version "2.17.1")
-- GIT_TAG / GIT_LOG_HASH : v1.7-rc1-151-g023c4abacde1 / 023c4abac
-- Source content hash: 91f261ea
whereas at https://github.com/thesofproject/sof/runs/2166298087,
xtensa-build-all:
-- Found Git: /usr/bin/git (found version "2.17.1")
-- GIT_TAG / GIT_LOG_HASH : v1.7-rc1-151-g023c4abacde1 / 023c4abac
-- Source content hash: 67f31697
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
The configuration names with _2017 are a consequence of the way I did
the internal configuration update. To keep it simple I update these
build scripts.
Signed-off-by: Paul Olaru <paul.olaru@nxp.com>
XTOBJCOPY and XTOBJDUMP were added in June 2018 by
commit 27795ece0f ("scripts: fix xt-xcc build with wrong config") for
`./configure CC=$XCC OBJCOPY=$XTOBJCOPY OBJDUMP=$XTOBJDUMP ...` that was
used at the time.
The build system was switched to CMake in January 2019, notably with
commit 9840ecbbfe ("cmake: update xtensa-build-all.sh") and
commit 0fd97adfb0 ("cmake: add utility scripts") that defined
CMAKE_OBJCOPY and CMAKE_OBJDUMP)
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
$ARCH was introduced in 2018 by commit b5af84deaa ("scripts: add smp
builds to xtensa-build-all.sh"). At the time used autotools and
`./configure --with-arch=$ARCH` were used.
In January 2019 the build system was migrated to CMake over several
commits, notably commit 9840ecbbfe ("cmake: update
xtensa-build-all.sh") that switched xtensa-build-all.sh to CMake and
removed its only use of ARCH. Also note commit 82b4da291b ("cmake:
defconfigs support") which added a few CONFIG_SMP defconfigs and
commit 905bad4252 ("scripts: xtensa-build-all: Add support to force
build UP Arch") using override.config.
Much more recently in June 2020: commit fd506970cc ("zephyr: kconfig:
rename CONFIG_SMP -> CONFIG_MULTICORE")
Zawinski's Law of Software Envelopment: adding lines of code is much
easier than removing the same amount and would be very impractical
without git. This one was easy enough.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
As CMake forks one compiler process for each source file, the XTensa
compiler spends much more time idle waiting for the license server over
the network than actually using CPU or disk.
On my VM with 16 virtual cores, rebuilding one platform from scratch
with this commit goes down from 12s to less than 9s: more than 25%
faster. With Ninja it goes down from 11s to less than 8s. My license
server is 25ms away: a closer server does not need as many threads while
a more distant server would obviously benefit for even more
threads... while already getting an even better improvement than 25%
from just 3 times more threads! It's complicated and we probably don't
want to start the build by measuring latency to the license server.
The entire, purely local _gcc_ build is so fast (~ 1s) that observing any
the difference between -j nproc and -j nproc*N is practically impossible
so let's not waste RAM when building with gcc.
Also: log the $XTENSA_SYSTEM variable as it is required for incremental
builds; remove one apostrophe in the here-doc usage as it breaks the
parser of some editor (jed).
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Notably:
- Log error and default MEU_OFFSET when meu -ver is not found.
- Explain rimage -s option
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Notably: xt-xcc --show-config failed with: No such file or directory
... when the directory exists but is wrong.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
There is no such compiler as RF-2017.8 that is in use. The correct
compiler is RG-2017.8 for all i.MX platforms.
Signed-off-by: Paul Olaru <paul.olaru@nxp.com>
Add cmake -DINIT_CONFIG= option that can point at any initial file.
"make clean" does not delete .config any more.
Note reconfiguration does NOT causes recompilation because -imacros
hides the generated .h from CMake's dependency scan. This is not a
regression, that problems exists since -imacros was introduced. At least
it's now possible to "make clean" and rebuild without losing the .config
file.
Fix for #3617
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Lets use the same version of Xtensa toolchain as Intel in order
to avoid compilation problems.
It is difficult to support multiple toolchain versions because specific
headers needs to be updated, so lets have the same version supported
between IMX and Intel.
Signed-off-by: Daniel Baluta <daniel.baluta@nxp.com>
The script already logs the full CMake command that is re-usable outside
this script... except for the PATH change. Expose that sneaky PATH
change.
Debugging every build issue starts with peeling the too many layers of
indirection.
Also fix some minor issue in the help message.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
This makes it possible to switch to Ninja with a single line change
which can be useful to test build changes and issues.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Currently xtensa-build-all.sh has restriction to use rimage for tgl
signing, because rimage didn't support tgl. As rimage now has support
for tgl signing let's remove this limitation.
Signed-off-by: Jaska Uimonen <jaska.uimonen@intel.com>
"make + make install" duplicates many steps, probably because of the
fatal combination of build timestamps + source hash, see PR #3353 for a
similar example.
A single, combined "make install" generates the exact same binary
outputs 30% faster and prints 30% shorter build logs.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Following commit 88b69cc2e5 ("scripts: xtensa-build-all: Add support
for building TGL and signing with MEU") we now have an unified and
flexible PRIVATE_KEY_OPTION that can do anything with very little code.
The -k option was never a good idea in the first place because it
already required an environment variable anyway (RIMAGE_PRIVATE_KEY)
instead of a command line argument so the code should have just checked
whether RIMAGE_PRIVATE_KEY was defined, that would have been
enough. Requiring the user to "double-confirm" with -k has been adding
extra complication for both the user interface and the implementation.
xtensa-build-all is effectively a "CMake configuration convenience"
script, however it shouldn't become an additional layer of indirection
and complexity and its interface should expose CMake (which is already a
layer of indirection!) as directly as possible.
See longer discussion in PR #3187
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
We switch to PCH-based names instead of CPU config names for tigerlake+
platforms, because we need to support multiple variants of PCHs based
on one CPU famile - for example TGL-H & TGL-LP.
Signed-off-by: Janusz Jankowski <janusz.jankowski@linux.intel.com>
As incredible as it sounds, some people run neither "git status" nor
"git diff" every few minutes and not even when their build fails. There
has been reports that they're puzzled when they miss a required
submodule update. This is an attempt to draw their attention based on
the assumption that they pay more attention to the CMake logs.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Pass PRIVATE_KEY_OPTION environment variable to docker to be able to
define external key for signing.
Suggested-by: Marc Herbert <marc.herbert@intel.com>
Signed-off-by: Jaska Uimonen <jaska.uimonen@intel.com>
Add support for building TGL FW and signing with MEU tool. Signing
with rimage is not supported atm.
Signed-off-by: Ranjani Sridharan <ranjani.sridharan@linux.intel.com>
Some platforms needs platform-specific configuration
that override the default config. For example, chrome platforms
need the LPS to be enabled by default for maximizing the
power savings for the WoV feature. Add a -o switch to allow
specifying the override config file.
Signed-off-by: Ranjani Sridharan <ranjani.sridharan@linux.intel.com>
This gets rid of about 600 lines of noisy "Entering directory..."
messages which cuts the size of the output of the script in half leaving
only useful stuff on the screen. This (and other changes before it) may
finally avoid bugs like 'commit f430addec7 ("Tools: Fuzzer: Do not use
illegal BUILD_COMMAND in CMakeLists.txt") evading scrutiny for months.
After so many years enjoying the convenience of make's '-C' option, I
finally did some research and realized it is the cause of this very
serious noise issue with no other easy way out :-(
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
... thanks to the docker --env option which is smart enough not to pass
anything when there is nothing to pass.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
To produce the exact same binaries than before this commit:
CMAKE_BUILD_TYPE=' ' ./scripts/build-tools.sh
To observe which CFLAGS are being used:
CMAKE_BUILD_TYPE=Release VERBOSE=1 ./scripts/build-tools.sh
Off topic: the logger fails to build with: -O3 -DNDEBUG -Wall -Werror
(all other tools build fine with these flags)
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Default is verbose enough. VERBOSE=1 can still be used later at make
time.
Also change nproc to nproc --all because we don't care about offline
processors.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Use newer rebuild-testbench.sh instead, added by
commit 46578cb103 ("scripts: Add a new script to rebuild testbench")
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
SOF_SRC_HASH always must have integer value, because of usage
them to initialize global variable in source code.
Variables, which may be empty should be used inside quotation
to prevent cmake incorect number of arguments error.
Signed-off-by: Karol Trzcinski <karolx.trzcinski@linux.intel.com>
Temporary files are useful to understand and debug the build.
Moving output to another directory, keeps binary directory clean.
Signed-off-by: Karol Trzcinski <karolx.trzcinski@linux.intel.com>
This is possible location of .git folder, instead of CMAKE_SOURCE_DIR.
Without this patch, checked condition always return false and
GIT_LOG_HASH is used instead source code hash.
Signed-off-by: Karol Trzcinski <karolx.trzcinski@linux.intel.com>
Currently, the testbench is built using the script named
'host-build-all.sh' under scripts directory. But the name of
the script, it's git history doesn't refer to what the script
is currently doing now.
The newly written script adds option to build the testbench.
This script was written in mind to remove the old script
used to build the testbench (i.e host-build-all.sh).
The script also adds an option to build the testbench with
AFL instrumentation. AFL fuzzer works well when the code
it's trying to fuzz is instrumented properly. The instrumentation
helps the fuzzer in generating unique inputs which cover
different paths in the code graph. When the option is set,
testbench is built using a compiler AFL provides, which does
the instrumentation work.
Signed-off-by: Mohana Datta Yelugoti <ymdatta.work@gmail.com>
GIT_TAG is user readable form of used source code version with
commit identifier, what is important for bugs reproducibility,
so it will be convenient to have this information in output logs.
Signed-off-by: Karol Trzcinski <karolx.trzcinski@linux.intel.com>
It may be used to check FW compatibility with ldc file.
It's much better than comparing DBG_ABI because logs content
may be updated without any DBG_ABI change in opposite to source
code hash value.
Signed-off-by: Karol Trzcinski <karolx.trzcinski@linux.intel.com>
Build ALSA locally under /home/sof/work and don't delete it. This will
allow ALSA development within the container.
Change the folder owner to sof to avoid permission issue.
Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Signed-off-by: Pan Xiuli <xiuli.pan@linux.intel.com>
This kconfig collides with one of the same name in Zephyr. Rename it
for clarity ("MULTICORE" was picked for symmetry with the existing
"CORE_COUNT", though it's admittedly a little long...).
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
In renaming of $j -> $platform one instance of variable was missed which
has led to wrong configs being used.
Fixes: 941dfcccb0 ("use switch case")
Signed-off-by: Shreeya Patel <shreeya.patel23498@gmail.com>
Stop drowning and missing other warnings.
Example:
./scripts/checkpatch.pl -g eaf49e26b4bb~0
- 40 "Please update ABI..." warnings
- 2 other warnings including a spelling mistake that were missed.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>