Fixes pylint warning `scripts/xtensa-build-zephyr.py:426:5: W0621:
Redefining name 'platform' from outer scope (line 14)
(redefined-outer-name)` and reduce "grep confusion": the script uses
`platform` A LOT.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
There's just too many of them and fixing them would cause a lot of churn
at a wrong time (big migration to west happening now)
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
shlex.join() requires 3.8, maybe others too. 3.8 is 3 years old and it's
the default Python version in Ubuntu 20.04
Fixes#6121
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
West tool returns on windows OS different white characters
than on Linux OS. Paths outputted from west tool had been
stripped for any trailing whitespaces.
Fixes "xtensa-build-zephyr.py: use west manifest"
(608833cf27)
Signed-off-by: Andrey Borisovich <andrey.borisovich@intel.com>
Stop immediately when `west build` fails. Do no keep going and show a
totally unrelated smex build failure.
Also fix error message when zephyr/ is missing.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Make the new -p pristine build option also delete the staging_dir. This
solves a problem where the following sequence was leaving a stale
platform1 build in staging:
xtensa-build-zephyr.py -p platform1
< code >
xtensa-build-zephyr.py -p platform2
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Added versioning to scripts/xtensa-build-zephyr.py to get version
information when incompatible changes are done to the script.
Added yml schema version number to west.yml manifest.
Signed-off-by: Andrey Borisovich <andrey.borisovich@intel.com>
xtensa-build-zephyr.py had been using some execute_command
calls when building rimage and using rimage keys over
symlink from modules/audio/sof. Changed it so that calls are
executed now over normal sof directory and symlink is not needed
for script execution.
Signed-off-by: Andrey Borisovich <andrey.borisovich@intel.com>
Added new flag for non-interactive mode that should be used
when script is invoked programmatically.
In default (interactive) mode user is asked using a prompt whether to
reinitialize west manifest if it is initialized to manifest other
than SOFs.
Signed-off-by: Andrey Borisovich <andrey.borisovich@intel.com>
Added usage of west manifest in the script to update both
SOF and zephyr dependencies.
Renamed "clone-mode" to "update" flag - the logic where
sof dependencies and zephyr with dependencies is cloned.
Removed "point-mode" from script as it will no longer work
(west manifests require fixed directory structure).
Removed "zephyr revision" or "-z" argument as it is no longer
needed (now revision may be set in west.yml manifest).
Added "-p" pristine flag to rebuild platforms while removing build
directory.
Co-developed-by: Krzysztof Frydryk <krzysztofx.frydryk@intel.com>
Signed-off-by: Krzysztof Frydryk <krzysztofx.frydryk@intel.com>
Signed-off-by: Andrey Borisovich <andrey.borisovich@intel.com>
This script had been rewritten to python xtensa-build-zephyr.py
script and is no longer used in CI.
Due to changes to sof processes related to Zephyr integration
this script will no longer be maintained.
Signed-off-by: Andrey Borisovich <andrey.borisovich@intel.com>
Remove overlays as they are the same board configurations available in
app/.
Keep common overlays and change script to reflect those changes.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
This will significantly speed up CI in some cases.
Fixes commit ffdf001eee ("xtensa-build-zephyr.py: clone zephyr with
--depth 5")
When I added --depth 5 to git clone in that commit I naively assumed git
fetch would "inherit" that. I think it does but only when fetching an
_existing_ (and shallow) branch, not when fetching something new.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
To track the docker images, need to keep all the hashes of the projects
builts in the image. Log git tree information to
/home/sof/work/sof_git_hash.txt in the docker image.
Signed-off-by: Fred Oh <fred.oh@linux.intel.com>
There are some restrictions when --fw-naming is set to 'AVS' for the
sake of providing reasonable FW outputs that make sense.
Restriction 1: When argument fw_naming is set as AVS, IPC4 ipc version
will be automatically chosen.
This is because --fw-naming 'AVS' is added for supporting IPC4 SOF build by
nature.
Restriction 2: When argument fw_naming is set as AVS, using platform
subdirectories will be automatically chosen.
The reason is that the platform subdirectories actually belong to the
"AVS" naming scheme, we must enable using platform subdirectories when
building FW in AVS naming convention.
This also ensure that we can tell firmwares of different platforms apart
by organizing them in different subdirectories.
Signed-off-by: Xiaoyun Wu(Iris) <xiaoyun.wu@intel.com>
Add argument option --fw-naming for specifying FW output with its
filename set with AVS style(filename as dsp_basefw.bin specifically)
OR SOF style(filename as sof-{platform}.ri). SOF style is by default.
This will be useful when building SOF+Zephyr IPC4 FW, because
dsp_basefw.bin will be loaded by default when snd_sof_pci parameter
ipc_type is set to 1 (which indicates running IPC4 mode).
Signed-off-by: Iris Wu <xiaoyun.wu@intel.com>
Quoting https://github.com/zephyrproject-rtos/zephyr/pull/46880#pullrequestreview-1020096140
> No complaints about the code here. Just a general comment:
> I do worry that as cavstool grows features (first to add the test
> integration support, now the new client/server architecture) that
> we're obscuring the core code that does the loading behind a big
> facade of other stuff that also has to be run In Just The Right Way to
> make things work. That impedes development flow. It might be time to
> split the code up so that a minimal "just run this file please" tool
> can continue to be maintained.`
Long story short: `cavstool.py` is likely to be split in the future.
To be readi-er, copy the entire `zephyr/soc/xtensa/intel_adsp/tools/`
directory.
Copying the entire directory also copies scripts that will never be used
on a test device like `cavstool_client.py`. They're small Python script
sso we don't care: keep it simple.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Local filters in ~/gitconfig, such as
[core]
autocrlf = input
can impact the result of git hash-object. Make sure no filters are
used so that the hash value remains unmodified across user setups.
BugLink: https://github.com/thesofproject/sof/issues/5917
Signed-off-by: Pierre-Louis Bossart <pierre-louis.bossart@linux.intel.com>
When neither -c nor -p is passed, the script defaults to '-p
SOF_TOP/zephyrproject', initializing the `west_top` global with
that. When everything is already cloned, this becomes
zephyrproject/modules/audio/sof/zephyrproject which is really not a
`west topdir`. Then west is pointed at:
real_west_top/modules/audio/sof/zephyrproject/zephyr/samples/subsys/audio/sof
which obviously fails.
This was found while testing
https://thesofproject.github.io/latest/getting_started/build-guide/build-with-zephyr.html
and trying (and failing) to switch away from the older .sh script.
Fix find_west_workspace() to return the output of `west topdir` and use
that to update the `west_top` global.
This bug was introduced from the very start when the script was
converted from shell to python. The shell script did not have this
problem because it could be used without using neither -c nor -p. This
logical difference was mentioned in the code review:
https://github.com/thesofproject/sof/pull/5299#discussion_r801158177
Also remove a couple of accidental creations of a sof/zephyrproject/
directory when sof is already inside a higher zephyrproject.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
If each platform has its own output subdirectory, each platform output
subdirectory will include a key subdirectory with same name. Function
show_installed_files() can fail to find the correct parent for a node
as it only compares the subdir name that can no longer be unique.
This patch uses Anynode to replace Node class to define a new
attribute 'long_name' for a node to store its unique relative path
from the output staging directory (e.g. 'sof/apl/community') and
search for its parent.
The attribute 'name' is still used to store the subdir or file name
for a node (e.g. 'community'), and to display the layout of the
installed firmware files.
Signed-off-by: mengdonglin <mengdong.lin@intel.com>
All firmware files are installed in the same staging directory by default.
This patch adds an option --use-platform-subdir to create an output
subdirectory for each platform.
e.g. $scripts/xtensa-build-zephyr.py apl jsl tgl --use-platform-subdir
The layout of installed files will be like this:
build-sof-staging
├── sof
│ ├── apl
│ │ ├── community
│ │ │ └── sof-apl.ri
│ │ └── sof-apl.ldc
│ ├── tgl
│ │ ├── community
│ │ │ └── sof-tgl.ri
│ │ └── sof-tgl.ldc
│ └── jsl
│ ├── sof-jsl.ldc
│ └── community
│ └── sof-jsl.ri
└── tools
├── sof-logger
└── cavstool.py
Signed-off-by: mengdonglin <mengdong.lin@intel.com>
Add argument option --key-type-subdir <key_type> to choose the output
subdirectory name based on the type of rimage signing key.
User can choose subdirectory name from 'community','dbgkey' and 'none'.
The default one is 'community', the type of default rimage signing key.
We can extend the list later.
When the key type is 'community' or 'dbgkey', the layout of SOF output
directory will be like:
build-sof-staging
├── sof
│ ├── sof-platform1.ldc
│ ├── sof-platform2.ldc
│ ├── ...
│ └── <key_type>
│ ├── sof-platform1.ri
│ ├── sof-platform2.ri
│ └── ...
└── tools
├── sof-logger
└── cavstool.py
When the key type is 'none', there will be no key type subdirectory:
build-sof-staging
├── sof
│ ├── sof-platform1.ldc
│ ├── sof-platform1.ri
│ ├── sof-platform2.ldc
│ ├── sof-platform2.ri
│ └── ...
└── tools
├── sof-logger
└── cavstool.py
Signed-off-by: mengdonglin <mengdong.lin@intel.com>
Do not fail when there's only an empty directory left over
because of some early interruption of this script or proxy
misconfiguration, wrong fetch location, etc.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Extract SOF firmware file version string major.minor.micro from the
generated sof_version.h, and pass the version string to rimage in the
signing phase. So rimage can write the version info to the firmware
manifest headers for cAVS platforms.
Signed-off-by: mengdonglin <mengdong.lin@intel.com>
This removes a small but conflicting difference with upstream.
Fixes commit d98a7ed103 ("checkpatch: check ABI updates before warning
on it")
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Use a separate if(!$SOF) to minimize differences with upstream.
Also remove some spurious whitespace differences with upstream.
Zero functional change.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
These attributes were added to Zephyr's include/zephyr/debug/sparse.h by
commit 17eb313a1b15 ("sparse: add an address space and a __sparse_force
annotation")
This gets rid of dozens of warnings in
https://github.com/thesofproject/sof/runs/6387526478
```
ERROR: need consistent spacing around '*' (ctx:WxV)
+ struct comp_buffer __sparse_cache *buffer_c, *sink_c;
^
ERROR: need consistent spacing around '*' (ctx:WxV)
+ struct comp_buffer __sparse_cache *source_c;
^
```
etc.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Add a new -d option to support SOF debug build. This
option will add CONFIG_DEBUG=y to SOF configuration,
which is aligned with xtensa-build-all.sh script.
The -d option is a shortcut for '-o debug_overlay_path'.
Signed-off-by: Chao Song <chao.song@linux.intel.com>
This can fail for various reasons listed in a comment.
Remove now redundant `west topdir` check.
Without this check, a not found SOF fails with very cryptic Kconfig
errors:
zephyr/samples/subsys/audio/sof/prj.conf:5:
warning: attempt to assign the value 'n' to undefined symbol HAVE_AGENT
zephyr/samples/subsys/audio/sof/prj.conf:8:
warning: attempt to assign the value 'n' to undefined symbol
DEBUG_MEMORY_USAGE_SCAN
zephyr/samples/subsys/audio/sof/boards/intel_adsp_cavs15.conf:1:
warning: attempt to assign the value 'y' to undefined symbol APOLLOLAKE
etc.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
This fixes SOF version.cmake which was just broken by a recent git
security update and started to fail like this:
```
-- SOF version.cmake starting at 2022-04-25T18:14:56Z UTC
-- /workdir/zephyr/.. is at git commit with parent(s):
fatal: unsafe repository ('/workdir' is owned by someone else)
To add an exception for this directory, call:
git config --global --add safe.directory /workdir
```
(example at https://github.com/thesofproject/sof/runs/6162885265)
chgrp -R was always an ugly hack because it was messing with
(persistent) file permissions on the host, outside the container. This
new adduser solution is unfortunately much more code but it does not
leak any side effect outside the container.
Do not fix scripts/docker-run.sh yet because there is still no UID
mismatch between Github Actions and the SOF container (they're both
1001) but add a warning + TODO.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
There was no actual "IPC3" value, it was ignored. The platform default
was used instead.
Stop ignoring invalid arguments, fail and report them.
Allow empty string as a no-op for scripting convenience.
Don't restrict IPC4 to tigerlake, it's not the job of
xtensa-build-all.sh to filter what's supported versus not.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Commit 475e09f17d ("xtensa-build-zephyr.py: default to
subprocess.run(check=True)") switched the default value of the
execute_command() function to "True". Remove all the "check=True"
arguments that are now redundant.
Zero functional change.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
There are warnings about bison and flex not available.
Interestingly these were not fatal errors but time to fix them.
Signed-off-by: Fred Oh <fred.oh@linux.intel.com>
Add ipc4 build support for tgl with IPC4_CONFIG_OVERLAY.
If IPC_VERSION is chosen as IPC4, then overlay configuration file
for different platforms will be used automatically without assigning
the configuration file with options.
Different configuration files will be used according to what platform
is building firmware for.
Signed-off-by: Xiaoyun Wu(Iris) <xiaoyun.wu@intel.com>
This reduces the number of warnings from about 600 to a more manageable
350.
Disable W0312 because the entire file uses tabs.
Disable C0103 because we have higher priority than naming conventions.
Disable C0116 because the script is small enough, shouldn't require
pydoc everywhere.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Building in parallel is much faster but it makes logs unreadable and
build failures impossible to understand. This is especially true when
building with recent ALSA that produces of deprecation warnings, see
examples below. For test topologies the problem is even worse: its
XARGS parallel build provides no log at all.
To find what actually fails, it is required to fall back on a single
threaded and verbose build and this is achieved with the variables
USE_XARGS, NO_PROCESSORS and VERBOSE. Pass these through docker-run.sh
and CMake.
Examples from #5608https://github.com/thesofproject/sof/runs/5717553555?check_suite_focus=truehttps://sof-ci.01.org/sofpr/PR5608/build12556/build/tools.txt
ALSA pcm.c:1523:(parse_hw_config) deprecated fsync value 'codec_slave',...
ALSA pcm.c:1471:(parse_hw_config) deprecated bclk value 'codec_slave',...
ALSA pcm.c:1523:(parse_hw_config) deprecated fsync value 'codec_slave',...
ALSA pcm.c:1471:(parse_hw_config) deprecated bclk value 'codec_slave',...
ALSA pcm.c:1523:(parse_hw_config) deprecated fsync value 'codec_slave',...
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Marc Herbert