Commit Graph

473 Commits

Author SHA1 Message Date
Marc Herbert 9915d1e1fd cmake: stop sneakily downloading missing submodules at build time
Note this change does NOT affect Zephyr builds in any shape or form as
Zephyr builds simply don't use the CMake files changed by this commit.

Downloading missing submodules at build time was never a good idea;
always a hack. Downloading and building should always be kept separate
from each other for version control and bill of materials reasons and to
support "off-line" builds; build inputs should always be available.

Now that Zephyr builds have just moved away from git submodules
(replaced by west), stop sneakily downloading missing submodules at
build time while the user cannot notice, overwhelmed by the volume of
build logs. Someone building XTOS first and Zephyr second could
unknowningly end up in a "hybrid" and undesired situation where git
submodules and west would be BOTH pointing at the same rimage clone.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-08-31 10:40:04 +01:00
Marc Herbert 44a5200c87 xtensa-build-zephyr: fix pylint "Redefine 'platform' from outer scope"
Fixes pylint warning `scripts/xtensa-build-zephyr.py:426:5: W0621:
Redefining name 'platform' from outer scope (line 14)
(redefined-outer-name)` and reduce "grep confusion": the script uses
`platform` A LOT.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-08-19 09:45:14 +01:00
Marc Herbert 730e861505 xtensa-build-zephyr.py: pylint:disable=bad-indentation W0311
There's just too many of them and fixing them would cause a lot of churn
at a wrong time (big migration to west happening now)

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-08-19 09:45:14 +01:00
Marc Herbert b94a5fbc54 xtensa-build-zephyr.py: fix two minor pylint f-string warnings
Fix two W1309 f-string-without-interpolation warnings

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-08-19 09:45:14 +01:00
Marc Herbert 3c6dcdb574 xtensa-build-zephyr.py: abort when python version < 3.8
shlex.join() requires 3.8, maybe others too. 3.8 is 3 years old and it's
the default Python version in Ubuntu 20.04

Fixes #6121

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-08-19 09:45:14 +01:00
Andrey Borisovich ecbbd1b5d1 scripts: xtensa-build-zephyr.py fix for windows
West tool returns on windows OS different white characters
than on Linux OS. Paths outputted from west tool had been
stripped for any trailing whitespaces.
Fixes "xtensa-build-zephyr.py: use west manifest"
(608833cf27)

Signed-off-by: Andrey Borisovich <andrey.borisovich@intel.com>
2022-08-12 15:02:18 +01:00
Fred Oh 4e85dfc95f docker: add AMD/Rembrandt gcc toolchain
Add AMD/Rembrandt (rmb) gcc toolchain to thesofproject/sof docker image.

Signed-off-by: Fred Oh <fred.oh@linux.intel.com>
2022-08-10 21:25:50 +01:00
Marc Herbert 473ec05b3b xtensa-build-zephyr.py: abort immediately on `west build` failure
Stop immediately when `west build` fails. Do no keep going and show a
totally unrelated smex build failure.

Also fix error message when zephyr/ is missing.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-08-05 15:53:50 +01:00
Marc Herbert b29242a2ad xtensa-build-zephyr.py: -p ristine deletes STAGING_DIR
Make the new -p pristine build option also delete the staging_dir. This
solves a problem where the following sequence was leaving a stale
platform1 build in staging:

xtensa-build-zephyr.py -p platform1
   < code >
xtensa-build-zephyr.py -p platform2

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-08-05 15:53:50 +01:00
Andrey Borisovich 70406a0db0 scripts: added versioning to xtensa-build-zephyr.py and west manifest
Added versioning to scripts/xtensa-build-zephyr.py to get version
information when incompatible changes are done to the script.
Added yml schema version number to west.yml manifest.

Signed-off-by: Andrey Borisovich <andrey.borisovich@intel.com>
2022-08-03 11:43:22 +01:00
Andrey Borisovich 4be98e4dc9 scripts: xtensa-build-zephyr.py removed calls over symlink
xtensa-build-zephyr.py had been using some execute_command
calls when building rimage and using rimage keys over
symlink from modules/audio/sof. Changed it so that calls are
executed now over normal sof directory and symlink is not needed
for script execution.

Signed-off-by: Andrey Borisovich <andrey.borisovich@intel.com>
2022-08-03 11:43:22 +01:00
Andrey Borisovich 627e29f287 scripts: xtensa-build-zephyr.py west reinitialization
Added new flag for non-interactive mode that should be used
when script is invoked programmatically.
In default (interactive) mode user is asked using a prompt whether to
reinitialize west manifest if it is initialized to manifest other
than SOFs.

Signed-off-by: Andrey Borisovich <andrey.borisovich@intel.com>
2022-08-03 11:43:22 +01:00
Andrey Borisovich 608833cf27 xtensa-build-zephyr.py: use west manifest
Added usage of west manifest in the script to update both
SOF and zephyr dependencies.
Renamed "clone-mode" to "update" flag - the logic where
sof dependencies and zephyr with dependencies is cloned.
Removed "point-mode" from script as it will no longer work
(west manifests require fixed directory structure).
Removed "zephyr revision" or "-z" argument as it is no longer
needed (now revision may be set in west.yml manifest).
Added "-p" pristine flag to rebuild platforms while removing build
directory.

Co-developed-by: Krzysztof Frydryk <krzysztofx.frydryk@intel.com>
Signed-off-by: Krzysztof Frydryk <krzysztofx.frydryk@intel.com>
Signed-off-by: Andrey Borisovich <andrey.borisovich@intel.com>
2022-08-03 11:43:22 +01:00
Andrey Borisovich ef028f634a Removed obsolete xtensa-build-zephyr.sh
This script had been rewritten to python xtensa-build-zephyr.py
script and is no longer used in CI.
Due to changes to sof processes related to Zephyr integration
this script will no longer be maintained.

Signed-off-by: Andrey Borisovich <andrey.borisovich@intel.com>
2022-07-19 09:51:29 +01:00
Anas Nashif afd28fb087 app: zephyr: remove overlays and adapt script
Remove overlays as they are the same board configurations available in
app/.
Keep common overlays and change script to reflect those changes.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-18 14:22:01 +01:00
Marc Herbert 33f27d3477 xtensa-build-zephyr.py: speed up thanks to an _actually_ shallow clone
This will significantly speed up CI in some cases.

Fixes commit ffdf001eee ("xtensa-build-zephyr.py: clone zephyr with
--depth 5")

When I added --depth 5 to git clone in that commit I naively assumed git
fetch would "inherit" that. I think it does but only when fetching an
_existing_ (and shallow) branch, not when fetching something new.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-07-11 09:22:32 +01:00
Fred Oh 9fc3833a5d docker: dump git log to a file before deleting projects
To track the docker images, need to keep all the hashes of the projects
builts in the image. Log git tree information to
/home/sof/work/sof_git_hash.txt in the docker image.

Signed-off-by: Fred Oh <fred.oh@linux.intel.com>
2022-07-08 21:27:27 +01:00
Fred Oh 85d68d0921 docker: change indentation and add HOME variable
No functionality change. Add some logs and minor clean up.

Signed-off-by: Fred Oh <fred.oh@linux.intel.com>
2022-07-08 21:27:27 +01:00
Xiaoyun Wu(Iris) 1e467a106f xtensa-build-zephyr: Auto pamam completion for --fw-naming
There are some restrictions when --fw-naming is set to 'AVS' for the
sake of providing reasonable FW outputs that make sense.

Restriction 1: When argument fw_naming is set as AVS, IPC4 ipc version
will be automatically chosen.

This is because --fw-naming 'AVS' is added for supporting IPC4 SOF build by
nature.

Restriction 2: When argument fw_naming is set as AVS, using platform
subdirectories will be automatically chosen.

The reason is that the platform subdirectories actually belong to the
"AVS" naming scheme, we must enable using platform subdirectories when
building FW in AVS naming convention.

This also ensure that we can tell firmwares of different platforms apart
by organizing them in different subdirectories.

Signed-off-by: Xiaoyun Wu(Iris) <xiaoyun.wu@intel.com>
2022-07-06 22:05:02 +01:00
Iris Wu 762f63bf8b xtensa-build-zephyr: add option to build FW in cavs naming style
Add argument option --fw-naming for specifying FW output with its
filename set with AVS style(filename as dsp_basefw.bin specifically)
OR SOF style(filename as sof-{platform}.ri). SOF style is by default.

This will be useful when building SOF+Zephyr IPC4 FW, because
dsp_basefw.bin will be loaded by default when snd_sof_pci parameter
ipc_type is set to 1 (which indicates running IPC4 mode).

Signed-off-by: Iris Wu <xiaoyun.wu@intel.com>
2022-07-06 22:05:02 +01:00
Marc Herbert 7c5bf651f8 xtensa-build-zephyr: copy ALL cavstool*.py scripts
Quoting https://github.com/zephyrproject-rtos/zephyr/pull/46880#pullrequestreview-1020096140

> No complaints about the code here. Just a general comment:

> I do worry that as cavstool grows features (first to add the test
> integration support, now the new client/server architecture) that
> we're obscuring the core code that does the loading behind a big
> facade of other stuff that also has to be run In Just The Right Way to
> make things work. That impedes development flow. It might be time to
> split the code up so that a minimal "just run this file please" tool
> can continue to be maintained.`

Long story short: `cavstool.py` is likely to be split in the future.

To be readi-er, copy the entire `zephyr/soc/xtensa/intel_adsp/tools/`
directory.

Copying the entire directory also copies scripts that will never be used
on a test device like `cavstool_client.py`. They're small Python script
sso we don't care: keep it simple.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-07-04 15:56:34 +01:00
Jaska Uimonen b0d71a23a6 xtensa-build-zephyr.py: add tgl-h ipc4 overlay
Add missing IPC4 overlay target and toml config for tgl-h.

Signed-off-by: Jaska Uimonen <jaska.uimonen@intel.com>
2022-06-28 11:11:53 +01:00
Pierre-Louis Bossart 3315681c69 cmake: make sure sha1 is computed without filters
Local filters in ~/gitconfig, such as

[core]
	autocrlf = input

can impact the result of git hash-object. Make sure no filters are
used so that the hash value remains unmodified across user setups.

BugLink: https://github.com/thesofproject/sof/issues/5917
Signed-off-by: Pierre-Louis Bossart <pierre-louis.bossart@linux.intel.com>
2022-06-16 20:17:27 +01:00
Balakishorepati c7fc085bd5 scripts: add rembrandt support
Add rembrandt build support.

Signed-off-by: Balakishorepati <balaKishore.pati@amd.com>
2022-06-15 14:52:59 +01:00
Marc Herbert 9d2e82adf0 xtensa-build-zephyr.py: fix find_west_workspace() to return west topdir
When neither -c nor -p is passed, the script defaults to '-p
SOF_TOP/zephyrproject', initializing the `west_top` global with
that. When everything is already cloned, this becomes
zephyrproject/modules/audio/sof/zephyrproject which is really not a
`west topdir`. Then west is pointed at:
 real_west_top/modules/audio/sof/zephyrproject/zephyr/samples/subsys/audio/sof
which obviously fails.

This was found while testing
https://thesofproject.github.io/latest/getting_started/build-guide/build-with-zephyr.html
and trying (and failing) to switch away from the older .sh script.

Fix find_west_workspace() to return the output of `west topdir` and use
that to update the `west_top` global.

This bug was introduced from the very start when the script was
converted from shell to python. The shell script did not have this
problem because it could be used without using neither -c nor -p. This
logical difference was mentioned in the code review:
https://github.com/thesofproject/sof/pull/5299#discussion_r801158177

Also remove a couple of accidental creations of a sof/zephyrproject/
directory when sof is already inside a higher zephyrproject.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-06-09 16:53:20 +01:00
mengdonglin 0d5bac87d3 xtensa-build-zephyr.py: avoid showing wrong layout of installed files
If each platform has its own output subdirectory, each platform output
subdirectory will include a key subdirectory with same name. Function
show_installed_files() can fail to find the correct parent for a node
as it only compares the subdir name that can no longer be unique.

This patch uses Anynode to replace Node class to define a new
attribute 'long_name' for a node to store its unique relative path
from the output staging directory (e.g. 'sof/apl/community') and
search for its parent.

The attribute 'name' is still used to store the subdir or file name
for a node (e.g. 'community'), and to display the layout of the
installed firmware files.

Signed-off-by: mengdonglin <mengdong.lin@intel.com>
2022-06-09 14:47:18 +01:00
mengdonglin be007d14ab xtensa-build-zephyr.py: add option to use an output subdirectory for each platform
All firmware files are installed in the same staging directory by default.

This patch adds an option --use-platform-subdir to create an output
subdirectory for each platform.

e.g. $scripts/xtensa-build-zephyr.py apl jsl tgl --use-platform-subdir
The layout of installed files will be like this:

build-sof-staging
├── sof
│   ├── apl
│   │   ├── community
│   │   │   └── sof-apl.ri
│   │   └── sof-apl.ldc
│   ├── tgl
│   │   ├── community
│   │   │   └── sof-tgl.ri
│   │   └── sof-tgl.ldc
│   └── jsl
│       ├── sof-jsl.ldc
│       └── community
│           └── sof-jsl.ri
└── tools
    ├── sof-logger
    └── cavstool.py

Signed-off-by: mengdonglin <mengdong.lin@intel.com>
2022-06-09 14:47:18 +01:00
mengdonglin 85a4c56d92 xtensa-build-zephyr.py: add option to specify key type subdir for output
Add argument option --key-type-subdir <key_type> to choose the output
subdirectory name based on the type of rimage signing key.

User can choose subdirectory name from 'community','dbgkey' and 'none'.
The default one is 'community', the type of default rimage signing key.
We can extend the list later.

When the key type is 'community' or 'dbgkey', the layout of SOF output
directory will be like:
build-sof-staging
├── sof
│   ├── sof-platform1.ldc
│   ├── sof-platform2.ldc
│   ├── ...
│   └── <key_type>
│       ├── sof-platform1.ri
│       ├── sof-platform2.ri
│       └── ...
└── tools
    ├── sof-logger
    └── cavstool.py

When the key type is 'none', there will be no key type subdirectory:
build-sof-staging
├── sof
│   ├── sof-platform1.ldc
│   ├── sof-platform1.ri
│   ├── sof-platform2.ldc
│   ├── sof-platform2.ri
│   └── ...
└── tools
    ├── sof-logger
    └── cavstool.py

Signed-off-by: mengdonglin <mengdong.lin@intel.com>
2022-06-09 14:47:18 +01:00
Marc Herbert 46a5de6f49 xtensa-build-zephyr.py: try to rmdir zephyrproject before cloning
Do not fail when there's only an empty directory left over
because of some early interruption of this script or proxy
misconfiguration, wrong fetch location, etc.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-06-07 21:39:25 +01:00
mengdonglin 55d8d90980 xtensa-build-zephyr.py: pass SOF firmware file version string to rimage
Extract SOF firmware file version string major.minor.micro from the
generated sof_version.h, and pass the version string to rimage in the
signing phase. So rimage can write the version info to the firmware
manifest headers for cAVS platforms.

Signed-off-by: mengdonglin <mengdong.lin@intel.com>
2022-05-31 15:23:27 +01:00
Tinghan Shen 34dda4ff6c docker_build: Add support for mt8195 and mt8186
Add support for building mt8186 and mt8195 toolchain in docker image.

Signed-off-by: Tinghan Shen <tinghan.shen@mediatek.com>
2022-05-25 11:43:20 +01:00
Marc Herbert af6586bd81 checkpatch: update to Linux version v5.17
Minimal, SOF-only differences with v5.17

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-05-13 09:58:49 +01:00
Marc Herbert d0fe4933f4 checkpatch: update to Linux version v5.12
Minimal, SOF-only differences with v5.12

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-05-13 09:58:49 +01:00
Marc Herbert 4f4fd1a425 checkpatch: remove spurious white line before "if MAINTAINERS"
This removes a small but conflicting difference with upstream.

Fixes commit d98a7ed103 ("checkpatch: check ABI updates before warning
on it")

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-05-13 09:58:49 +01:00
Marc Herbert d19009ab8c checkpatch: do not combine if (!$SOF) with other ifs
Use a separate if(!$SOF) to minimize differences with upstream.

Also remove some spurious whitespace differences with upstream.

Zero functional change.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-05-13 09:58:49 +01:00
Marc Herbert 7c4a53f802 checkpatch: add __sparse_cache and __sparse_force attributes
These attributes were added to Zephyr's include/zephyr/debug/sparse.h by
commit 17eb313a1b15 ("sparse: add an address space and a __sparse_force
annotation")

This gets rid of dozens of warnings in
https://github.com/thesofproject/sof/runs/6387526478

```
ERROR: need consistent spacing around '*' (ctx:WxV)
+	struct comp_buffer __sparse_cache *buffer_c, *sink_c;
 	                                  ^

ERROR: need consistent spacing around '*' (ctx:WxV)
+	struct comp_buffer __sparse_cache *source_c;
 	                                  ^
```

etc.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-05-13 09:29:37 +01:00
Chao Song de352e0beb xtensa-build-zephyr.py: add -d option for debug build
Add a new -d option to support SOF debug build. This
option will add CONFIG_DEBUG=y to SOF configuration,
which is aligned with xtensa-build-all.sh script.

The -d option is a shortcut for '-o debug_overlay_path'.

Signed-off-by: Chao Song <chao.song@linux.intel.com>
2022-05-11 09:56:02 +01:00
Chao Song 30431a19a9 xtensa-build-zephyr.py: add -o option for overlay
Add -o option for user to specify overlays while
building SOF.

Signed-off-by: Chao Song <chao.song@linux.intel.com>
2022-05-11 09:56:02 +01:00
Marc Herbert 464fb0e280 xtensa-build-zephyr.py: make sure that west can find SOF
This can fail for various reasons listed in a comment.

Remove now redundant `west topdir` check.

Without this check, a not found SOF fails with very cryptic Kconfig
errors:

zephyr/samples/subsys/audio/sof/prj.conf:5:
  warning: attempt to assign the value 'n' to undefined symbol HAVE_AGENT

zephyr/samples/subsys/audio/sof/prj.conf:8:
  warning: attempt to assign the value 'n' to undefined symbol
  DEBUG_MEMORY_USAGE_SCAN

zephyr/samples/subsys/audio/sof/boards/intel_adsp_cavs15.conf:1:
 warning: attempt to assign the value 'y' to undefined symbol APOLLOLAKE

etc.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-05-08 10:01:57 +01:00
Pin-chih Lin 141da0793f scripts: update xtensa core name and version for renoir
Update xtensa core to RI-2019.1 for AMD platform Renoir

Signed-off-by: Pin-chih Lin <johnylin@google.com>
2022-05-06 13:20:50 +01:00
Marc Herbert d09844ab98 zephyr/docker-build.sh: match UID with 'adduser' instead of 'chgrp -R'
This fixes SOF version.cmake which was just broken by a recent git
security update and started to fail like this:

```
  -- SOF version.cmake starting at 2022-04-25T18:14:56Z UTC
  -- /workdir/zephyr/.. is at git commit with parent(s):
  fatal: unsafe repository ('/workdir' is owned by someone else)
  To add an exception for this directory, call:

   git config --global --add safe.directory /workdir
```
(example at https://github.com/thesofproject/sof/runs/6162885265)

chgrp -R was always an ugly hack because it was messing with
(persistent) file permissions on the host, outside the container. This
new adduser solution is unfortunately much more code but it does not
leak any side effect outside the container.

Do not fix scripts/docker-run.sh yet because there is still no UID
mismatch between Github Actions and the SOF container (they're both
1001) but add a warning + TODO.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-04-26 22:08:53 +01:00
Marc Herbert 0c5dcaca04 xtensa-build-all.sh: actually implement CONFIG_IPC_MAJOR_3=y
There was no actual "IPC3" value, it was ignored. The platform default
was used instead.

Stop ignoring invalid arguments, fail and report them.

Allow empty string as a no-op for scripting convenience.

Don't restrict IPC4 to tigerlake, it's not the job of
xtensa-build-all.sh to filter what's supported versus not.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-04-25 16:39:50 +01:00
Marc Herbert 236de3f142 xtensa-build-zephyr.py: drop check=True, now the default
Commit 475e09f17d ("xtensa-build-zephyr.py: default to
subprocess.run(check=True)") switched the default value of the
execute_command() function to "True". Remove all the "check=True"
arguments that are now redundant.

Zero functional change.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-04-21 11:30:09 +01:00
Fred Oh af1e38a995 docker_build: add bison and flex to apt install list
There are warnings about bison and flex not available.
Interestingly these were not fatal errors but time to fix them.

Signed-off-by: Fred Oh <fred.oh@linux.intel.com>
2022-04-20 17:13:42 +01:00
Xiaoyun Wu(Iris) 62ead96614 xtensa-build-all.sh: add ipc4 build support for tgl
Add ipc4 build support for tgl with IPC4_CONFIG_OVERLAY.

If IPC_VERSION is chosen as IPC4, then overlay configuration file
for different platforms will be used automatically without assigning
the configuration file with options.

Different configuration files will be used according to what platform
is building firmware for.

Signed-off-by: Xiaoyun Wu(Iris) <xiaoyun.wu@intel.com>
2022-04-20 16:36:43 +01:00
Marc Herbert 475e09f17d xtensa-build-zephyr.py: default to subprocess.run(check=True)
The default value is wrong, change it. See pylint warning W1510

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-04-19 10:52:20 +01:00
Marc Herbert 35e588080d xtensa-build-zephyr.py: add #pylint:disable=W0312,C0103,C0116
This reduces the number of warnings from about 600 to a more manageable
350.

Disable W0312 because the entire file uses tabs.
Disable C0103 because we have higher priority than naming conventions.
Disable C0116 because the script is small enough, shouldn't require
              pydoc everywhere.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-04-19 10:52:20 +01:00
Marc Herbert c9ee109296 xtensa-build-zephyr.py: add cavstool.py to staging directory
This is required to read Zephyr logs.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-04-08 13:15:38 +01:00
Marc Herbert f9c461009a xtensa-build-zephyr.py: add for loop to install files into staging
Preparation to copy multiple files. Zero functional change.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-04-08 13:15:38 +01:00
Marc Herbert bde4858625 topologies: docker --env USE_XARGS, NO_PROCESSORS and VERBOSE
Building in parallel is much faster but it makes logs unreadable and
build failures impossible to understand. This is especially true when
building with recent ALSA that produces of deprecation warnings, see
examples below. For test topologies the problem is even worse: its
XARGS parallel build provides no log at all.

To find what actually fails, it is required to fall back on a single
threaded and verbose build and this is achieved with the variables
USE_XARGS, NO_PROCESSORS and VERBOSE. Pass these through docker-run.sh
and CMake.

Examples from #5608
https://github.com/thesofproject/sof/runs/5717553555?check_suite_focus=true
https://sof-ci.01.org/sofpr/PR5608/build12556/build/tools.txt

ALSA pcm.c:1523:(parse_hw_config) deprecated fsync value 'codec_slave',...
ALSA pcm.c:1471:(parse_hw_config) deprecated bclk value 'codec_slave',...
ALSA pcm.c:1523:(parse_hw_config) deprecated fsync value 'codec_slave',...
ALSA pcm.c:1471:(parse_hw_config) deprecated bclk value 'codec_slave',...
ALSA pcm.c:1523:(parse_hw_config) deprecated fsync value 'codec_slave',...

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-04-05 14:37:28 +01:00
Marc Herbert 93f20d577c xtensa-build-zephyr.py: forbid shell=True
The entire purpose of replacing the older shell script with Python was
to achieve cross platform compatibility. shell=True would lose that.

See also
https://docs.python.org/3/library/subprocess.html#security-considerations

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-04-04 16:24:43 +01:00
Marc Herbert 0bbdbb111f test/topology: make tplg-build.sh callable from anywhere
This removes two layers of indirection when trying to find what fails.
The lack of this proved especially painful when working on #5632 and
similar backports.

Also move outputs to a subdirectory by default. No change when invoked
from CMake which does not use the default output directory.

Also show the exact m4 and alsatplg commands run when not using XARGS
parallelization. Again this is critical when something fails.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-04-02 17:51:04 +01:00
Andrey Borisovich ce3264a507 platform: Move configuration from Zephyr to SOF
Move software configuration from Zephyr samples directory
to SOF platforms directory using Zephyr overlay mechanism.
How it works:
1. Gets zephyr board defconfig values
2. Applies Zephyr samples prj.conf values on top of it
3. Then applies our new overlay.conf on top of it
4. Final "summary" of Kconfig settings is written to autoconf.h header

Signed-off-by: Andrey Borisovich <andrey.borisovich@intel.com>
2022-03-18 16:14:19 +00:00
Marc Herbert a62fc4b9af xtensa-build-zephyr.py: restore ability to override CI
This has been useful to test Zephyr work in progress.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-03-15 13:39:12 +00:00
Marc Herbert ffdf001eee xtensa-build-zephyr.py: clone zephyr with --depth 5
This restores the performance of the original shell script and makes a
big difference in CI. CI clones over and over again whereas developers
clone rarely and can run a simple git fetch --unshallow.

Don't use --depth 0 to provide more context in the logs and also make it
more obvious that this is a shallow clone and not just git log -n 1.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-03-15 13:39:12 +00:00
Marc Herbert 5a80db78af xtensa-build-zephyr.py: give -z option a long form --zephyr-ref
This makes the code more readable and other scripts possibly too.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-03-15 13:39:12 +00:00
Marc Herbert df6b48c889 xtensa-build-zephyr.py: make "unsupported OS" actually a warning
Running on a mac crashed with "undefined xtensa_tools_version_postfix",
that should not be fatal (there are other toolchains).

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-03-15 13:39:12 +00:00
Marc Herbert d8572420d0 xtensa-build-zephyr.py: un-hardcode execute_command() wrapper
Duplicating all parameters of subprocess.run() is not just tedious and
error-prone, it makes the script compatible only with a range of
specific Python versions.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-03-14 15:04:45 +00:00
Kai Vehmanen 8f3a78dc6a xtensa-build-zephyr.sh: fix shellsheck warning
In scripts/xtensa-build-zephyr.sh line 199:

  if test $platform = tgl-h ; then
           ^-------^ SC2086: Double quote to prevent globbing
			     and word splitting.

Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
2022-03-14 11:43:32 +00:00
Kai Vehmanen 6d0ecc37cc xtensa-build-all: add a separate target for JSL
Use intel_adsp_cavs20_jsl Zephyr board when building for
Intel Jasper Lake.

BugLink: https://github.com/thesofproject/sof/issues/4539
Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
2022-03-14 11:35:35 +00:00
Marc Herbert 177e18f701 xtensa-build-zephyr.sh: add new -u zephyr_remote option
Same -u option as the new Python script.

We need this for the new sof/stable-v2.1 branch on
https://github.com/thesofproject/zephyr while our CI has not entirely
switched to the python script yet.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-03-14 11:32:59 +00:00
Marc Herbert 0ee59c317e xtensa-build-zephyr.py: do not update SOF submodules after west init
This script is run from an existing SOF repo that is assumed to be fully
initialized. Updating existing submodules is a potentially destructive
operation.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-03-11 12:52:41 +00:00
Marc Herbert 78667c028d xtensa-build-zephyr.py: do not modify submodules after building!
The older shell script had a deprecated convenience hack to clone
submodules before building rimage but _only if submodules were missing_!

While trying to preserve that, the new script changed that to an
unconditional git submodule update that can be destructive when
submodules are already present.

Generally speaking, using git and building must always be two very
distinct activities. No one wants the source code to change quietly from
one build to the next.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-03-11 12:52:41 +00:00
Marc Herbert 7a847a5bd8 xtensa-build-zephyr.py: de-duplicate common code
De-duplicate this code:

	if args.platforms:
		build_platforms()
		show_installed_files()

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-03-10 13:15:11 +00:00
Marc Herbert fc3bc3c805 xtensa-build-zephyr.py: don't git_submodules_update() twice
A second git_submodules_update() immediately after the first one does
not make sense. This looks like something leftover from a past
experiment.

Fixes initial commit 1de3ef3675 ("Rewritten xtensa-build-zephyr.sh to
python")

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-03-10 13:15:11 +00:00
Adrian Warecki b9d971920e xtensa-build-all.sh: Use appropriate defconfig for tgl-h platform
Recent combination of tgl and tgl-h building resulted in use of an invalid
defconfig file for tgl-h platform. This commit reverts that change.

Fixes: f1e6e1fdd3

Signed-off-by: Adrian Warecki <adrianx.warecki@intel.com>
2022-03-09 11:38:29 -08:00
Marc Herbert 2ac78a0603 version.cmake: make XTOS and Zephyr .ldc file different from each other
This is especially important considering some sof-bin releases are now
"hybrid": with a mix of XTOS and Zephyr.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-03-08 22:02:36 +00:00
Marc Herbert f052cbffaa zephyr/cmake: fix SOF_ROOT_SOURCE_DIR to fix the .ldc checksum
Fixes copy/paste of commit de41202f8f ("zephyr: build: Add initial
build support for SOF application.")

This fixes the dictionary hash when using Zephyr; no more fallback on
the git SHA1.

When using Zephyr, SOF_ROOT_SOURCE_DIR (and SOF_ROOT_BINARY_DIR) are used
only by version.cmake

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-03-08 22:02:36 +00:00
Marc Herbert 302ebe3b03 version.cmake: fix git -C sof/ log command when building Zephyr
When building Zephyr, CMAKE_CURRENT_SOURCE_DIR does not point at
SOF. Use SOF_ROOT_SOURCE_DIRECTORY instead.

No impact besides the logs.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-03-08 22:02:36 +00:00
Marc Herbert d722ee15b1 version.cmake: --no-abbrev-commit cause github has allowAnySHA1InWant
Some git servers like Github allow fetching by full length SHA so this
useful information to share.

(Others forbid this entirely, see last page of `git help fetch-pack`)

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-03-08 22:02:36 +00:00
Marc Herbert 23d2a01777 version.cmake: add comments and fix logs to clarify recursive invocation
version.cmake has a not very intuitive logic to make sure sof_versions.h
is always up to date without triggering a full rebuild. Add comments and
rephrase some logs to make it less hard to follow.

Absolutely zero functional change.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-03-08 22:02:36 +00:00
Adrian Warecki f1e6e1fdd3 xtensa-build-all.sh: Code style correction
Combined case label tgl and tgl-h similarly as in xtensa-build-zephyr.sh
Corrected code formatting.

Signed-off-by: Adrian Warecki <adrianx.warecki@intel.com>
2022-03-08 09:25:28 +00:00
Adrian Warecki 95d887251e xtensa-build-all.sh: Don't override user private key path
If the user doesn't set the path to own private key, the default path is
placed in the variable containing the user's key. This causes a leak of the
default key of one platform to the others platforms, which will treat it as
a key supplied from the user.

Signed-off-by: Adrian Warecki <adrianx.warecki@intel.com>
2022-03-08 09:25:28 +00:00
Chao Song f645b36cc8 xtensa-build-zephyr.py: use string path for rimage_desc
PosixPath doesn't convert to string automatically, and
it is not acceptable in sign_cmd, convert it to string
explicitly.

Signed-off-by: Chao Song <chao.song@linux.intel.com>
2022-03-07 12:02:33 +00:00
Chao Song de7016edd4 platforms: tgl: renamme tgl_ipc4_overlay.conf
Rename tgl_ipc4_overlay.conf to ipc4_overlay.conf,
because it is under tgl folder.

Signed-off-by: Chao Song <chao.song@linux.intel.com>
2022-03-07 12:02:33 +00:00
Kai Vehmanen 26e08d4d55 xtensa-build-zephyr: use separate Zephyr config for TGPH builds
Zephyr recently added support for separate configs for 2-core and 4-core
variants of cAVS2.5. Use the new "tgph" configuration for when building
SOF for TigerLake-H variant.

BugLink: https://github.com/thesofproject/sof/issues/5018
Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
2022-03-01 22:49:36 +00:00
Chao Song 6f71808e3e xtensa-build-zephyr.py: add ipc4 build support for tgl
Add ipc4 build support for tgl with OVERLAY_CONFIG.

Signed-off-by: Chao Song <chao.song@linux.intel.com>
2022-02-26 11:06:59 +00:00
Marc Herbert ce8bc4a394 xtensa-build-zephyr.py: important -v verbosity fixes
The most important fix is the addition of the "flush" argument which
means are commands are now printed when they are run and not all at the
end when everything is done.

Commands run by this script are very few so they are always shown.  This
makes it possible to debug just this script without being completely
drown in build noise. All '-v's are passed to west.

Pass -v to `west sign`, not just to `west build`.

Drop the manual "-DCMAKE_VERBOSE_MAKEFILE=ON", west knows how to do that.

Print directories first so commands are aligned, more visible and can be
copied. Use shlex to quote whitespace.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-02-24 14:46:17 +00:00
Marc Herbert 9fb7a607eb xtensa-build-zephyr.py: fix -C option so it can support whitespace
Make it possible to invoke -C multiple times which is required to
support whitespace as in:

  -C=--warn-uninitialized  -C '-DEXTRA_FLAGS=-Werror -g0'

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-02-21 09:23:40 +00:00
Marc Herbert 7d9aa8e04b xtensa-build-zephyr.py: switch to RawTextHelpFormatter and """
We need control to format add_argument(help=...) strings manually too.

Zero functional change.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-02-21 09:23:40 +00:00
Andrey Borisovich bb314fca61 Fix for xtensa-build-zephyr.py. This fixes 1de3ef3
Existing xtensa-build-zephyr.py failed to copy sof logger
executable to staging directory due to missing '.exe' file extension.
This fixes the script.

Signed-off-by: Andrey Borisovich <andrey.borisovich@intel.com>
2022-02-18 11:25:47 +00:00
Andrey Borisovich 1de3ef3675 Rewritten xtensa-build-zephyr.sh to python
Created new script with similar functionality to allow SOF + Zephyr
builds on both Windows and Linux operating systems.

Signed-off-by: Andrey Borisovich <andrey.borisovich@intel.com>
2022-02-16 12:02:40 +00:00
Marc Herbert 0bf7b73dc8 cmake: move ALSA_CONFIG_DIR from .sh file to topology/CMakeLists.txt
Fixes incremental builds of topology2 that were missing ALSA_CONFIG_DIR.

This was making it impossible to troubleshoot alsatplg issues like #5249.
Now incremental builds fail or pass the same as builds from scratch.

Fixes commit 308a24a92b ("topology2: Add build support")

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-01-26 10:14:15 -08:00
Tinghan Shen 59a32d5bbe platform: Add mt8186 to Kconfig
Add mt8186 to Kconfig

Signed-off-by: Tinghan Shen <tinghan.shen@mediatek.com>
Signed-off-by: Allen-KH Cheng <allen-kh.cheng@mediatek.com>
2022-01-26 12:24:24 +00:00
Marc Herbert 0031664403 xtensa-build-all.sh: distinguish default/working platform and WIP
The -a option was recently broken by the addition of platform(s) that
don't have a toolchain available in the Docker image. Every platform can
be built by someone but no one can built -a(ll) platforms right now.

Add a new platform array to make the distinction between default
platforms built by the -a option versus work in progress.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-01-21 17:17:09 +00:00
Marc Herbert da1d1b041e xtensa-build-all.sh: rn: remove odd ARCH=xtensa definition
No one else has it.

Gets rid of the only shellcheck warning.

Fixes commit 7453e3d571 ("scripts: add renoir support")

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-01-21 17:17:09 +00:00
Marc Herbert 5de4a09756 version.cmake: include scripts/ and zephyr/ in the .ldc dictionary hash
scripts/ has kconfig defaults and CMake code that can affect the
dictionary. Note this does not fix #3890 because .config (and maybe
others) are still not hashed but it helps a bit.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-01-14 16:59:01 +00:00
Marc Herbert 58d3551960 version.cmake: more robust logging of the current git commit and parents
execute_process() runs in the current directory of the process invoking
cmake. This can be completely outside the git repo.

Fixes commit a5899812b7 ("version.cmake: don't trust CI to record time
and versions and log ourselves")

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-01-14 16:59:01 +00:00
Marc Herbert 805e57b28f xtensa-build-zephyr: include DEFAULT_PLATFORMS in SUPPORTED_PLATFORMS
Include DEFAULT_PLATFORMS in SUPPORTED_PLATFORMS instead of the other
way round. SUPPORTED_PLATFORMS will always be bigger than
DEFAULT_PLATFORMS and it's easier to add a platform than remove one.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-01-14 16:57:35 +00:00
Marc Herbert ac1a09afdb xtensa-build-zephyr: de-dup. first vs incremental west invocation
Passing spurious --board and source directory arguments to an already
build directory does not help but it does not hurt either and it
simplifies the code. It also provides a more consistent west command in
the set -x "logs", one that can be re-used in any circumstance.

The only restriction is to make sure CMAKE_ARGS is empty on all but the
first invocation but that's not new and unchanged here.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-01-14 16:57:35 +00:00
Marc Herbert deb0e7340e Rename generated version.h to sof_versions.h
Zephyr also a version.h too so the situation was a buggy #include mess
that (among others) dropped the SOF git version from the SOF banner when
using Zephyr. It's absolutely impossible to fix this mess while keeping
two .h files both named exactly the same and even included with the
exact same '#include version.h' line, I mean no sof/version.h or other
prefix.

As a bonus, this rename also reduces the XTOS-only confusion between
"version.h" and cavs/version.h".

Generating a file in the source tree is a serious bug (to be fixed
later), it means this file is never cleaned. Do not hide this bug with a
.gitignore rule.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-01-14 16:56:29 +00:00
Marc Herbert 950728c095 Docker: sof: add ninja-build
We should gradually switch to ninja as a default.

ninja-build is apparently missing from build-essential

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-01-10 11:52:04 +00:00
Marc Herbert b339ca95e9 Remove scripts/host-build-all.sh
scripts/host-build-all.sh has been replaced and deprecated for more
than a year.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-01-07 15:29:14 +00:00
Marc Herbert 55e862c92e docker-run.sh: switch from "sof" shortcut to full "thesoftproject/sof"
It's super annoying and time-consuming to update the image and wonder
why the previous one is still used. Shortcuts are nice in interactive
use but pointless in scripts.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-01-07 15:25:53 +00:00
Marc Herbert f79d2341af docker-run.sh: log command run and image digests
There's frequent confusion between image ID and image digest: display
both.

Show both sof and thesofproject/sof.

Use set -x to display the full command that is run.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-01-07 15:25:53 +00:00
Ranjani Sridharan 308a24a92b topology2: Add build support
Add build support for topology2. Topology2 will be built only if the
alsatplg version if 1.2.6 or greater.

Signed-off-by: Ranjani Sridharan <ranjani.sridharan@linux.intel.com>
2022-01-06 21:38:43 +00:00
Kai Vehmanen b95d995bd2 xtensa-build-zephyr: add -v option for verbose build
Align with xtensa-build-all.sh and add a "-v" option to toggle
verbose build log ("west -v build").

Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
2022-01-05 13:23:00 +00:00
Marc Herbert f45f5dffec xtensa-build-zephyr.sh: remove x-prefix in comparisons
Fixes new shellcheck 0.8.0 warning:

SC2268 (style): Avoid x-prefix in comparisons as it no longer serves a
purpose.

This script requires bash anyway.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-01-04 11:55:21 +02:00
Marc Herbert 0d39ff6b91 xtensa-build-zephyr.sh: add missing "${SOF_TOP}" quotes
Fixes commit a8c333b606 ("xtensa-build-zephyr: add deterministic
sha256sum of final .ri file")

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2022-01-04 11:55:21 +02:00
Marc Herbert 5d66f1dac4 cmake: fix .ldc chksum fallback when no SOURCE_DIRECTORY/.git
Fixes old (Aug 2020) and untested commit b2a325a3b7 ("cmake: Handle
empty SOF_SRC_HASH_LONG") which tried to use GIT_LOG_HASH as a fallback
for the SOF_SRC_HASH .ldc checksum but always fell back on "0" instead
because of a misunderstanding of cmake's surprisingly complex "if"
operator.

Before Zephyr this was not an issue in practice because GIT_LOG_HASH was
empty anyway when SOURCE_DIRECTORY/.git was missing, so there was
nothing to lose.

For the Zephyr builds this will now use the SOF SHA1 as the .ldc checksum
for now. Probably not ideal yet but far better than the current constant
"0" which means no check at all.

Also log the SOF_SRC_HASH fallback value now and change the second
fallback (when GIT_LOG_HASH is also missing) from "0" to the
searchable hexspeak "baadf00d".

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2021-12-24 22:15:13 +00:00