For rimage parameters, parse new, static versions.json file instead of
generated sof_versions.h file. This will make possible to configure
rimage before west build.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
This replaces and simplifies commit a823958a8d ("xtensa-build-zephyr:
pass sof build version to rimage") with a constant 1.
That commit was submitted to avoid a test failure in a broken, internal
test looking for an SOF_BUILD value... hardcoded to 1! (internal FW issue
257, test TestLoadFwExtended::()::test_00_01_load_fw_and_check_version")
The final goal is for this script to stop extracting anything from
`generated/sof_versions.h` so rimage can be configured _before_ the build
has started. Other commits will deal with other parts of sof_versions.h
SOF_BUILD can be replaced by a constant because it has always been one
when building with Zephyr. The optional BLD_COUNTERS feature - disabled
by default in XTOS since commit 9f8cce1522 ("Disable __TIME__ and the
non-reproducible build counter by default") for build reproducibility
reasons - has never worked with Zephyr for the following reasons:
- The sof_add_build_counter_rule() invocation is located in the
top-level sof/CMakeLists.txt file which has never been used by the
Zephyr build.
- sof_add_build_counter_rule() registers a POST_BUILD command for the
top-level "sof" CMake target but there is no "sof" build target in the
Zephyr build.
- Variables like VERSION_BUILD_COUNTER_CMAKE_PATH are not defined in the
Zephyr build for some unknown reason.
- Probably others.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
As discussed in #6952, relying on `git describe` for defining SOF_MAJOR
and friends is very brittle and unreliable. Switch to a static
versions.json file instead.
Note the full `git describe --tags` output is _still_ present in the
binary, this is useful and left unchanged. It's just not used any more
to guess SOF_MAJOR, SOF_MINOR and SOF_MICRO.
Use JSON because CMake and pretty much every other piece of software has
a JSON parser out of the box.
This aligns with linux/Makefile, Zephyr/VERSIONS and probably many
others. - except we use JSON because we're in 2023 so we don't spend
time having fun re-implementing parsers any more.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Remove all support for cAVS 1.5 platformsm including Apollo Lake,
Sky Lake, Kaby Lake, Broxton and Gemini Lake, they aren't supported
any more.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
Commit 3315681c69 ("cmake: make sure sha1 is computed without
filters") added the `--no-filters` option to the `git hash-object`
invocation used to produce a `src/` checksum embedded in all firmware
binaries and .ldc files.
This fixed a small mismatch for the (very few) people using the
_asymmetric_ `autocrlf = input`. However it created a huge mismatch for
the numerous people using the _symetric_ `autocrlf = true`.
So revert that commit, remove `--no-filters` and replace it with a build
time warning.
`--no-filters` produces a checksum of the files as they are on the
filesystem, as opposed to how they would be input into git after
optional CRLF -> LF conversion.
When `--no-filters` was added, no one was compiling in Windows. Most
developers were not performing any conversion because there is no
`autocrlf` conversion by default on Linux. Files were identical inside
and outside of git and life was simple.
Simple life except for at least one indomitable Linux developer who had
the _asymmetric_ `autocrlf = input` setting. Also, the SOF git repo
always had two exceptional (and generated) files with CRLF end of lines;
see issue #5917 for details (all other files have LF end of lines). So
for that "asymmetric" developer only and these two files only, `autocrlf
= input` converted these 2 files to LF before hashing and this caused a
different checksum. `--no-filters` stopped that conversion and "solved"
that mismatch.
But now Windows has entered the stage. On Windows, the default is
usually (and unfortunately) the symmetric `autocrlf = true`. It is the
default for Github's Windows runner. This symmetric default converts ALL
other files to CRLF when extracting them out of git. This causes `git
hash-object --no-filters` to produce a _different hash for all but 2
files on Windows_!
The only solution is to:
1. Drop the `--no-filters` to align everything on what is _stored in
git_. What is stored in git is the only thing we are sure all users have
in common.
2. Request users to use only _symmetric_ `autocrlf` settings so any
optional input+output conversion cancels itself out.
3. Convert odd files and make sure they all have LF end of lines.
4. In the future, drop this entire git hash-object technique which is
flawed in multiple ways and has been causing multiple issues. Much,
much more work than this very small revert though.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Note this change does NOT affect Zephyr builds in any shape or form as
Zephyr builds simply don't use the CMake files changed by this commit.
Downloading missing submodules at build time was never a good idea;
always a hack. Downloading and building should always be kept separate
from each other for version control and bill of materials reasons and to
support "off-line" builds; build inputs should always be available.
Now that Zephyr builds have just moved away from git submodules
(replaced by west), stop sneakily downloading missing submodules at
build time while the user cannot notice, overwhelmed by the volume of
build logs. Someone building XTOS first and Zephyr second could
unknowningly end up in a "hybrid" and undesired situation where git
submodules and west would be BOTH pointing at the same rimage clone.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Local filters in ~/gitconfig, such as
[core]
autocrlf = input
can impact the result of git hash-object. Make sure no filters are
used so that the hash value remains unmodified across user setups.
BugLink: https://github.com/thesofproject/sof/issues/5917
Signed-off-by: Pierre-Louis Bossart <pierre-louis.bossart@linux.intel.com>
This is especially important considering some sof-bin releases are now
"hybrid": with a mix of XTOS and Zephyr.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Fixes copy/paste of commit de41202f8f ("zephyr: build: Add initial
build support for SOF application.")
This fixes the dictionary hash when using Zephyr; no more fallback on
the git SHA1.
When using Zephyr, SOF_ROOT_SOURCE_DIR (and SOF_ROOT_BINARY_DIR) are used
only by version.cmake
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
When building Zephyr, CMAKE_CURRENT_SOURCE_DIR does not point at
SOF. Use SOF_ROOT_SOURCE_DIRECTORY instead.
No impact besides the logs.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Some git servers like Github allow fetching by full length SHA so this
useful information to share.
(Others forbid this entirely, see last page of `git help fetch-pack`)
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
version.cmake has a not very intuitive logic to make sure sof_versions.h
is always up to date without triggering a full rebuild. Add comments and
rephrase some logs to make it less hard to follow.
Absolutely zero functional change.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
scripts/ has kconfig defaults and CMake code that can affect the
dictionary. Note this does not fix#3890 because .config (and maybe
others) are still not hashed but it helps a bit.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
execute_process() runs in the current directory of the process invoking
cmake. This can be completely outside the git repo.
Fixes commit a5899812b7 ("version.cmake: don't trust CI to record time
and versions and log ourselves")
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Fixes old (Aug 2020) and untested commit b2a325a3b7 ("cmake: Handle
empty SOF_SRC_HASH_LONG") which tried to use GIT_LOG_HASH as a fallback
for the SOF_SRC_HASH .ldc checksum but always fell back on "0" instead
because of a misunderstanding of cmake's surprisingly complex "if"
operator.
Before Zephyr this was not an issue in practice because GIT_LOG_HASH was
empty anyway when SOURCE_DIRECTORY/.git was missing, so there was
nothing to lose.
For the Zephyr builds this will now use the SOF SHA1 as the .ldc checksum
for now. Probably not ideal yet but far better than the current constant
"0" which means no check at all.
Also log the SOF_SRC_HASH fallback value now and change the second
fallback (when GIT_LOG_HASH is also missing) from "0" to the
searchable hexspeak "baadf00d".
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
When compiling with Zephyr, SOF_ROOT_SOURCE_DIRECTORY is changed to
sof/zephyr/ and sof/zephyr/.git does not exist. Change error message to
show SOF_ROOT_SOURCE_DIRECTORY/.git
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Without CMP0079 we cannot conditionally include libraries against SOF in
sub directories without seriously restructuring the project. This is
because the old policy requires the link target must be created in the
same folder. This does not work well from a configuration standpoint for
3P audio libraries trying to keep their config in src/audio/*. Rather
than enable the policy, lets simply upgrade since 3.13 is widely
available.
With this upgrade we can also remove the two version dependent checks at
the top of our scripts.
Signed-off-by: Curtis Malainey <cujomalainey@chromium.org>
This is motivated by the zephyr.elf build that does not need rimage at
all.
Also build smex later, only when it's needed.
After building and deleting the rimage/ clone, a plain SOF clean +
RE-build now fails much later: it now fails AFTER successfully building
the sof binary, only when it tries to build either bootloader,
boot_module or base_module that actually need rimage:
[ 87%] Performing configure step for 'rimage_ep'
CMake Error: The source directory "/home/SOF/sof/rimage" does not appear
to contain CMakeLists.txt.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
In an ideal world, every CI engine records and shares the most important
CI information:
- current date and time in a well identified timezone
- git version of the pull request
- git version of the moving branch the PR is being merged with
In the real world we have multiple CI solutions and they unfortunately
cannot not all be trusted to perform their most basic job
correctly. Fortunately, they all make at least build logs available so
these very few lines of code adding very few lines of output cost near
zero extra build time and solve the problem once for all. I feel stupid
I didn't do this sooner, this would have saved me hours and hours in
vain requests and discussions and in trying to puzzle that information
together.
Sample output:
-- Preparing Xtensa toolchain
version.cmake starting SOF build at 2021-03-31T18:09:46Z UTC
Building git commit with parent(s):
150fd1e4c968 4249bdb1b305 [other parent if merge] (HEAD -> main) cmake: ...
-- GIT_TAG / GIT_LOG_HASH : v1.7-rc1-174-g150fd1e4c968-dirty / 150fd1e4c968
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
For at least two reasons:
- exposes sneaky change(s) performed by automation if/when any
- solves the mystery of the Source content hash (printed on the next
line) changing while the git version does not.
Example, at https://sof-ci.01.org/sofpr/PR3941/build8429/build/bdw_gcc.txt
-- Found Git: /usr/bin/git (found version "2.17.1")
-- GIT_TAG / GIT_LOG_HASH : v1.7-rc1-151-g023c4abacde1 / 023c4abac
-- Source content hash: 91f261ea
whereas at https://github.com/thesofproject/sof/runs/2166298087,
xtensa-build-all:
-- Found Git: /usr/bin/git (found version "2.17.1")
-- GIT_TAG / GIT_LOG_HASH : v1.7-rc1-151-g023c4abacde1 / 023c4abac
-- Source content hash: 67f31697
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Notably: xt-xcc --show-config failed with: No such file or directory
... when the directory exists but is wrong.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Add cmake -DINIT_CONFIG= option that can point at any initial file.
"make clean" does not delete .config any more.
Note reconfiguration does NOT causes recompilation because -imacros
hides the generated .h from CMake's dependency scan. This is not a
regression, that problems exists since -imacros was introduced. At least
it's now possible to "make clean" and rebuild without losing the .config
file.
Fix for #3617
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
As incredible as it sounds, some people run neither "git status" nor
"git diff" every few minutes and not even when their build fails. There
has been reports that they're puzzled when they miss a required
submodule update. This is an attempt to draw their attention based on
the assumption that they pay more attention to the CMake logs.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
SOF_SRC_HASH always must have integer value, because of usage
them to initialize global variable in source code.
Variables, which may be empty should be used inside quotation
to prevent cmake incorect number of arguments error.
Signed-off-by: Karol Trzcinski <karolx.trzcinski@linux.intel.com>
Temporary files are useful to understand and debug the build.
Moving output to another directory, keeps binary directory clean.
Signed-off-by: Karol Trzcinski <karolx.trzcinski@linux.intel.com>
This is possible location of .git folder, instead of CMAKE_SOURCE_DIR.
Without this patch, checked condition always return false and
GIT_LOG_HASH is used instead source code hash.
Signed-off-by: Karol Trzcinski <karolx.trzcinski@linux.intel.com>
GIT_TAG is user readable form of used source code version with
commit identifier, what is important for bugs reproducibility,
so it will be convenient to have this information in output logs.
Signed-off-by: Karol Trzcinski <karolx.trzcinski@linux.intel.com>
It may be used to check FW compatibility with ldc file.
It's much better than comparing DBG_ABI because logs content
may be updated without any DBG_ABI change in opposite to source
code hash value.
Signed-off-by: Karol Trzcinski <karolx.trzcinski@linux.intel.com>
dist.cmake is (for now) the only .cmake file that uses GIT_TAG at _cmake
configure time_. This means even "make clean" is not enough for
dist.cmake to take into account some git tag or version change. It's
very confusing because on the other hand "make check_version_h" and
everything else always shows up-to-date git information.
"make rebuild_cache" addresses this situation so mention it in a
warning.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Before this fix, any random, later git tag like for instance
"releases/jsl/v3.0-rc1" would be picked up and then immediately
discarded and replaced with v0000 because it didn't start with 'v*'
Also log all the versions always and not just when there is already a
.tarball-version file.
Increase --abbrev=12 because --abbrev=4 doesn't seem to make sense.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
A "tarbomb" is an archive file that has more than one directory at the
top level, which is (fortunately) unusual for .tar files. By default,
tar extracts archive members "as is" in the current directory. If the
archive has many top level members then it becomes very hard to make the
difference between what was just extracted versus what was already
there. As extraction is recursive this gets much worse because the
problem can repeat itself in subdirectories. As the last but not least
nail in the coffin, tar silently overwrites existing files by default.
This commit is admittedly an "incompatible build API change" meaning any
higher level build or packaging script that was consuming source release
tarballs (as opposed to git cloning the source) will require a small
change. Consumers of release tarballs must be rare though because the
output of "make dist" has already been broken by commit
c00d39c71b ("Add rimage as a git submodule") anyway yet no one noticed
yet.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
- Add a simple test in cmake to download submodules only when rimage is
missing. This stops randomly changing the source code from one build
to the next and overwriting any submodule work in progress.
- Remove the GIT_SUBMODULE option which is now useless because rimage is
not optional for sof
- Add a --merge option for safety. It makes zero difference right now
but could save work in progress if we ever add more
submodules (hopefully not) and someone struggles with submodules and
ends up in a partially initialized state.
Git submodules are rarely ever the answer; these few changes make them a
bit more usable.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
This adds the necessary bits to update git submodules when
cmake is run, with the option to turn this behavior off if
needed. This is in preparation to use fw.h and manifest.h
from the rimage repo to prevent having two copies of each
file in two different repos. Obviously, the files in
the submodules must exist before building the firmware,
so run git submodule update to checkout the files.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
The git commands to get version strings need to run in the SOF source
directory. This is not automatically set when building for Zephyr so
fix it.
Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Add targets that are meant to be used after defconfig,
to apply configs from <arch>/configs/override on top of defconfigs.
Signed-off-by: Janusz Jankowski <janusz.jankowski@linux.intel.com>
CMake regeneration condition should depend on directory to automatically
update when new defconfigs are added.
Signed-off-by: Janusz Jankowski <janusz.jankowski@linux.intel.com>
When variable length array, filled with string will be
placed in sucha manner that null terminator address will
be divisible by four, then it will be lost in output
binary file. It leads to troubles during scanning content
of such a section. Such a problem occur in firmware
and produce logger and FW debug ABI mismatch and it's why
logger output is broken.
After change length of XCC_TOOLS_VERSION to be none of
number four multiplication problem with logger disappear.
Signed-off-by: Karol Trzcinski <karolx.trzcinski@linux.intel.com>
Add support for compiling FW with settings acceptable for clang,
in order to let it perform static analysis on the code.
Clang works mostly on ASTs made out of C code, so there is
no need to build complete signed binary for it.
Signed-off-by: Janusz Jankowski <janusz.jankowski@linux.intel.com>
Command git describe looks only for annotated tags, but we should
get any tag reachable from master, that's why --tags flag is needed.
Signed-off-by: Janusz Jankowski <janusz.jankowski@linux.intel.com>
Unify calls for compiler version and name for XCC and GCC.
Separate conversion of optimization level from Kconfig settings
to compiler flags to separate cmake function, because of usage
in two places.
Read XCC_TOOLS_VERSION from compiler.
Signed-off-by: Karol Trzcinski <karolx.trzcinski@linux.intel.com>
Add helper function sof_append_relative_path_definitions that
defines RELATIVE_PATH per source file for given target.
__FILE is not always suitable as C standard states that __FILE__ expands to
input file name, that usually is absolute path what will cause f.e. .rodata
size to be dependent on where project is physically located on the disk.
Signed-off-by: Janusz Jankowski <janusz.jankowski@linux.intel.com>
When there are changes made to Kconfig files it's better
to run olddefconfig automatically to prevent undesired behaviour,
like for example depending on added config that has some default value
that is not present because developer didn't run olddefconfig
after pulling changes.
Signed-off-by: Janusz Jankowski <janusz.jankowski@linux.intel.com>
Adding static libraries properly may be troublesome for developers
that are not familiar with CMake, so function that makes it easier
should be useful.
Usually developer will just add sources directly to the target.
Using static libraries should be limited just to closed / precompiled
3rd party components.
Signed-off-by: Janusz Jankowski <janusz.jankowski@linux.intel.com>
Marc Herbert