This script lives in a sof.git/ clone yet it was systematically cloning
a second sof.git/. Besides the obvious confusion and risk of editing the
wrong files, this meant it was not possible to build code that has not
been merged yet! This was a problem for both CI and developers. Fixed by
using symbolic links to ourselves instead.
Note it is _still_ possible to build from another sof.git clone if
desired, however this script will never git re-clone a second sof.git
itself, that second clone has to be created (e.g.: by west) before this
script runs.
When cloning a brand new zephyrproject, use a shallow zephyr clone and
download only the two zephyr modules we actually use. This speeds up
automation considerably and makes it much faster for non-Zephyr
developers to reproduce Zephyr issues. Developers can always git
unshallow and west update once if they want to.
Rename the default west top to "zephyrproject" to not just match the
zephyr documentation but to also avoid creating a double zephyr/zephyr/
directory.
See the new print_usage() for a few more implementation details.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
It still looks smart but the \n addition compared to the original
version in xtensa-build-all.sh broke it for more advanced cases:
die '%s %d' str 5
-bash ERROR: -bash: printf: 5\n: invalid number
As reported by shellcheck. shellcheck saves lives.
In scripts/xtensa-build-zephyr.sh line 22:
>&2 printf "$@\n"
^-- SC2145: Argument mixes string and array.
Use * or separate argument.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
... because the latter does not print on stderr when not found; so IDEs
and CIs don't display this as a warning.
Also remove the "... and ignore set -e" comment that made sense at the
time of commit cad86dc340 ("scripts: xtensa-build: fix alias detection
for bxt/apl gcc") but not anymore.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Allows the mocks to be quickly run on the host with full access to host
debug and development tooling.
Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
In an ideal world, every CI engine records and shares the most important
CI information:
- current date and time in a well identified timezone
- git version of the pull request
- git version of the moving branch the PR is being merged with
In the real world we have multiple CI solutions and they unfortunately
cannot not all be trusted to perform their most basic job
correctly. Fortunately, they all make at least build logs available so
these very few lines of code adding very few lines of output cost near
zero extra build time and solve the problem once for all. I feel stupid
I didn't do this sooner, this would have saved me hours and hours in
vain requests and discussions and in trying to puzzle that information
together.
Sample output:
-- Preparing Xtensa toolchain
version.cmake starting SOF build at 2021-03-31T18:09:46Z UTC
Building git commit with parent(s):
150fd1e4c968 4249bdb1b305 [other parent if merge] (HEAD -> main) cmake: ...
-- GIT_TAG / GIT_LOG_HASH : v1.7-rc1-174-g150fd1e4c968-dirty / 150fd1e4c968
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Required by #3459 / #3975 "elfsize" proof of concept and probably by
other things too in the future - we use both ELF and Python
everywhere; it's surprising they haven't met each other yet.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
For at least two reasons:
- exposes sneaky change(s) performed by automation if/when any
- solves the mystery of the Source content hash (printed on the next
line) changing while the git version does not.
Example, at https://sof-ci.01.org/sofpr/PR3941/build8429/build/bdw_gcc.txt
-- Found Git: /usr/bin/git (found version "2.17.1")
-- GIT_TAG / GIT_LOG_HASH : v1.7-rc1-151-g023c4abacde1 / 023c4abac
-- Source content hash: 91f261ea
whereas at https://github.com/thesofproject/sof/runs/2166298087,
xtensa-build-all:
-- Found Git: /usr/bin/git (found version "2.17.1")
-- GIT_TAG / GIT_LOG_HASH : v1.7-rc1-151-g023c4abacde1 / 023c4abac
-- Source content hash: 67f31697
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
The configuration names with _2017 are a consequence of the way I did
the internal configuration update. To keep it simple I update these
build scripts.
Signed-off-by: Paul Olaru <paul.olaru@nxp.com>
XTOBJCOPY and XTOBJDUMP were added in June 2018 by
commit 27795ece0f ("scripts: fix xt-xcc build with wrong config") for
`./configure CC=$XCC OBJCOPY=$XTOBJCOPY OBJDUMP=$XTOBJDUMP ...` that was
used at the time.
The build system was switched to CMake in January 2019, notably with
commit 9840ecbbfe ("cmake: update xtensa-build-all.sh") and
commit 0fd97adfb0 ("cmake: add utility scripts") that defined
CMAKE_OBJCOPY and CMAKE_OBJDUMP)
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
$ARCH was introduced in 2018 by commit b5af84deaa ("scripts: add smp
builds to xtensa-build-all.sh"). At the time used autotools and
`./configure --with-arch=$ARCH` were used.
In January 2019 the build system was migrated to CMake over several
commits, notably commit 9840ecbbfe ("cmake: update
xtensa-build-all.sh") that switched xtensa-build-all.sh to CMake and
removed its only use of ARCH. Also note commit 82b4da291b ("cmake:
defconfigs support") which added a few CONFIG_SMP defconfigs and
commit 905bad4252 ("scripts: xtensa-build-all: Add support to force
build UP Arch") using override.config.
Much more recently in June 2020: commit fd506970cc ("zephyr: kconfig:
rename CONFIG_SMP -> CONFIG_MULTICORE")
Zawinski's Law of Software Envelopment: adding lines of code is much
easier than removing the same amount and would be very impractical
without git. This one was easy enough.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
As CMake forks one compiler process for each source file, the XTensa
compiler spends much more time idle waiting for the license server over
the network than actually using CPU or disk.
On my VM with 16 virtual cores, rebuilding one platform from scratch
with this commit goes down from 12s to less than 9s: more than 25%
faster. With Ninja it goes down from 11s to less than 8s. My license
server is 25ms away: a closer server does not need as many threads while
a more distant server would obviously benefit for even more
threads... while already getting an even better improvement than 25%
from just 3 times more threads! It's complicated and we probably don't
want to start the build by measuring latency to the license server.
The entire, purely local _gcc_ build is so fast (~ 1s) that observing any
the difference between -j nproc and -j nproc*N is practically impossible
so let's not waste RAM when building with gcc.
Also: log the $XTENSA_SYSTEM variable as it is required for incremental
builds; remove one apostrophe in the here-doc usage as it breaks the
parser of some editor (jed).
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Notably:
- Log error and default MEU_OFFSET when meu -ver is not found.
- Explain rimage -s option
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Notably: xt-xcc --show-config failed with: No such file or directory
... when the directory exists but is wrong.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
There is no such compiler as RF-2017.8 that is in use. The correct
compiler is RG-2017.8 for all i.MX platforms.
Signed-off-by: Paul Olaru <paul.olaru@nxp.com>
Add cmake -DINIT_CONFIG= option that can point at any initial file.
"make clean" does not delete .config any more.
Note reconfiguration does NOT causes recompilation because -imacros
hides the generated .h from CMake's dependency scan. This is not a
regression, that problems exists since -imacros was introduced. At least
it's now possible to "make clean" and rebuild without losing the .config
file.
Fix for #3617
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Lets use the same version of Xtensa toolchain as Intel in order
to avoid compilation problems.
It is difficult to support multiple toolchain versions because specific
headers needs to be updated, so lets have the same version supported
between IMX and Intel.
Signed-off-by: Daniel Baluta <daniel.baluta@nxp.com>
The script already logs the full CMake command that is re-usable outside
this script... except for the PATH change. Expose that sneaky PATH
change.
Debugging every build issue starts with peeling the too many layers of
indirection.
Also fix some minor issue in the help message.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
This makes it possible to switch to Ninja with a single line change
which can be useful to test build changes and issues.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Currently xtensa-build-all.sh has restriction to use rimage for tgl
signing, because rimage didn't support tgl. As rimage now has support
for tgl signing let's remove this limitation.
Signed-off-by: Jaska Uimonen <jaska.uimonen@intel.com>
"make + make install" duplicates many steps, probably because of the
fatal combination of build timestamps + source hash, see PR #3353 for a
similar example.
A single, combined "make install" generates the exact same binary
outputs 30% faster and prints 30% shorter build logs.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Following commit 88b69cc2e5 ("scripts: xtensa-build-all: Add support
for building TGL and signing with MEU") we now have an unified and
flexible PRIVATE_KEY_OPTION that can do anything with very little code.
The -k option was never a good idea in the first place because it
already required an environment variable anyway (RIMAGE_PRIVATE_KEY)
instead of a command line argument so the code should have just checked
whether RIMAGE_PRIVATE_KEY was defined, that would have been
enough. Requiring the user to "double-confirm" with -k has been adding
extra complication for both the user interface and the implementation.
xtensa-build-all is effectively a "CMake configuration convenience"
script, however it shouldn't become an additional layer of indirection
and complexity and its interface should expose CMake (which is already a
layer of indirection!) as directly as possible.
See longer discussion in PR #3187
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
We switch to PCH-based names instead of CPU config names for tigerlake+
platforms, because we need to support multiple variants of PCHs based
on one CPU famile - for example TGL-H & TGL-LP.
Signed-off-by: Janusz Jankowski <janusz.jankowski@linux.intel.com>
As incredible as it sounds, some people run neither "git status" nor
"git diff" every few minutes and not even when their build fails. There
has been reports that they're puzzled when they miss a required
submodule update. This is an attempt to draw their attention based on
the assumption that they pay more attention to the CMake logs.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Pass PRIVATE_KEY_OPTION environment variable to docker to be able to
define external key for signing.
Suggested-by: Marc Herbert <marc.herbert@intel.com>
Signed-off-by: Jaska Uimonen <jaska.uimonen@intel.com>
Add support for building TGL FW and signing with MEU tool. Signing
with rimage is not supported atm.
Signed-off-by: Ranjani Sridharan <ranjani.sridharan@linux.intel.com>
Some platforms needs platform-specific configuration
that override the default config. For example, chrome platforms
need the LPS to be enabled by default for maximizing the
power savings for the WoV feature. Add a -o switch to allow
specifying the override config file.
Signed-off-by: Ranjani Sridharan <ranjani.sridharan@linux.intel.com>
This gets rid of about 600 lines of noisy "Entering directory..."
messages which cuts the size of the output of the script in half leaving
only useful stuff on the screen. This (and other changes before it) may
finally avoid bugs like 'commit f430addec7 ("Tools: Fuzzer: Do not use
illegal BUILD_COMMAND in CMakeLists.txt") evading scrutiny for months.
After so many years enjoying the convenience of make's '-C' option, I
finally did some research and realized it is the cause of this very
serious noise issue with no other easy way out :-(
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
... thanks to the docker --env option which is smart enough not to pass
anything when there is nothing to pass.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
To produce the exact same binaries than before this commit:
CMAKE_BUILD_TYPE=' ' ./scripts/build-tools.sh
To observe which CFLAGS are being used:
CMAKE_BUILD_TYPE=Release VERBOSE=1 ./scripts/build-tools.sh
Off topic: the logger fails to build with: -O3 -DNDEBUG -Wall -Werror
(all other tools build fine with these flags)
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Default is verbose enough. VERBOSE=1 can still be used later at make
time.
Also change nproc to nproc --all because we don't care about offline
processors.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Use newer rebuild-testbench.sh instead, added by
commit 46578cb103 ("scripts: Add a new script to rebuild testbench")
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
SOF_SRC_HASH always must have integer value, because of usage
them to initialize global variable in source code.
Variables, which may be empty should be used inside quotation
to prevent cmake incorect number of arguments error.
Signed-off-by: Karol Trzcinski <karolx.trzcinski@linux.intel.com>
Temporary files are useful to understand and debug the build.
Moving output to another directory, keeps binary directory clean.
Signed-off-by: Karol Trzcinski <karolx.trzcinski@linux.intel.com>
This is possible location of .git folder, instead of CMAKE_SOURCE_DIR.
Without this patch, checked condition always return false and
GIT_LOG_HASH is used instead source code hash.
Signed-off-by: Karol Trzcinski <karolx.trzcinski@linux.intel.com>