The main change is the elimination of the bootstrapper, a design flaw
/ misfeature.
Update the documentation to be compatible with the 0.6.x releases as
well. This has to be done atomically, as there were incompatible
changes. Make use of the versionchanged and versionadded directives
to begin keeping track of how these APIs are evolving.
(Note that west 0.6.0 will remain compatible with the extension
commands in Zephyr v1.14 LTS as long as that is still alive. This
change is targeted towards Zephyr 2.0 users.)
This requires a bump in the shippable container and allows us to
simplify the west_commands test procedure.
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
Clean up the INPUT paths for doxygen scanning:
- directory scans include sub-directories so no need
to include them explicitly
- remove excluded (legacy) includes that no longer exist
Sync directory changes in the CMakeLists.txt (as noted in the comments)
Add content to the home page of the doxygen-generated HTML, and add a
Zephyr branding theme.
Signed-off-by: David B. Kinder <david.b.kinder@intel.com>
Fixes: #6773
The module processing logic in Zephyr requires Python, since it uses a
Python script to locate and process the modules. It is therefore
required to find the Python executable using CMake before actually
including the CMake script that processes the modules. In order to
prevent this from happening again, move the inclusion of version.cmake
as well in case that script chooses to use Python in the future.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
With the addition of a local CMake variable ZEPHYR_BASE that mirrors the
value of the environment variable the documentation build broke on
Windows. This is because CMake was incorrectly interpreting backslashes
present in the Windows path as escaping characters. In order to avoid
this, use CMake's built-in file(TO_CMAKE_PATH) functionality, which
converts the path to a canonical version using forward slashes only.
At the same time rework boilerplate.cmake to use the same mechanism
instead of manually replacing backslashes and removing trailing ones.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
This fixes a corner case.
The ZEPHYR_BASE variable might be set in the environment at generation
time, but unset at build time. Save it in the cmake -E env calls to
scripts that need it.
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
Update the files which contain no license information with the
'Apache-2.0' SPDX license identifier. Many source files in the tree are
missing licensing information, which makes it harder for compliance
tools to determine the correct license.
By default all files without license information are under the default
license of Zephyr, which is Apache version 2.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
When generating the Kconfig reference documentation, enable the warnings
for undefined Kconfig options in order to make sure all references are
met.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
In order to parse all the Kconfig files that make up the whole Kconfig
tree we need to process the modules, since this can optionally provide
Kconfig files. In order to do so, include the modules CMake file so that
they are listed and a Kconfig.modules is generated.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Currently, the Kconfig.modules file is placed in the build directory
relative to the CMake "project". But technically, the file is not
project-specific, but global, or build-directory specific.
So we move it up one level to the CMAKE_BINARY_DIR instead. Currently,
there is only one project, so this change has no effect, but this
enables us to have multiple projects in the future, which again
enables multi-image builds.
Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
Signed-off-by: Håkon Øye Amundsen <haakon.amundsen@nordicsemi.no>
Avoids having it look like a Kconfig symbol reference (which gets
detected by an upcoming CI check).
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Do not load the GENERATED_DTS_BOARD_CONF if in doc mode, since it will
not defined as it doesn't apply. No need to defined it to a dummy value.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Use file(GLOB_RECURSE ...) to generate a superset of doxygen input
files. Use this list as a dependency for a new empty file
doc/_build/last_doxy_run_tstamp.
The incremental build time to re-run "make htmldocs-fast" after a small
.rst doc change is brought down from from ~8s to ~4s on my Linux system
and from ~30s to less than 10s on my MacBook.
(Doxygen changes are still slow for a couple reasons recently isolated
and documented in doc/CMakeLists.txt)
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Clearly shows where time is spent. More specifically helps work on the
next commit while removing noise from it.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Use a variable to set the doxygen output directory so we can refer to it
later when fixing mtimes instead of hardcoding it.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Doxygen doesn't support incremental builds. It could be ok because it
doesn't take much time in this codebase. However it's not because it
makes old output look new which has a cascade effect on sphinx:
https://sourceforge.net/p/doxygen/mailman/message/36580807/
Make doxygen artificially smarter by saving and restoring modify
timestamps on the filesystem for doxygen output files that haven't
changed.
On my system this brings down the time to run an incremental "make
htmldocs" from 75s down to 8s (cmake -DKCONFIG_TURBO_MODE=1 used in both
cases)
This optimization speeds-up non-doxygen documentation work only.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Make kconfigfunctions.py agnostic to where GENERATED_DTS_BOARD_CONF is
located.
We don't want to encode the path of GENERATED_DTS_BOARD_CONF into more
places than necessary. Also, re-using GENERATED_DTS_BOARD_CONF makes
it easier to change where generated_dts_board.conf is located.
Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
This file is now needed by Kconfig and was not created during doc
creation. So touch an empty file to make doc build go through.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
On (at least) Fedora the /usr/bin/sphinx-build binary always belong to
python2-sphinx which results in a confusing "cannot import name
DEVNULL" error below (DEVNULL is available from python 3.3).
Tell find_program() to look for sphinx-build-3 first and then
sphinx-build next.
(shorten) Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/sphinx/config.py",
line 161, in __init__ execfile_(filename, config)
...
File "conf.py", line 17, in <module>
from subprocess import CalledProcessError, check_output, DEVNULL
ImportError: cannot import name DEVNULL
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Introduces the ARCH_ROOT argument, similar to BOARD_ROOT and SOC_ROOT.
This enables support for out-of-tree architectures.
The ARCH_ROOT out-of-tree layout is expected to be the following:
* ${ARCH_ROOT}/arch/${ARCH}/
* ${ARCH_ROOT}/include/arch/${ARCH}/ (Optional)
Signed-off-by: Klaus Petersen <kape@oticon.com>
Export a new KCONFIG_DOC_MODE environment variable when building the doc
and invoking Kconfig, so that the functions that expect a build folder
can accordingly return a hardcoded value.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Building the documentation for all the Kconfig options significantly
adds to the total doc build time. When making and testing major changes
to the documentation, we provide an option to temporarily stub-out
the auto-generated configuration documentation so the doc build process
runs much faster.
To enable this mode, set the following option when invoking cmake
-DKCONFIG_TURBO_MODE=1
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
The 'doc' build has been invoking 'project' with the argument "VERSION
${PROJECT_VERSION}". This is cargo-cult code inherited from the
toplevel build system.
This can be safely removed because in the doc build the versioning
variables that are set by 'project' are unused and PROJECT_VERSION is
not available to be de-referenced at that point in time any way ...
This fixes#12282
Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
Move to latest cmake version with many bug fixes and enhancements.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Adding ninja targets based on the files that are to be copied into the
build folder doesn't seem to speed up the build, so simplify the CMake
script by relying on extract_content.py to do the copy.
Numbers on my machine:
* master
clean: 6m45.541s, 6m8.113s
incremental: 1m24.233s, 1m32.720s
* revert
clean: 6m25.221s, 6m19.751s
incremental:1m20.893s, 1m20.337s
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
In a clean build, Ninja is creating copies of the extracted
documentation outputs regardless of whether they already exist or
not. I can't explain this behavior from the documentation of the build
command, but it's so, even if extract_content.py is patched to use
copyfile() instead of copy2() so that outputs (copied .rst files in
the build directory) have strictly newer mtimes than inputs (source
tree .rst files).
This extra copying doesn't happen for incremental builds, for some
reason.
To account for this behavior, add and use an extract_content.py option
that skips content extraction completely and leaves that all up to the
add_custom_command() which tracks each individual input->output build
command.
Reported-by: Carles Cufi <carles.cufi@nordicsemi.no>
Signed-off-by: Marti Bolivar <marti@foundries.io>
Most of time (i.e. when just writing to .rst files or editing images,
rather than editing Doxygen or Kconfig), a full rebuild of the docs
pipeline is not necessary.
This is unfortunate, because it takes over a minute to re-run doxygen
etc. (on my system), whereas an incremental sphinx-build run should
only take a few seconds.
As a band-aid to help developers who have already generated a docs
build and know the consequences of their actions, add "sphinx-html"
and "sphinx-latex" targets. These just re-extract content files into
the build directory and run sphinx-build. This makes for much faster
documentation edit/compile/read loops.
Add some extra COMMENTs while we're here, to make it more obvious
what's going on when the docs build system crunches on something.
A "proper" solution for changing "ninja html" to have similar
performance would need, at least, to:
- only re-run doxygen if any source files have changed
- only re-generate Kconfig if any of those files have changed
That's all left to future work, as this is good enough to make
rebuilding the docs bearable for me.
Signed-off-by: Marti Bolivar <marti@foundries.io>
The logic which copies source documentation files into the build
directory could use some improvements to its dependency management, so
that when a source file changes, extract_content.py gets re-run.
Make these changes as follows:
- Add an --outputs flag to extract_content.py, so that the
sources it depends on and generates can be saved into a file and
thus made known to the build system
- Change the way the sources and destination are specified in the
extract_content.py command line so that the entire job can be done
in a single command, rather than multiple (to avoid having to
collate multiple --outputs files in CMake)
- Extract the content at configure time with execute_process(),
tracking all inputs and outputs within the build system itself. Use
this information to make sure that each individual output depends on
just its exact input file, ensuring updated inputs produce updated
outputs without having to call extract_content.py again.
- Ensure that the "content" build system target depends on all the
outputs, transitively triggering a rebuild any time an input
file (e.g. .rst documentation file or included image/source file)
changes.
Signed-off-by: Marti Bolivar <marti@foundries.io>
Minor cleanups:
- Allow the user to run extract.content.py -h with ZEPHYR_BASE unset
without crashing
- Replace the use of the ZEPHYR_BUILD environment variable with an
--ignore argument (that can be given multiple times) which specifies
a path to a directory whose files should *not* be copied, and use
it to ignore the build directory
- rename, reorder, and tweak get_files() arguments a bit to go from
source to dest, with extra configuration at the end, getting rid of
local shadowing of global built-in functions
Signed-off-by: Marti Bolivar <marti@foundries.io>
In order to make PDF generation work on Windows, search for a generic
Latex package and use full quotes for the latexmk environment variable.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
The documentation has been setting the environment variable
KERNELVERSION to the PROJECT_VERSION of the documentation. But
PROJECT_VERSION is not set during the documentation build so this has
no effect.
Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
The following error appeared on documentation builds when enabling the
USES_TERMINAL flag:
USES_TERMINAL may not be specified without any COMMAND
This is because targets without a command should not be using the flag
at all, so remove its usage from those. Instead, use it in the
add_custom_command() that actualy invokes sphinx.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
The prefixes might be a leftover from the old 'option env="..."' symbols
(which are no longer needed). Since environment variables can be
referenced directly now, there's no point in having a prefix.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
The chain of dependencies works for make but not for Ninja.
Make it explicit so Ninja will work to generate PDF.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This adds new targets to generate build documentation through
LaTEX to PDF.
There are a few notes:
1. pdflatex complains about the tex file generated by doxygen
so it needs to be fixed with a Python script before feeding
in through pdflatex.
2. SVG files are not recognized by pdflatex so they are converted
to known good format on the fly, only for producing PDF. This
uses the libRSVG's rsvg-convert tool.
Relates to #6782.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Due a design goof, the top-level Kconfig file passed to
Kconfig.__init__() wasn't looked up relative to $srctree. This broke a
planned fix for an issue with the CI check for references to undefined
Kconfig files in external projects.
Update Kconfiglib to upstream revision 953cc12464 to fix it.
Also update doc/CMakeLists.txt to pass 'Kconfig' instead of '../Kconfig'
to genrest.py. $srctree is already set to $ZEPHYR_BASE.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
In order to be able to place the generated HTML content directly in a
folder of the user's choice, make the Sphinx HTML output folder
configurable.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Since we're now copying the whole doc/ folder into the output folder,
there's no need anymore to copy certain files by hand.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
To avoid having to refer to non-rst files artificially in order to get
them copied into the destination folder, make a raw full copy of the
doc/ folder instead of copying only the .rst files.
Fixes#9128
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
The following changes have been made to support out-of-tree builds:
* In order to avoid using relative hardcoded paths, use CMake's
configure_file() to replace the paths in the doxygen input file
so that the output directory is set correctly.
* All .rst and additional required files are now copied from the doc/
folder into the build/rst folder using extract_content.py. The
samples/ and boards/ folder are copied twice (once into build/rst
and another into build/doc/rst) to manage relative paths.
* All paths are absolute where possible, including themes and static
content.
This patch ensures that the Zephyr repo is not contaminated by the
build at all.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
When using Ninja, output will be buffered until the command completes
unless USES_TERMINAL is used. Use USES_TERMINAL here so that output is
sent directly to the terminal without having to wait. This is a bit of
an abuse of the Ninja console pool, but for building documentation this
should not be an issue since it's a self-contained CMake project.
Note that this is done conditionally, only when Sphinx output is not
disabled by using its "-q" flag.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
In order to be able to use this CMakeLists.txt file from other projects,
use the proper variables so that they refer to their own paths.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Fix the way a list is constructed during argument parsing so that CMake
propagates the options correctly to Sphinx.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Move options to the place they belong ([OPTIONS] SOURCEDIR OUTPUTDIR)
according to sphinx itself.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Add a new CMakeLists.txt that is capable of building the documentation.
Note that the error checkin currently only works on Unix platforms since
it's tied to a shell script.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Marti Bolivar