Add an option --separate-all-index that makes genrest.py generate a
separate index-all.rst index page that lists all symbols, instead of
listing all symbols in index.rst. index-all.rst is linked from
index.rst.
This was originally motivated by an external project where index.rst
becomes the top-level page, which runs into a Sphinx bottleneck with
sphinx_rtd_theme. See https://github.com/sphinx-doc/sphinx/issues/6909.
This turned out pretty nice after some feedback from Ruth Fuchss, so use
it for Zephyr too.
Also unclutter the generated documentation a bit by removing some
headings. This makes the navigation menu on the left nicer too.
Piggyback making genrest.py executable, which is handy when running it
manually.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Using ':' as the separator in --modules breaks module specifications
like
nrfxlib:nrfxlib:C:/Users/Carles/src/ncs/nrfxlib
The ':' in 'C:' gets seen as a separator.
Use ',' instead, which is unlikely to appear in paths (at least in
--modules). Treat a doubled ',,' as a literal ','.
Another option would be ';', but it clashes with how CMake represents
lists, which is awkward.
Also make quoting of arguments with spaces more robust by passing
VERBATIM to add_custom_target().
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Use --modules to split the Kconfig reference up by where symbols are
defined. See the argparse docstring for --modules in
doc/scripts/genrest.py.
Not sure what the best way to split things up is, so feedback would be
appreciated. I just pulled out some top-level directories. It could
always be tweaked later.
If a symbol is defined in more than one module (by being defined in
multiple locations), it appears on multiple index pages.
To build the documentation, do
$ mkdir doc/b && cd doc/b
$ cmake -GNinja ..
$ ninja htmldocs
(output in html/reference/kconfig/index.html)
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Need to add the kernel_arch_interface.h header here
to match zephyr.doxyfile.in.
This is the header which contains the formal kernel-to-
architecture interface and we want docs generated for it.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Paths like soc//arm/... showed up in the output due to the extra '/' at
the end of the SOC_DIR value. Remove the redundant '/' in SOC_DIR and
other environment variables that get referenced in the Kconfig files.
Kconfiglib never normalizes paths, because it avoids some gotchas, e.g.
with symlinks.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Add a --modules flag to genrest.py for generating separate index pages
for symbols defined within certain paths.
Passing
--modules Shell🐚subsys/shell Storage:storage:subsys/storage
will generate these index pages, instead of a single index.rst file:
- index-shell.rst: Lists the symbols defined in subsys/shell
- index-storage.rst: Lists the symbols defined in subsys/storage
- index-main.rst: Lists all symbols that are not in subsys/shell or
subsys/storage
- index-all.rst: Lists all symbols
- index.rst: Contains links to the other index pages
The string before the first ':' ('Shell' and 'Storage' above) is used
when generating the title of the index page. 'Shell' gives
'Shell Configuration Options', for example.
The title for index-main.rst can be set by passing
'--non-module-title <title>'. It defaults to "Zephyr".
By default, paths in symbol information pages that are within modules
are shown as '<title>/<path relative to module>'. This can be disabled
by passing --keep-module-paths.
--keep-module-paths would make sense for the example above, but
stripping the path to the module is nice when dealing with modules
defined outside the Zephyr repository.
If a symbol is defined in multiple modules (or both in a module and
outside all modules), it will appear on multiple index pages.
This commit also simplifies how genrest.py is called a bit, making the
Kconfig filename optional (default: Kconfig).
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Correctly set the separator as a semicolon on Windows when
constructing the PYTHONPATH environment variable so that CMake
doesn't intepret it and swallow it.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
The main change is the elimination of the bootstrapper, a design flaw
/ misfeature.
Update the documentation to be compatible with the 0.6.x releases as
well. This has to be done atomically, as there were incompatible
changes. Make use of the versionchanged and versionadded directives
to begin keeping track of how these APIs are evolving.
(Note that west 0.6.0 will remain compatible with the extension
commands in Zephyr v1.14 LTS as long as that is still alive. This
change is targeted towards Zephyr 2.0 users.)
This requires a bump in the shippable container and allows us to
simplify the west_commands test procedure.
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
Clean up the INPUT paths for doxygen scanning:
- directory scans include sub-directories so no need
to include them explicitly
- remove excluded (legacy) includes that no longer exist
Sync directory changes in the CMakeLists.txt (as noted in the comments)
Add content to the home page of the doxygen-generated HTML, and add a
Zephyr branding theme.
Signed-off-by: David B. Kinder <david.b.kinder@intel.com>
Fixes: #6773
The module processing logic in Zephyr requires Python, since it uses a
Python script to locate and process the modules. It is therefore
required to find the Python executable using CMake before actually
including the CMake script that processes the modules. In order to
prevent this from happening again, move the inclusion of version.cmake
as well in case that script chooses to use Python in the future.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
With the addition of a local CMake variable ZEPHYR_BASE that mirrors the
value of the environment variable the documentation build broke on
Windows. This is because CMake was incorrectly interpreting backslashes
present in the Windows path as escaping characters. In order to avoid
this, use CMake's built-in file(TO_CMAKE_PATH) functionality, which
converts the path to a canonical version using forward slashes only.
At the same time rework boilerplate.cmake to use the same mechanism
instead of manually replacing backslashes and removing trailing ones.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
This fixes a corner case.
The ZEPHYR_BASE variable might be set in the environment at generation
time, but unset at build time. Save it in the cmake -E env calls to
scripts that need it.
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
Update the files which contain no license information with the
'Apache-2.0' SPDX license identifier. Many source files in the tree are
missing licensing information, which makes it harder for compliance
tools to determine the correct license.
By default all files without license information are under the default
license of Zephyr, which is Apache version 2.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
When generating the Kconfig reference documentation, enable the warnings
for undefined Kconfig options in order to make sure all references are
met.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
In order to parse all the Kconfig files that make up the whole Kconfig
tree we need to process the modules, since this can optionally provide
Kconfig files. In order to do so, include the modules CMake file so that
they are listed and a Kconfig.modules is generated.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Currently, the Kconfig.modules file is placed in the build directory
relative to the CMake "project". But technically, the file is not
project-specific, but global, or build-directory specific.
So we move it up one level to the CMAKE_BINARY_DIR instead. Currently,
there is only one project, so this change has no effect, but this
enables us to have multiple projects in the future, which again
enables multi-image builds.
Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
Signed-off-by: Håkon Øye Amundsen <haakon.amundsen@nordicsemi.no>
Avoids having it look like a Kconfig symbol reference (which gets
detected by an upcoming CI check).
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Do not load the GENERATED_DTS_BOARD_CONF if in doc mode, since it will
not defined as it doesn't apply. No need to defined it to a dummy value.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Use file(GLOB_RECURSE ...) to generate a superset of doxygen input
files. Use this list as a dependency for a new empty file
doc/_build/last_doxy_run_tstamp.
The incremental build time to re-run "make htmldocs-fast" after a small
.rst doc change is brought down from from ~8s to ~4s on my Linux system
and from ~30s to less than 10s on my MacBook.
(Doxygen changes are still slow for a couple reasons recently isolated
and documented in doc/CMakeLists.txt)
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Clearly shows where time is spent. More specifically helps work on the
next commit while removing noise from it.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Use a variable to set the doxygen output directory so we can refer to it
later when fixing mtimes instead of hardcoding it.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Doxygen doesn't support incremental builds. It could be ok because it
doesn't take much time in this codebase. However it's not because it
makes old output look new which has a cascade effect on sphinx:
https://sourceforge.net/p/doxygen/mailman/message/36580807/
Make doxygen artificially smarter by saving and restoring modify
timestamps on the filesystem for doxygen output files that haven't
changed.
On my system this brings down the time to run an incremental "make
htmldocs" from 75s down to 8s (cmake -DKCONFIG_TURBO_MODE=1 used in both
cases)
This optimization speeds-up non-doxygen documentation work only.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Make kconfigfunctions.py agnostic to where GENERATED_DTS_BOARD_CONF is
located.
We don't want to encode the path of GENERATED_DTS_BOARD_CONF into more
places than necessary. Also, re-using GENERATED_DTS_BOARD_CONF makes
it easier to change where generated_dts_board.conf is located.
Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
This file is now needed by Kconfig and was not created during doc
creation. So touch an empty file to make doc build go through.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
On (at least) Fedora the /usr/bin/sphinx-build binary always belong to
python2-sphinx which results in a confusing "cannot import name
DEVNULL" error below (DEVNULL is available from python 3.3).
Tell find_program() to look for sphinx-build-3 first and then
sphinx-build next.
(shorten) Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/sphinx/config.py",
line 161, in __init__ execfile_(filename, config)
...
File "conf.py", line 17, in <module>
from subprocess import CalledProcessError, check_output, DEVNULL
ImportError: cannot import name DEVNULL
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Introduces the ARCH_ROOT argument, similar to BOARD_ROOT and SOC_ROOT.
This enables support for out-of-tree architectures.
The ARCH_ROOT out-of-tree layout is expected to be the following:
* ${ARCH_ROOT}/arch/${ARCH}/
* ${ARCH_ROOT}/include/arch/${ARCH}/ (Optional)
Signed-off-by: Klaus Petersen <kape@oticon.com>
Export a new KCONFIG_DOC_MODE environment variable when building the doc
and invoking Kconfig, so that the functions that expect a build folder
can accordingly return a hardcoded value.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Building the documentation for all the Kconfig options significantly
adds to the total doc build time. When making and testing major changes
to the documentation, we provide an option to temporarily stub-out
the auto-generated configuration documentation so the doc build process
runs much faster.
To enable this mode, set the following option when invoking cmake
-DKCONFIG_TURBO_MODE=1
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
The 'doc' build has been invoking 'project' with the argument "VERSION
${PROJECT_VERSION}". This is cargo-cult code inherited from the
toplevel build system.
This can be safely removed because in the doc build the versioning
variables that are set by 'project' are unused and PROJECT_VERSION is
not available to be de-referenced at that point in time any way ...
This fixes#12282
Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
Move to latest cmake version with many bug fixes and enhancements.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Adding ninja targets based on the files that are to be copied into the
build folder doesn't seem to speed up the build, so simplify the CMake
script by relying on extract_content.py to do the copy.
Numbers on my machine:
* master
clean: 6m45.541s, 6m8.113s
incremental: 1m24.233s, 1m32.720s
* revert
clean: 6m25.221s, 6m19.751s
incremental:1m20.893s, 1m20.337s
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
In a clean build, Ninja is creating copies of the extracted
documentation outputs regardless of whether they already exist or
not. I can't explain this behavior from the documentation of the build
command, but it's so, even if extract_content.py is patched to use
copyfile() instead of copy2() so that outputs (copied .rst files in
the build directory) have strictly newer mtimes than inputs (source
tree .rst files).
This extra copying doesn't happen for incremental builds, for some
reason.
To account for this behavior, add and use an extract_content.py option
that skips content extraction completely and leaves that all up to the
add_custom_command() which tracks each individual input->output build
command.
Reported-by: Carles Cufi <carles.cufi@nordicsemi.no>
Signed-off-by: Marti Bolivar <marti@foundries.io>
Most of time (i.e. when just writing to .rst files or editing images,
rather than editing Doxygen or Kconfig), a full rebuild of the docs
pipeline is not necessary.
This is unfortunate, because it takes over a minute to re-run doxygen
etc. (on my system), whereas an incremental sphinx-build run should
only take a few seconds.
As a band-aid to help developers who have already generated a docs
build and know the consequences of their actions, add "sphinx-html"
and "sphinx-latex" targets. These just re-extract content files into
the build directory and run sphinx-build. This makes for much faster
documentation edit/compile/read loops.
Add some extra COMMENTs while we're here, to make it more obvious
what's going on when the docs build system crunches on something.
A "proper" solution for changing "ninja html" to have similar
performance would need, at least, to:
- only re-run doxygen if any source files have changed
- only re-generate Kconfig if any of those files have changed
That's all left to future work, as this is good enough to make
rebuilding the docs bearable for me.
Signed-off-by: Marti Bolivar <marti@foundries.io>
The logic which copies source documentation files into the build
directory could use some improvements to its dependency management, so
that when a source file changes, extract_content.py gets re-run.
Make these changes as follows:
- Add an --outputs flag to extract_content.py, so that the
sources it depends on and generates can be saved into a file and
thus made known to the build system
- Change the way the sources and destination are specified in the
extract_content.py command line so that the entire job can be done
in a single command, rather than multiple (to avoid having to
collate multiple --outputs files in CMake)
- Extract the content at configure time with execute_process(),
tracking all inputs and outputs within the build system itself. Use
this information to make sure that each individual output depends on
just its exact input file, ensuring updated inputs produce updated
outputs without having to call extract_content.py again.
- Ensure that the "content" build system target depends on all the
outputs, transitively triggering a rebuild any time an input
file (e.g. .rst documentation file or included image/source file)
changes.
Signed-off-by: Marti Bolivar <marti@foundries.io>
Minor cleanups:
- Allow the user to run extract.content.py -h with ZEPHYR_BASE unset
without crashing
- Replace the use of the ZEPHYR_BUILD environment variable with an
--ignore argument (that can be given multiple times) which specifies
a path to a directory whose files should *not* be copied, and use
it to ignore the build directory
- rename, reorder, and tweak get_files() arguments a bit to go from
source to dest, with extra configuration at the end, getting rid of
local shadowing of global built-in functions
Signed-off-by: Marti Bolivar <marti@foundries.io>
In order to make PDF generation work on Windows, search for a generic
Latex package and use full quotes for the latexmk environment variable.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
The documentation has been setting the environment variable
KERNELVERSION to the PROJECT_VERSION of the documentation. But
PROJECT_VERSION is not set during the documentation build so this has
no effect.
Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
The following error appeared on documentation builds when enabling the
USES_TERMINAL flag:
USES_TERMINAL may not be specified without any COMMAND
This is because targets without a command should not be using the flag
at all, so remove its usage from those. Instead, use it in the
add_custom_command() that actualy invokes sphinx.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
The prefixes might be a leftover from the old 'option env="..."' symbols
(which are no longer needed). Since environment variables can be
referenced directly now, there's no point in having a prefix.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
The chain of dependencies works for make but not for Ninja.
Make it explicit so Ninja will work to generate PDF.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This adds new targets to generate build documentation through
LaTEX to PDF.
There are a few notes:
1. pdflatex complains about the tex file generated by doxygen
so it needs to be fixed with a Python script before feeding
in through pdflatex.
2. SVG files are not recognized by pdflatex so they are converted
to known good format on the fly, only for producing PDF. This
uses the libRSVG's rsvg-convert tool.
Relates to #6782.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Due a design goof, the top-level Kconfig file passed to
Kconfig.__init__() wasn't looked up relative to $srctree. This broke a
planned fix for an issue with the CI check for references to undefined
Kconfig files in external projects.
Update Kconfiglib to upstream revision 953cc12464 to fix it.
Also update doc/CMakeLists.txt to pass 'Kconfig' instead of '../Kconfig'
to genrest.py. $srctree is already set to $ZEPHYR_BASE.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
In order to be able to place the generated HTML content directly in a
folder of the user's choice, make the Sphinx HTML output folder
configurable.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Since we're now copying the whole doc/ folder into the output folder,
there's no need anymore to copy certain files by hand.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
To avoid having to refer to non-rst files artificially in order to get
them copied into the destination folder, make a raw full copy of the
doc/ folder instead of copying only the .rst files.
Fixes#9128
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
The following changes have been made to support out-of-tree builds:
* In order to avoid using relative hardcoded paths, use CMake's
configure_file() to replace the paths in the doxygen input file
so that the output directory is set correctly.
* All .rst and additional required files are now copied from the doc/
folder into the build/rst folder using extract_content.py. The
samples/ and boards/ folder are copied twice (once into build/rst
and another into build/doc/rst) to manage relative paths.
* All paths are absolute where possible, including themes and static
content.
This patch ensures that the Zephyr repo is not contaminated by the
build at all.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
When using Ninja, output will be buffered until the command completes
unless USES_TERMINAL is used. Use USES_TERMINAL here so that output is
sent directly to the terminal without having to wait. This is a bit of
an abuse of the Ninja console pool, but for building documentation this
should not be an issue since it's a self-contained CMake project.
Note that this is done conditionally, only when Sphinx output is not
disabled by using its "-q" flag.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Ulf Magnusson