The dependencies in the scripts are figures, images, includes or literal
includes. Typically those dependencies are placed relative to the .rst
file that is being copied, but in some instances you might want them to
refer to an absolute path which, in fact, is relative to Sphinx's
top-level source directory. In this case, and as long as the top-level
source directory is provided, do not emit a warning.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Previously, symbols not defined with 'menuconfig' with children weren't
listed in the children's menu paths. It was deliberate, but it's
probably an anti-feature in retrospect, because it can make it harder to
find stuff by following the menu path.
Don't try to be clever and just list all the parent nodes in the menu
path.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Fixes pylint warnings like this one:
doc/conf.py:325:0: W1401: Anomalous backslash in string: '\s'.
String constant might be missing an r prefix.
(anomalous-backslash-in-string)
The reason for this warning is that backslash escapes are interpreted in
non-raw (non-r-prefixed) strings. For example, '\a' and r'\a' are not
the same string (first one has a single ASCII bell character, second one
has two characters).
It just happens that there's no \s (or \., or \/) escape for example,
and '\s' turns into two characters (as needed for a regex). It's risky
to rely on stuff like that regexes though. Best to make them raw strings
unless they're super trivial.
Also note that '\s' and '\\s' turn into the same string.
Another tip: A literal ' can be put into a string with "blah'blah"
instead of 'blah\'blah'.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Doxygen doesn't support incremental builds. It could be ok because it
doesn't take much time in this codebase. However it's not because it
makes old output look new which has a cascade effect on sphinx:
https://sourceforge.net/p/doxygen/mailman/message/36580807/
Make doxygen artificially smarter by saving and restoring modify
timestamps on the filesystem for doxygen output files that haven't
changed.
On my system this brings down the time to run an incremental "make
htmldocs" from 75s down to 8s (cmake -DKCONFIG_TURBO_MODE=1 used in both
cases)
This optimization speeds-up non-doxygen documentation work only.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Building the documentation for all the Kconfig options significantly
adds to the total doc build time. When making and testing major changes
to the documentation, we provide an option to temporarily stub-out
the auto-generated configuration documentation so the doc build process
runs much faster.
To enable this mode, set the following option when invoking cmake
-DKCONFIG_TURBO_MODE=1
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Normalize the path of ignored folders so that variations in paths
formats on Windows are detected correctly.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
In a clean build, Ninja is creating copies of the extracted
documentation outputs regardless of whether they already exist or
not. I can't explain this behavior from the documentation of the build
command, but it's so, even if extract_content.py is patched to use
copyfile() instead of copy2() so that outputs (copied .rst files in
the build directory) have strictly newer mtimes than inputs (source
tree .rst files).
This extra copying doesn't happen for incremental builds, for some
reason.
To account for this behavior, add and use an extract_content.py option
that skips content extraction completely and leaves that all up to the
add_custom_command() which tracks each individual input->output build
command.
Reported-by: Carles Cufi <carles.cufi@nordicsemi.no>
Signed-off-by: Marti Bolivar <marti@foundries.io>
The logic which copies source documentation files into the build
directory could use some improvements to its dependency management, so
that when a source file changes, extract_content.py gets re-run.
Make these changes as follows:
- Add an --outputs flag to extract_content.py, so that the
sources it depends on and generates can be saved into a file and
thus made known to the build system
- Change the way the sources and destination are specified in the
extract_content.py command line so that the entire job can be done
in a single command, rather than multiple (to avoid having to
collate multiple --outputs files in CMake)
- Extract the content at configure time with execute_process(),
tracking all inputs and outputs within the build system itself. Use
this information to make sure that each individual output depends on
just its exact input file, ensuring updated inputs produce updated
outputs without having to call extract_content.py again.
- Ensure that the "content" build system target depends on all the
outputs, transitively triggering a rebuild any time an input
file (e.g. .rst documentation file or included image/source file)
changes.
Signed-off-by: Marti Bolivar <marti@foundries.io>
Separate the phases of creating the list of outputs (and output
directories) from the actual creation of these outputs. This lays the
groundwork for just printing the inputs and outputs in a subsequent
patch, enabling CMake to track dependencies properly.
In the now-separate output generation phase, use shutil.copy2() to
preserve metadata like access and modification times when possible,
rather than re-reading every file and seeing if the contents are
different. This has the same semantics but is clearer and potentially
more efficient, as the initial copies have more accurate relative
timestamps.
Signed-off-by: Marti Bolivar <marti@foundries.io>
Minor cleanups:
- Allow the user to run extract.content.py -h with ZEPHYR_BASE unset
without crashing
- Replace the use of the ZEPHYR_BUILD environment variable with an
--ignore argument (that can be given multiple times) which specifies
a path to a directory whose files should *not* be copied, and use
it to ignore the build directory
- rename, reorder, and tweak get_files() arguments a bit to go from
source to dest, with extra configuration at the end, getting rid of
local shadowing of global built-in functions
Signed-off-by: Marti Bolivar <marti@foundries.io>
Update menuconfig (and Kconfiglib, just to sync) to upstream revision
256e5b3e38e92 to get the fix below in, for an issue in an external
project. Also update genrest.py with a similar fix (the genrest issue
was what originally prompted it).
menuconfig: Improve/fix promptless choice handling
The code assumed that all parent (interface) menus always have a
prompt, which is false for items in promptless choices. This led to
a crash e.g. when viewing the symbol information for a symbol within
a promptless choice.
Promptless choices with children can show up "legitimately" when
people define choices in multiple locations to add symbols, though
this is broken in the C tools.
Use standard_sc_expr_str(node.item) instead of the non-existing
prompt for promptless choices. That way they show up as
'<choice (name if any>)>', which is consistent with how they're
shown elsewhere.
This commit also changes how choice names are displayed in
show-name/show-all mode, to the standard_sc_expr_str() format.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
This adds new targets to generate build documentation through
LaTEX to PDF.
There are a few notes:
1. pdflatex complains about the tex file generated by doxygen
so it needs to be fixed with a Python script before feeding
in through pdflatex.
2. SVG files are not recognized by pdflatex so they are converted
to known good format on the fly, only for producing PDF. This
uses the libRSVG's rsvg-convert tool.
Relates to #6782.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Update Kconfiglib and menuconfig to upstream revision a28bc4da9762e,
which adds include path information to menuconfig, showing how the
Kconfig file of the symbol got 'source'd in addition to showing the
definition location.
Add include path information to the auto-generated Kconfig documentation
too.
Some small Kconfiglib bugs are fixed to, like error reporting for
recursive 'source's being broken (crashing instead of printing the
error), a minor file descriptor leak, and syntax checking not catching
extra trailing tokens after 'if <expr>' and 'depends on <expr>'.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Previously, you had to go to the Kconfig dump at the bottom of the
symbol reference page to see selected symbols (as opposed to the
select*ing* symbols, which were already listed). Might be easy to miss.
Use a format similar to the list of defaults
('- SELECTED_SYM if CONDITION').
Also list 'imply's, though I don't think those are used anywhere in
Zephyr yet.
Simplify the code generating the list of selecting symbols a bit as
well.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Use the new Kconfiglib expression printing customization functionality
to get rid of the hacky expr_str() overriding.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
To avoid having to refer to non-rst files artificially in order to get
them copied into the destination folder, make a raw full copy of the
doc/ folder instead of copying only the .rst files.
Fixes#9128
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
The following changes have been made to support out-of-tree builds:
* In order to avoid using relative hardcoded paths, use CMake's
configure_file() to replace the paths in the doxygen input file
so that the output directory is set correctly.
* All .rst and additional required files are now copied from the doc/
folder into the build/rst folder using extract_content.py. The
samples/ and boards/ folder are copied twice (once into build/rst
and another into build/doc/rst) to manage relative paths.
* All paths are absolute where possible, including themes and static
content.
This patch ensures that the Zephyr repo is not contaminated by the
build at all.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
The filter-doc-log.sh script is no longer used since its functionality
has been moved to the CMakeLists.txt and filter-known-issues.py.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
It's common for people to put in a 'default n' "just in case", because
it might not be obvious that bool symbols implicitly default to 'n'. To
make it clearer, mention the implicit default value on the reference
pages of symbols without defaults.
Also mention that choices without defaults default to the first
(visible) symbol in the choice.
Note: Adding to the confusion, Kconfig used to generate a
'# CONFIG_FOO is not set' line for 'default n', but no output when there
was no default. I changed that recently in both the C tools and
Kconfiglib. There's no output either case now (unless the symbol is
visible).
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
The genrest.py script (used to create the Kconfig configuration option
documentation) was depending on the environment's locale settings to
properly handle UTF-8 character encoding in the reST files it creates.
When we reused this script for another project, that environment's
LC_CTYPE was set to C, telling Python to use ASCII as the (default)
encoding. This patch is a prophylactic measure to prevent potential
failures by adding the encoding option on the open() calls.
It also includes a small tweak to the introduction.
Signed-off-by: David B. Kinder <david.b.kinder@intel.com>
Improve error reporting to include the filename being processed when a
problem occurs during a pre-generation phase when boards and samples
files are temporarily copied (by doc/scripts/extract_content.py) into
the documentation area for processing.
Two recent problems were noticed:
Some new files were using window-1252 encoding and included windows
printer quote marks and hyphens, for example 0x92 in window-1252
encoding is Unicode 0x2019 for 'RIGHT SINGLE QUOTATION MARK'.
An image file reference by a reST file was missing
Both of these threw an exception reporting the error, but did not
include any information about the file currently being processed, making
it hard to fix the problem (e.g., change the Windows right quote
character to an ASCII ').
Signed-off-by: David B. Kinder <david.b.kinder@intel.com>
Until now, choices have kinda been a black box in the Kconfig reference,
and hid the dependencies of choice symbols (because choice symbols
depend directly on the choice rather than on whatever the choice
'depends on').
Generate separate information pages for choices and turn <choice>
dependencies into links.
One complication is that choices (usually) don't have names. Use the
index of each choice in the Kconfig files (first choice seen = choice 0,
then choice 1, etc.) instead to identify each choice.
Choice reference pages include the same information as symbol reference
pages (minus some things that don't apply for choices), and also list
the choice symbols contained in the choice.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Sphinx defaults to Python highlighting, but a Kconfig lexer is available
as well
(http://pygments.org/docs/lexers/#pygments.lexers.configs.KconfigLexer).
Use it.
This only highlights Kconfig definitions without links (references to
other symbols), as Sphinx doesn't highlight '.. parsed-literal::' blocks
with links. It's an improvement over Python stuff getting highlighted at
least.
Side note: '.. highlight:: none' still gives '.. parsed-literal::'
blocks without links a different background color, for whatever reason.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Make all menu paths ("(top menu) -> menu -> submenu -> ...") exclude
implicit submenus, which are shown indented in the menuconfig interface
and are created when items depend on the symbol before them.
Previously, implicit submenus were excluded in the menu path at the top
of the menuconfig interface, but were included in the menuconfig symbol
information dialog and in the docs generated by genrest.py.
This makes it consistent, and un-spams the documentation for some
symbols a bit.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Leading/trailing whitespace in prompts requires ugly workarounds in
genrest.py, as e.g. *prompt * is invalid RST. strip() all prompts in
Kconfiglib and get rid of the genrest.py workarounds. Add a warning too.
The Kconfiglib update has some unrelated cleanups and fixes (that won't
affect Zephyr).
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Skip writing index and symbol RST files whose content hasn't changed, to
avoid updating their timestamps. This makes documentation rebuilds much
faster, as Sphinx looks at the timestamp to determine if an RST file has
been updated.
Rebuilding docs with symbol reference up-to-date, before:
$ time make html
real 4m52.838s
user 4m46.242s
sys 0m4.249s
After:
$ time make html
real 0m48.731s
user 0m47.571s
sys 0m0.908s
Testing was done with 'make VERBOSE=1 SPHINXOPTS= html', suggested by
Marti Bolivar.
Piggyback a small cleanup to the code generating the select/imply
information.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Previously, symbols defined in multiple locations (e.g.,
Kconfig.defconfig symbols) would incorrectly get all their defaults,
selects, implies, and ranges displayed on the first definition.
This commit uses the Kconfiglib functionality added in 53af8db920066
("kconfiglib: Record which MenuNode has each property") to show such
symbols with all properties in the right location.
The definitions are now listed one-by-one, with the filename/linenr
displayed before each definition. The menu path (e.g.,
'(top menu) -> Device Drivers') is now listed for each definition as
well.
Fixes: #6879
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
The logic for walking reverse dependencies (from 'select's and 'imply's)
is a bit tricky to read (and write), and was essentially duplicated in a
few different spots (including in the upcoming menuconfig
implementation).
Use a new split_expr() helper function that's been added to Kconfiglib
to reduce code duplication and make the code more readable.
This also has another nice side effect: The list of symbols that select
a particular symbol now has the symbols in the order that they appear in
the Kconfig files.
Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
The new version of the reference page for a symbol lists the following
essential and generally useful information at the top of the page:
- Prompt
- Type
- Help text
- Direct dependencies (from 'depends on', and inherited from if's and
menus)
- Defaults
- Other symbols selecting or implying the symbol
The current symbol value and visibility are no longer displayed. They
don't make much sense in a reference document, as they depend on user
values from configuration files.
The bottom of the reference page also includes the symbol represented in
Kconfig format (including propagated dependencies).
All references to defined symbols appearing in expressions are turned
into links, which can be clicked on to jump to the reference page for
that symbol.
The following (slightly outdated) screenshot gives an idea of the
format:
https://www.dropbox.com/s/a34tlk2ncyus8po/promptandtype.png?dl=0
This commit also removes Kconfiglib 1, since it is no longer used.
Fixes#5622Fixes#6821
Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
Running 'make html' in doc/ when doc.log is missing or empty gives the
following error if sphinx-build doesn't write anything on stdout/stderr:
Error in ./scripts/filter-doc-log.sh: logfile "doc.log" not found.
Makefile:84: recipe for target ”html” failed
make: *** [html] Error 1
The problem is that scripts/filter-doc-log.sh tests for the existence of
the log file with [ -s ${LOG_FILE} ], which requires it to be nonempty.
Fix it by using -e instead, which only checks if the log file exists.
scripts/filter-known-issues.py ($KI_SCRIPT) seems to be able to deal
with empty files (and runs quickly).
Fixes#6854
Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
Decode Kconfig sources as UTF-8 instead of decoding them according to
the system locale (which might be ascii-only).
Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
Checking for TERM being undefined before doing tput with colors is not
enough; if TERM is defined as 'dumb', this thing also behaves dumbly.
In fact, the whole trying to do colors thing is dumb and causes all
kinds of headache in corner cases, so just wrap anything smelling like
color in a check for TERM being undefined or 'dumb' and be done with
it.
It shall take care of different automation mechanisms that don't
invoke 'make htmldocs' from a user console.
Signed-off-by: Inaky Perez-Gonzalez <inaky.perez-gonzalez@intel.com>
When running on jenkins and other automation environment, TERM will
not be defined and thus tput errors out.
Signed-off-by: Inaky Perez-Gonzalez <inaky.perez-gonzalez@intel.com>
The genrest.py script used to create configuration options documentation
from the KConfig files generates slightly incorrect OPTION directives
(with an extra colon at the end). Sphinx 1.5 didn't care, but Sphinx
1.6 does, so fix this now in preparation for upgrading Sphinx to the
current version.
Signed-off-by: David B. Kinder <david.b.kinder@intel.com>
Because of known issues with Sphinx/Breathe tools we're using to
generate doxygen-based comments for our API documentation, we're getting
a bevy of warning messages written out. As a workaround for our CI
system, we created a filter-known-issues.py script to remove "expected"
warnings from the output.
This patch moves calling that filter script into the doc generation
Makefile so folks making local builds of the docs won't be tripped up by
all the warning messages either. Output of the "make htmldocs" command
is now filtered so only "unexpected" errors and warnings will be shown.
(See https://github.com/sphinx-doc/sphinx/issue/2682 and
sphinx-doc/sphinx#2683i for the Sphinx/Breathe issues.)
Fixes#1527
Signed-off-by: David B. Kinder <david.b.kinder@intel.com>
The script used to generate Kconfig documentation (genrest.py)
was creating .rst files without a final newline.
Signed-off-by: David B. Kinder <david.b.kinder@intel.com>
Instead of a configuration options index in "discovered" order
during the walk of the Kconfig files, create the index in alphabetic
order.
Also added a more descriptive name above the displayed table and added
table headings.
jira: ZEP-2310
Signed-off-by: David B. Kinder <david.b.kinder@intel.com>
Depending on the version of Python 3 the user has installed, 'open'
may default to using 'ascii' encoding. Since the documentation is in
UTF-8, and contains characters beyond 'ascii', this will result in
failures when using these versions of Python.
Fix this by being explicit about the encoding to be used when reading
these files.
Signed-off-by: David Brown <david.brown@linaro.org>
Change-Id: I54c69334c6bf377f1135cec04f4e0ea96a8e9a5b
Replace the existing Apache 2.0 boilerplate header with an SPDX tag
throughout the zephyr code tree. This patch was generated via a
script run over the master branch.
Also updated doc/porting/application.rst that had a dependency on
line numbers in a literal include.
Manually updated subsys/logging/sys_log.c that had a malformed
header in the original file. Also cleanup several cases that already
had a SPDX tag and we either got a duplicate or missed updating.
Jira: ZEP-1457
Change-Id: I6131a1d4ee0e58f5b938300c2d2fc77d2e69572c
Signed-off-by: David B. Kinder <david.b.kinder@intel.com>
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Running documentation scripts on the top directory shifted all links one
level dowwn and is breaking all incoming links.
Use a script to copy all RST files into the doc/ directory before
running sphinx and keep structure intact.
Jira: ZEP-1579
Change-Id: Iccff068430e2ddb29e172cd8ae920475815d199e
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Use the ReST metadata document title directive instead of
embedded javascript to assign a title to the configuration
options (Kconfig) reference guide pages. This will generate
a static <title> directive in the generated HTML instead
of relying on javascript.
Change-Id: Ib70a8b1f641a5ed72be774f0f5b2a93a2d1c9b8c
Signed-off-by: David B. Kinder <david.b.kinder@intel.com>
Add a document.title='xxx' via on-page javascript for the
.rst files generated by the genrest.py script
Jira: ZEP-459
Change-Id: I2b3dbb97758cfa232006a0cd98c4bd8394d8183e
Signed-off-by: David Kinder <david.b.kinder@intel.com>
This allows us to cross ref from code with :option:`CONFIG_XYZ`
generating a useful link and avoiding an 'undefined target` warning.
Hyperlink using :option: instead of :ref:.
Change-Id: I6cc0daec012dfcca504faa47d591885e69c8e521
Signed-off-by: Inaky Perez-Gonzalez <inaky.perez-gonzalez@intel.com>
Do not use a TOC, instead put all options in a table and use references.
Jira: ZEP-149
Change-Id: I23821759c64ce28241ee8260ad7cba235df72d86
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Carles Cufi
