zephyr/doc/guides/build/index.rst

.. _build_overview:

Build and Configuration Systems
###############################


.. _cmake-details:

Build System (CMake)
********************


CMake is used to build your application together with the Zephyr kernel. A
CMake build is done in two stages. The first stage is called
**configuration**. During configuration, the CMakeLists.txt build scripts are
executed. After configuration is finished, CMake has an internal model of the
Zephyr build, and can generate build scripts that are native to the host
platform.

CMake supports generating scripts for several build systems, but only Ninja and
Make are tested and supported by Zephyr. After configuration, you begin the
**build** stage by executing the generated build scripts. These build scripts
can recompile the application without involving CMake following
most code changes. However, after certain changes, the configuration step must
be executed again before building. The build scripts can detect some of these
situations and reconfigure automatically, but there are cases when this must be
done manually.

Zephyr uses CMake's concept of a 'target' to organize the build. A
target can be an executable, a library, or a generated file. For
application developers, the library target is the most important to
understand. All source code that goes into a Zephyr build does so by
being included in a library target, even application code.

Library targets have source code, that is added through CMakeLists.txt
build scripts like this:

.. code-block:: cmake

   target_sources(app PRIVATE src/main.c)

In the above :file:`CMakeLists.txt`, an existing library target named ``app``
is configured to include the source file :file:`src/main.c`. The ``PRIVATE``
keyword indicates that we are modifying the internals of how the library is
being built. Using the keyword ``PUBLIC`` would modify how other
libraries that link with app are built. In this case, using ``PUBLIC``
would cause libraries that link with ``app`` to also include the
source file :file:`src/main.c`, behavior that we surely do not want. The
``PUBLIC`` keyword could however be useful when modifying the include
paths of a target library.


Build and Configuration Phases
==============================

The Zephyr build process can be divided into two main phases: a configuration
phase (driven by CMake) and a build phase (driven by Make or Ninja).

.. _build_configuration_phase:

Configuration Phase
-------------------

The configuration phase begins when the user invokes *CMake*,
specifying a source application directory and a board target.

.. figure:: build-config-phase.svg
    :align: center
    :alt: Zephyr's build configuration phase
    :figclass: align-center
    :width: 80%

CMake begins by processing the :file:`CMakeLists.txt` file in the application
directory, which refers to the :file:`CMakeLists.txt` file in the Zephyr
top-level directory, which in turn refers to :file:`CMakeLists.txt` files
throughout the build tree (directly and indirectly). Its primary output is a
set of Makefiles or Ninja files to drive the build process, but the CMake
scripts also do some processing of their own:

Devicetree
   :file:`*.dts` (*devicetree source*) and :file:`*.dtsi` (*devicetree source
   include*) files are collected from the target's architecture, SoC, board,
   and application directories.

   :file:`*.dtsi` files are included by :file:`*.dts` files via the C
   preprocessor (often abbreviated *cpp*, which should not be confused with
   C++). The C preprocessor is also used to merge in any devicetree
   :file:`*.overlay` files, and to expand macros in :file:`*.dts`,
   :file:`*.dtsi`, and :file:`*.overlay` files.

   The preprocessed devicetree sources (stored in :file:`*.dts.pre.tmp`) are
   parsed by :zephyr_file:`gen_defines.py <scripts/dts/gen_defines.py>` to
   generate a :file:`devicetree_unfixed.h` header with preprocessor macros.

   As a debugging aid, :file:`gen_defines.py` writes the final devicetree to
   :file:`zephyr.dts`. This file is just for reference. It is not used
   anywhere.

   The ``dtc`` devicetree compiler also gets run on the preprocessed devicetree
   sources to catch any extra warnings and errors generated by it. The output
   from ``dtc`` is unused otherwise.

   The above is just a brief overview. For more information on devicetree, see
   :ref:`dt-guide`.

Devicetree fixups
   Files named :file:`dts_fixup.h` from the target’s architecture, SoC, board,
   and application directories are concatenated into a single
   :file:`devicetree_fixups.h` file. :file:`dts_fixup.h` files are used to
   rename generated macros to names expected by the source code.

   Source code accesses preprocessor macros generated from devicetree by
   including the :zephyr_file:`devicetree.h <include/devicetree.h>` header,
   which includes :file:`devicetree_unfixed.h` and :file:`devicetree_fixups.h`.

Kconfig
   :file:`Kconfig` files define available configuration options for for the
   target architecture, SoC, board, and application, as well as dependencies
   between options.

   Kconfig configurations are stored in *configuration files*. The initial
   configuration is generated by merging configuration fragments from the board
   and application (e.g. :file:`prj.conf`).

   The output from Kconfig is an :file:`autoconf.h` header with preprocessor
   assignments, and a :file:`.config` file that acts both as a saved
   configuration and as configuration output (used by CMake).

   Information from devicetree is available to Kconfig, through the functions
   defined in :zephyr_file:`kconfigfunctions.py
   <scripts/kconfig/kconfigfunctions.py>`.

   See :ref:`the Kconfig section of the manual <kconfig>` for more information.

Build Phase
-----------

The build phase begins when the user invokes ``make`` or ``ninja``. Its
ultimate output is a complete Zephyr application in a format suitable for
loading/flashing on the desired target board (:file:`zephyr.elf`,
:file:`zephyr.hex`, etc.) The build phase can be broken down, conceptually,
into four stages: the pre-build, first-pass binary, final binary, and
post-processing.

Pre-build
+++++++++

Pre-build occurs before any source files are compiled, because during
this phase header files used by the source files are generated.

Offset generation
   Access to high-level data structures and members is sometimes
   required when the definitions of those structures is not
   immediately accessible (e.g., assembly language). The generation of
   *offsets.h* (by *gen_offset_header.py*) facilitates this.

System call boilerplate
   The *gen_syscall.py* and *parse_syscalls.py*  scripts work
   together to bind potential system call functions with their
   implementations.

.. figure:: build-build-phase-1.svg
    :align: center
    :alt: Zephyr's build stage I
    :figclass: align-center
    :width: 80%

First-pass binary
+++++++++++++++++

Compilation proper begins with the first-pass binary. Source files (C
and assembly) are collected from various subsystems (which ones is
decided during the configuration phase), and compiled into archives
(with reference to header files in the tree, as well as those
generated during the configuration phase and the pre-build stage).

If memory protection is enabled, then:

Partition grouping
   The *gen_app_partitions.py* script scans all the
   generated archives and outputs linker scripts to ensure that
   application partitions are properly grouped and aligned for the
   target’s memory protection hardware.

Then *cpp* is used to combine linker script fragments from the target’s
architecture/SoC, the kernel tree, optionally the partition output if
memory protection is enabled, and any other fragments selected during
the configuration process, into a *linker.cmd* file. The compiled
archives are then linked with *ld* as specified in the
*linker.cmd*.

In some configurations, this is the final binary, and the next stage
is skipped.

.. figure:: build-build-phase-2.svg
    :align: center
    :alt: Zephyr's build stage II
    :figclass: align-center
    :width: 80%

Final binary
++++++++++++

The binary from the previous stage is incomplete, with empty and/or
placeholder sections that must be filled in by, essentially, reflection.

Device dependencies
   The *gen_handles.py* script scans the first-pass binary to determine
   relationships between devices that were recorded from devicetree data,
   and replaces the encoded relationships with values that are optimized to
   locate the devices actually present in the application.

When :ref:`usermode_api` is enabled:

Kernel object hashing
   The *gen_kobject_list.py* scans the *ELF DWARF*
   debug data to find the address of the all kernel objects. This
   list is passed to *gperf*, which generates a perfect hash function and
   table of those addresses, then that output is optimized by
   *process_gperf.py*, using known properties of our special case.

Then, the link from the previous stage is repeated, this time with the
missing pieces populated.

.. figure:: build-build-phase-3.svg
    :align: center
    :alt: Zephyr's build stage III
    :figclass: align-center
    :width: 80%


Post processing
+++++++++++++++

Finally, if necessary, the completed kernel is converted from *ELF* to
the format expected by the loader and/or flash tool required by the
target. This is accomplished in a straightforward manner with *objdump*.

.. figure:: build-build-phase-4.svg
    :align: center
    :alt: Zephyr's build final stage
    :figclass: align-center
    :width: 80%


.. _build_system_scripts:

Supporting Scripts and Tools
============================

The following is a detailed description of the scripts used during the build process.

.. _gen_syscalls.py:

:zephyr_file:`scripts/gen_syscalls.py`
--------------------------------------

.. include:: ../../../scripts/gen_syscalls.py
   :start-after: """
   :end-before: """

.. _gen_handles.py:

:zephyr_file:`scripts/gen_handles.py`
--------------------------------------

.. include:: ../../../scripts/gen_handles.py
   :start-after: """
   :end-before: """

.. _gen_kobject_list.py:

:zephyr_file:`scripts/gen_kobject_list.py`
------------------------------------------

.. include:: ../../../scripts/gen_kobject_list.py
   :start-after: """
   :end-before: """

.. _gen_offset_header.py:

:zephyr_file:`scripts/gen_offset_header.py`
-------------------------------------------

.. include:: ../../../scripts/gen_offset_header.py
   :start-after: """
   :end-before: """

.. _parse_syscalls.py:

:zephyr_file:`scripts/parse_syscalls.py`
----------------------------------------


.. include:: ../../../scripts/parse_syscalls.py
   :start-after: """
   :end-before: """

.. _gen_idt.py:

:zephyr_file:`arch/x86/gen_idt.py`
----------------------------------

.. include:: ../../../arch/x86/gen_idt.py
   :start-after: """
   :end-before: """

.. _gen_gdt.py:

:zephyr_file:`arch/x86/gen_gdt.py`
----------------------------------

.. include:: ../../../arch/x86/gen_gdt.py
   :start-after: """
   :end-before: """

.. _gen_relocate_app.py:

:zephyr_file:`scripts/gen_relocate_app.py`
-------------------------------------------

.. include:: ../../../scripts/gen_relocate_app.py
   :start-after: """
   :end-before: """

.. _process_gperf.py:

:zephyr_file:`scripts/process_gperf.py`
---------------------------------------

.. include:: ../../../scripts/process_gperf.py
   :start-after: """
   :end-before: """

:zephyr_file:`scripts/gen_app_partitions.py`
--------------------------------------------

.. include:: ../../../scripts/gen_app_partitions.py
   :start-after: """
   :end-before: """

.. _kconfig:

Configuration System (Kconfig)
*******************************

The Zephyr kernel and subsystems can be configured at build time to adapt them
for specific application and platform needs. Configuration is handled through
Kconfig, which is the same configuration system used by the Linux kernel. The
goal is to support configuration without having to change any source code.

Configuration options (often called *symbols*) are defined in :file:`Kconfig`
files, which also specify dependencies between symbols that determine what
configurations are valid. Symbols can be grouped into menus and sub-menus to
keep the interactive configuration interfaces organized.

The output from Kconfig is a header file :file:`autoconf.h` with macros that
can be tested at build time. Code for unused features can be compiled out to
save space.

The following sections explain how to set Kconfig configuration options, go
into detail on how Kconfig is used within the Zephyr project, and have some
tips and best practices for writing :file:`Kconfig` files.

.. toctree::
   :maxdepth: 1

   kconfig/menuconfig.rst
   kconfig/setting.rst
   kconfig/tips.rst
   kconfig/preprocessor-functions.rst
   kconfig/extensions.rst