Fix the dependencies of `CONFIG_EXCEPTION_STACK_TRACE`:
- Architecture-specific Kconfig, i.e.
`X86_EXCEPTION_STACK_TRACE`, will be enabled automatically
when all the dependencies are met.
- `EXCEPTION_STACK_TRACE` depends on architecture-specific
Kconfig to be enabled.
- The stack trace implementations should be compiled only if
user enables `CONFIG_EXCEPTION_STACK_TRACE`.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Currently, the stack trace in ARM64 implementation depends on
frame pointer Kconfigs combo to be enabled. Create a dedicated
Kconfig for that instead, so that it is consistent with x86 and
riscv, and update the source accordingly.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
In some cases, the `fp` will never be `NULL` and the stack
unwinding can go on and on forever, limit the max depth so that
this will not happen.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Namespaced the generated headers with `zephyr` to prevent
potential conflict with other headers.
Introduce a temporary Kconfig `LEGACY_GENERATED_INCLUDE_PATH`
that is enabled by default. This allows the developers to
continue the use of the old include paths for the time being
until it is deprecated and eventually removed. The Kconfig will
generate a build-time warning message, similar to the
`CONFIG_TIMER_RANDOM_GENERATOR`.
Updated the includes path of in-tree sources accordingly.
Most of the changes here are scripted, check the PR for more
info.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Selecting `CONFIG_SYMTAB` will
enable the symtab generation which will be used in the
stack trace to print the function name of the return
address.
Added `arm64` to the `arch.common.stack_unwind.symtab` test.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Change the GCC toolchain configuration to make use of the Cortex-R82
target. When Cortex-R82 was added as a GCC toolchain option, the GCC
version of the Zephyr SDK did not support Cortex-R82 tuning. Zephyr was
therefore compiled compiled for the Armv8.4-A architecture. Since Zephyr
SDK 0.15.0 (which updated GCC from 10.3.0 to 12.1.0) coupled with Zephyr
3.2, the Cortex-R82 target is supported.
The Armv8-R AArch64 architecture does not support the EL3 exception level.
EL3 support is therefore made conditional on Armv8-R vs Armv8-A.
Signed-off-by: Debbie Martin <Debbie.Martin@arm.com>
Simple rename to align the kernel naming scheme. This is being
used throughout the tree, especially in the architecture code.
As this is not a private API internal to kernel, prefix it
appropriately with K_.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
arch_interface.h is for architecture and should not be
under sys/. So move it under include/zephyr/arch/.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
There is a #endif comment which was incorrectly marked with
CONFIG_HW_STACK_PROTECTION instead of
CONFIG_ARM64_STACK_PROTECTION, which is used at #if.
So update it.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This fix removes the zephyr/ prefix from linker included files.
With this prefix the build works only for Ninja and not for
other build tools.
Linking in Zephyr / CMake:
- Ninja invokes linking directly from <build>.
- Make invokes linking form <build>/zephyr.
The linker default uses cwd for looking up INCLUDE directives if not found
in list of includes.
Zephyr always adds <build>/zephyr as link include using CMake,
and this is passed to ld as -L<build>/zephyr therefore using
INCLUDE isr_tables_swi.ld ensures it will be correctly found in all cases.
Signed-off-by: Radosław Koppel <radoslaw.koppel@nordicsemi.no>
This commit removes the need of swi_tables.ld file if the
ISR table generator is not configured to use it.
Signed-off-by: Radosław Koppel <radoslaw.koppel@nordicsemi.no>
This commit updates the arm and arm64 architecture files
to support the new ISR handlers creation parser.
Signed-off-by: Radosław Koppel <radoslaw.koppel@nordicsemi.no>
The arm64_cpu_boot_params will be read on other cores
call sys_cache_data_flush_range flush the data from the cache to RAM.
This ensures that other cores can access the correct data.
Signed-off-by: honglin leng <a909204013@gmail.com>
The interface to flush fpu is not unique to one architecture, make it a
generic, optional interface that can be implemented (and overriden) by a
platform.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Different architecture are doing this in custom ways and using different
naming conventions, unify this interface and make it part of the arch
implementation for SMP.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
The Cortex ARM documentation states that the DC IVAC instruction
requires write access permission to the virtual address (VA);
otherwise, it may generate a permission fault.
Therefore, it is needed to avoid invalidating read-only memory
after the memory map operation.
This issue has been produced by commit c9b534c.
This commit resolves the issue #64758.
Signed-off-by: Mykola Kvach <mykola_kvach@epam.com>
HCR_EL2 is configured to certain value by some
loaders such as Uboot on some arm64 boards(such as roc_rk3568_pc),
When HCR_EL2.TGE, HCR_EL2.AMO and HCR_EL2.IMO bits are
set to 1, some unpredictable behaviors may occur during
zephyr boot. So we clear these bits to avoid it.
Signed-off-by: Charlie Xiong <1981639884@qq.com>
The exclusive load/store instructions don't work well when MMU and cache
are disabled on some cores e.g. Cortex-A72. Change it to voting lock[1]
to select the primary core when multi-cores boot simultaneously.
The voting lock has reasonable but minimal requirements on the memory
system.
[1] https://www.kernel.org/doc/html/next/arch/arm/vlocks.html
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Let's make this official: we use the suffix `_MASK` for the define
carrying the GENMASK for the attributes, and the suffix `_GET(x)` for
the actual macro extracting the attributes.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
To make the stack guard works well, clean and refine the MPU code. To
save the MPU regions (the number of MPU regions are limited), we choose
to remove the guard region. Comparing to add an individual region to
guard the stack, removing the guard region can at least save 2 regions
per core.
Similarly with userspace, the stack guard will leverage the dynamic
regions switching mechanism which means we need a region switch during
the context switch. Otherwise, the other option is using stack guard
region, but this is very limited since the number of MPU regions is
limited.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Refactor the stack relevant macros to prepare to introduce the stack
guard. Also add comments about the changes related to stack layout.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Add the stack check function z_arm64_stack_corruption_check at
z_arm64_fatal_error to handle the stack overflow triggered by the
hardware region.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Introduce the ARM64_STACK_PROTECTION config. This option leverages the
MMU or MPU to cause a system fatal error if the bounds of the current
process stack are overflowed. This is done by preceding all stack areas
with a fixed guard region. The config depends on MPU for now since MMU
stack protection is not ready.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Clean the thread->arch during the arch_new_thread to avoid unexpected
behavior. If the thread struct is allocated from heap or in stack, the
data in thread->arch might be dirty.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Accessing mem before mmu or mpu init will cause a cache coherence issue.
To avoid such a problem, move the safe exception stack init function
after the mmu or mpu is initiated.
Also change the data section attribute from INNER_SHAREABLE to
OUTER_SHAREABLE. Otherwise there will be a cache coherence issue during
the memory regions switch. Because we are using background region to do
the regions switch, and the default background region is
OUTER_SHAREABLE, if we use INNER_SHAREABLE as the foreground region,
then we have to flush all cache regions to make sure the cached values
are right. However, flushing all regions is too heavy, so we set
OUTER_SHAREABLE to fix this issue.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Dom0less is Xen mode without privileged domain. All guests are created
according to hypervisor device tree configuration on boot. Thus, there
is no Dom0 with console daemon, that usually manages console output
from domains.
Zephyr OS contains 2 serial drivers related to Xen hypervisor: regular
with console shared page and consoleio-based. The first one is for
setups with console daemon and usually was used for Zephyr DomU guests.
The second one previously was used only for Zephyr Dom0 and had
corresponding Kconfig options. But consoleio is also used as interface
for DomU output on Dom0less setups and should be configurable without
XEN_DOM0 option.
Add corresponding XEN_DOM0LESS config to Xen Kconfig files and proper
dependencies in serial drivers.
Signed-off-by: Dmytro Firsov <dmytro_firsov@epam.com>
Co-authored-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
This is the final step in making the `zephyr,memory-attr` property
actually useful.
The problem with the current implementation is that `zephyr,memory-attr`
is an enum type, this is making very difficult to use that to actually
describe the memory capabilities. The solution proposed in this PR is to
use the `zephyr,memory-attr` property as an OR-ed bitmask of memory
attributes.
With the change proposed in this PR it is possible in the DeviceTree to
mark the memory regions with a bitmask of attributes by using the
`zephyr,memory-attr` property. This property and the related memory
region can then be retrieved at run-time by leveraging a provided helper
library or the usual DT helpers.
The set of general attributes that can be specified in the property are
defined and explained in
`include/zephyr/dt-bindings/memory-attr/memory-attr.h` (the list can be
extended when needed).
For example, to mark a memory region in the DeviceTree as volatile,
non-cacheable, out-of-order:
mem: memory@10000000 {
compatible = "mmio-sram";
reg = <0x10000000 0x1000>;
zephyr,memory-attr = <( DT_MEM_VOLATILE |
DT_MEM_NON_CACHEABLE |
DT_MEM_OOO )>;
};
The `zephyr,memory-attr` property can also be used to set
architecture-specific custom attributes that can be interpreted at run
time. This is leveraged, among other things, to create MPU regions out
of DeviceTree defined memory regions on ARM, for example:
mem: memory@10000000 {
compatible = "mmio-sram";
reg = <0x10000000 0x1000>;
zephyr,memory-region = "NOCACHE_REGION";
zephyr,memory-attr = <( DT_ARM_MPU(ATTR_MPU_RAM_NOCACHE) )>;
};
See `include/zephyr/dt-bindings/memory-attr/memory-attr-mpu.h` to see
how an architecture can define its own special memory attributes (in
this case ARM MPU).
The property can also be used to set custom software-specific
attributes. For example we can think of marking a memory region as
available to be used for memory allocation (not yet implemented):
mem: memory@10000000 {
compatible = "mmio-sram";
reg = <0x10000000 0x1000>;
zephyr,memory-attr = <( DT_MEM_NON_CACHEABLE |
DT_MEM_SW_ALLOCATABLE )>;
};
Or maybe we can leverage the property to specify some alignment
requirements for the region:
mem: memory@10000000 {
compatible = "mmio-sram";
reg = <0x10000000 0x1000>;
zephyr,memory-attr = <( DT_MEM_CACHEABLE |
DT_MEM_SW_ALIGN(32) )>;
};
The conventional and recommended way to deal and manage with memory
regions marked with attributes is by using the provided `mem-attr`
helper library by enabling `CONFIG_MEM_ATTR` (or by using the usual DT
helpers).
When this option is enabled the list of memory regions and their
attributes are compiled in a user-accessible array and a set of
functions is made available that can be used to query, probe and act on
regions and attributes, see `include/zephyr/mem_mgmt/mem_attr.h`
Note that the `zephyr,memory-attr` property is only a descriptive
property of the capabilities of the associated memory region, but it
does not result in any actual setting for the memory to be set. The
user, code or subsystem willing to use this information to do some work
(for example creating an MPU region out of the property) must use either
the provided `mem-attr` library or the usual DeviceTree helpers to
perform the required work / setting.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Add Xen domctl API implementation for Zephyr as control domain.
Previously Zephyr OS was used as unprivileged Xen domain (Domain-U),
but it also can be used as lightweight Xen control domain (Domain-0).
To implement such fuctionality additional Xen interfaces are needed.
One of them is Xen domain controls (domctls) - it allows to create,
configure and manage Xen domains.
Also, used it as a possibility to update files copyright and licenses
identifiers in touched files.
Signed-off-by: Dmytro Firsov <dmytro_firsov@epam.com>
Xen-related Kconfig options were highly dependand on BOARD/SOC xenvm.
It is not correct because Xen support may be used on any board and SoC.
So, Kconfig structure was refactored, now CONFIG_XEN is located in
arch/ directory (same as in Linux kernel) and can be selected for
any Cortex-A arm64 setup (no other platforms are currently supported).
Also remove confusion in Domain 0 naming: Domain-0, initial domain,
Dom0, privileged domain etc. Now all options related to Xen Domain 0
will be controlled by CONFIG_XEN_DOM0.
Signed-off-by: Dmytro Firsov <dmytro_firsov@epam.com>
This header is private and included only in architecture code, no need for
it to be in the top of the public include directory.
Note: This might move to a more private location later. For now just
cleaning up the obvious issues.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
During Zephyr boot with SMP enabled,
while z_smp_init is not completed yet and only boot core is running,
incorrect code in broadcast_ipi API will cause following:
1. Generate IPI even if other cores are not booted.
2. Incorrect setting of sgi1r register.
3. All the affinity(1/2/3) value will be incorrect.
Signed-off-by: Chirag Kochar <chirag.kochar@intel.com>
Enhanced arch_start_cpu so if a core is not available based on pm_cpu_on
return value, booting does not halt. Instead the next core in
cpu_node_list will be tried. If the number of CPU nodes described in the
device tree is greater than CONFIG_MP_MAX_NUM_CPUS then the extra cores
will be reserved and used if any previous cores in the cpu_node_list fail
to power on. If the number of cores described in the device tree matches
CONFIG_MP_MAX_NUM_CPUS then no cores are in reserve and booting will
behave as previous, it will halt.
Signed-off-by: Chad Karaginides <quic_chadk@quicinc.com>
In some shared-memory use cases between Zephyr and other parallel
running OS, for data coherent, the non-cacheable normal memory
mapping is needed.
Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
Move the function prototype before declaration of the function itself.
Maybe the prototype could be removed altogether?
Signed-off-by: Florian La Roche <Florian.LaRoche@gmail.com>
The current mechanism of the MPU region switching configures and
reprograms the regions (including inserting, splitting the dynamic
region, and flushing the regions to the registers) every time during the
context switch. This, not only causes a large usage of the kernel stack
but also a lower performance.
To improve it, move the configuration operations ahead to make sure the
context swtich only flushes the current thread regions to the registers
and does not configure the regions anymore. To achieve this, configure
the regions during any operations related to partitions (partition
add/remove, and domain add/remove thread), flush the sys_dyn_regions if
the current thread is the privileged thread, and flush the thread's own
regions if it's a user thread.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Using BR(background region) during the flushing regions instead of
enabling/disabling the MPU which is a heavy operation.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Architecturally, Set/Way operations are not guaranteed to affect all
caches prior to the PoC, and may require other IMPLEMENTATION DEFINED
maintenance (e.g. MMIO control of system-level caches).
First of all this patch was designed for Xen domain Zephyr build, set/way
ops are not easily virtualized by Xen. S/W emulation is disabled, because
IP-MMU is active for Dom0. IP-MMU is a IO-MMU made by Renesas, as any good
IO-MMU, it shares page-tables with CPU. Trying to emulate S/W with IP-MMU
active will lead to IO-MMU faults. So if we build Zephyr as a Xen Initial
domain, it won't work with cache management support enabled.
Exposing set/way cache maintenance to a virtual machine is unsafe, not
least because the instructions are not permission-checked, but also
because they are not broadcast between CPUs.
In this commit, VA data invalidate invoked after every mapping instead of
using set/way instructions on init MMU. So, it was easy to delete
sys_cache_data_invd_all from enable MMU function, becase every adding of
a new memory region to xclat tabes will cause invalidating of this memory
and in this way we sure that there are not any stale data inside.
Signed-off-by: Mykola Kvach <mykola_kvach@epam.com>
Yong Cong Sin