Change for loops of the form:
for (i = 0; i < CONFIG_MP_NUM_CPUS; i++)
...
to
unsigned int num_cpus = arch_num_cpus();
for (i = 0; i < num_cpus; i++)
...
We do the call outside of the for loop so that it only happens once,
rather than on every iteration.
Signed-off-by: Kumar Gala <kumar.gala@intel.com>
Change automated searching for files using "IRQ_CONNECT()" API not
including <zephyr/irq.h>.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
GCC may generate ldp/stp instructions with the Advanced SIMD Qn
registers for consecutive 32-byte loads and stores.
This commit disables this GCC behaviour because saving and restoring
the Advanced SIMD context is very expensive, and it is preferable to
keep it turned off by not emitting these instructions for better
context switching performance.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
On GICv3, when we send an IPI interrupt, aff3, aff2 and aff1 should
be assigned a value corespond to a PE for which interrupt will be
generated. target_list only corresponds to aff0.
On real hardware, aff3, aff2, aff1 and aff0 should be treated as a
whole to determine a PE.
Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
VMPIDR_EL2 is assigned the value returned by EL2 reads of MPIDR_EL1
MPIDR_EL1 is the register holding the Multiprocessor ID which is to
identify different cores. Because of the virtualization requirements
for AArch64, MPIDR_EL1 should be virtualized (the different virtualized
cores can run on the same physical core). Thus the value of MPIDR_EL1
should be switched when the VM is switched. Setting the VMPIDR_EL2 is
the way to change the value returned by EL1 reads of MPIDR_EL1. Even
without virtualization, we still need to set VMPIDR_EL2 during booting
at EL2 or EL3. Otherwise, all cores' IDs are zero at the EL1 stage
which will break the SMP system.
Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
When a cache API function is called from userspace, this results on
ARM64 in an OOPS (bad syscall error). This is due to at least two
different factors:
- the location of the cache handlers is preventing the linker to
actually find the handlers
- specifically for ARM64 and ARC some cache handling functions are not
implemented (when userspace is not used the compiler simply optimizes
out these calls)
Fix the problem by:
- moving the userspace cache handlers to a their logical and proper
location (in the drivers directory)
- adding the missing handlers for ARM64 and ARC
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
When CONFIG_INIT_STACKS is enabled all stacks should be filled with 0xaa
so that the thread analyzer can measure stack utilization, but the IRQ
stack was not filled and so `kernel stacks` on the shell would show that
the stack had been fully used and inferring an IRQ stack overflow
regardless of the IRQ stack size.
Fill the IRQ stack before it gets used so that we can have precise usage
reports.
Signed-off-by: Jamie Iles <quic_jiles@quicinc.com>
Signed-off-by: Dave Aldridge <quic_daldridg@quicinc.com>
Following zephyr's style guideline, all if statements, including single
line statements shall have braces.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Targets with text or data addresses above the 4GB boundary may need to use
the large code model to ensure relocations in the linker work correctly.
Signed-off-by: Keith Packard <keithp@keithp.com>
Enable single-threaded support for the arm64 archtecture.
This mode of execution is supported on an soc under
development and is validated regularly.
Signed-off-by: Eugene Cohen <quic_egmc@quicinc.com>
In performing a double check of Zephyr arm64 MMU config
against edk2, a different in the programming of the
Translation Control Register (TCR) was found. TCR.TG[1]
should be set to address Cortex-A57 erratum 822227:
"Using unsupported 16K translation granules might cause
Cortex-A57 to incorrectly trigger a domain fault"
Signed-off-by: Eugene Cohen <quic_egmc@quicinc.com>
On platforms where reset vector catch is not possible
it is useful to have a compile-time option to spin
at the reset vector allowing a debugger to be attached
and then to manually resume execution.
Define a config option for arm64 to spin at the
reset vectdor so a debugger can be attached.
Signed-off-by: Eugene Cohen <quic_egmc@quicinc.com>
This commit selectively disables the infinite recursion warning
(`-Winfinite-recursion`), which may be reported by GCC 12 and above,
for the `disable_table` function because no actual infinite recursion
will occur under normal circumstances.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
arch_mem_map() on ARM64 is currently not supporting the K_MEM_PERM_USER
parameter so we cannot allocate userspace accessible memory using the
memory helpers. Fix this.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
<soc.h> has been traditionally been used as a proxy to HAL headers,
register definitions, etc. Nowadays, <soc.h> is anarchy. It serves a
different purpose depending on the SoC. In some cases it includes HALs,
in some others it works as a header sink/proxy (for no good reason), as
a register definition when there's no HAL... To make things worse, it is
being included in code that is, in theory, non-SoC specific.
This patch is part of a series intended to improve the situation by
removing <soc.h> usage when not needed, and by eventually removing it.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
The init stack of the secondary core should use KERNEL_STACK_BUFFER + sz
Using Z_THREAD_STACK_BUFFER will calculate the wrong stack size.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
The current SMP boot code doesn't consider that the cores can boot at
the same time. Possibly, more than one core can boot into primary core
boot sequence. Fix it by using the atomic operation to make sure only
one core act as the primary core.
Correspondingly, sgi_raise_ipi should transfer CPU id to mpidr.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Fix writing of ICC_SRE_EL3 to or-in bits to align
with original intent to read-modify-write this
register.
Also disable FIQ and IRQ bypass so interrupt delivery
occurs through GIC. Platforms may choose to override
this behavior in z_arm64_el3_plat_init implementations.
Remove ICC_SRE_EL3 config from viper and qemu since
this is now handled in the arm64 arch core.
Signed-off-by: Eugene Cohen <quic_egmc@quicinc.com>
Assembler files were not migrated with the new <zephyr/...> prefix.
Note that the conversion has been scripted, refer to #45388 for more
details.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
In order to bring consistency in-tree, migrate all arch code to the new
prefix <zephyr/...>. Note that the conversion has been scripted, refer
to zephyrproject-rtos#45388 for more details.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
Commit d8f186aa4a ("arch: common: semihost: add semihosting
operations") encapsulated semihosting invocation in a per-arch
semihost_exec() function. There is a fixed register variable declaration
for the return value but this variable is not listed as an output
operand to respective inline assembly segments which is an error.
This is not reported as such by gcc and the generated code is still OK
in those particular instances but this is not guaranteed, and clang
does complain about such cases.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Add an API that utilizes the ARM semihosting mechanism to interact with
the host system when a device is being emulated or run under a debugger.
RISCV is implemented in terms of the ARM implementation, and therefore
the ARM definitions cross enough architectures to be defined 'common'.
Functionality is exposed as a separate API instead of syscall
implementations (`_lseek`, `_open`, etc) due to various quirks with
the ARM mechanisms that means function arguments are not standard.
For more information see:
https://developer.arm.com/documentation/dui0471/m/what-is-semihosting-
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
impl
In ARM parlance, the subroutine call return address is stored in the
"link register" or simply lr. Refer to it as lr which is clearer than
the anonymous x30 designation.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
ARM64 supports more memory mapping types for device memory (nGnRnE,
nGnRE, GRE), add these mapping support for os common mapping API
function z_phys_map().
Signed-off-by: Jiafei Pan <Jiafei.Pan@nxp.com>
It is not necessary to go through the full exception exit code.
This is simpler, smaller and faster.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Make it optimal without the need for an SVC/exception roundtrip on
every context switch. Performance numbers from tests/benchmarks/sched:
Before:
unpend 85 ready 58 switch 258 pend 231 tot 632 (avg 699)
After:
unpend 85 ready 59 switch 115 pend 138 tot 397 (avg 478)
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Get rid of all those global variables and scheduler locking.
Use the reguler IRQ exit path to let tests properly validate preemption.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Add CONFIG_SMP to fvp_baser_aemv8r_smp board.
Fix compile warnings by adding missing header file in arm_mpu.c.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
This commit mainly fixes the broadcast_ipi issue when one core broadcast
ipi to other cores using gic_raise_sgi. The issue doesn't affect the
functionality of Zephyr SMP but will happen when Zephyr runs on Xen.
Suppose Xen provides 4 CPUs to the Zephyr guest, for example, when cpu0
broadcasts ipi to the rest of the cores, the mask should be 0xE(0b1110),
but for now, Zephyr will send 0xFFFE. So for Xen, it will receive a
target list containing many invalid CPUs which don't exist. My solution
is: to generate the target list according to the online CPUs.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
In the Armv8R AArch64 profile[1], the Armv8R AArch64 is always in secure
mode. But the FVP_BaseR_AEMv8R before version 11.16.16 doesn't strictly
follow this rule. It still has some non-secure registers
(e.g. CNTHP_CTL_EL2).
Since version 11.16.16, the FVP_BaseR_AEMv8R has fixed this issue. The
CNTHP_XXX_EL2 registers have been changed to CNTHPS_XXX_EL2. So the
FVP_BaseR_AEMv8R (version >= 11.16.16) cannot boot Zephyr. This patch
will fix it.
[1] https://developer.arm.com/documentation/ddi0600/latest/
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Change-Id: If986f34dc080ae7a8b226bba589b6fe616a4260b
Currently, the DCACHE range invalidation can cause data corruption when
the ends of the given range is not aligned to a full cache line.
Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
Avoid executing ISRs using the thread stack as it might not be sized
for that. Plus, we do have IRQ stacks already set up for us.
The non-nested IRQ context is still (and has to be) saved on the thread
stack as the thread could be preempted.
The irq_offload case is never nested and always invoked with the
sched_lock held so it can be simplified a bit.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
This is an uint32_t so the proper register width must be used, otherwise
the adjacent structure member will be overwritten (didn't happen in
practice because of struct member alignment but still). This makes the
inc_nest_counter and dec_nest_counter macros rather unwieldy, especially
with upcoming changes, so let's just remove them.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Let's provide our own z_early_memset() and z_early_memcpy() rather than
making our own .bss clearing function that risk missing out on updates
to the main version.
Also remove extra stuff already provided by kernel_internal.h.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Avoid setting the switch_handler in the z_get_next_switch_handle() code
when the context is not fully saved yet to avoid a race against other
cores waiting on wait_for_switch().
See issue #40795 and discussion in #41840
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
For functions returning nothing, there is no need to document
with @return, as Doxgen complains about "documented empty
return type of ...".
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This commit adds support of Xen Enlighten page and initial support for
Xen event channels. It is needed for future Xen PV drivers
implementation.
Now enlighten page is mapped to the prepared memory area on
PRE_KERNEL_1 stage. In case of success event channel logic gets
inited and can be used ASAP after Zephyr start. Current implementation
allows to use only pre-defined event channels (PV console/XenBus) and
works only in single CPU mode (without VCPUOP_register_vcpu_info).
Event channel allocation will be implemented in future versions.
Signed-off-by: Dmytro Firsov <dmytro_firsov@epam.com>
Kumar Gala