This commit activates the built-in stack guard on the main_thread
before jumping to it upon system initialization. Stack guard is
activated if BUILTIN_STACK_GUARD is enabled by the user. The
commit also activates built-in thread stack guards at every
context switch, if BUILTIN_STACK_GUARD enabled by the user.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
This commit implements the build-in interrupt stack protection
via the native MSPLIM register in ARMv8-M MCUs that support the
Main Extension. Main stack pointer overflows will cause Stack
Overflow UsageFaults. In addition, the commit introduces a new
config option, BUILTIN_STACK_GUARD, allowing the user to enable
and use the built-in stack protection in ARMv8-M firmware.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
This commit contributes the implementation of the SecureFault
handling for ARMv8-M-based Cortex-M33. The implementation is
compiled conditionally with compile-time directive
CONFIG_ARM_SECURE_FIRMWARE, which is to signify the intention to
build a Secure image on ARMv8-M with Security Extensions.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
This commit removes the unnecessary asm inline header for ARM.
It also adapts the stack.h and exc.h to use the ARM CMSIS inline
functions to access the IPSR and MSP registers.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
This patch fixes a hole in the stack guard configuration. The initial
branch to main is missing the stack guard configuration.
Fixes: Issue #3718
Signed-off-by: Andy Gross <andy.gross@linaro.org>
This patch fixes calculations for the top of the interrupt and main
stacks. Due to power of two alignment requirements for certain MPUs,
the guard size must be taken into account due to the guard being
counted against the initial stack size.
Signed-off-by: Andy Gross <andy.gross@linaro.org>
This patch adds support for userspace on ARM architectures. Arch
specific calls for transitioning threads to user mode, system calls,
and associated handlers.
Signed-off-by: Andy Gross <andy.gross@linaro.org>
This PR includes the required changes in order to support
conditional compilation for Armv8-M architecture. Two
variants of the Armv8-M architecture are defined:
- the Armv8-M Baseline (backwards compatible with ARMv6-M),
- the Armv8-M Mainline (backwards compatible with ARMv7-M).
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
This patch adds the generation and incorporation of privileged stack
regions that are used by ARM user mode threads. This patch adds the
infrastructure for privileged stacks. Later patches will utilize the
generated stacks and helper functions.
Signed-off-by: Chunlin Han <chunlin.han@linaro.org>
Signed-off-by: Andy Gross <andy.gross@linaro.org>
Rename the nano_internal.h to kernel_internal.h and modify the
header file name accordingly wherever it is used.
Signed-off-by: Ramakrishna Pallala <ramakrishna.pallala@intel.com>
Currently this is defined as a k_thread_stack_t pointer.
However this isn't correct, stacks are defined as arrays. Extern
references to k_thread_stack_t doesn't work properly as the compiler
treats it as a pointer to the stack array and not the array itself.
Declaring as an unsized array of k_thread_stack_t doesn't work
well either. The least amount of confusion is to leave out the
pointer/array status completely, use pointers for function prototypes,
and define K_THREAD_STACK_EXTERN() to properly create an extern
reference.
The definitions for all functions and struct that use
k_thread_stack_t need to be updated, but code that uses them should
be unchanged.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
In various places, a private _thread_entry_t, or the full prototype
were being used. Be consistent and use the same typedef everywhere.
Signen-off-by: Andrew Boie <andrew.p.boie@intel.com>
This patch fixes a couple of issues with the stack guard size and
properly constructs the STACK_ALIGN and STACK_ALIGN_SIZE definitions.
The ARM AAPCS requires that the stack pointers be 8 byte aligned. The
STACK_ALIGN_SIZE definition is meant to contain the stack pointer
alignment requirements. This is the required alignment at public API
boundaries (ie stack frames).
The STACK_ALIGN definition is the required alignment for the start
address for stack buffer storage. STACK_ALIGN is used to validate
the allocation sizes for stack buffers.
The MPU_GUARD_ALIGN_AND_SIZE definition is the minimum alignment and
size for the MPU. The minimum size and alignment just so happen to be
32 bytes for vanilla ARM MPU implementations.
When defining stack buffers, the stack guard alignment requirements
must be taken into consideration when allocating the stack memory.
The __align() must be filled in with either STACK_ALIGN_SIZE or the
align/size of the MPU stack guard. The align/size for the guard region
will be 0 when CONFIG_MPU_STACK_GUARD is not set, and 32 bytes when it
is.
The _ARCH_THREAD_STACK_XXXXXX APIs need to know the minimum alignment
requirements for the stack buffer memory and the stack guard size to
correctly allocate and reference the stack memory. This is reflected
in the macros with the use of the STACK_ALIGN definition and the
MPU_GUARD_ALIGN_AND_SIZE definition.
Signed-off-by: Andy Gross <andy.gross@linaro.org>
Historically, stacks were just character buffers and could be treated
as such if the user wanted to look inside the stack data, and also
declared as an array of the desired stack size.
This is no longer the case. Certain architectures will create a memory
region much larger to account for MPU/MMU guard pages. Unfortunately,
the kernel interfaces treat both the declared stack, and the valid
stack buffer within it as the same char * data type, even though these
absolutely cannot be used interchangeably.
We introduce an opaque k_thread_stack_t which gets instantiated by
K_THREAD_STACK_DECLARE(), this is no longer treated by the compiler
as a character pointer, even though it really is.
To access the real stack buffer within, the result of
K_THREAD_STACK_BUFFER() can be used, which will return a char * type.
This should catch a bunch of programming mistakes at build time:
- Declaring a character array outside of K_THREAD_STACK_DECLARE() and
passing it to K_THREAD_CREATE
- Directly examining the stack created by K_THREAD_STACK_DECLARE()
which is not actually the memory desired and may trigger a CPU
exception
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Unline k_thread_spawn(), the struct k_thread can live anywhere and not
in the thread's stack region. This will be useful for memory protection
scenarios where private kernel structures for a thread are not
accessible by that thread, or we want to allow the thread to use all the
stack space we gave it.
This requires a change to the internal _new_thread() API as we need to
provide a separate pointer for the k_thread.
By default, we still create internal threads with the k_thread in stack
memory. Forthcoming patches will change this, but we first need to make
it easier to define k_thread memory of variable size depending on
whether we need to store coprocessor state or not.
Change-Id: I533bbcf317833ba67a771b356b6bbc6596bf60f5
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Historically, space for struct k_thread was always carved out of the
thread's stack region. However, we want more control on where this data
will reside; in memory protection scenarios the stack may only be used
for actual stack data and nothing else.
On some platforms (particularly ARM), including kernel_arch_data.h from
the toplevel kernel.h exposes intractable circular dependency issues.
We create a new per-arch header "kernel_arch_thread.h" with very limited
scope; it only defines the three data structures necessary to instantiate
the arch-specific bits of a struct k_thread.
Change-Id: I3a55b4ed4270512e58cf671f327bb033ad7f4a4f
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Put the reason code in r0 and make a SVC #2 call, which will be
propagated to _fatal_error_handler as an exception.
The _is_in_isr() implementation had to be tweaked a bit. User-generated
SVC exception no longer just used for irq_offload(); just because we are
in it does not mean we are in interrupt context. Instead, have the
irq_offload code set and clear the offload_routine global; it will be
non-NULL only if it's in use. Upcoming changes to support memory
protection (which will require system calls) will need this too.
We free up some small amount of ROM deleting _default_esf struct as it's
no longer needed.
Issue: ZEP-843
Change-Id: Ie82bd708575934cffe41e64f5c128c8704ca4e48
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
The ICSR RETTOBASE bit is improperly implemented in QEMU (the polarity
is flipped) and the fix for it has not yet made it into a QEMU release,
although it is present in upstream master branch.
The symptom is that if we are not in thread mode, the system always
believes were are in a nested exception state, causing _IsInIsr() to
always return true.
Skip the nested exception check if we are building for QEMU.
This is a workaround until SDK-54 is resolved.
Issue: SDK-54
Change-Id: I06eafcc85fb76a9b23b4ba85ed6e111a08516231
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Convert code to use u{8,16,32,64}_t and s{8,16,32,64}_t instead of C99
integer types. There are few places we dont convert over to the new
types because of compatiability with ext/HALs or for ease of transition
at this point. Fixup a few of the PRI formatters so we build with newlib.
Jira: ZEP-2051
Change-Id: I7d2d3697cad04f20aaa8f6e77228f502cd9c8286
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
This is a start to move away from the C99 {u}int{8,16,32,64}_t types to
Zephyr defined u{8,16,32,64}_t and s{8,16,32,64}_t. This allows Zephyr
to define the sized types in a consistent manor across all the
architectures we support and not conflict with what various compilers
and libc might do with regards to the C99 types.
We introduce <zephyr/types.h> as part of this and have it include
<stdint.h> for now until we transition all the code away from the C99
types.
We go with u{8,16,32,64}_t and s{8,16,32,64}_t as there are some
existing variables defined u8 & u16 as well as to be consistent with
Zephyr naming conventions.
Jira: ZEP-2051
Change-Id: I451fed0623b029d65866622e478225dfab2c0ca8
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
The previous code incorrectly used the value 0xfe to clear the mem and
bus faults. It attempted to handle the address register valid bits
separately, but reversed the bit order.
Jira: ZEP-1568
Change-Id: I240d072610af9979ca93c0081ed862df08929372
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Signed-off-by: Maureen Helm <maureen.helm@nxp.com>
A number of SoCs clear out the Mem/Bus/Usage and Hard Fault exceptions
during init. Lets refactor that into a common function so we don't have
to keep duplicating it over and over.
Change-Id: Ida908a9092db37447abcf3c9872f36937982f729
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Replace _ScbExcPrioSet with calls to NVIC_SetPriority as it handles both
interrupt and exception priorities. We don't need to shift around the
priority values for NVIC_SetPriority.
Jira: ZEP-1568
Change-Id: Iccd68733c3f7faa82b7ccb17200eef328090b6da
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Replace the existing Apache 2.0 boilerplate header with an SPDX tag
throughout the zephyr code tree. This patch was generated via a
script run over the master branch.
Also updated doc/porting/application.rst that had a dependency on
line numbers in a literal include.
Manually updated subsys/logging/sys_log.c that had a malformed
header in the original file. Also cleanup several cases that already
had a SPDX tag and we either got a duplicate or missed updating.
Jira: ZEP-1457
Change-Id: I6131a1d4ee0e58f5b938300c2d2fc77d2e69572c
Signed-off-by: David B. Kinder <david.b.kinder@intel.com>
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
The cortex-m7 is an implementation of armv7-m. Adjust the Kconfig
support for cortex-m7 to reflect this and drop the unnecessary,
explicit, conditional compilation.
Change-Id: I6ec20e69c8c83c5a80b1f714506f7f9e295b15d5
Signed-off-by: Marcus Shawcroft <marcus.shawcroft@arm.com>
Precursor patches have arranged that conditional compilation hanging
on CONFIG_CPU_CORTEX_M3_M4 provides support for ARMv7-M, rename the
config variable to reflect this.
Change-Id: Ifa56e3c1c04505d061b2af3aec9d8b9e55b5853d
Signed-off-by: Marcus Shawcroft <marcus.shawcroft@arm.com>
Precursor patches have arranged all conditional compilation hanging on
CONFIG_CPU_CORTEX_M0_M0PLUS such that it actually represents support
for ARM ARMv6-M, rename the config variable to reflect this.
Change-Id: I553fcf3e606b350a9e823df31bac96636be1504f
Signed-off-by: Marcus Shawcroft <marcus.shawcroft@arm.com>
The ARM code base provides for three mutually exclusive ARM
architecture related conditional compilation choices. M0_M0PLUS,
M3_M4 and M7. Throughout the code base we have conditional
compilation gated around these three choices. Adjust the form of this
conditional compilation to adopt a uniform structure. The uniform
structure always selects code based on the definition of an
appropriate config option rather the the absence of a definition.
Removing the extensive use of #else ensures that when support for
other ARM architecture versions is added we get hard compilation
failures rather than attempting to compile inappropriate code for the
added architecture with unexpected runtime consequences.
Adopting this uniform structure makes it straight forward to replace
the adhoc CPU_CORTEX_M3_M4 and CPU_CORTEX_M0_M0PLUS configuration
variables with ones that directly represent the actual underlying ARM
architectures we provide support for. This change also paves the way
for folding adhoc conditional compilation related to CPU_CORTEX_M7
directly in support for ARMv7-M.
This change is mechanical in nature involving two transforms:
1)
#if !defined(CONFIG_CPU_CORTEX_M0_M0PLUS)
...
is transformed to:
#if defined(CONFIG_CPU_CORTEX_M0_M0PLUS)
#elif defined(CONFIG_CPU_CORTEX_M3_M4) || defined(CONFIG_CPU_CORTEX_M7)
...
2)
#if defined(CONFIG_CPU_CORTEX_M0_M0PLUS)
...
#else
...
#endif
is transformed to:
#if defined(CONFIG_CPU_CORTEX_M0_M0PLUS)
...
#elif defined(CONFIG_CPU_CORTEX_M3_M4) || defined(CONFIG_CPU_CORTEX_M7)
...
#else
#error Unknown ARM architecture
#endif
Change-Id: I7229029b174da3a8b3c6fb2eec63d776f1d11e24
Signed-off-by: Marcus Shawcroft <marcus.shawcroft@arm.com>
The Cortex-M0(+) and in general processors that support only the ARMv6-M
instruction set have a reduced set of registers and fields compared to
the ARMv7-M compliant processors.
This change goes through all core registers and disables or removes
everything that is not part of the ARMv6-M architecture when compiling
for Cortex-M0.
Jira: ZEP-1497
Change-id: I13e2637bb730e69d02f2a5ee687038dc69ad28a8
Signed-off-by: Vinayak Chettimada <vinayak.kariappa.chettimada@nordicsemi.no>
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Also remove some old cflags referencing directories that do not exist
anymore.
Also replace references to legacy APIs in doxygen documentation of
various functions.
Change-Id: I8fce3d1fe0f4defc44e6eb0ae09a4863e33a39db
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
nano_cpu_idle/nano_cpu_atomic_idle were not ported to the unified
kernel, and only the old APIs were available. There was no real impact
since, in the unified kernel, only the idle thread should really be
doing power management. However, with a single-threaded kernel, these
functions can be useful again.
The kernel internals now make use of these APIs instead of the legacy
ones.
Change-Id: Ie8a6396ba378d3ddda27b8dd32fa4711bf53eb36
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
The way the ready thread cache was implemented caused it to not always
be "hot", i.e. there could be some misses, which happened when the
cached thread was taken out of the ready queue. When that happened, it
was not replaced immediately, since doing so could mean that the
replacement might not run because the flow could be interrupted and
another thread could take its place. This was the more conservative
approach that insured that moving a thread to the cache would never be
wasted.
However, this caused two problems:
1. The cache could not be refilled until another thread context-switched
in, since there was no thread in the cache to compare priorities
against.
2. Interrupt exit code would always have to call into C to find what
thread to run when the current thread was not coop and did not have the
scheduler locked. Furthermore, it was possible for this code path to
encounter a cold cache and then it had to find out what thread to run
the long way.
To fix this, filling the cache is now more aggressive, i.e. the next
thread to put in the cache is found even in the case the current cached
thread is context-switched out. This ensures the interrupt exit code is
much faster on the slow path. In addition, since finding the next thread
to run is now always "get it from the cache", which is a simple fetch
from memory (_kernel.ready_q.cache), there is no need to call the more
complex C code.
On the ARM FRDM K64F board, this improvement is seen:
Before:
1- Measure time to switch from ISR back to interrupted task
switching time is 215 tcs = 1791 nsec
2- Measure time from ISR to executing a different task (rescheduled)
switch time is 315 tcs = 2625 nsec
After:
1- Measure time to switch from ISR back to interrupted task
switching time is 130 tcs = 1083 nsec
2- Measure time from ISR to executing a different task (rescheduled)
switch time is 225 tcs = 1875 nsec
These are the most dramatic improvements, but most of the numbers
generated by the latency_measure test are improved.
Fixes ZEP-1401.
Change-Id: I2eaac147048b1ec71a93bd0a285e743a39533973
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
The Cortex-M3/4 kernel was reserving priorities 0 and 1 for itself, but
was not registering any exception on priority 0. Only reserve priority 0
and use it for SVC and fault exceptions instead of priority 1.
Change-Id: Iff2405e27fd4bed4e49ab90ec2ae984f2c0a83a6
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
Also remove NO_METRIC, which is not referenced anywhere anymore.
Change-Id: Ieaedf075af070a13aa3d975fee9b6b332203bfec
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
The ARM Cortex-M early boot was using a custom stack at the end of the
SRAM instead of the interrupt stack. This works as long as no static
data that needs a known initial value occupies that stack space. This
has probably not been an issue because the .noinit section is at the
very end of the image, but it was still wrong to use that region of
memory for that initial stack.
To be able to use the interrupt stack during early boot, the stack has
to be released before an interrupt can happen. Since ARM Cortex-M uses
PendSV as a very low priority exception for context switching, if a
device driver installs and enables an interrupt during the PRE_KERNEL
initialization points, an interrupt could take precedence over PendSV
while the initial dummy thread has not yet been context switched of and
thus released the interrupt stack. To address this, rather than using
_Swap() and thus triggering PendSV, the initialization logic switches to
the main stack and branches to _main() directly instead.
Fixes ZEP-1309
Change-Id: If0b62cc66470b45b601e63826b5b3306e6a25ae9
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
There was a possible race condition when setting the return value of a
thread that is pending, from an ISR.
A kernel function causes a thread to pend, with the following series of
steps:
- disable interrupts
- move current thread to wait_q
- call _Swap
Depending if running on M3/4 or M0+, _Swap will either issue a svc #0,
or pend PendSV directly. The same problem exists in both cases.
M3/4:
__svc will:
- enable interrupts
- trigger __pendsv
M0+:
_Swap() will enable interrupts.
__pendsv will:
- save register context including PSP into the thread struct
If an interrupt occurs between interrupts being enabled them and
__pendsv saving PSP, and the ISR sets the pending thread's return value,
this will happen:
- sees the thread in a wait_q
- removes it
- makes it ready
- calls _set_thread_return_value
- _set_thread_return_value looks at the thread's saved PSP to poke
the value
In this scenario, PSP hasn't yet been updated by __pendsv so it's a
stale value from the previous context switch, resulting in unpredictable
word on the stack getting set to the return value.
There is no way to fix this issue and still have the return value being
delivered directly in the pending thread's exception stack frame, in the
M0+ case. There will always be a window between the unlocking of
interrupts and PendSV being handled. On M3/4, it could be possible with
the mix of SVC and PendSV, since the exception stack frame is created in
the __svc handler. However, because we want to keep the two
implementations as close as possible, and there were talks of moving
M3/4 to using PendSV only, to save an exception, the approach taken
solves both cases.
The approach taken is similar to the ARC and Nios2 ports, where
there is a field in the thread structure that holds the return value.
_Swap() then loads r0/a1 with that value just before returning.
Fixes ZEP-1289.
Change-Id: Iee7e06fe3f8ded84aff918fd43408c7f589344d9
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
There was a lot of duplication between architectures for the definition
of threads and the "nanokernel" guts. These have been consolidated.
Now, a common file kernel/unified/include/kernel_structs.h holds the
common definitions. Architectures provide two files to complement it:
kernel_arch_data.h and kernel_arch_func.h. The first one contains at
least the struct _thread_arch and struct _kernel_arch data structures,
as well as the struct _callee_saved and struct _caller_saved register
layouts. The second file contains anything that needs what is provided
by the common stuff in kernel_structs.h. Those two files are only meant
to be included in kernel_structs.h in very specific locations.
The thread data structure has been separated into three major parts:
common struct _thread_base and struct k_thread, and arch-specific struct
_thread_arch. The first and third ones are included in the second.
The struct s_NANO data structure has been split into two: common struct
_kernel and arch-specific struct _kernel_arch. The latter is included in
the former.
Offsets files have also changed: nano_offsets.h has been renamed
kernel_offsets.h and is still included by the arch-specific offsets.c.
Also, since the thread and kernel data structures are now made of
sub-structures, offsets have to be added to make up the full offset.
Some of these additions have been consolidated in shorter symbols,
available from kernel/unified/include/offsets_short.h, which includes an
arch-specific offsets_arch_short.h. Most of the code include
offsets_short.h now instead of offsets.h.
Change-Id: I084645cb7e6db8db69aeaaf162963fe157045d5a
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
Symbols now use the K_ prefix which is now standard for the
unified kernel. Legacy support for these symbols is retained
to allow existing applications to build successfully.
Change-Id: I3ff12c96f729b535eecc940502892cbaa52526b6
Signed-off-by: Allan Stephens <allan.stephens@windriver.com>
Adds standard prefix to symbolic option that flags a thread
as essential to system operation.
Change-Id: Ia904a81ce343fdd1cd44caaaeae641d822777f9b
Signed-off-by: Allan Stephens <allan.stephens@windriver.com>
They were the same, standardize on the lowercase one.
Change-Id: I8bca080e45f3e0970697d4451e468b9081f96f5f
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
Not disabling SysTick as it is optional by the spec.
SVC not used as there is no priority-based interrupt masking (only
PendSV is used).
Largely based on a previous work done by Euan Mutch <euan@abelon.com>.
Jira: ZEP-783
Change-Id: I38e29bfcf0624c1aea5f9fd7a74230faa1b59e8b
Signed-off-by: Ricardo Salveti <ricardo.salveti@linaro.org>
When adding a thread to the ready queue, it is often known at that time
if the thread added will be the next one to run or not. So, instead of
simply updating the ready queues and the bitmask, also cache what that
thread is, so that when the scheduler is invoked, it can simply fetch it
from there. This is only done if there is a thread in the cache, since
the way the cache is updated is by comparing the priorities of the
thread being added and the cached thread.
When a thread is removed from the ready queue, if it is currently the
cached thread, it is also removed from the cache. The cache is not
updated at this time, since this would be a preemptive fetching that
could be overriden before the newly cached thread would even be
scheduled in.
Finally, when a thread is scheduled in, it now becomes the cached thread
since the fact that it is running means that by definition it was the
next one to run.
Doing this can speed up considerably some context switch times,
especially when a thread is preempted by an interrupt and the same
thread is scheduled when the interrupt exits.
Change-Id: I6dc8391cfca566699bb9b217eafe6bc6a063c8bb
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
Ioannis Glaropoulos