This adds the necessary modifier to the stdin/stdout/stderr
variables in picolib, and putting into the z_libc_partition.
This allows userspace applications to utilize these variables
for console I/O.
Fixes#51343
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Compliance check complains about static not being the first
modifier. So move them so there are no more complains
in the future.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
The newlib nano variant is currently enabled by default when
`CONFIG_NEWLIB_LIBC=y` and the selected toolchain-architecture
combination includes the newlib nano variant support, even if
`CONFIG_NEWLIB_LIBC_NANO` is not selected by the user.
When `CONFIG_NEWLIB_LIBC=y`, this results in the newlib nano variant
being selected for some architectures (e.g. ARC, ARM and RISC-V), while
the full variant is selected for the rest of the architectures.
The above behaviour is problematic because there exist functional
differences between the newlib full and nano variants (e.g. C99 format
modifiers such as `hh`, `ll`, `z`, `j` and `t` are not available in the
newlib nano variant), and this effectively leads to different level of
C standard support across different architectures when
`CONFIG_NEWLIB_LIBC=y`.
This commit fixes this problem by making the `CONFIG_NEWLIB_LIBC_NANO`
not `default y` and requiring its user to explicitly set this symbol to
`y` when they want to use the newlib nano variant.
Signed-off-by: Stephanos Ioannidis <stephanos.ioannidis@nordicsemi.no>
Add a new Kconfig and build this code conditionally, so we do not end up
with this file being built for each zephyr app.
Partial fix for #50654
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Extend capabilities of a minimal libc to support C11 capability
to allocate memory with requested alignment.
Signed-off-by: Piotr Pryga <piotr.pryga@nordicsemi.no>
The picolibc heap size configuration (`CONFIG_PICOLIBC_HEAP_SIZE`) is
used to set the statically allocated malloc heap size when userspace is
enabled.
The current default heap size of 1048576 bytes (MMU) and 65536 bytes
(MPU) is too large for most platforms that Zephyr supports and may
result in the picolibc tests being filtered out due to the increased
memory footprint of the compiled image (i.e. SRAM overflow).
This commit updates the default picolibc heap size to a more reasonable
16384 bytes for MMU platforms and 1024 bytes for MPU platforms.
Signed-off-by: Stephanos Ioannidis <stephanos.ioannidis@nordicsemi.no>
Add build assert to make sure _RETARGETABLE_LOCKING is enabled in
toolchain, When _RETARGETABLE_LOCKING is enabled, "_LOCK_T" is "__lock"
pointer type, otherwise "_LOCK_T" is "int" type, so there will be the
following compile warnings when toolchain doesn't enable
_RETARGETABLE_LOCKING:
zephyr/lib/libc/newlib/libc-hooks.c:416:13: warning: cast to pointer from
integer of different size [-Wint-to-pointer-cast]
416 | k_sem_take((struct k_sem *)lock, K_FOREVER);
| ^
zephyr/lib/libc/newlib/libc-hooks.c: In function '__retarget_lock_acquire
_recursive':
zephyr/lib/libc/newlib/libc-hooks.c:423:15: warning: cast to pointer from
integer of different size [-Wint-to-pointer-cast]
423 | k_mutex_lock((struct k_mutex *)lock, K_FOREVER);
| ^
...
Signed-off-by: Jiafei Pan <Jiafei.Pan@nxp.com>
When CONFIG_POSIX_CLOCK is enabled, we should have implementation
of gettimeofday() and therefore time(NULL) should return correct
time, instead of -1.
Signed-off-by: Seppo Takalo <seppo.takalo@nordicsemi.no>
Static packaging is using only argument types to build a package. There
is one case where analysing argument type only is not enough to
determine package content. That is %p with (unsigned) char pointer vs
%s. In case of %s a string might need to be appended to the package
and in case of %p it must be avoided. Format string analysis is required
to distinguish those two cases.
In order to speed up the runtime inspection, additional information is
added to a static package. That is index of the string argument (where
first argument has index 0). This information allows quick format string
inspection where nth format specifier is found and checked if it is a
pointer format specifier.
Inspection algorithm is added to cbprintf_package_convert() and if %p
is found then data for that argument is discarded. Additionally, log
warning is printed with suggestion to cast pointer argument to void *
to avoid confusion. It is desired to get rid of this ambiguity because
there are going to be logging configurations where strings are stripped
from a binary and runtime inspection cannot be performed.
Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
As of today <zephyr/zephyr.h> is 100% equivalent to <zephyr/kernel.h>.
This patch proposes to then include <zephyr/kernel.h> instead of
<zephyr/zephyr.h> since it is more clear that you are including the
Kernel APIs and (probably) nothing else. <zephyr/zephyr.h> sounds like a
catch-all header that may be confusing. Most applications need to
include a bunch of other things to compile, e.g. driver headers or
subsystem headers like BT, logging, etc.
The idea of a catch-all header in Zephyr is probably not feasible
anyway. Reason is that Zephyr is not a library, like it could be for
example `libpython`. Zephyr provides many utilities nowadays: a kernel,
drivers, subsystems, etc and things will likely grow. A catch-all header
would be massive, difficult to keep up-to-date. It is also likely that
an application will only build a small subset. Note that subsystem-level
headers may use a catch-all approach to make things easier, though.
NOTE: This patch is **NOT** removing the header, just removing its usage
in-tree. I'd advocate for its deprecation (add a #warning on it), but I
understand many people will have concerns.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
At some point, package copy function was extended and renamed
to cbprintf_package_convert. However, flags used by this
function were not renamed and used contained COPY idiom.
Deprecating flags with COPY and replacing them with flags
with CONVERT idiom to match function which is utilizing them.
Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
Minor cleanup in allocation function. Using define instead of
sizeof(uint32_t) to better explain the purpose. Adding few
comments.
Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
Allocator is adding padding in 2 steps. First padding mark is written
to the buffer and then write index is reset to 0. Consumer may interrupt
this operation (another thread or core) and fail since it was only checking
padding marker and that alone was enough to consume the padding. If that
happen before write index got updated, buffer reading become corrupted.
Fixing it by adding write index check when padding is found.
Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
Wrong value was used for free space calculation. Updating test which
previously was hiding this bug.
Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
Wire this up the same way ASAN works. Right now it's support only by
recent clang versions (not gcc), and only in 64 bit mode. But it's
capable of detecting uninitialized data reads, which ASAN is not.
This support is wired into the sys_heap (and thus k_heap/k_malloc)
layers, allowing detection of heap misuse like use-after-free. Note
that there is one false negative lurking: due to complexity, in the
case where a sys_heap_realloc() call is able to shrink memory in
place, the now-unused suffix is not marked uninitialized immediately,
making it impossible to detect use-after-free of those particular
bytes. But the system will recover cleanly the next time the memory
gets allocated.
Also no attempt was made to integrate this handling into the newlib or
picolibc allocators, though that should hopefully be possible via
similar means.
Signed-off-by: Andy Ross <andyross@google.com>
The `off_t` type, which is specified by the POSIX standard as a signed
integer type representing file sizes, was defined as `long` or `int`
depending on the target architecture without a clear explanation on why
it was defined as such.
While the POSIX standard does not specify the size requirement of the
`off_t` type, it generally corresponds to the size of a pointer in
practice, mainly because the optimal file handling size is closely tied
to the native pointer size.
For this reason, this commit removes the per-architecture `off_t`
definition and defines it as `intptr_t` such that its size always
matches the native pointer size.
Note that the toolchain-defined `__INTPTR_TYPE__` macro is used instead
of the `intptr_t` typedef as per the common convention used in the C
standard library headers.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
The cbprintf formatter cbvprintf_exteral_formatter_func has a typo
in it with a missing 'n'. So add it.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Currently picolib isn't compatible with ARC MWDT toolchain,
so don't try to build picolib tests in case of ARC MWDT toolchain
usage.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
If the architecture has TLS support, but the toolchain doesn't, then
don't enable Zephyr TLS support when selecting picolibc.
Closes: #47275.
Signed-off-by: Keith Packard <keithp@keithp.com>
When the heap is of a fixed size and there isn't a special malloc partition
in use, place the heap in uninitialized memory so that the application
doesn't spend time at startup erasing it. Picolibc malloc always clears
memory before returning it to applications, so this change will not be
visible to applications.
Signed-off-by: Keith Packard <keithp@keithp.com>
There is a spinlock used in the EFI console code for printing
one line string. It already made the log output being in order
under multi-cores, so it doesn't need a printk sync spinlock
again. Disable it while using EFI console as a log backend.
Fixes#47512.
Signed-off-by: Enjia Mai <enjia.mai@intel.com>
Fixing bug in free space calculation which was assuming 1 byte
padding and not 32 bit word padding. Bug could result in the
data corruption in certain scenario.
Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
xcc compiler complains about how fdtable variable is initialized:
"""
warning: missing braces around initialize
"""
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Use CONFIG_DCACHE_LINE_SIZE to determine distance between
fields modified by different cores.
Add option which specifies what is the data cache line
of the remote core. Maximum from local and remote cache
line sizes is used as distance and alignement.
Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
Any project with Kconfig option CONFIG_LEGACY_INCLUDE_PATH set to n
couldn't be built because some files were missing zephyr/ prefix in
includes
Re-run the migrate_includes.py script to fix all legacy include paths
Signed-off-by: Tomislav Milkovic <milkovic@byte-lab.com>
add a sys_mem_blocks_is_region_free procedure to test
if the block in question is free or taken
Signed-off-by: Marcin Szkudlinski <marcin.szkudlinski@intel.com>
Signed-off-by: Tomasz Leman <tomasz.m.leman@intel.com>
Since the retrieved heap memory statistics are identical to those of
the retrieved mem_block statistics, it makes sense to use a single
generic-named structure for both instead of two identical structures
with different names.
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
Both the current and maximum number of allocations in a given memory
blocked are tracked (and can be queried) when the
CONFIG_SYS_MEM_BLOCKS_RUNTIME_STATS Kconfig option is selected.
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
If a thread is joined using `pthread_join()`, then the
internal state would be set to `PTHREAD_EXITED`.
Previously, `pthread_create()` would only consider pthreads
with internal state `PTHREAD_TERMINATED` as candidates for new
threads. However, that causes a descriptor leak.
We should be able to reuse a single thread an infinite number
of times.
Here, we also consider threads with internal state
`PTHREAD_EXITED` as candiates in `pthread_create()`.
Fixes#47609
Signed-off-by: Christopher Friedt <cfriedt@fb.com>
Move scripts needed by the build system and not designed to be run
individually or standalone into the build subfolder.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
In C++ applications, methods which are meant to be implemented in C
should be wrapped in a `extern "C"` scope at the point of declaration.
This enables the correct symbol table mangling, which fixes current
linker errors in C++.
Signed-off-by: Alexander Dengg <dornbirndevelops@gmail.com>
Following zephyr's style guideline, all if statements, including single
line statements shall have braces.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
This commit changes the invocation of the picolibc malloc heap
initialisation function such that it is executed during the POST_KERNEL
phase instead of the APPLICATION phase.
This is necessary in order to ensure that the application
initialisation functions (i.e. the functions called during the
APPLICATIION phase) can make use of the libc heap.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
This commit changes the invocation of the newlib malloc heap
initialisation function such that it is executed during the POST_KERNEL
phase instead of the APPLICATION phase.
This is necessary in order to ensure that the application
initialisation functions (i.e. the functions called during the
APPLICATIION phase) can make use of the libc heap.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
This commit changes the invocation of the minimal libc malloc
initialisation function such that it is executed during the POST_KERNEL
phase instead of the APPLICATION phase.
This is necessary in order to ensure that the application
initialisation functions (i.e. the functions called during the
APPLICATIION phase) can make use of the libc heap.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
The function used an assert if n was 0. Instead
of using an assert, the function will now just
not do anything. The documentation has also
been updated to reflect this.
The reasoning for this is that the strlcpy function
this (sort of) implements for utf8 works the same way.
Signed-off-by: Emil Gydesen <emil.gydesen@nordicsemi.no>
MISRA C:2012 Rule 21.13 (Any value passed to a function in <ctype.h>
shall be representable as an unsigned char or be the value EOF).
Functions in <ctype.h> have undefined behavior if they are called with
any other value. Callers affected by this change are not prepared to
handle EOF anyway. The addition of these casts avoids the issue
and does not result in any performance penalty.
Signed-off-by: Abramo Bagnara <abramo.bagnara@bugseng.com>
Signed-off-by: Simon Hein <SHein@baumer.com>
MISRA C:2012 Rule 9.3 (Arrays shall not be partially initialized.)
Systematically use `{0}' to specify full 0 initialization
(not `{}', not `{0U}').
Signed-off-by: Abramo Bagnara <abramo.bagnara@bugseng.com>
Signed-off-by: Simon Hein <SHein@baumer.com>
Picolibc inherited its malloc arena configuration from newlib instead of
from minimal libc. This ended up making it a bit too fragile to run the
full set of zephyr tests. In particular:
* Z_MALLOC_PARTITION_EXISTS would get set when not used
* Setting an arena size depended on a bunch of other values, including
whether the system had an MMU or MPU, and whether the MPU required
power-of-two alignment or not.
This patch cleans things up so that there is a single heap size specifier,
PICOLIBC_HEAP_SIZE.
* If PICOLIBC_HEAP_SIZE is positive, this sets the size of the heap. On
MMU systems, picolibc will only use the remaining memory if that's
smaller.
* If PICOLIBC_HEAP_SIZE is zero, then there is no heap available and
malloc will always fail. This also disables Z_MALLOC_PARTITION_EXISTS.
* If PICOLIBC_HEAP_SIZE is negative, then picolibc uses all remaining
memory for the malloc heap.
The defaults are designed to allow tests to work without requiring
additional settings.
* For MMU enabled systems, the default value is 1048576. It would be nice
to have this use 'all available memory', but that's difficult to manage
as the API which returns free memory (k_mem_free_get) doesn't take into
account the amount of free virtual address space.
* For MPU enabled systems which require power-of-two aligned MPU regions,
the default value is 64kB.
* For other systems, the default value is -1, indicating that all
available memory be used for the malloc arena.
Signed-off-by: Keith Packard <keithp@keithp.com>
Added functions which allow to use zero copy model for handling
data within the packet buffer.
Additionally, added handling of cache by adding option to keep rd_idx
in different cache line than wr_idx and data.
Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
Picolibc doesn't have the non-standard printf support required for tagged
arguments in cbprintf. Disable this and use the format string parsing code
instead.
Signed-off-by: Keith Packard <keithp@keithp.com>
Picolibc already provides the functionality offered by cbprintf, so
there's no reason to use the larger and less functional version included
in zephyr.
Signed-off-by: Keith Packard <keithp@keithp.com>
When using picolibc, change vprintk to use picolibc's vfprintf instead of
cbvprintf.
Building samples/hello-world for qemu-cortex-m3.
old:
Memory region Used Size Region Size %age Used
FLASH: 9056 B 256 KB 3.45%
SRAM: 3960 B 64 KB 6.04%
IDT_LIST: 0 GB 2 KB 0.00%
new:
Memory region Used Size Region Size %age Used
FLASH: 7900 B 256 KB 3.01%
SRAM: 3960 B 64 KB 6.04%
IDT_LIST: 0 GB 2 KB 0.00%
Signed-off-by: Keith Packard <keithp@keithp.com>
Daniel Leung