The mempool mistakenly considers the heap memory as its own
The recurring scenario only occurs in mempool_deinit
test code in hello_main:
int main(int argc, FAR char *argv[])
{
void *a = malloc(1024*64);
void *d[16];
void *heap = mm_initialize("123", a, 1024 * 64);
for (int i = 0; i < 16; i++)
{
d[i] = mm_malloc(heap,32);
}
for (int i = 0; i < 16; i++)
{
if (d[i] != NULL)
mm_free(heap,d[i]);
}
mm_uninitialize(heap);
free(a);
return 0;
}
and crash backtrace
0 _assert (filename=0x4ea20 "mempool/mempool.c", linenum=373, msg=0x0 <up_perf_convert>, regs=0x0 <up_perf_convert>)
at misc/assert.c:551
1 0x0000a32c in __assert (filename=0x4ea20 "mempool/mempool.c", linenum=373, msg=0x0 <mempool_multiple_foreach>)
at assert/lib_assert.c:36
2 0x0000f92c in mempool_release (pool=0x100e7a0, blk=0x100ff80) at mempool/mempool.c:373
3 0x000109ce in mempool_multiple_free (mpool=0x100e6f8, blk=0x100ff80) at mempool/mempool_multiple.c:648
4 0x0000deac in mm_delayfree (heap=0x100e090, mem=0x1010000, delay=false) at mm_heap/mm_free.c:83
5 0x0000e21c in mm_free (heap=0x100e090, mem=0x1010000) at mm_heap/mm_free.c:242
6 0x0001021c in mempool_multiple_free_chunk (mpool=0x100e6f8, ptr=0x1010000) at mempool/mempool_multiple.c:222
7 0x0001048e in mempool_multiple_free_callback (pool=0x100e7a0, addr=0x1010080) at mempool/mempool_multiple.c:291
8 0x0000ff6e in mempool_deinit (pool=0x100e7a0) at mempool/mempool.c:644
9 0x00010cba in mempool_multiple_deinit (mpool=0x100e6f8) at mempool/mempool_multiple.c:883
10 0x0000dd0c in mm_uninitialize (heap=0x100e090) at mm_heap/mm_initialize.c:326
11 0x0002c742 in hello_main (argc=1, argv=0x100d050) at hello_main.c:54
12 0x0000a83e in nxtask_startup (entrypt=0x2c6a5 <hello_main>, argc=1, argv=0x100d050) at sched/task_startup.c:70
13 0x00005272 in nxtask_start () at task/task_start.c:112
Signed-off-by: anjiahao <anjiahao@xiaomi.com>
remove alist, switch to a convenient way to traverse
the physical address directly.
At the same time, we can use `gurad` to mark whether
it is free or alloc or out of bounds check
Signed-off-by: anjiahao <anjiahao@xiaomi.com>
the mempool_alloc and mempool_free function are duplicate with the
mempool method that provided in optee_os, in order to port optee_os, so
we rename the mempool related functions in nuttx
Signed-off-by: guoshichao <guoshichao@xiaomi.com>
Fixes build error:
map/vm_region.c: In function 'vm_map_region':
map/vm_region.c:128:16: error: pointer of type 'void *' used in arithmetic [-Werror=pointer-arith]
128 | return vaddr + (MM_PGMASK & paddr);
| ^
cc1: all warnings being treated as errors
This adds explicit `void mm_free_delaylist(heap)` interface so that
to force freeing the heap's delaylist.
Signed-off-by: Yanfeng Liu <yfliu2008@qq.com>
Add a common method to format backtrace to buffer, so it can be used by both mm, fs and other possoble modules.
Signed-off-by: fangpeina <fangpeina@xiaomi.com>
- `g_iob_sem.semcount` must be equal to the total number of free IOBs.
It can also be negative if there are no free IOBs and there are threads
waiting for an IOB.
- g_throttle_sem.semcount represents the number of IOBs available for
throttled IOB allocations. Like any other semaphore, it should only go
negative if there is a thread waiting for it.
- Both semaphores are related to the same resource (free IOBs), hence,
they must be incremented/decremented simultaneously:
- Whenever a IOB buffer is freed, if a thread is waiting for a
non-throttled IOB or a thread is waiting for a throttled IOB and we
have at least `CONFIG_IOB_THROTTLE` buffers available, the IOB is put
in the committed list (`g_iob_committed`). Otherwise, it is put in the
common free list (`g_iob_freelist`).
- `g_iob_sem` is always incremented when an IOB buffer is freed, but
`g_throttle_sem` is incremented only if we have at least CONFIG_IOB_THROTTLE
buffers free.
- Both semaphores are posted with the schedule locked to avoid any
mismatches in the semaphores count.
- If a task is waiting for an IOB semaphore (`iob_allocwait`) is
awakened and would check the `g_iob_committed`. The highest priority
task waiting for a semaphore will be awakened first.
make possible enable pool, but disabled in system heap.
also compatible for further extra heap use/not use pool.
Now use MM_HEAP_MEMPOOL_THRESHOLD -1 to indicate pool disabled.
Signed-off-by: buxiasen <buxiasen@xiaomi.com>
If the CAN stack receiving packets fast, but the application layer reading packets slow. Then `conn->readahead` will continue to grow, leading to memory leaks. Finally CAN stack potentially starve out all IOB buffers. To prevent memory leaks, users can restrict can socket buffer length.
Signed-off-by: gaohedong <gaohedong@xiaomi.com>
GRAN_ALIGNED should check that the memory block's alignment (log2align)
is correct, not that the memory block is aligned with the granule size.
This fixes DEBUGASSERT() in mm_granfree:
_assert: Assertion failed : at file: mm_gran/mm_granfree.c:49
The assertion triggers if granule size != alignment.
This revises vm_map_region() by accepting unaligned paddr, which is
aligned-down before mapping and in-page offset is then added to vaddr
before returning. It also moves vm_map_region() and vm_unmap_region()
to vm_region.c.
Signed-off-by: Yanfeng Liu <yfliu2008@qq.com>
This patch adds definitions to support user space device mappings
that allows devices like frame buffer to be accessible from user
space in kernel mode.
The are mainly two changes:
- in `mm/`:
added vm_map_region(), vm_unmap_region() for drivers to do
device mapping easily.
- in `arch/`:
extended ARCH_SHM_NPAGES as user-space mapping region size.
decoupled ARCH_SHM_MAXREGIONS from region size calculations and
limit its usage only for SysV shm purposes.
Signed-off-by: Yanfeng Liu <yfliu2008@qq.com>
Support the network interface card driver to receive zero copies of packets and send and receive giant frame packets, allowing drivers to initialize the DMA buffer to the iob structure, and we can apply for IOB with large memory
Signed-off-by: zhanghongyu <zhanghongyu@xiaomi.com>
This patch refactors granule allocator to remove the 32 granules
limitation with the help of a gran_range_s structure and related
functions, see "mm_grantable.h" for details.
Below are the major functions explaining how this works:
- The gran_match() checks if a gran range all in the given state.
it gives last mismatch position when fails free range matching.
- The gran_search() tries to find the position of a free range.
It leverages last mismatch position from gran_match() to speed
up the search.
range size handling is mainly in gran_match() and gran_set_().
Signed-off-by: Yanfeng Liu <yfliu2008@qq.com>
- Add ARCH_KVMA_MAPPING to guard kernel mapping.
- Set dependency from MM_KMAP to ARCH_KVMA_MAPPING, as per commit
70de321de3.
Signed-off-by: Yanfeng Liu <yfliu2008@qq.com>
Extracting global variable information using scripts:
kasan_global.py:
1. Extract the global variable information provided by the -- param asan globals=1 option
2. Generate shadow regions for global variable out of bounds detection
Makefile:
1. Implement multiple links, embed the shadow area into the program, and call it by the Kasan module
Signed-off-by: W-M-R <mike_0528@163.com>
In SMP mode, up_cpu_index()/this_cpu() are the same, both return the index of the physical core.
In AMP mode, up_cpu_index() will return the index of the physical core, and this_cpu() will always return 0
| #ifdef CONFIG_SMP
| # define this_cpu() up_cpu_index()
| #elif defined(CONFIG_AMP)
| # define this_cpu() (0)
| #else
| # define this_cpu() (0)
| #endif
Signed-off-by: chao an <anchao@lixiang.com>
These variables will trigger variable 'ret' set but not used warnings due to different configurations.
Signed-off-by: yinshengkai <yinshengkai@xiaomi.com>
ctc E246: ["map/mm_map.c" 67/41] left side of '.' or '->' is not struct or union
ctc E260: ["map/mm_map.c" 67/25] not an lvalue
ctc E246: ["map/mm_map.c" 80/3] left side of '.' or '->' is not struct or union
ctc E260: ["map/mm_map.c" 80/3] not an lvalue
Signed-off-by: chao an <anchao@lixiang.com>
After this, RISC-V fully supports the kmap interface.
Due to the current design limitations of having only a single L2 table
per process, the kernel kmap area cannot be mapped via any user page
directory, as they do not contain the page tables to address that range.
So a "kernel address environment" is added, which can do the mapping. The
mapping is reflected to every process as only the root page directory (L1)
is copied to users, which means every change to L2 / L3 tables will be
seen by every user.
Mapping a physical page to a kernel virtual page is very simple and does
not need the kernel vma list, just get the kernel addressable virtual
address for the page.
is_kmap_vaddr is added and used to test that a given (v)addr is actually
inside the kernel map area. This gives a speed optimization for kmm_unmap,
as it is no longer necessary to take the mm_map_lock to check if such a
mapping exists; obviously if the address is not within the kmap area, it
won't be in the list either.
User pages are mapped from the currently active address environment. If
the process is running on a borrowed address environment, then the
mapping should be created from there.
This happens during (new) process creation only.
rndis header length is 36, L2 header is 14, IPv6 header is 40, tcp header is 56 when sack option count is 4(default max_ofosegs is 4). so the iob bufsize should greater than we need.
Signed-off-by: zhanghongyu <zhanghongyu@xiaomi.com>
Modification based on opengroup's description for shmget: "When the shared memory segment is created, it shall be initialized with all zero values."
Link to documentation page for shmget: https://pubs.opengroup.org/onlinepubs/9699919799/
If we apply `iob_reserve` on an IOB with `io_offset != 0`, the `head->io_pktlen` and `iob->io_len` will become wrong value, because we only need to trim `offset - iob->io_offset`.
Signed-off-by: Zhe Weng <wengzhe@xiaomi.com>
if there are two throttled wait, when iob_free occurs, one of wait
will be awakened to execute iob_alloc_committed, but it will fail
to execute, sem will be posted at this time, then another wait will
be awakened. after the other wait thread is awakened, This step is
repeated. the two threads are in the critical_section state and
cannot be switched to other threads. then cpu will busy util timeout.
Signed-off-by: zhanghongyu <zhanghongyu@xiaomi.com>
If malloc chunk fails, and if malloc fails to dump all memory,
it will cause deadlock in multiple_mempool_info
Signed-off-by: anjiahao <anjiahao@xiaomi.com>
No memory map count limit that will exhaust memory and cause
the system hang. Also that fix pass LTP posix case mmap/24-1.c
Signed-off-by: fangxinyong <fangxinyong@xiaomi.com>
When other code use nuttx memory manager by call mm_xx api directly,
it better to let other code to control weather dump or panic when
malloc failed.
Signed-off-by: Bowen Wang <wangbowen6@xiaomi.com>
The circbuf_peekat should copy valid content from the specfied position
of buffer, so skip the area from this position to the tail.
Signed-off-by: dongjiuzhu1 <dongjiuzhu1@xiaomi.com>
1. command "memdump leak" can dump the leacked memory node;
2. fix the leak memory stat bug in memory manager;
Signed-off-by: wangbowen6 <wangbowen6@xiaomi.com>
1. Update all CMakeLists.txt to adapt to new layout
2. Fix cmake build break
3. Update all new file license
4. Fully compatible with current compilation environment(use configure.sh or cmake as you choose)
------------------
How to test
From within nuttx/. Configure:
cmake -B build -DBOARD_CONFIG=sim/nsh -GNinja
cmake -B build -DBOARD_CONFIG=sim:nsh -GNinja
cmake -B build -DBOARD_CONFIG=sabre-6quad/smp -GNinja
cmake -B build -DBOARD_CONFIG=lm3s6965-ek/qemu-flat -GNinja
(or full path in custom board) :
cmake -B build -DBOARD_CONFIG=$PWD/boards/sim/sim/sim/configs/nsh -GNinja
This uses ninja generator (install with sudo apt install ninja-build). To build:
$ cmake --build build
menuconfig:
$ cmake --build build -t menuconfig
--------------------------
2. cmake/build: reformat the cmake style by cmake-format
https://github.com/cheshirekow/cmake_format
$ pip install cmakelang
$ for i in `find -name CMakeLists.txt`;do cmake-format $i -o $i;done
$ for i in `find -name *\.cmake`;do cmake-format $i -o $i;done
Co-authored-by: Matias N <matias@protobits.dev>
Signed-off-by: chao an <anchao@xiaomi.com>
anjiahao