OrgZephyr/zephyr
OrgZephyr
/
zephyr
mirror of https://github.com/zephyrproject-rtos/zephyr.git
1
0
Fork 0
Code Issues Releases Wiki Activity
zephyr/subsys/rtio/rtio_init.c

34 lines
738 B
C
Raw Normal View History

rtio: Use mpsc for submission and completion queue Rather than the rings, which weren't shared between userspace and kernel space in Zephyr like they are in Linux with io_uring, use atomic mpsc queues for submission and completion queues. Most importantly this removes a potential head of line blocker in the submission queue as the sqe would be held until a task is completed. As additional bonuses this avoids some additional locks and restrictions about what can be submitted and where. It also removes the need for two executors as all chains/transactions are done concurrently. Lastly this opens up the possibility for a common pool of sqe's to allocate from potentially saving lots of memory. Signed-off-by: Tom Burdick <thomas.burdick@intel.com>
2023-04-29 04:52:16 +08:00
/*
* Copyright (c) 2023 Google LLC
* SPDX-License-Identifier: Apache-2.0
*/
rtio: add missing init.h include File was using init.h API without directly including the header. Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
2023-08-29 17:34:41 +08:00
#include <zephyr/init.h>
rtio: Use mpsc for submission and completion queue Rather than the rings, which weren't shared between userspace and kernel space in Zephyr like they are in Linux with io_uring, use atomic mpsc queues for submission and completion queues. Most importantly this removes a potential head of line blocker in the submission queue as the sqe would be held until a task is completed. As additional bonuses this avoids some additional locks and restrictions about what can be submitted and where. It also removes the need for two executors as all chains/transactions are done concurrently. Lastly this opens up the possibility for a common pool of sqe's to allocate from potentially saving lots of memory. Signed-off-by: Tom Burdick <thomas.burdick@intel.com>
2023-04-29 04:52:16 +08:00
#include <zephyr/rtio/rtio.h>
#include <zephyr/sys/util.h>
#include <zephyr/app_memory/app_memdomain.h>
iterable_sections: move to specific header Until now iterable sections APIs have been part of the toolchain (common) headers. They are not strictly related to a toolchain, they just rely on linker providing support for sections. Most files relied on indirect includes to access the API, now, it is included as needed. Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2023-05-15 21:50:28 +08:00
#include <zephyr/sys/iterable_sections.h>
rtio: Use mpsc for submission and completion queue Rather than the rings, which weren't shared between userspace and kernel space in Zephyr like they are in Linux with io_uring, use atomic mpsc queues for submission and completion queues. Most importantly this removes a potential head of line blocker in the submission queue as the sqe would be held until a task is completed. As additional bonuses this avoids some additional locks and restrictions about what can be submitted and where. It also removes the need for two executors as all chains/transactions are done concurrently. Lastly this opens up the possibility for a common pool of sqe's to allocate from potentially saving lots of memory. Signed-off-by: Tom Burdick <thomas.burdick@intel.com>
2023-04-29 04:52:16 +08:00
#ifdef CONFIG_USERSPACE
K_APPMEM_PARTITION_DEFINE(rtio_partition);
#endif
int rtio_init(void)
{
rtio: Cleanup the various define macros Reworks the zephyr macros and pools to be objects in their own right. Each pool can be statically defined with a Z_ private macro. The objects can then be initialized with an rtio instance statically. This cleans up a lot of code that was otherwise doing little bits of management around allocation/freeing and reduces the scope those functions has to the data it needs. This should enable sharing the pools of sqe, cqe, and mem blocks among rtio instances in a future improvement easily. Signed-off-by: Tom Burdick <thomas.burdick@intel.com>
2023-05-01 06:02:39 +08:00
STRUCT_SECTION_FOREACH(rtio_sqe_pool, sqe_pool) {
for (int i = 0; i < sqe_pool->pool_size; i++) {
rtio_mpsc_push(&sqe_pool->free_q, &sqe_pool->pool[i].q);
rtio: Use mpsc for submission and completion queue Rather than the rings, which weren't shared between userspace and kernel space in Zephyr like they are in Linux with io_uring, use atomic mpsc queues for submission and completion queues. Most importantly this removes a potential head of line blocker in the submission queue as the sqe would be held until a task is completed. As additional bonuses this avoids some additional locks and restrictions about what can be submitted and where. It also removes the need for two executors as all chains/transactions are done concurrently. Lastly this opens up the possibility for a common pool of sqe's to allocate from potentially saving lots of memory. Signed-off-by: Tom Burdick <thomas.burdick@intel.com>
2023-04-29 04:52:16 +08:00
}
rtio: Cleanup the various define macros Reworks the zephyr macros and pools to be objects in their own right. Each pool can be statically defined with a Z_ private macro. The objects can then be initialized with an rtio instance statically. This cleans up a lot of code that was otherwise doing little bits of management around allocation/freeing and reduces the scope those functions has to the data it needs. This should enable sharing the pools of sqe, cqe, and mem blocks among rtio instances in a future improvement easily. Signed-off-by: Tom Burdick <thomas.burdick@intel.com>
2023-05-01 06:02:39 +08:00
}
STRUCT_SECTION_FOREACH(rtio_cqe_pool, cqe_pool) {
for (int i = 0; i < cqe_pool->pool_size; i++) {
rtio_mpsc_push(&cqe_pool->free_q, &cqe_pool->pool[i].q);
rtio: Use mpsc for submission and completion queue Rather than the rings, which weren't shared between userspace and kernel space in Zephyr like they are in Linux with io_uring, use atomic mpsc queues for submission and completion queues. Most importantly this removes a potential head of line blocker in the submission queue as the sqe would be held until a task is completed. As additional bonuses this avoids some additional locks and restrictions about what can be submitted and where. It also removes the need for two executors as all chains/transactions are done concurrently. Lastly this opens up the possibility for a common pool of sqe's to allocate from potentially saving lots of memory. Signed-off-by: Tom Burdick <thomas.burdick@intel.com>
2023-04-29 04:52:16 +08:00
}
}
rtio: Cleanup the various define macros Reworks the zephyr macros and pools to be objects in their own right. Each pool can be statically defined with a Z_ private macro. The objects can then be initialized with an rtio instance statically. This cleans up a lot of code that was otherwise doing little bits of management around allocation/freeing and reduces the scope those functions has to the data it needs. This should enable sharing the pools of sqe, cqe, and mem blocks among rtio instances in a future improvement easily. Signed-off-by: Tom Burdick <thomas.burdick@intel.com>
2023-05-01 06:02:39 +08:00
rtio: Use mpsc for submission and completion queue Rather than the rings, which weren't shared between userspace and kernel space in Zephyr like they are in Linux with io_uring, use atomic mpsc queues for submission and completion queues. Most importantly this removes a potential head of line blocker in the submission queue as the sqe would be held until a task is completed. As additional bonuses this avoids some additional locks and restrictions about what can be submitted and where. It also removes the need for two executors as all chains/transactions are done concurrently. Lastly this opens up the possibility for a common pool of sqe's to allocate from potentially saving lots of memory. Signed-off-by: Tom Burdick <thomas.burdick@intel.com>
2023-04-29 04:52:16 +08:00
return 0;
}
SYS_INIT(rtio_init, POST_KERNEL, 0);