OrgZephyr/zephyr
OrgZephyr
/
zephyr
mirror of https://github.com/zephyrproject-rtos/zephyr.git
1
0
Fork 0
Code Issues Releases Wiki Activity
zephyr/kernel/futex.c

102 lines
2.1 KiB
C
Raw Normal View History

kernel: add futex support A k_futex is a lightweight mutual exclusion primitive designed to minimize kernel involvement. Uncontended operation relies only on atomic access to shared memory. k_futex structure lives in application memory. And when using futexes, the majority of the synchronization operations are performed in user mode. A user-mode thread employs the futex wait system call only when it is likely that the program has to block for a longer time until the condition becomes true. When the condition comes true, futex wake operation will be used to wake up one or more threads waiting on that futex. This patch implements two futex operations: k_futex_wait and k_futex_wake. For k_futex_wait, the comparison with the expected value, and starting to sleep are performed atomically to prevent lost wake-ups. If different context changed futex's value after the calling use-mode thread decided to block himself based on the old value, the comparison will help observing the value change and will not start to sleep. And for k_futex_wake, it will wake at most num_waiters of the waiters that are sleeping on that futex. But no guarantees are made on which threads are woken, that means scheduling priority is not taken into consideration. Fixes: #14493. Signed-off-by: Wentong Wu <wentong.wu@intel.com>
2019-06-20 23:51:27 +08:00
/*
* Copyright (c) 2019 Intel corporation
*
* SPDX-License-Identifier: Apache-2.0
*/
#include <kernel.h>
#include <kernel_structs.h>
#include <spinlock.h>
#include <kswap.h>
#include <syscall_handler.h>
#include <init.h>
#include <ksched.h>
static struct z_futex_data *k_futex_find_data(struct k_futex *futex)
{
struct _k_object *obj;
obj = z_object_find(futex);
if (obj == NULL || obj->type != K_OBJ_FUTEX) {
return NULL;
}
return (struct z_futex_data *)obj->data;
}
int z_impl_k_futex_wake(struct k_futex *futex, bool wake_all)
{
k_spinlock_key_t key;
unsigned int woken = 0;
struct k_thread *thread;
struct z_futex_data *futex_data;
futex_data = k_futex_find_data(futex);
if (futex_data == NULL) {
return -EINVAL;
}
key = k_spin_lock(&futex_data->lock);
do {
thread = z_unpend_first_thread(&futex_data->wait_q);
if (thread) {
z_ready_thread(thread);
kernel: rename z_arch_ to arch_ Promote the private z_arch_* namespace, which specifies the interface between the core kernel and the architecture code, to a new top-level namespace named arch_*. This allows our documentation generation to create online documentation for this set of interfaces, and this set of interfaces is worth treating in a more formal way anyway. Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-11-08 04:43:29 +08:00
arch_thread_return_value_set(thread, 0);
kernel: add futex support A k_futex is a lightweight mutual exclusion primitive designed to minimize kernel involvement. Uncontended operation relies only on atomic access to shared memory. k_futex structure lives in application memory. And when using futexes, the majority of the synchronization operations are performed in user mode. A user-mode thread employs the futex wait system call only when it is likely that the program has to block for a longer time until the condition becomes true. When the condition comes true, futex wake operation will be used to wake up one or more threads waiting on that futex. This patch implements two futex operations: k_futex_wait and k_futex_wake. For k_futex_wait, the comparison with the expected value, and starting to sleep are performed atomically to prevent lost wake-ups. If different context changed futex's value after the calling use-mode thread decided to block himself based on the old value, the comparison will help observing the value change and will not start to sleep. And for k_futex_wake, it will wake at most num_waiters of the waiters that are sleeping on that futex. But no guarantees are made on which threads are woken, that means scheduling priority is not taken into consideration. Fixes: #14493. Signed-off-by: Wentong Wu <wentong.wu@intel.com>
2019-06-20 23:51:27 +08:00
woken++;
}
} while (thread && wake_all);
z_reschedule(&futex_data->lock, key);
return woken;
}
userspace: Support for split 64 bit arguments System call arguments, at the arch layer, are single words. So passing wider values requires splitting them into two registers at call time. This gets even more complicated for values (e.g k_timeout_t) that may have different sizes depending on configuration. This patch adds a feature to gen_syscalls.py to detect functions with wide arguments and automatically generates code to split/unsplit them. Unfortunately the current scheme of Z_SYSCALL_DECLARE_* macros won't work with functions like this, because for N arguments (our current maximum N is 10) there are 2^N possible configurations of argument widths. So this generates the complete functions for each handler and wrapper, effectively doing in python what was originally done in the preprocessor. Another complexity is that traditional the z_hdlr_*() function for a system call has taken the raw list of word arguments, which does not work when some of those arguments must be 64 bit types. So instead of using a single Z_SYSCALL_HANDLER macro, this splits the job of z_hdlr_*() into two steps: An automatically-generated unmarshalling function, z_mrsh_*(), which then calls a user-supplied verification function z_vrfy_*(). The verification function is typesafe, and is a simple C function with exactly the same argument and return signature as the syscall impl function. It is also not responsible for validating the pointers to the extra parameter array or a wide return value, that code gets automatically generated. This commit includes new vrfy/msrh handling for all syscalls invoked during CI runs. Future commits will port the less testable code. Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2019-08-07 04:34:31 +08:00
static inline int z_vrfy_k_futex_wake(struct k_futex *futex, bool wake_all)
kernel: add futex support A k_futex is a lightweight mutual exclusion primitive designed to minimize kernel involvement. Uncontended operation relies only on atomic access to shared memory. k_futex structure lives in application memory. And when using futexes, the majority of the synchronization operations are performed in user mode. A user-mode thread employs the futex wait system call only when it is likely that the program has to block for a longer time until the condition becomes true. When the condition comes true, futex wake operation will be used to wake up one or more threads waiting on that futex. This patch implements two futex operations: k_futex_wait and k_futex_wake. For k_futex_wait, the comparison with the expected value, and starting to sleep are performed atomically to prevent lost wake-ups. If different context changed futex's value after the calling use-mode thread decided to block himself based on the old value, the comparison will help observing the value change and will not start to sleep. And for k_futex_wake, it will wake at most num_waiters of the waiters that are sleeping on that futex. But no guarantees are made on which threads are woken, that means scheduling priority is not taken into consideration. Fixes: #14493. Signed-off-by: Wentong Wu <wentong.wu@intel.com>
2019-06-20 23:51:27 +08:00
{
if (Z_SYSCALL_MEMORY_WRITE(futex, sizeof(struct k_futex)) != 0) {
return -EACCES;
}
userspace: Support for split 64 bit arguments System call arguments, at the arch layer, are single words. So passing wider values requires splitting them into two registers at call time. This gets even more complicated for values (e.g k_timeout_t) that may have different sizes depending on configuration. This patch adds a feature to gen_syscalls.py to detect functions with wide arguments and automatically generates code to split/unsplit them. Unfortunately the current scheme of Z_SYSCALL_DECLARE_* macros won't work with functions like this, because for N arguments (our current maximum N is 10) there are 2^N possible configurations of argument widths. So this generates the complete functions for each handler and wrapper, effectively doing in python what was originally done in the preprocessor. Another complexity is that traditional the z_hdlr_*() function for a system call has taken the raw list of word arguments, which does not work when some of those arguments must be 64 bit types. So instead of using a single Z_SYSCALL_HANDLER macro, this splits the job of z_hdlr_*() into two steps: An automatically-generated unmarshalling function, z_mrsh_*(), which then calls a user-supplied verification function z_vrfy_*(). The verification function is typesafe, and is a simple C function with exactly the same argument and return signature as the syscall impl function. It is also not responsible for validating the pointers to the extra parameter array or a wide return value, that code gets automatically generated. This commit includes new vrfy/msrh handling for all syscalls invoked during CI runs. Future commits will port the less testable code. Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2019-08-07 04:34:31 +08:00
return z_impl_k_futex_wake(futex, wake_all);
kernel: add futex support A k_futex is a lightweight mutual exclusion primitive designed to minimize kernel involvement. Uncontended operation relies only on atomic access to shared memory. k_futex structure lives in application memory. And when using futexes, the majority of the synchronization operations are performed in user mode. A user-mode thread employs the futex wait system call only when it is likely that the program has to block for a longer time until the condition becomes true. When the condition comes true, futex wake operation will be used to wake up one or more threads waiting on that futex. This patch implements two futex operations: k_futex_wait and k_futex_wake. For k_futex_wait, the comparison with the expected value, and starting to sleep are performed atomically to prevent lost wake-ups. If different context changed futex's value after the calling use-mode thread decided to block himself based on the old value, the comparison will help observing the value change and will not start to sleep. And for k_futex_wake, it will wake at most num_waiters of the waiters that are sleeping on that futex. But no guarantees are made on which threads are woken, that means scheduling priority is not taken into consideration. Fixes: #14493. Signed-off-by: Wentong Wu <wentong.wu@intel.com>
2019-06-20 23:51:27 +08:00
}
userspace: Support for split 64 bit arguments System call arguments, at the arch layer, are single words. So passing wider values requires splitting them into two registers at call time. This gets even more complicated for values (e.g k_timeout_t) that may have different sizes depending on configuration. This patch adds a feature to gen_syscalls.py to detect functions with wide arguments and automatically generates code to split/unsplit them. Unfortunately the current scheme of Z_SYSCALL_DECLARE_* macros won't work with functions like this, because for N arguments (our current maximum N is 10) there are 2^N possible configurations of argument widths. So this generates the complete functions for each handler and wrapper, effectively doing in python what was originally done in the preprocessor. Another complexity is that traditional the z_hdlr_*() function for a system call has taken the raw list of word arguments, which does not work when some of those arguments must be 64 bit types. So instead of using a single Z_SYSCALL_HANDLER macro, this splits the job of z_hdlr_*() into two steps: An automatically-generated unmarshalling function, z_mrsh_*(), which then calls a user-supplied verification function z_vrfy_*(). The verification function is typesafe, and is a simple C function with exactly the same argument and return signature as the syscall impl function. It is also not responsible for validating the pointers to the extra parameter array or a wide return value, that code gets automatically generated. This commit includes new vrfy/msrh handling for all syscalls invoked during CI runs. Future commits will port the less testable code. Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2019-08-07 04:34:31 +08:00
#include <syscalls/k_futex_wake_mrsh.c>
kernel: add futex support A k_futex is a lightweight mutual exclusion primitive designed to minimize kernel involvement. Uncontended operation relies only on atomic access to shared memory. k_futex structure lives in application memory. And when using futexes, the majority of the synchronization operations are performed in user mode. A user-mode thread employs the futex wait system call only when it is likely that the program has to block for a longer time until the condition becomes true. When the condition comes true, futex wake operation will be used to wake up one or more threads waiting on that futex. This patch implements two futex operations: k_futex_wait and k_futex_wake. For k_futex_wait, the comparison with the expected value, and starting to sleep are performed atomically to prevent lost wake-ups. If different context changed futex's value after the calling use-mode thread decided to block himself based on the old value, the comparison will help observing the value change and will not start to sleep. And for k_futex_wake, it will wake at most num_waiters of the waiters that are sleeping on that futex. But no guarantees are made on which threads are woken, that means scheduling priority is not taken into consideration. Fixes: #14493. Signed-off-by: Wentong Wu <wentong.wu@intel.com>
2019-06-20 23:51:27 +08:00
int z_impl_k_futex_wait(struct k_futex *futex, int expected, s32_t timeout)
{
int ret;
k_spinlock_key_t key;
struct z_futex_data *futex_data;
futex_data = k_futex_find_data(futex);
if (futex_data == NULL) {
return -EINVAL;
}
key = k_spin_lock(&futex_data->lock);
if (atomic_get(&futex->val) != (atomic_val_t)expected) {
k_spin_unlock(&futex_data->lock, key);
return -EAGAIN;
}
ret = z_pend_curr(&futex_data->lock,
key, &futex_data->wait_q, timeout);
if (ret == -EAGAIN) {
ret = -ETIMEDOUT;
}
return ret;
}
kernel: syscalls: Whitespace fixups The semi-automated API changes weren't checkpatch aware. Fix up whitespace warnings that snuck into the previous patches. Really this should be squashed, but that's somewhat difficult given the structure of the series. Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2019-08-14 03:58:38 +08:00
static inline int z_vrfy_k_futex_wait(struct k_futex *futex, int expected,
s32_t timeout)
kernel: add futex support A k_futex is a lightweight mutual exclusion primitive designed to minimize kernel involvement. Uncontended operation relies only on atomic access to shared memory. k_futex structure lives in application memory. And when using futexes, the majority of the synchronization operations are performed in user mode. A user-mode thread employs the futex wait system call only when it is likely that the program has to block for a longer time until the condition becomes true. When the condition comes true, futex wake operation will be used to wake up one or more threads waiting on that futex. This patch implements two futex operations: k_futex_wait and k_futex_wake. For k_futex_wait, the comparison with the expected value, and starting to sleep are performed atomically to prevent lost wake-ups. If different context changed futex's value after the calling use-mode thread decided to block himself based on the old value, the comparison will help observing the value change and will not start to sleep. And for k_futex_wake, it will wake at most num_waiters of the waiters that are sleeping on that futex. But no guarantees are made on which threads are woken, that means scheduling priority is not taken into consideration. Fixes: #14493. Signed-off-by: Wentong Wu <wentong.wu@intel.com>
2019-06-20 23:51:27 +08:00
{
if (Z_SYSCALL_MEMORY_WRITE(futex, sizeof(struct k_futex)) != 0) {
return -EACCES;
}
userspace: Support for split 64 bit arguments System call arguments, at the arch layer, are single words. So passing wider values requires splitting them into two registers at call time. This gets even more complicated for values (e.g k_timeout_t) that may have different sizes depending on configuration. This patch adds a feature to gen_syscalls.py to detect functions with wide arguments and automatically generates code to split/unsplit them. Unfortunately the current scheme of Z_SYSCALL_DECLARE_* macros won't work with functions like this, because for N arguments (our current maximum N is 10) there are 2^N possible configurations of argument widths. So this generates the complete functions for each handler and wrapper, effectively doing in python what was originally done in the preprocessor. Another complexity is that traditional the z_hdlr_*() function for a system call has taken the raw list of word arguments, which does not work when some of those arguments must be 64 bit types. So instead of using a single Z_SYSCALL_HANDLER macro, this splits the job of z_hdlr_*() into two steps: An automatically-generated unmarshalling function, z_mrsh_*(), which then calls a user-supplied verification function z_vrfy_*(). The verification function is typesafe, and is a simple C function with exactly the same argument and return signature as the syscall impl function. It is also not responsible for validating the pointers to the extra parameter array or a wide return value, that code gets automatically generated. This commit includes new vrfy/msrh handling for all syscalls invoked during CI runs. Future commits will port the less testable code. Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2019-08-07 04:34:31 +08:00
return z_impl_k_futex_wait(futex, expected, timeout);
kernel: add futex support A k_futex is a lightweight mutual exclusion primitive designed to minimize kernel involvement. Uncontended operation relies only on atomic access to shared memory. k_futex structure lives in application memory. And when using futexes, the majority of the synchronization operations are performed in user mode. A user-mode thread employs the futex wait system call only when it is likely that the program has to block for a longer time until the condition becomes true. When the condition comes true, futex wake operation will be used to wake up one or more threads waiting on that futex. This patch implements two futex operations: k_futex_wait and k_futex_wake. For k_futex_wait, the comparison with the expected value, and starting to sleep are performed atomically to prevent lost wake-ups. If different context changed futex's value after the calling use-mode thread decided to block himself based on the old value, the comparison will help observing the value change and will not start to sleep. And for k_futex_wake, it will wake at most num_waiters of the waiters that are sleeping on that futex. But no guarantees are made on which threads are woken, that means scheduling priority is not taken into consideration. Fixes: #14493. Signed-off-by: Wentong Wu <wentong.wu@intel.com>
2019-06-20 23:51:27 +08:00
}
userspace: Support for split 64 bit arguments System call arguments, at the arch layer, are single words. So passing wider values requires splitting them into two registers at call time. This gets even more complicated for values (e.g k_timeout_t) that may have different sizes depending on configuration. This patch adds a feature to gen_syscalls.py to detect functions with wide arguments and automatically generates code to split/unsplit them. Unfortunately the current scheme of Z_SYSCALL_DECLARE_* macros won't work with functions like this, because for N arguments (our current maximum N is 10) there are 2^N possible configurations of argument widths. So this generates the complete functions for each handler and wrapper, effectively doing in python what was originally done in the preprocessor. Another complexity is that traditional the z_hdlr_*() function for a system call has taken the raw list of word arguments, which does not work when some of those arguments must be 64 bit types. So instead of using a single Z_SYSCALL_HANDLER macro, this splits the job of z_hdlr_*() into two steps: An automatically-generated unmarshalling function, z_mrsh_*(), which then calls a user-supplied verification function z_vrfy_*(). The verification function is typesafe, and is a simple C function with exactly the same argument and return signature as the syscall impl function. It is also not responsible for validating the pointers to the extra parameter array or a wide return value, that code gets automatically generated. This commit includes new vrfy/msrh handling for all syscalls invoked during CI runs. Future commits will port the less testable code. Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2019-08-07 04:34:31 +08:00
#include <syscalls/k_futex_wait_mrsh.c>