acrn-hypervisor/devicemodel/include/iothread.h

55 lines
1.3 KiB
C
Raw Normal View History

/* Copyright (C) 2022 Intel Corporation.
*
* SPDX-License-Identifier: BSD-3-Clause
*
*/
#ifndef _iothread_CTX_H_
#define _iothread_CTX_H_
dm: improve the flexibility of the iothread support Prior to this patch, one single iothread instance is created and initialized in the `main` function. This single iothread monitors all the registered fds and handles all the corresponding requests. It leads to the limited flexibility of the iothread support. To improve the flexibility of the iothread support, this patch does: - add the support of multiple iothread instances. `iothread_create` is introduced to create a certain number of iothread instances. It shall be called at first by each virtual device owner (such as virtio-blk BE) on initialization phase. Then, `iothread_add` can be called to add the to be monitored fd to the specified iothread. - update virtio-blk BE to let the acrn-dm option `iothread` accept a number as the number of iothread instances to be created. If `iothread` is contained in the parameters, but the number is not specified, one iothread instance would be created by default. Examples to specify the number of iothread instances: 1. Create 2 iothread instances `add_virtual_device 9 virtio-blk iothread=2,mq=2,/dev/nvme1n1,writeback,aio=io_uring` 2. Create 1 iothread instances (by default) `add_virtual_device 9 virtio-blk iothread,mq=2,/dev/nvme1n1,writeback,aio=io_uring` - update virtio-blk BE to separate the request handling of different virtqueues to different iothreads. The request from one or more virtqueues can be handled in one iothread. The mapping between virtqueues and iothreads is based on round robin. v1 -> v2: * add a mutex to protect the free ioctx slot allocation Tracked-On: #8612 Signed-off-by: Shiqing Gao <shiqing.gao@intel.com> Acked-by: Wang, Yu1 <yu1.wang@intel.com>
2023-09-06 00:58:03 +08:00
#define IOTHREAD_NUM 40
/*
* The pthread_setname_np() function can be used to set a unique name for a thread,
* which can be useful for debugging multithreaded applications.
* The thread name is a meaningful C language string,
* whose length is restricted to 16 characters, including the terminating null byte ('\0').
*/
#define PTHREAD_NAME_MAX_LEN 16
struct iothread_mevent {
void (*run)(void *);
void *arg;
int fd;
};
dm: improve the flexibility of the iothread support Prior to this patch, one single iothread instance is created and initialized in the `main` function. This single iothread monitors all the registered fds and handles all the corresponding requests. It leads to the limited flexibility of the iothread support. To improve the flexibility of the iothread support, this patch does: - add the support of multiple iothread instances. `iothread_create` is introduced to create a certain number of iothread instances. It shall be called at first by each virtual device owner (such as virtio-blk BE) on initialization phase. Then, `iothread_add` can be called to add the to be monitored fd to the specified iothread. - update virtio-blk BE to let the acrn-dm option `iothread` accept a number as the number of iothread instances to be created. If `iothread` is contained in the parameters, but the number is not specified, one iothread instance would be created by default. Examples to specify the number of iothread instances: 1. Create 2 iothread instances `add_virtual_device 9 virtio-blk iothread=2,mq=2,/dev/nvme1n1,writeback,aio=io_uring` 2. Create 1 iothread instances (by default) `add_virtual_device 9 virtio-blk iothread,mq=2,/dev/nvme1n1,writeback,aio=io_uring` - update virtio-blk BE to separate the request handling of different virtqueues to different iothreads. The request from one or more virtqueues can be handled in one iothread. The mapping between virtqueues and iothreads is based on round robin. v1 -> v2: * add a mutex to protect the free ioctx slot allocation Tracked-On: #8612 Signed-off-by: Shiqing Gao <shiqing.gao@intel.com> Acked-by: Wang, Yu1 <yu1.wang@intel.com>
2023-09-06 00:58:03 +08:00
struct iothread_ctx {
pthread_t tid;
int epfd;
bool started;
pthread_mutex_t mtx;
int idx;
dm: update the `iothread` option to specify the CPU affinity This patch updates the `iothread` option to specify the CPU affinity of the iothread. Setting the iothread's CPU affinity could benefit the Service VM's CPU utilization when Service VM owns limited dedicated CPUs. It could be helpful to ensure the I/O mediator Quality of Service (QoS). Once the performance tuning is done, the specific CPU affinity config could pass to acrn-dm directly, letting the deployment more easily. The format looks like below: iothread=<num_iothread>@<cpu_affinity> "@" is used to separate the following two settings: - the number of iothread instances - the CPU affinity settings for each iothread instance. The format of `cpu_affinity` looks like below: <cpu_affinity_0>/<cpu_affinity_1>/<cpu_affinity_2>/... 1. "/" is used to separate the CPU affinity setting for each iothread instance (sequentially). 2. char '*' can be used to skip the setting for the specific iothread instance. 3. the number of cpu_affinity_x vs. the number of iothread instances - If # of cpu_affinity_x is less than # of iothread instances, no CPU affinity settings for the last few iothread instances. - If # of cpu_affinity_x is more than # of iothread instances, the extra cpu_affinity_x are discarded. 4. ":" is used to separate different CPU cores for each CPU affinity setting. Examples to specify the CPU affinity of the iothread: 1. iothread=3@0:1:2/0:1 `add_virtual_device 9 virtio-blk iothread=3@0:1:2/0:1,mq=3,/dev/nvme1n1` a) 3 iothread instances are created. b) CPU affinity of iothread instances for this virtio-blk device: - 1st iothread instance <-> pins to Service VM CPU 0,1,2 - 2nd iothread instance <-> pins to Service VM CPU 0,1 - 3rd iothread instance <-> No CPU affinity settings 2. iothread=3@0/*/1 `add_virtual_device 9 virtio-blk iothread=3@0/*/1,mq=3,/dev/nvme1n1` a) 3 iothread instances are created. b) CPU affinity of iothread instances for this virtio-blk device: - 1st iothread instance <-> pins to Service VM CPU 0 - 2nd iothread instance <-> No CPU affinity settings - 3rd iothread instance <-> pins to Service VM CPU 1 v1 -> v2: * encapsulate one API in iothread.c to parse the iothread options, so that other BE can also use it. v2 -> v3: * introduce one API iothread_free_options to free the elements that are allocated dynamically in iothread_parse_options(). Tracked-On: #8612 Signed-off-by: Shiqing Gao <shiqing.gao@intel.com> Acked-by: Wang, Yu1 <yu1.wang@intel.com>
2023-10-31 00:45:17 +08:00
cpu_set_t cpuset;
char name[PTHREAD_NAME_MAX_LEN];
dm: improve the flexibility of the iothread support Prior to this patch, one single iothread instance is created and initialized in the `main` function. This single iothread monitors all the registered fds and handles all the corresponding requests. It leads to the limited flexibility of the iothread support. To improve the flexibility of the iothread support, this patch does: - add the support of multiple iothread instances. `iothread_create` is introduced to create a certain number of iothread instances. It shall be called at first by each virtual device owner (such as virtio-blk BE) on initialization phase. Then, `iothread_add` can be called to add the to be monitored fd to the specified iothread. - update virtio-blk BE to let the acrn-dm option `iothread` accept a number as the number of iothread instances to be created. If `iothread` is contained in the parameters, but the number is not specified, one iothread instance would be created by default. Examples to specify the number of iothread instances: 1. Create 2 iothread instances `add_virtual_device 9 virtio-blk iothread=2,mq=2,/dev/nvme1n1,writeback,aio=io_uring` 2. Create 1 iothread instances (by default) `add_virtual_device 9 virtio-blk iothread,mq=2,/dev/nvme1n1,writeback,aio=io_uring` - update virtio-blk BE to separate the request handling of different virtqueues to different iothreads. The request from one or more virtqueues can be handled in one iothread. The mapping between virtqueues and iothreads is based on round robin. v1 -> v2: * add a mutex to protect the free ioctx slot allocation Tracked-On: #8612 Signed-off-by: Shiqing Gao <shiqing.gao@intel.com> Acked-by: Wang, Yu1 <yu1.wang@intel.com>
2023-09-06 00:58:03 +08:00
};
dm: update the `iothread` option to specify the CPU affinity This patch updates the `iothread` option to specify the CPU affinity of the iothread. Setting the iothread's CPU affinity could benefit the Service VM's CPU utilization when Service VM owns limited dedicated CPUs. It could be helpful to ensure the I/O mediator Quality of Service (QoS). Once the performance tuning is done, the specific CPU affinity config could pass to acrn-dm directly, letting the deployment more easily. The format looks like below: iothread=<num_iothread>@<cpu_affinity> "@" is used to separate the following two settings: - the number of iothread instances - the CPU affinity settings for each iothread instance. The format of `cpu_affinity` looks like below: <cpu_affinity_0>/<cpu_affinity_1>/<cpu_affinity_2>/... 1. "/" is used to separate the CPU affinity setting for each iothread instance (sequentially). 2. char '*' can be used to skip the setting for the specific iothread instance. 3. the number of cpu_affinity_x vs. the number of iothread instances - If # of cpu_affinity_x is less than # of iothread instances, no CPU affinity settings for the last few iothread instances. - If # of cpu_affinity_x is more than # of iothread instances, the extra cpu_affinity_x are discarded. 4. ":" is used to separate different CPU cores for each CPU affinity setting. Examples to specify the CPU affinity of the iothread: 1. iothread=3@0:1:2/0:1 `add_virtual_device 9 virtio-blk iothread=3@0:1:2/0:1,mq=3,/dev/nvme1n1` a) 3 iothread instances are created. b) CPU affinity of iothread instances for this virtio-blk device: - 1st iothread instance <-> pins to Service VM CPU 0,1,2 - 2nd iothread instance <-> pins to Service VM CPU 0,1 - 3rd iothread instance <-> No CPU affinity settings 2. iothread=3@0/*/1 `add_virtual_device 9 virtio-blk iothread=3@0/*/1,mq=3,/dev/nvme1n1` a) 3 iothread instances are created. b) CPU affinity of iothread instances for this virtio-blk device: - 1st iothread instance <-> pins to Service VM CPU 0 - 2nd iothread instance <-> No CPU affinity settings - 3rd iothread instance <-> pins to Service VM CPU 1 v1 -> v2: * encapsulate one API in iothread.c to parse the iothread options, so that other BE can also use it. v2 -> v3: * introduce one API iothread_free_options to free the elements that are allocated dynamically in iothread_parse_options(). Tracked-On: #8612 Signed-off-by: Shiqing Gao <shiqing.gao@intel.com> Acked-by: Wang, Yu1 <yu1.wang@intel.com>
2023-10-31 00:45:17 +08:00
struct iothreads_option {
char tag[PTHREAD_NAME_MAX_LEN];
int num;
cpu_set_t *cpusets;
};
dm: improve the flexibility of the iothread support Prior to this patch, one single iothread instance is created and initialized in the `main` function. This single iothread monitors all the registered fds and handles all the corresponding requests. It leads to the limited flexibility of the iothread support. To improve the flexibility of the iothread support, this patch does: - add the support of multiple iothread instances. `iothread_create` is introduced to create a certain number of iothread instances. It shall be called at first by each virtual device owner (such as virtio-blk BE) on initialization phase. Then, `iothread_add` can be called to add the to be monitored fd to the specified iothread. - update virtio-blk BE to let the acrn-dm option `iothread` accept a number as the number of iothread instances to be created. If `iothread` is contained in the parameters, but the number is not specified, one iothread instance would be created by default. Examples to specify the number of iothread instances: 1. Create 2 iothread instances `add_virtual_device 9 virtio-blk iothread=2,mq=2,/dev/nvme1n1,writeback,aio=io_uring` 2. Create 1 iothread instances (by default) `add_virtual_device 9 virtio-blk iothread,mq=2,/dev/nvme1n1,writeback,aio=io_uring` - update virtio-blk BE to separate the request handling of different virtqueues to different iothreads. The request from one or more virtqueues can be handled in one iothread. The mapping between virtqueues and iothreads is based on round robin. v1 -> v2: * add a mutex to protect the free ioctx slot allocation Tracked-On: #8612 Signed-off-by: Shiqing Gao <shiqing.gao@intel.com> Acked-by: Wang, Yu1 <yu1.wang@intel.com>
2023-09-06 00:58:03 +08:00
struct iothreads_info {
struct iothread_ctx *ioctx_base;
int num;
};
int iothread_add(struct iothread_ctx *ioctx_x, int fd, struct iothread_mevent *aevt);
int iothread_del(struct iothread_ctx *ioctx_x, int fd);
void iothread_deinit(void);
dm: update the `iothread` option to specify the CPU affinity This patch updates the `iothread` option to specify the CPU affinity of the iothread. Setting the iothread's CPU affinity could benefit the Service VM's CPU utilization when Service VM owns limited dedicated CPUs. It could be helpful to ensure the I/O mediator Quality of Service (QoS). Once the performance tuning is done, the specific CPU affinity config could pass to acrn-dm directly, letting the deployment more easily. The format looks like below: iothread=<num_iothread>@<cpu_affinity> "@" is used to separate the following two settings: - the number of iothread instances - the CPU affinity settings for each iothread instance. The format of `cpu_affinity` looks like below: <cpu_affinity_0>/<cpu_affinity_1>/<cpu_affinity_2>/... 1. "/" is used to separate the CPU affinity setting for each iothread instance (sequentially). 2. char '*' can be used to skip the setting for the specific iothread instance. 3. the number of cpu_affinity_x vs. the number of iothread instances - If # of cpu_affinity_x is less than # of iothread instances, no CPU affinity settings for the last few iothread instances. - If # of cpu_affinity_x is more than # of iothread instances, the extra cpu_affinity_x are discarded. 4. ":" is used to separate different CPU cores for each CPU affinity setting. Examples to specify the CPU affinity of the iothread: 1. iothread=3@0:1:2/0:1 `add_virtual_device 9 virtio-blk iothread=3@0:1:2/0:1,mq=3,/dev/nvme1n1` a) 3 iothread instances are created. b) CPU affinity of iothread instances for this virtio-blk device: - 1st iothread instance <-> pins to Service VM CPU 0,1,2 - 2nd iothread instance <-> pins to Service VM CPU 0,1 - 3rd iothread instance <-> No CPU affinity settings 2. iothread=3@0/*/1 `add_virtual_device 9 virtio-blk iothread=3@0/*/1,mq=3,/dev/nvme1n1` a) 3 iothread instances are created. b) CPU affinity of iothread instances for this virtio-blk device: - 1st iothread instance <-> pins to Service VM CPU 0 - 2nd iothread instance <-> No CPU affinity settings - 3rd iothread instance <-> pins to Service VM CPU 1 v1 -> v2: * encapsulate one API in iothread.c to parse the iothread options, so that other BE can also use it. v2 -> v3: * introduce one API iothread_free_options to free the elements that are allocated dynamically in iothread_parse_options(). Tracked-On: #8612 Signed-off-by: Shiqing Gao <shiqing.gao@intel.com> Acked-by: Wang, Yu1 <yu1.wang@intel.com>
2023-10-31 00:45:17 +08:00
struct iothread_ctx *iothread_create(struct iothreads_option *iothr_opt);
int iothread_parse_options(char *str, struct iothreads_option *iothr_opt);
void iothread_free_options(struct iothreads_option *iothr_opt);
#endif