uacce: Add documents for uacce
Uacce (Unified/User-space-access-intended Accelerator Framework) is a kernel module targets to provide Shared Virtual Addressing (SVA) between the accelerator and process. This patch add document to explain how it works. Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com> Signed-off-by: Kenneth Lee <liguozhu@hisilicon.com> Signed-off-by: Zaibo Xu <xuzaibo@huawei.com> Signed-off-by: Zhou Wang <wangzhou1@hisilicon.com> Signed-off-by: Zhangfei Gao <zhangfei.gao@linaro.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This commit is contained in:
Kenneth Lee
Herbert Xu
parent
41ccdbfd54
commit
aa017ab97a
|
|
@ -0,0 +1,176 @@
|
|||
.. SPDX-License-Identifier: GPL-2.0
|
||||
|
||||
Introduction of Uacce
|
||||
---------------------
|
||||
|
||||
Uacce (Unified/User-space-access-intended Accelerator Framework) targets to
|
||||
provide Shared Virtual Addressing (SVA) between accelerators and processes.
|
||||
So accelerator can access any data structure of the main cpu.
|
||||
This differs from the data sharing between cpu and io device, which share
|
||||
only data content rather than address.
|
||||
Because of the unified address, hardware and user space of process can
|
||||
share the same virtual address in the communication.
|
||||
Uacce takes the hardware accelerator as a heterogeneous processor, while
|
||||
IOMMU share the same CPU page tables and as a result the same translation
|
||||
from va to pa.
|
||||
|
||||
::
|
||||
|
||||
__________________________ __________________________
|
||||
| | | |
|
||||
| User application (CPU) | | Hardware Accelerator |
|
||||
|__________________________| |__________________________|
|
||||
|
||||
| |
|
||||
| va | va
|
||||
V V
|
||||
__________ __________
|
||||
| | | |
|
||||
| MMU | | IOMMU |
|
||||
|__________| |__________|
|
||||
| |
|
||||
| |
|
||||
V pa V pa
|
||||
_______________________________________
|
||||
| |
|
||||
| Memory |
|
||||
|_______________________________________|
|
||||
|
||||
|
||||
|
||||
Architecture
|
||||
------------
|
||||
|
||||
Uacce is the kernel module, taking charge of iommu and address sharing.
|
||||
The user drivers and libraries are called WarpDrive.
|
||||
|
||||
The uacce device, built around the IOMMU SVA API, can access multiple
|
||||
address spaces, including the one without PASID.
|
||||
|
||||
A virtual concept, queue, is used for the communication. It provides a
|
||||
FIFO-like interface. And it maintains a unified address space between the
|
||||
application and all involved hardware.
|
||||
|
||||
::
|
||||
|
||||
___________________ ________________
|
||||
| | user API | |
|
||||
| WarpDrive library | ------------> | user driver |
|
||||
|___________________| |________________|
|
||||
| |
|
||||
| |
|
||||
| queue fd |
|
||||
| |
|
||||
| |
|
||||
v |
|
||||
___________________ _________ |
|
||||
| | | | | mmap memory
|
||||
| Other framework | | uacce | | r/w interface
|
||||
| crypto/nic/others | |_________| |
|
||||
|___________________| |
|
||||
| | |
|
||||
| register | register |
|
||||
| | |
|
||||
| | |
|
||||
| _________________ __________ |
|
||||
| | | | | |
|
||||
------------- | Device Driver | | IOMMU | |
|
||||
|_________________| |__________| |
|
||||
| |
|
||||
| V
|
||||
| ___________________
|
||||
| | |
|
||||
-------------------------- | Device(Hardware) |
|
||||
|___________________|
|
||||
|
||||
|
||||
How does it work
|
||||
----------------
|
||||
|
||||
Uacce uses mmap and IOMMU to play the trick.
|
||||
|
||||
Uacce creates a chrdev for every device registered to it. New queue is
|
||||
created when user application open the chrdev. The file descriptor is used
|
||||
as the user handle of the queue.
|
||||
The accelerator device present itself as an Uacce object, which exports as
|
||||
a chrdev to the user space. The user application communicates with the
|
||||
hardware by ioctl (as control path) or share memory (as data path).
|
||||
|
||||
The control path to the hardware is via file operation, while data path is
|
||||
via mmap space of the queue fd.
|
||||
|
||||
The queue file address space:
|
||||
|
||||
::
|
||||
|
||||
/**
|
||||
* enum uacce_qfrt: qfrt type
|
||||
* @UACCE_QFRT_MMIO: device mmio region
|
||||
* @UACCE_QFRT_DUS: device user share region
|
||||
*/
|
||||
enum uacce_qfrt {
|
||||
UACCE_QFRT_MMIO = 0,
|
||||
UACCE_QFRT_DUS = 1,
|
||||
};
|
||||
|
||||
All regions are optional and differ from device type to type.
|
||||
Each region can be mmapped only once, otherwise -EEXIST returns.
|
||||
|
||||
The device mmio region is mapped to the hardware mmio space. It is generally
|
||||
used for doorbell or other notification to the hardware. It is not fast enough
|
||||
as data channel.
|
||||
|
||||
The device user share region is used for share data buffer between user process
|
||||
and device.
|
||||
|
||||
|
||||
The Uacce register API
|
||||
----------------------
|
||||
|
||||
The register API is defined in uacce.h.
|
||||
|
||||
::
|
||||
|
||||
struct uacce_interface {
|
||||
char name[UACCE_MAX_NAME_SIZE];
|
||||
unsigned int flags;
|
||||
const struct uacce_ops *ops;
|
||||
};
|
||||
|
||||
According to the IOMMU capability, uacce_interface flags can be:
|
||||
|
||||
::
|
||||
|
||||
/**
|
||||
* UACCE Device flags:
|
||||
* UACCE_DEV_SVA: Shared Virtual Addresses
|
||||
* Support PASID
|
||||
* Support device page faults (PCI PRI or SMMU Stall)
|
||||
*/
|
||||
#define UACCE_DEV_SVA BIT(0)
|
||||
|
||||
struct uacce_device *uacce_alloc(struct device *parent,
|
||||
struct uacce_interface *interface);
|
||||
int uacce_register(struct uacce_device *uacce);
|
||||
void uacce_remove(struct uacce_device *uacce);
|
||||
|
||||
uacce_register results can be:
|
||||
|
||||
a. If uacce module is not compiled, ERR_PTR(-ENODEV)
|
||||
|
||||
b. Succeed with the desired flags
|
||||
|
||||
c. Succeed with the negotiated flags, for example
|
||||
|
||||
uacce_interface.flags = UACCE_DEV_SVA but uacce->flags = ~UACCE_DEV_SVA
|
||||
|
||||
So user driver need check return value as well as the negotiated uacce->flags.
|
||||
|
||||
|
||||
The user driver
|
||||
---------------
|
||||
|
||||
The queue file mmap space will need a user driver to wrap the communication
|
||||
protocol. Uacce provides some attributes in sysfs for the user driver to
|
||||
match the right accelerator accordingly.
|
||||
More details in Documentation/ABI/testing/sysfs-driver-uacce.
|
||||
Loading…
Reference in New Issue