Skip to content

Latest commit

 

History

History
64 lines (36 loc) · 3.75 KB

README.md

File metadata and controls

64 lines (36 loc) · 3.75 KB

virtio-fpga-bridge

A platform for emulating Virtio frontend and backend bridging with FPGAs.

Introduction

With the Qemu-HDL Cosim framework, it is now possible to run the full system emulation of BM-Hive (Alibaba Cloud's X-Dragon, "Shenlong" in Chinese) on a single x86 PC.

BM-Hive architecture

BM-Hive architecture

virtio-fpga-bridge

The Virtio frontend runs on the 1st x86(Qemu), and the Virtio backend runs on the 2nd x86(Qemu). FPGA(HDL Sim) plays the role of bridging/synchronizing the frontend and backend, with shadow vrings and DMAs.

virtio-fpga-bridge architecture

There are two PCIe-EP IPs instantiated in the FPGA logic. The EP facing the 1st x86 is an implementation of the standard Virtio protocol. And there is no driver modification needed in the 1st x86 OS. The EP interfacing with the 2nd x86 is an implementation of a custom protocol, handling CSR/Mailbox registers and DMAs. And user space drivers, such as ixy, DPDK and SPDK, are needed for this interface.

Discussion

System topology

With Virtio's Virtqueue handling implemented in FPGA, there are many possible combinations of how the system can be constructed.

Topology comparison

HW limitations

BM-Hive requires special HW PCB board design, on which the x86 system is not available for many PCIe FPGA development boards. Adding a soft-core CPU inside the FPGA is an interesting option.

SoC FPGA alternative

Backend custom protocol design

The backend protocol of the FPGA bridge follows the same design principle of vhost-user protocol, which is customized to run solely over PCIe.

Custom protocol design following vhost-user

Related Projects

qemu-hdl-cosim

virtio-fpga

ixy Virtio driver with PCIe MMIO

Useful Links

High-density Multi-tenant Bare-metal Cloud

阿里云神龙架构入选ASPLOS 20'论文解读

阿里云弹性裸金属服务器-神龙架构(X-Dragon)揭秘

Vhost-user Protocol

Red Hat Blog: A journey to the vhost-users realm

Red Hat Blog: Virtio-networking series 2019

Red Hat Blog: Blogs on Virtualization

KVM Forum 2019: VirtIO without the Virt

Project ACRN: Virtio devices high-level design

VIRTIO-USER: A New Versatile Channel for Kernel-Bypass Networks

IOVTee: A Fast and Pragmatic Software-based Zero-copy/Pass-through Mechanism for NFV-nodes