[lustre-discuss] Lustre Clients on KVM Virtual Guests on CentOS 7

[Ault, James R (Contractor)]

Does anyone on this list have experience running lustre clients on Virtual Guests running in a QEMU/KVM environment (using CentOS 7)?
The desired configuration for the base host is Mellanox Connect-X 5 for 100GB Ethernet, but the KVM Guests are running over a Bridged Ethernet using the virtio from KVM.

The device shows on the Virtual Guest as eth0, and we have not been successful enabling RDMA over this Linux Ethernet Bridge device so far.

Other KVM networking options for this network device displayed in virt-manager are "e1000", "rtl8139" and "Hypervisor default" instead of "virtio".

Another possibility could be to use an LNET router to allow these Virtual Guests to talk to a Lustre server over the TCP driver.

Servers: CentOS 7.6, Lustre 2.12.1, ZFS devices
Clients: CentOS 7.6, Lustre 2.12.2


Error messages displayed include:

# lnetctl add --net o2ib --if eth0
add:
   net:
      errno: -100
      descr: "cannot add network: Network is down"
# lnetctl net show
net:
   - net type: lo
     local NI(s):
         - nid: 0 at lo
          status: up

and nothing else.

Thanks in advance for any helpful recommendations.

-Jim Ault, HPC Future Studies, Naval Nuclear Laboratory