Sr-Iov: Configuration And Use Case Examples; Linux Use Case - Broadcom NetXreme-C User Manual

NetXtreme-E User's Manual
System 1
PxPy: ifconfig PxPy 1.1.1.4/24
ip link add vxlan10 type vxlan id 10 group 239.0.0.10
dev PxPy dstport 4789
ip addr add 192.168.1.5/24 broadcast 192.168.1.255
dev vxlan10
ip link set vxlan10 up
ip –d link show vxlan10
Ping 192.168.1.10
Note: x represents the PCIe bus number of the physical adapter found in the system. y represents the port
number on the physical adapter.

SR-IOV: Configuration and Use Case Examples

SR-IOV can be configured, enabled, and used on 10Gb and 25Gb Broadcom NetExtreme-E NICs.

Linux Use Case

1. Enable SR-IOV in the NIC cards:
a. SR-IOV in the NIC card can be enabled using the HII menu. During system boot, access the system
BIOS -> Device Settings -> NetXtreme-E NIC -> Device Level Configuration.
b. Set the Virtualization mode to SR-IOV.
c. Set the number of virtual functions per physical function.
d. Set the number of MSI-X vectors per the VF and Max number of physical function MSI-X vectors. If the
VF is running out of resources, balance the number of MSI-X vectors per VM using CCM.
2. Enable virtualization in the BIOS:
a. During system boot, enter the system BIOS -> Processor settings -> Virtualization Technologies and
set it to Enabled.
b. During system boot, enter the system BIOS -> Integrated Devices -> SR-IOV Global and set it to
Enabled.
3. Install the desired Linux version with Virtualization enabled (libvirt and Qemu).
4. Enable the iommu kernel parameter.
a. The IOMMU kernel parameter is enabled by editing
-o /boot/grub2/grub.cfg
mkconfig -o /etc/grub2-efi.cfg
Linuxefi /vmlinuz-3.10.0-229.el7.x86_64 root=/dev/mapper/rhel-root ro rd.lvm.lv=rhel/swap
crashkernel=auto rd.lvm.lv=rhel/root rhgb intel_iommu=on quiet LANG=en_US.UTF.8
5. Install bnxt_en driver:
a. Copy the
bnxt_en
February 26, 2018 • NetXtreme-E-UG100
Table 36: VXLAN Command and Configuration Examples
for legacy mode. For UEFI mode, edit
. Refer to the following example:
driver on to the OS and run
SR-IOV: Configuration and Use Case Examples
System 2
PxPy: ifconfig PxPy 1.1.1.2/24
ip link add vxlan10 type vxlan id 10 group 239.0.0.10
dev PxPy dstport 4789
ip addr add 192.168.1.10/24 broadcast 192.168.1.255
dev vxlan10
ip link set vxlan10 up
ifconfig vxlan10 (MTU 1450) (SUSE and RHEL)
/etc/default/grub.cfg
/etc/default/grub.cfg
make; make install; modprobe bnxt_en
and running
grub2-mkconfig
and run
grub2-
.
Page 57