Installing the VMware Driver

The ESX drivers are provided in VMware standard VIB format and can be downloaded from VMware.com.
  1. To install the Ethernet and RDMA driver, issue the following commands:
    $ esxcli software vib install -v <bnxtnet>-<driver version>.vib $ esxcli software vib install -v <bnxtroce>-<driver version>.vib
  2. A system reboot is required for the new driver to take effect.
Other useful VMware commands are shown in the following table.
In the following table, replace vmnicX with the actual interface name.
$ kill -HUP $(cat /var/run/vmware/vmkdevmgr.pid)
This command is required after vmkload_mod bnxtnet for successful module bring up.
NPAR + SR-IOV and NPAR + MultiRSS are currently not supported due to resource constraints.
VMware Commands
Command
Description
esxcli software vib list |grep bnx
List the VIBs installed to see whether the bnxt driver installed successfully.
esxcfg-module –I bnxtnet
Print module info on to screen.
esxcli network get –n vmnicX
Get vmnicX properties.
esxcfg-module –g bnxtnet
Print module parameters.
esxcfg-module –s ‘multi_rx_filters=2 disable_tap=0 max_vfs=0,0 RSS=0’
Set the module parameters.
vmkload_mod –u bnxtnet
Unload bnxtnet module.
vmkload_mod bnxtnet
Load bnxtnet module.
esxcli network nic set –n vmnicX –D full –S 25000
Set the speed and duplex of vmnicX.
esxcli network nic down –n vmnicX
Disable vmnicX.
esxcli network nic up –n vmnic6
Enable vmnicX.
bnxtnetcli –s –n vmnic6 –S “25000”
Set the link speed. Bnxtnetcli is needed for older ESX versions to support the 25G speed setting.

Limitations

The current version of the RoCE supported driver requires ESXi-6.5.0 GA build 4564106 or above.

BNXT RoCE Driver Requirements

The BNXTNET L2 driver must be installed with the
disable_roce=0
module parameter before installing the driver. To set the module parameter, use the following command:
esxcfg-module -s "disable_roce=0" bnxtnet
Use ESX6.5 L2 driver version 20.6.9.0 (RoCE supported L2 driver) or above.

Installing the RoCE Driver

To install the RoCE driver:
  1. Copy the
    <bnxtroce>-<driver version>.vib
    file in
    /var/log/vmware
    using the following commands:
    $ cd /var/log/vmware $ esxcli software vib install --no-sig-check -v <bnxtroce>-<driver version>.vib
  2. Reboot the machine.
  3. Verify that the drivers are correctly installed using the following command:
esxcli software vib list | grep bnxtroceTo disable ECN (enabled by default) for RoCE traffic use the tos_ecn=0 module parameter for bnxtroce.