Monday, December 28, 2015

Mellanox ConnectX-2 10GB Interface on FreeBSD 10.2

Although Mellanox's FreeBSD driver for the ConnectX-2 is included in the kernel source, the kernel modules are not included with the generic kernel. To use this card, build a custom kernel.

1. update the FreeBSD source using freebsd-update

sudo freebsd-update fetch
sudo freebsd-update install

2. copy the generic kernel config to a custom kernel
cd /usr/src/sys/amd64/conf


4. edit the config file (sudo vi MYKERNEL01.) Add these lines to the bottom:

#### Mellanox ConnectX-2 support
options  OFED
options  IPOIB_CM
device  ipoib
device  mlx4ib
device  mlxen 

5. Compile the kernel and install it

cd /usr/src
sudo make buildkernel KERNCONF=MYKERNEL01
sudo make installkernel KERNCONF=MYKERNEL01

6. reboot

sudo shutdown -r now

Your new interface should show up when the machine comes back:

mlxen0: flags=8802 metric 0 mtu 1500
ether 00:02:c9:52:ad:23
nd6 options=29
media: Ethernet autoselect (autoselect )
status: active

You can configure however you'd like (IP, MTU, etc.)

If you run freebsd-update on a regular basis, you may want to append a  kernel rebuild (step #5) to a script so you can rebuild the kernel automatically if there is a kernel source update.