IRDMA(4)

NAME
irdma - RDMA FreeBSD driver for Intel(R) Ethernet Controller E810

SYNOPSIS
This module relies on ice(4)

The following kernel options should be included in the configuration:
    options OFED
    options OFED_DEBUG_INIT
    options COMPAT_LINUXKPI
    options SDP
    options IPOIB_CM

DESCRIPTION

  Features
The irdma driver provides RDMA protocol support on RDMA-capable Intel Ethernet 800 Series NICs which are supported by ice(4)

The driver supports both iWARP and RoCEv2 protocols.

CONFIGURATION

  TUNABLES
Tunables can be set at the loader(8) prompt before booting the kernel or stored in loader.conf(5).

dev.irdma<interface_number>.roce_enable
    enables RoCEv2 protocol usage on <interface_numer> interface.

    By default RoCEv2 protocol is used.

dev.irdma<interface_number>.dcqcn_cc_cfg_valid
    indicates that all DCQCN parameters are valid and should be updated in registers or QP context.

    Setting this parameter to 1 means that settings in dcqcn_min_dec_factor, dcqcn_min_rate_MBps, dcqcn_F, dcqcn_T, dcqcn_B, dcqcn_rai_factor, dcqcn_hai_factor, dcqcn_rreduce_mperiod are taken into account. Otherwise default values are used.

    Note: "roce_enable" must also be set for this tunable to take effect.

dev.irdma<interface_number>.dcqcn_min_dec_factor
    The minimum factor by which the current transmit rate can be changed when processing a CNP. Value is given as a percentage (1-100).

    Note: "roce_enable" and "dcqcn_cc_cfg_valid" must also be set for this tunable to take effect.

dev.irdma<interface_number>.dcqcn_min_rate_MBps
    The minimum value, in Mbits per second, for rate to limit.

    Note: "roce_enable" and "dcqcn_cc_cfg_valid" must also be set for this tunable to take effect.

dev.irdma<interface_number>.dcqcn_F
    The number of times to stay in each stage of bandwidth recovery.

    Note: "roce_enable" and "dcqcn_cc_cfg_valid" must also be set for this tunable to take effect.

dev.irdma<interface_number>.dcqcn_T
    The number of microseconds that should elapse before increasing the CWND in DCQCN mode.

    Note: "roce_enable" and "dcqcn_cc_cfg_valid" must also be set for this tunable to take effect.

dev.irdma<interface_number>.dcqcn_B
    The number of bytes to transmit before updating CWND in DCQCN mode.

    Note: "roce_enable" and "dcqcn_cc_cfg_valid" must also be set for this tunable to take effect.

dev.irdma<interface_number>.dcqcn_rai_factor
    The number of MSS to add to the congestion window in additive increase mode.

    Note: "roce_enable" and "dcqcn_cc_cfg_valid" must also be set for this tunable to take effect.

dev.irdma<interface_number>.dcqcn_hai_factor
    The number of MSS to add to the congestion window in hyperactive increase mode.

    Note: "roce_enable" and "dcqcn_cc_cfg_valid" must also be set for this tunable to take effect.

dev.irdma<interface_number>.dcqcn_rreduce_mperiod
    The minimum time between 2 consecutive rate reductions for a single flow. Rate reduction will occur only if a CNP is received during the relevant time interval.

    Note: "roce_enable" and "dcqcn_cc_cfg_valid" must also be set for this tunable to take effect.

  SYSCTL PROCEDURES
Sysctl controls are available for runtime adjustments.

dev.irdma<interface_number>.debug
    defines level of debug messages.

    Typical value: 1 for errors only, 0x7fffffff for full debug.

dev.irdma<interface_number>.dcqcn_enable
    enables the DCQCN algorithm for RoCEv2.

    Note: "roce_enable" must also be set for this sysctl to take effect.

    Note: The change may be set at any time, but it will be applied only to newly created QPs.

  TESTING

1.  To load the irdma driver, run:

        kldload irdma

    If if_ice is not already loaded, the system will load it on its own. Please check whether the value of sysctl hw.ice.irdma is 1, if the irdma driver is not loading. To change the value put:

        hw.ice.irdma=1

    in /boot/loader.conf and reboot.

2.  To check that the driver was loaded, run:

        sysctl -a | grep infiniband

    Typically, if everything goes well, around 190 entries per PF will appear.

3.  Each interface of the card may work in either iWARP or RoCEv2 mode. To enable RoCEv2 compatibility, add:

        dev.irdma<interface_number>.roce_enable=1

    where <interface_number> is a desired ice interface number on which RoCEv2 protocol needs to be enabled, into: /boot/loader.conf , for instance:

    dev.irdma0.roce_enable=0

    dev.irdma1.roce_enable=1
    will keep iWARP mode on ice0 and enable RoCEv2 mode on interface ice1. The RoCEv2 mode is the default.

    To check irdma roce_enable status, run:

        sysctl dev.irdma<interface_number>.roce_enable

    for instance:

        sysctl dev.irdma2.roce_enable

    with returned value of '0' indicate the iWARP mode, and the value of '1' indicate the RoCEv2 mode.

    Note: An interface configured in one mode will not be able to connect to a node configured in another mode.

    Note: RoCEv2 has currently limited support, for functional testing only. DCB and Priority Flow Controller (PFC) are not currently supported which may lead to significant performance loss or connectivity issues.

4.  Enable flow control in the ice driver:

        sysctl dev.ice.<interface_number>.fc=3

    Enable flow control on the switch your system is connected to. See your switch documentation for details.

5.  The source code for krping software is provided with the kernel in /usr/src/sys/contrib/rdma/krping/. To compile the software, change directory to /usr/src/sys/modules/rdma/krping/ and invoke the following:

    make clean

    make

    make install

    kldload krping

6.  Start a krping server on one machine:

        echo size=64,count=1,port=6601,addr=100.0.0.189,server > /dev/krping

7.  Connect a client from another machine:

        echo size=64,count=1,port=6601,addr=100.0.0.189,client > /dev/krping

SUPPORT
For general information and support, go to the Intel support website at: http://support.intel.com/.

If an issue is identified with this driver with a supported adapter, email all the specific information related to the issue to freebsd@intel.com.

SEE ALSO
ice(4)

AUTHORS
-nosplit The irdma driver was prepared by Bartosz Sobczak <bartosz.sobczak@intel.com>.