Documentation/infiniband/ipoib.rst

Source file repositories/reference/linux-study-clean/Documentation/infiniband/ipoib.rst

File Facts

System
Linux kernel
Corpus path
Documentation/infiniband/ipoib.rst
Extension
.rst
Size
4524 bytes
Lines
116
Domain
Support Tooling And Documentation
Bucket
Documentation
Inferred role
Support Tooling And Documentation: documentation
Status
atlas-only

Why This File Exists

Repository support layer: documentation, build tooling, samples, user-space helper tools, generated initramfs support, licenses, and validation utilities.

Dependency Surface

Detected Declarations

Annotated Snippet

==================
IP over InfiniBand
==================

  The ib_ipoib driver is an implementation of the IP over InfiniBand
  protocol as specified by RFC 4391 and 4392, issued by the IETF ipoib
  working group.  It is a "native" implementation in the sense of
  setting the interface type to ARPHRD_INFINIBAND and the hardware
  address length to 20 (earlier proprietary implementations
  masqueraded to the kernel as ethernet interfaces).

Partitions and P_Keys
=====================

  When the IPoIB driver is loaded, it creates one interface for each
  port using the P_Key at index 0.  To create an interface with a
  different P_Key, write the desired P_Key into the main interface's
  /sys/class/net/<intf name>/create_child file.  For example::

    echo 0x8001 > /sys/class/net/ib0/create_child

  This will create an interface named ib0.8001 with P_Key 0x8001.  To
  remove a subinterface, use the "delete_child" file::

    echo 0x8001 > /sys/class/net/ib0/delete_child

  The P_Key for any interface is given by the "pkey" file, and the
  main interface for a subinterface is in "parent."

  Child interface create/delete can also be done using IPoIB's
  rtnl_link_ops, where children created using either way behave the same.

Datagram vs Connected modes
===========================

  The IPoIB driver supports two modes of operation: datagram and
  connected.  The mode is set and read through an interface's
  /sys/class/net/<intf name>/mode file.

  In datagram mode, the IB UD (Unreliable Datagram) transport is used
  and so the interface MTU has is equal to the IB L2 MTU minus the
  IPoIB encapsulation header (4 bytes).  For example, in a typical IB
  fabric with a 2K MTU, the IPoIB MTU will be 2048 - 4 = 2044 bytes.

  In connected mode, the IB RC (Reliable Connected) transport is used.
  Connected mode takes advantage of the connected nature of the IB
  transport and allows an MTU up to the maximal IP packet size of 64K,
  which reduces the number of IP packets needed for handling large UDP
  datagrams, TCP segments, etc and increases the performance for large
  messages.

  In connected mode, the interface's UD QP is still used for multicast
  and communication with peers that don't support connected mode. In
  this case, RX emulation of ICMP PMTU packets is used to cause the
  networking stack to use the smaller UD MTU for these neighbours.

Stateless offloads
==================

  If the IB HW supports IPoIB stateless offloads, IPoIB advertises
  TCP/IP checksum and/or Large Send (LSO) offloading capability to the
  network stack.

  Large Receive (LRO) offloading is also implemented and may be turned
  on/off using ethtool calls.  Currently LRO is supported only for
  checksum offload capable devices.

  Stateless offloads are supported only in datagram mode.

Interrupt moderation

Annotation

Implementation Notes