Documentation/networking/scaling.rst

Source file repositories/reference/linux-study-clean/Documentation/networking/scaling.rst

File Facts

System
Linux kernel
Corpus path
Documentation/networking/scaling.rst
Extension
.rst
Size
27199 bytes
Lines
599
Domain
Support Tooling And Documentation
Bucket
Documentation
Inferred role
Support Tooling And Documentation: documentation
Status
atlas-only

Why This File Exists

Repository support layer: documentation, build tooling, samples, user-space helper tools, generated initramfs support, licenses, and validation utilities.

Dependency Surface

Detected Declarations

Annotated Snippet

.. SPDX-License-Identifier: GPL-2.0

=====================================
Scaling in the Linux Networking Stack
=====================================


Introduction
============

This document describes a set of complementary techniques in the Linux
networking stack to increase parallelism and improve performance for
multi-processor systems.

The following technologies are described:

- RSS: Receive Side Scaling
- RPS: Receive Packet Steering
- RFS: Receive Flow Steering
- Accelerated Receive Flow Steering
- XPS: Transmit Packet Steering


RSS: Receive Side Scaling
=========================

Contemporary NICs support multiple receive and transmit descriptor queues
(multi-queue). On reception, a NIC can send different packets to different
queues to distribute processing among CPUs. The NIC distributes packets by
applying a filter to each packet that assigns it to one of a small number
of logical flows. Packets for each flow are steered to a separate receive
queue, which in turn can be processed by separate CPUs. This mechanism is
generally known as “Receive-side Scaling” (RSS). The goal of RSS and
the other scaling techniques is to increase performance uniformly.
Multi-queue distribution can also be used for traffic prioritization, but
that is not the focus of these techniques.

The filter used in RSS is typically a hash function over the network
and/or transport layer headers-- for example, a 4-tuple hash over
IP addresses and TCP ports of a packet. The most common hardware
implementation of RSS uses an indirection table where each entry
stores a queue number. The receive queue for a packet is determined
by indexing the indirection table with the low order bits of the
computed hash for the packet (usually a Toeplitz hash).

The indirection table helps even out the traffic distribution when queue
count is not a power of two. NICs should provide an indirection table
at least 4 times larger than the queue count. 4x table results in ~16%
imbalance between the queues, which is acceptable for most applications.

Some NICs support symmetric RSS hashing where, if the IP (source address,
destination address) and TCP/UDP (source port, destination port) tuples
are swapped, the computed hash is the same. This is beneficial in some
applications that monitor TCP/IP flows (IDS, firewalls, ...etc) and need
both directions of the flow to land on the same Rx queue (and CPU). The
"Symmetric-XOR" and "Symmetric-OR-XOR" are types of RSS algorithms that
achieve this hash symmetry by XOR/ORing the input source and destination
fields of the IP and/or L4 protocols. This, however, results in reduced
input entropy and could potentially be exploited.

Specifically, the "Symmetric-XOR" algorithm XORs the input
as follows::

    # (SRC_IP ^ DST_IP, SRC_IP ^ DST_IP, SRC_PORT ^ DST_PORT, SRC_PORT ^ DST_PORT)

The "Symmetric-OR-XOR" algorithm, on the other hand, transforms the input as
follows::

    # (SRC_IP | DST_IP, SRC_IP ^ DST_IP, SRC_PORT | DST_PORT, SRC_PORT ^ DST_PORT)

Annotation

Implementation Notes