Documentation/kernel-hacking/false-sharing.rst

Source file repositories/reference/linux-study-clean/Documentation/kernel-hacking/false-sharing.rst

File Facts

System
Linux kernel
Corpus path
Documentation/kernel-hacking/false-sharing.rst
Extension
.rst
Size
9446 bytes
Lines
207
Domain
Support Tooling And Documentation
Bucket
Documentation
Inferred role
Support Tooling And Documentation: documentation
Status
atlas-only

Why This File Exists

Repository support layer: documentation, build tooling, samples, user-space helper tools, generated initramfs support, licenses, and validation utilities.

Dependency Surface

Detected Declarations

Annotated Snippet

struct foo {
		refcount_t refcount;
		...
		char name[16];
	} ____cacheline_internodealigned_in_smp;

Member 'refcount'(A) and 'name'(B) _share_ one cache line like below::

                +-----------+                     +-----------+
                |   CPU 0   |                     |   CPU 1   |
                +-----------+                     +-----------+
               /                                        |
              /                                         |
             V                                          V
         +----------------------+             +----------------------+
         | A      B             | Cache 0     | A       B            | Cache 1
         +----------------------+             +----------------------+
                             |                  |
  ---------------------------+------------------+-----------------------------
                             |                  |
                           +----------------------+
                           |                      |
                           +----------------------+
              Main Memory  | A       B            |
                           +----------------------+

'refcount' is modified frequently, but 'name' is set once at object
creation time and is never modified.  When many CPUs access 'foo' at
the same time, with 'refcount' being only bumped by one CPU frequently
and 'name' being read by other CPUs, all those reading CPUs have to
reload the whole cache line over and over due to the 'sharing', even
though 'name' is never changed.

There are many real-world cases of performance regressions caused by
false sharing.  One of these is a rw_semaphore 'mmap_lock' inside
mm_struct struct, whose cache line layout change triggered a
regression and Linus analyzed in [2]_.

There are two key factors for a harmful false sharing:

* A global datum accessed (shared) by many CPUs
* In the concurrent accesses to the data, there is at least one write
  operation: write/write or write/read cases.

The sharing could be from totally unrelated kernel components, or
different code paths of the same kernel component.


False Sharing Pitfalls
======================
Back in time when one platform had only one or a few CPUs, hot data
members could be purposely put in the same cache line to make them
cache hot and save cacheline/TLB, like a lock and the data protected
by it.  But for recent large system with hundreds of CPUs, this may
not work when the lock is heavily contended, as the lock owner CPU
could write to the data, while other CPUs are busy spinning the lock.

Looking at past cases, there are several frequently occurring patterns
for false sharing:

* lock (spinlock/mutex/semaphore) and data protected by it are
  purposely put in one cache line.
* global data being put together in one cache line. Some kernel
  subsystems have many global parameters of small size (4 bytes),
  which can easily be grouped together and put into one cache line.
* data members of a big data structure randomly sitting together
  without being noticed (cache line is usually 64 bytes or more),
  like 'mem_cgroup' struct.

Following 'mitigation' section provides real-world examples.

Annotation

Implementation Notes