Documentation/gpu/rfc/gpusvm.rst

Source file repositories/reference/linux-study-clean/Documentation/gpu/rfc/gpusvm.rst

File Facts

System
Linux kernel
Corpus path
Documentation/gpu/rfc/gpusvm.rst
Extension
.rst
Size
5288 bytes
Lines
119
Domain
Support Tooling And Documentation
Bucket
Documentation
Inferred role
Support Tooling And Documentation: documentation
Status
atlas-only

Why This File Exists

Repository support layer: documentation, build tooling, samples, user-space helper tools, generated initramfs support, licenses, and validation utilities.

Dependency Surface

Detected Declarations

Annotated Snippet

.. SPDX-License-Identifier: (GPL-2.0+ OR MIT)

===============
GPU SVM Section
===============

Agreed upon design principles
=============================

* migrate_to_ram path
	* Rely only on core MM concepts (migration PTEs, page references, and
	  page locking).
	* No driver specific locks other than locks for hardware interaction in
	  this path. These are not required and generally a bad idea to
	  invent driver defined locks to seal core MM races.
	* An example of a driver-specific lock causing issues occurred before
	  fixing do_swap_page to lock the faulting page. A driver-exclusive lock
	  in migrate_to_ram produced a stable livelock if enough threads read
	  the faulting page.
	* Partial migration is supported (i.e., a subset of pages attempting to
	  migrate can actually migrate, with only the faulting page guaranteed
	  to migrate).
	* Driver handles mixed migrations via retry loops rather than locking.
* Eviction
	* Eviction is defined as migrating data from the GPU back to the
	  CPU without a virtual address to free up GPU memory.
	* Only looking at physical memory data structures and locks as opposed to
	  looking at virtual memory data structures and locks.
	* No looking at mm/vma structs or relying on those being locked.
	* The rationale for the above two points is that CPU virtual addresses
	  can change at any moment, while the physical pages remain stable.
	* GPU page table invalidation, which requires a GPU virtual address, is
	  handled via the notifier that has access to the GPU virtual address.
* GPU fault side
	* mmap_read only used around core MM functions which require this lock
	  and should strive to take mmap_read lock only in GPU SVM layer.
	* Big retry loop to handle all races with the mmu notifier under the gpu
	  pagetable locks/mmu notifier range lock/whatever we end up calling
          those.
	* Races (especially against concurrent eviction or migrate_to_ram)
	  should not be handled on the fault side by trying to hold locks;
	  rather, they should be handled using retry loops. One possible
	  exception is holding a BO's dma-resv lock during the initial migration
	  to VRAM, as this is a well-defined lock that can be taken underneath
	  the mmap_read lock.
	* One possible issue with the above approach is if a driver has a strict
	  migration policy requiring GPU access to occur in GPU memory.
	  Concurrent CPU access could cause a livelock due to endless retries.
	  While no current user (Xe) of GPU SVM has such a policy, it is likely
	  to be added in the future. Ideally, this should be resolved on the
	  core-MM side rather than through a driver-side lock.
* Physical memory to virtual backpointer
	* This does not work, as no pointers from physical memory to virtual
	  memory should exist. mremap() is an example of the core MM updating
	  the virtual address without notifying the driver of address
	  change rather the driver only receiving the invalidation notifier.
	* The physical memory backpointer (page->zone_device_data) should remain
	  stable from allocation to page free. Safely updating this against a
	  concurrent user would be very difficult unless the page is free.
* GPU pagetable locking
	* Notifier lock only protects range tree, pages valid state for a range
	  (rather than seqno due to wider notifiers), pagetable entries, and
	  mmu notifier seqno tracking, it is not a global lock to protect
          against races.
	* All races handled with big retry as mentioned above.

Overview of baseline design
===========================

.. kernel-doc:: drivers/gpu/drm/drm_gpusvm.c

Annotation

Implementation Notes