Documentation/arch/powerpc/pci_iov_resource_on_powernv.rst

Source file repositories/reference/linux-study-clean/Documentation/arch/powerpc/pci_iov_resource_on_powernv.rst

File Facts

System
Linux kernel
Corpus path
Documentation/arch/powerpc/pci_iov_resource_on_powernv.rst
Extension
.rst
Size
14926 bytes
Lines
313
Domain
Support Tooling And Documentation
Bucket
Documentation
Inferred role
Support Tooling And Documentation: documentation
Status
atlas-only

Why This File Exists

Repository support layer: documentation, build tooling, samples, user-space helper tools, generated initramfs support, licenses, and validation utilities.

Dependency Surface

Detected Declarations

Annotated Snippet

===================================================
PCI Express I/O Virtualization Resource on Powerenv
===================================================

Wei Yang <weiyang@linux.vnet.ibm.com>

Benjamin Herrenschmidt <benh@au1.ibm.com>

Bjorn Helgaas <bhelgaas@google.com>

26 Aug 2014

This document describes the requirement from hardware for PCI MMIO resource
sizing and assignment on PowerKVM and how generic PCI code handles this
requirement. The first two sections describe the concepts of Partitionable
Endpoints and the implementation on P8 (IODA2). The next two sections talks
about considerations on enabling SRIOV on IODA2.

1. Introduction to Partitionable Endpoints
==========================================

A Partitionable Endpoint (PE) is a way to group the various resources
associated with a device or a set of devices to provide isolation between
partitions (i.e., filtering of DMA, MSIs etc.) and to provide a mechanism
to freeze a device that is causing errors in order to limit the possibility
of propagation of bad data.

There is thus, in HW, a table of PE states that contains a pair of "frozen"
state bits (one for MMIO and one for DMA, they get set together but can be
cleared independently) for each PE.

When a PE is frozen, all stores in any direction are dropped and all loads
return all 1's value. MSIs are also blocked. There's a bit more state that
captures things like the details of the error that caused the freeze etc., but
that's not critical.

The interesting part is how the various PCIe transactions (MMIO, DMA, ...)
are matched to their corresponding PEs.

The following section provides a rough description of what we have on P8
(IODA2).  Keep in mind that this is all per PHB (PCI host bridge).  Each PHB
is a completely separate HW entity that replicates the entire logic, so has
its own set of PEs, etc.

2. Implementation of Partitionable Endpoints on P8 (IODA2)
==========================================================

P8 supports up to 256 Partitionable Endpoints per PHB.

  * Inbound

    For DMA, MSIs and inbound PCIe error messages, we have a table (in
    memory but accessed in HW by the chip) that provides a direct
    correspondence between a PCIe RID (bus/dev/fn) with a PE number.
    We call this the RTT.

    - For DMA we then provide an entire address space for each PE that can
      contain two "windows", depending on the value of PCI address bit 59.
      Each window can be configured to be remapped via a "TCE table" (IOMMU
      translation table), which has various configurable characteristics
      not described here.

    - For MSIs, we have two windows in the address space (one at the top of
      the 32-bit space and one much higher) which, via a combination of the
      address and MSI value, will result in one of the 2048 interrupts per
      bridge being triggered.  There's a PE# in the interrupt controller
      descriptor table as well which is compared with the PE# obtained from
      the RTT to "authorize" the device to emit that specific interrupt.

    - Error messages just use the RTT.

Annotation

Implementation Notes