Gpu P2P Underperforming; Pcie Link Health Error - H3 Falcon PCIe User Manual

GPU P2P underperforming

Make sure that your GPU supports peer-to-peer function.
Disable the PCI Access Control Services (ACS).
IO virtualization (VT-d for Intel platform, or IOMMU for AMD platform) can interfere with GPU Direct by
redirecting all PCI point-to-point traffic to the CPU root complex, causing a significant performance reduction
or even a hang. You can check whether ACS is enabled on PCI bridges by executing following commands:
# sudo lspci -vvv | grep ACSCtl
If it shows "SrcValid+", then ACS might be enabled. Looking at the full output of lspci, one can check if a PCI
bridge has ACS enabled.
If PCI switches have ACS enabled, it needs to be disabled. On some systems this can be done from the BIOS
by disabling IO virtualization or VT-d and ACS.
Disabling IO virtualization
Host BIOS → IO or Advanced
Disable VT for Direct IO (VT-d) for Intel platforms.
Disable IOMMU for AMD platforms.
Other platforms may have different name for the IO virtualization function. Please ask your server vendor if the function
cannot be found.

PCIe link health error

If you find the status of PCIe link health showing "Error", there may be Physical signal issue or PCIe
TLP(Transaction Layer Packet) error between the PCIe slot and your PCIe device. It may have an impact on
performance (e.g., latency and bandwidth), but no data/information is lost and PCIe fabric remains reliable.
Such errors are corrected by hardware and no software intervention is required. You may try the following
steps to improve it:
Re-install the PCIe device – The error may be caused by incorrection installment of the PCIe device card.
Please unplug the card and plug it again.
Change a slot – The PCIe signal is slight varied due to different internal length within PCB board. Please
install the card on another slot to check whether it is improved.
Make sure that the device is on the compatible list of your Falcon model.
46