26 November 2013

Multi GPU Machines and PCIE In-Band vs. Out-of-Band Presence Detection

TLDR: 
To improve PCIE bus initialization during boot when trying to run x16 GPUs via various PCIE risers, short pin A1 to B17 on ALL PCIE x1 risers (in the unlikely event you are using x4/x8 to x16 risers, look up the proper x4/x8 PRSNT#2 pin and short that one to A1 instead).

I guess I should preface everything below by saying, I'd be happy to hear how PCIE bus initialization really works (in relation to the problem described below) from someone who knows PCIE buses.


Full Story

Last night I had a problem getting multiple Radeon 7970 GPUs running in a single motherboard, 4 cards good --> 5 cards pure chaos, lspci showing 3 cards only, swapping slots, risers, cards all showed that everything was good, just not in any combination over 4 and in fact, some combinations of 4 no good. (Maybe even some with 3).

7970 GPUs