Page 3
(2) this device must accept any interference received, including interference that may cause undesired operation. CAN ICES-3(A)/NMB-3(A) For more information on REACH or conflicting minerals, please contact the Molex Compliance Group. CHAPTER 1 │ XUP-VV8 Hardware Reference Guide...
Page 4
CONTENTS 1 About this Guide 1.1 Document History 2 Board Setup 2.1 Handling, Shipping, and Warranty 2.1.1 Handling the XUP-VV8 2.1.2 Warranty 2.1.3 RMA - Returning Your Product to BittWare 2.2 Installing the Board 2.3 Software and Examples 2.3.1 Software Development Tools 2.3.2 FPGA Development Resources 3 Board Overview 3.1 Block Diagram...
Page 5
4.1.1 Supported Devices 4.1.2 FPGA Bank Usage 4.1.3 Hard IP Usage 4.1.4 I2C Interface 4.1.5 Step Load Requirements 4.1.6 FPGA Configuration 4.2 PCIe Interface 4.2.1 Tandem Support 4.3 External Interfaces 4.3.1 QSFP-DD 4.3.2 OCuLink Connectors XUP-VV8 Board Ordering Configurations OCuLink GPIOs OCuLink I2C buses 4.3.3 Validated Cables and Transceiver Settings 4.4 Memory...
Page 6
PCIe Clock USB Clocks 4.6 USB and JTAG 4.6.1 USB-UART 4.7 LEDs 4.8 Headers 4.9 Board Management Controller 4.9.1 Unused FPGA Pins 4.9.2 I2C Interface 4.9.3 Card Initialization and Reset 4.9.4 FPGA Reset 4.9.5 Board Monitoring and Control 4.10 Power 4.10.1 Power Inputs 4.10.2 Power Requirements 4.10.3 External Power Connectors...
Page 7
1 About this Guide This guide covers Revision 0 of the XUP-VV8 board and describes the XUP-VV8’s components and interfaces. The Getting Started Guide also explains how to load the FPGA. 1.1 Document History Document Revi- Release Date Notes/Changes sion 11/22/19 Preliminary release 11/26/19...
Page 8
Document Revi- Release Date Notes/Changes sion 1/18/22 photo with power connectors labeled 5/22/23 Correction to Clock Circuitry block diagram 9/5/23 Add detail on function of BMC LEDs 11/13/23 Clarify FPGA control of I2C bus, update I2C block diagram...
Page 10
2.1 Handling, Shipping, and Warranty 2.1.1 Handling the XUP-VV8 Warning! The XUP-VV8 contains electro-static discharge (ESD) sensitive devices. Be sure to follow the standard hand- ling procedures for ESD sensitive devices, taking proper precautions to ground yourself and the work area before removing the board from its static shielding bag.
Page 11
Hot Surface Warning When the XUP-VV8 is operational, the card generates heat. The heat is dispersed using various heatsinks and the rear stiffening frame. Take care on any XUP-VV8 exposed metal surface as it may be too hot to touch. 2.1.2 Warranty This section outlines the handling guidelines that are allowed within the warranty.
Page 12
2.2 Installing the Board The XUP-VV8 requires a x16 PCIe slot and two 6- or 8-pin power supply cables. For complete installation instruc- tions, refer to the XUP-VV8 Getting Started Guide. 2.3 Software and Examples Complete software development tools and FPGA examples are available for the XUP-VV8: Software development tools FPGA development resources 2.3.1 Software Development Tools...
Page 14
3.1 Block Diagram XUP-VV8 System Architecture shows the key functional features of the XUP-VV8. The FPGA is used to imple- ment the PCIe Gen3, DDR4, QSFP-DD, and Board Management Controller interfaces while leaving ample room for user applications. The board is also designed to efficiently handle cooling for higher powered FPGA applic- ations while maintaining high throughput.
Page 15
2 OCuLink, each connected to 4x transceivers 1 PPS and 10MHz Ref Clk inputs for timestamping (optional) Debug: USB and JTAG Board Management Controller Size 3/4-length, standard-height PCIe dual-slot card 10 x 4.37 inches (254 x 111.15 mm) 3.3 Components and Interfaces Component Reference Table below lists the components and interfaces on the board, their manufacturer, and the section(s) where you will find more details about them.
Page 16
Board Layout (Bottom View) Front Panel Table 1 Component Reference Table Component Manufacturer Part Number Section Reference FPGA UltraScale+ FPGA Xilinx Part # is order dependent Xilinx Ultrascale+ FPGA Connectors QSFP-DD connectors Molex 202718-0100 QSFP-DD...
Page 17
Component Manufacturer Part Number Section Reference Micro USB interface TE Connectivity 2013499-1 OCuLink Molex 171982-1142 External power Molex 45586-0005 Power DIMM connectors Molex 78730-1002 Memory SMA (PPS and Clk) Amphenol 908-22101TC Clocks 48 MHz crystal Kyocera CX3225SB48000D0FPJC1 Clock Circuitry Programmable clock generator...
Page 18
3.4 Board Configuration Options The XUP-VV8 is available in many configurations. This document describes all available features of the XUP-VV8 board. Keep in mind that your board may be configured differently and that some information and settings described in this guide may not apply to your hardware. Feature Options VU9P...
Page 19
3.5 Firmware The following table describes the programmable non-volatile devices on the XUP-VV8 board. Table 2 XUP-VV8 Firmware Programming Device Description Usage Method(s) Local micro- BittWare factory-pro- Stores code and configuration options. Clock configuration stored in local flash. controller, flash grammed FRU data.
Page 20
BittWare supports a variety of leading server vendors. As part of Molex, we can handle global supply chain and logistics. We can cater to a range of application environments such as data center and at the edge. Lastly, through OEM programs, you have the option of purchasing selected TeraBox servers directly from server vendors such as Dell, HPE, and Lenovo.
Page 22
4.1 Xilinx Ultrascale+ FPGA The FPGA on the XUP-VV8 handles all of the board's I/O. 4.1.1 Supported Devices The XUP-VV8 supports the following UltraScale+ devices: Table 3 Supported Virtex UltraScale+ Devices Family Part Number Virtex XCVU9P Virtex XCVU13P 4.1.2 FPGA Bank Usage The "FPGA Bank"...
Page 23
FPGA annotated DIE view (VU13P) Note: Please contact BittWare Support with any questions about special handling to cross the SLR (as men- tioned in the diagram above). CHAPTER 4 │ XUP-VV8 Hardware Reference Guide...
Page 25
FPGA Bank and Clock Connections (VU13P) CHAPTER 4 │ XUP-VV8 Hardware Reference Guide...
Page 26
FPGA Bank and Clock Connections (VU9P) 4.1.3 Hard IP Usage FPGA annotated DIE view (VU13P) FPGA annotated DIE view (VU9P) diagrams above shows spe- cific connections, but the table below gives an overview of how the Hard IP is used for a VU13P or VU9P FPGA. Connector VU13P VU9P...
Page 27
Connector VU13P VU9P x4 same SLR 2 CMACs (no QSFP conflict) 1 PCIe Hard IP w/o tandem x8 OCuLink-2 PCIe Hard IP w/o tandem x4 same SLR PCIe Fingers PCIe Hard IP with Tandem optional PCIe Hard IP with Tandem optional 4.1.4 I C Interface The FPGA directly connects to a number of I...
Page 28
slower slew rates. A general rule of thumb for high-current designs can be considered to be 0.25 ns per amp (or 4 A/ns) of step current. The Xilinx Power Estimator (XPE) tool is used to estimate the current on each power rail. Kintex UltraScale FPGAs Data Sheet: DC and AC Switching Characteristics (DS892) [Ref 1] and Virtex UltraScale FPGAs Data Sheet: DC and AC Switching Characteristics (DS893) [Ref 2] provide the operating range for all the various power rails.
Page 29
4.2 PCIe Interface The XUP-VV8 features a PCI x16 electrical and mechanical interface, which supports a PCIe Gen1, Gen2, and Gen3 connection in a standard slot. The PCIe interface connects to the FPGA via 16 transceivers. 4.2.1 Tandem Support Tandem is a feature of Xilinx UltraScale+ FPGA devices. BittWare cards are designed to provide support for some Tandem modes.
Page 30
Faceplate with 4x QSFP-DDs Cables and Transceiver Modules Breakout cables allow you to connect QSFP-DDs to QSFPs and SFPs. Contact BittWare for details. The XUP-VV8 supports a power consumption of up to 28W across the four QSFP-DD sites. For example, you can populate four class 3 QSFP-DD modules (7W), or two class 7 (14W), etc.
Page 31
QSFP-DD Connector Pinouts Table 5 QSFP-DD Port 1 Connector Pinout QSFP-DD Port 1 QSFP-DD Port 1 Lanes 7:4 Lanes 3:0 QSFP-DD Signal FPGA Pin Pin QSFP-DD Signal FPGA Pin QSFP1_TX_N_1 BB43 QSFP1_TX_N_5 AT39 QSFP1_TX_P_1 BB42 QSFP1_TX_P_5 AT38 QSFP1_TX_N_3 AW41 QSFP1_TX_N_7 AP39 QSFP1_TX_P_3 AW40 QSFP1_TX_P_7 AP38...
Page 35
QSFP-DD Port 3 QSFP-DD Port 3 Lanes 7:4 Lanes 3:0 QSFP3_TX_P_2 AC40 QSFP3_TX_P_6 QSFP3_TX_N_2 AC41 QSFP3_TX_N_6 QSFP3_TX_P_0 AE40 QSFP3_TX_P_4 AA40 QSFP3_TX_N_0 AE41 QSFP3_TX_N_4 AA41 Table 8 QSFP-DD Port 4 Connector Pinout QSFP-DD Port 4 QSFP-DD Port 4 Lanes 7:4 Lanes 3:0 QSFP-DD Signal FPGA Pin Pin QSFP-DD Signal...
Page 36
QSFP-DD Port 4 QSFP-DD Port 4 Lanes 7:4 Lanes 3:0 QSFP4_RX_N_1 QSFP4_RX_N_5 QSFP4_RX_P_1 QSFP4_RX_P_5 QSFP3_RX_N_3 QSFP4_RX_N_7 QSFP3_RX_P_3 QSFP4_RX_P_7 QSFP_PRSNT_L QSFP_INT_L RSVD P3V3_QSFP_VCCT P3V3_QSFP_VCCT P3V3_QSFP_VCC P3V3_QSFP_VCC QSFP_LP RSVD QSFP3_TX_P_2 QSFP4_TX_P_6 QSFP3_TX_N_2 QSFP4_TX_N_6 QSFP4_TX_P_0 QSFP4_TX_P_4 QSFP4_TX_N_0 QSFP4_TX_N_4 QSFP-DD Control Signals All QSFP-DD I C signals are connected to the FPGA via an I C expander.
Page 37
Connector Pin FPGA Signal Default Description Number The FPGA_I2C_MASTER_L signal is monitored in software on the BMC as a way to cooperatively share the I C interface. It is a static signal that allows either the FPGA or the BMC to ‘own’ this interface completely.
Page 38
QSFP-DD Control Signal Connections to I C Expander...
Page 39
4.3.2 OCuLink Connectors The XUP-VV8 has two optional x4 Gbps OCuLink connectors (Molex Nano-Pitch I/O Interconnect System) at the rear of the board. Depending on your board's configuration options, the two connectors can be used for PCIe or for serial expansion options, such as board-to-board interconnect or connecting to accessory boards. They also include some general purpose I/O signals.
Page 42
How to control the OCuLink GPIO sidebands and I C buses The XUP-VV8 features an NXP PCA9555BS I C controllable GPIO device used to control both OCuLink con- nectors’ sideband signals and I C buses (OCU1/J8 and OCU2/J11). The card uses this device to: 1.
Page 43
GPIO Configuration OCuLink GPIOs The XUP-VV8 performs functions 1 and 2 using three 4-bidirectional-user-GPIO buffers (Texas Instruments SN74AVC4T774 - U39, U47 & U42). Each of the bidirectional GPIO buffers has four inputs used to control the dir- ection of four user IOs. The XUP-VV8 circuitry uses each of the bidirectional GPIO buffer’s four user IOs to connect an FPGA IO to a OCuLink connector IO pin.
Page 44
First, the PCA9555 is used to drive the user IOs’ direction as an FPGA input or output. Twelve of the PCA9555BS's outputs control the direction of the corresponding twelve OCuLink sideband signals (see table below for details). If the Pxx / IOx_x signal is configured as an output and driven low (logic ‘0’), then the corresponding OCuLink side- band is an output to the card, and the FPGA must drive the corresponding signal as an FPGA output.
Page 45
inverted on the card so the I C buses are disabled by default. If the P17 / IO1_7 signal is configured as an output and driven low (logic ‘0’), then the voltage-level shifter device is active. Upon card power on, the P17 / IO1_7 signal is configured as an input, and, therefore, the I C buses will be disabled.
Page 49
4.4 Memory The XUP-VV8 features four DIMM sites that support standard DDR4 and proprietary QDR-II+ DIMMs. Additional on-board memory includes Flash with factory default and support for multiple FPGA images. Note: The single slot version of the XUP-VV8 does not have any DIMM connectors. 4.4.1 DIMM The XUP-VV8 features four 288-pin DIMM connectors that support DIMMs in the following configurations: DDR4: x72 w/ECC...
Page 50
DIMM Power Architecture Table 16 DIMM Power Configuration Options Configuration DIMM connector installed, but no DIMM inserted 1.2V DDR4 installed 1.2V QDRII+ installed 1.8V 4.4.3 DIMM Support BittWare supports multiple DDR4 DIMMs in addition to a QDR-II+ DIMM as documented earlier in this section. Bitt- Ware does not support customers with non-standard or non-validated DIMMs installed.
Page 51
4.4.4 Flash Memory A 2 Gbit bank of flash memory is used to store FPGA configuration code. The flash contains a factory image and a user image. The configuration code can be loaded via the USB interface, using the BwConfig utility, which is included in BittWare's BittWorks II Toolkit, or it can be loaded via PCIe (requires BMC IP in the FPGA).
Page 52
4.5 Clock Circuitry The XUP-VV8 features the following on-board clocks: Two Si5341-A high-performance programmable clock synthesizers Two Si5346-A jitter cleaners 48MHz crystal oscillator Note: Refer to the Component Reference Table for a list of manufacturers and part numbers for the on- board clocks.
Page 53
4.5.3 System Clocks The XUP-VV8 uses a 48 MHz crystal and a programmable oscillator (Si5341-A) to provide the clocks to the FPGA. FPGA Bank Usage for clock circuitry diagrams. If you need a different clock configuration, contact BittWare. Clock Descriptions User Clock A 100MHz miscellaneous LVDS fabric clock is sourced from the Si5341-A clock generator to the FPGA.
Page 54
The Si5341-A programmable clock The GT RxClk The fabric clock The 10MHz external clock input FPGA pin name: MGT_120_RXCLK_C1_x (to Si5346-A), MGT_122_RXCLK_C1_x (to Si5346-B)
Page 56
4.6 USB and JTAG The XUP-VV8 provides a USB 2.0 port (J11) for access to the BMC and JTAG access to the FPGA (for Vivado). The USB port connects via a hub to the BMC, USB-JTAG, and USB-UART. The USB interface can be used for the following: Reading/writing Xilinx UltraScale+ configuration files in Flash Reading manufacturing and configuration data via USB...
Page 57
XUP-VV8 LEDs Table 17 FPGA LED Details FPGA Description Signal Name Color Pin # BMC LEDs: Solid green, red off - BMC is in standby power mode, waiting for 12v power to be applied. Blink green, solid red - BMC has turned off card power, possibly due to a D1:Green sensor crossing a fatal threshold.
Page 58
4.8 Headers The XUP-VV8 has four jumpers/headers: JP1: Connection for an optional "normally open" pushbutton reset switch for standalone operation (header not installed) JP2: Connection for optional rechargable battery (header installed) JP3: Fansink power connection (header not installed) JP4: Factory use only (header not installed) 4.9 Board Management Controller The XUP-VV8 features an advanced system monitoring subsystem, similar to those typically found on today’s server platforms.
Page 59
C Connections XUP-VV8 I C Connections C Expander The XUP-VV8 features an I C expander from NXP Semiconductor (PCA9548ABS). 4.9.3 Card Initialization and Reset When installed in a PCIe slot, the XUP-VV8 is a slave to the host processor that is the root complex of the PCIe bus.
Page 60
1. The card powers up, and supplies are sequenced by the BMC, after which the board comes out of reset. 2. When the power supplies are stable, the BMC Power Good (green) LED comes on (D1), the configuration logic is released, and the FPGA configuration starts. 3.
Page 61
4.10 Power This section covers the board's power and standalone setup features: Power requirements External power connector Standalone setup 4.10.1 Power Inputs The XUP-VV8 uses 3.3V aux from the PCIe connector if it is present. If both power connectors are populated on the board, they must both be connected.
Page 62
XUP-VV8 Power Connector Locations Table 18 Power connector pinout Signal +12V +12V +12V Sense1 Sense0 Table 19 Sense pins decoding by the XUP-VV8 Sense1 Sense0 Description A 2x4 auxiliary power connector is plugged into the card. The XUP-VV8 can draw up to 150W from the Ground Ground auxilliary power connector.
Page 63
Sense1 Sense0 Description Ground Open Reserved A 2x3 auxiliary power connector is plugged into the card. The XUP-VV8 can only draw up to 75W from the Open Ground auxiliary power connector. Open Open No auxiliary power connector is plugged in. 4.10.4 Standalone Setup To set the board up for standalone use, connect two external power cables.
Page 65
5.1 Chassis Requirements 5.1.1 PCIe Express The XUP-VV8 is capable of PCIe Gen 1/2/3 with 1/2/4/8/16 lanes.The board requires a x16 PCIe slot for mech- anical compatibility. 5.1.2 PCIe Bracket The board ships with a PCIe faceplate bracket matching the board configuration. The example below shows the faceplate for a board configured with four QSFP-DDs and the 10MHz clock and 1 PPS inputs.
Page 66
5.3.1 Environmental Conditions The XUP-VV8 is passively cooled (where the airflow is provided by the host server system. BittWare recommend operating the card in the following environmental parameters. Table 20 Environmental Parameters Parameter Comments FPGA Die Temperature 100C For extended temperature FPGAs Ambient Air Temperature Measured at the card when inside server (Card Inlet Temperature)
Page 67
Cooling Performances for VU9P (single-slot passive) The following instructions are used for estimating FPGA junction temperature, T 1. Choose the curve closest to your application’s actual FPGA power. 2. Estimate q from the chosen curve based on expected airflow in LFM. For example, a 60W FPGA with 1000 LFM has q = 0.90.
Page 68
Cooling Performances for VU9P (dual-slot passive) The following instructions are used for estimating FPGA junction temperature, T 1. Choose the curve closest to your application’s actual FPGA power. 2. Estimate q from the chosen curve based on expected airflow in LFM. For example, a 120W FPGA with 800 LFM has q = 0.44.
Page 69
Cooling Performances for VU13P (dual-slot passive) The following instructions are used for estimating FPGA junction temperature, T 1. Choose the curve closest to your application’s actual FPGA power. 2. Estimate q from the chosen curve based on expected airflow in LFM. For example, a 120W FPGA with 800 LFM has q = 0.42.
Need help?
Do you have a question about the BittWare XUP-VV8 and is the answer not in the manual?
Questions and answers