Page 1
S260-NF0 S260-NF1 2U NVMe Over Fabric Storage User Manual Rev. 1.0...
Page 2
GIGABYTE's prior written permission. Documentation Classifications In order to assist in the use of this product, GIGABYTE provides the following types of documentation: User Manual: detailed information & steps about the installation, configuration and use of this product (e.g.
Page 3
Conventions The following conventions are used in this user's guide: NOTE! Gives bits and pieces of additional information related to the current topic. CAUTION! Gives precautionary measures to avoid possible hardware or software problems. WARNING! Alerts you to any damage that might result from doing or not doing specific actions.
Page 4
Server Warnings and Cautions Before installing a server, be sure that you understand the following warnings and cautions. WARNING! To reduce the risk of electric shock or damage to the equipment: • Do not disable the power cord grounding plug. The grounding plug is an important safety feature.
Page 5
Electrostatic Discharge (ESD) CAUTION! ESD CAN DAMAGE DRIVES, BOARDS, AND OTHER PARTS. WE RECOMMEND THAT YOU PERFORM ALL PROCEDURES AT AN ESD WORKSTATION. IF ONE IS NOT AVAILABLE, PROVIDE SOME ESD PROTECTION BY WEARING AN ANTI-STATIC WRIST STRAP AT- TACHED TO CHASSIS GROUND -- ANY UNPAINTED METAL SURFACE -- ON YOUR SERVER WHEN HANDLING PARTS.
Page 6
CAUTION! Risk of explosion if battery is replaced incorrectly or with an incorrect type. Replace the battery only with the same or equivalent type recommended by the manufacturer. Dispose of used bat- teries according to the manufacturer’s instructions.
Chapter 1 Hardware Installation Installation Precautions The motherboard/system contain numerous delicate electronic circuits and components which can become damaged as a result of electrostatic discharge (ESD). Prior to installation, carefully read the service guide and follow these procedures: • Prior to installation, do not remove or break motherboard S/N (Serial Number) sticker or warranty sticker provided by your dealer.
Motherboard 2 x CPBD530 Embedded Microsemi PM8536B-FEI PICe switch with 96 Lanes (S260-NF0) Pre-installed 1 x Western Digital Onyx NVMe Host Bus Adapter* in each CPBD530 Each CPBD530 board controls 12 x NVMe SSD devices with PCIe x4 bandwidth...
Page 11
1 x BMC Reset button 1 x System Reset button 1 x System status LED Backplane I/O Bandwidth: PCIe Gen3 x4 per port (S260-NF0) (S260-NF1) Bandwidth: PCIe Gen3 x2 per port System Aspeed® AST2520 management controller...
Chapter 2 System Appearance Front View NODE1 NODE2 NODE1 NODE2 Description Front Panel Buttons and LEDs Orange HDD Latches Support NVMe • Refer to section 2-3 Front Panel LEDs and Buttons for a detailed description of the function of the LEDs. - 15 - System Appearance...
Rear View PSU2 PSU1 1 2 3 4 5 6 7 Description Power Button ID Button System Reset Button BMC Reset Button System Status LED Server Management LAN Port COM Jack • For HDD Font Color, yellow indicates SATA, blue indicates NVMe. •...
Front Panel LEDs and Buttons NODE1 NODE2 Name Color Status Description Green System is operating normally. Critical condition, may indicates: System fan failure System temperature Amber System Non-critical condition, may indicates: Status LED Blink Redundant power module failure Temperature and voltage issue Non-critical condition, may indicates: Redundant power module failure Temperature and voltage issue...
Rear System LAN LEDs Name Color Status Description Yellow 1 Gbps data rate 1GbE Green 100 Mbps data rate Speed LED 10 Mbps data rate Link between system and network or no access 1GbE Green Link/ Blink Data transmission or receiving is occurring Activity No data transmission or receiving is occurring...
Power Supply Unit LED PSU LED State Description No AC power to all power supplies 1Hz Green Blinking AC present / only standby on / Cold redundant mode 2Hz Green Blinking Power supply firmware updateing mode AC cord unplugged or AC power lost; with a second power supply in parallel still with AC input power Amber Power supply critical event causing shut down:...
Hard Disk Drive LEDs HDD Present RAID SKU LED1 Locate HDD Fault Rebuilding Access (No Access) Disk LED Green ON(*1) BLINK (*2) (LED on Back Panel) Amber No RAID configuration ON(*1) Green (via HBA) Removed HDD Slot (LED on Back Panel) Amber Green BLINK (*2)
Chapter 3 System Hardware Installation Pre-installation Instructions Computer components and electronic circuit boards can be damaged by discharges of static electricity. Working on computers that are still connected to a power supply can be extremely dangerous. Follow the simple guidelines below to avoid damage to your computer or injury to yourself.
Installing the Hard Disk Drive Read the following guidelines before you begin to install the Hard disk drive: • Take note of the drive tray orientation before sliding it out. • The tray will not fit back into the bay if inserted incorrectly. •...
Removing the Node Follow these instructions to remove a node: Press the release retaining clip on the left side of the node along the direction of the arrow, Pulling out the node using its handle. Press - 23 - System Hardware Installation...
Installing the PCI Expansion Card • The PCI riser assembly does not include a riser card or any cabling as standard. To install a PCI card, a riser card must be installed. Follow these instructions to install the PCI Expansion card: Remove the three screws securing the riser bracket to the system.
Chapter 4 Western Digital NIC Configuration Introduction This chapter describes the evaluation and use of the Western Digital Networks NVMe-oFTM bridge adapter board codenamed Onyx, which contains the ASIC device codename Fuji. Adapter Board Specification 4-2-1 Adapter Block Diagram 4-2-2 Block Decription Industry-standard Ethernet cage, designed to accept both copper and QSFP28 optical cables.
Adapter Board LEDs LEDs on the Onyx bulkhead indicate various board state and activity. • • LEDs labeled “1” and “2” indicate Ethernet lane activity. • The LED labeled ‘F’ indicates the firmware state. • The LED labeled ‘H’ indicates hardware and management state. 4-3-1 Firmware State LED The following table describes the meaning of the “F”...
• Core Dumps in Flash • Power-on Selftest (POST) Failure • PCIe Link is up, not configured correctly • Additional “attention” state where the user should access Fuji’s Management Interface for status 4-3-2 Reset EMP LED The secondary Ethernet Management Port (EMP) and reset state are represented on a dual LED in the same package.
Onyx Setup Onyx 2.0 shipped with the default configuration: Ethernet 2x50Gbps, PCIe 2x8. While customers can change the settings as required, we changed the settings to be Ethernet 1x100Gbps, and set the IP address to be 3.3.3.1/24. set-autoneg 1 1 1 2 8; set-autoneg 2 1 1 2 0 Configure the Ethernet to be 100G RS FEC, here is a high-level description of these parameters: set-autoneg <Port>...
Page 27
It’s strongly recommended to check the Ethernet link status, PCIe link status and drives presented before further operations. show-version 1 Check the firmware version to make sure the firmware is up to date. show-pcie 0xff Check the PCIe link status and also the number of drives discovered. show-port 0xff Check the Ethernet link status.
Chapter 5 Appendix Ethernet/PCIe Configuration Changing PCIe bifurcation mode: To change the PCIe mode to either 2 x8 or 1 x16 (that is, either the PCIe interface is bifurcated in to 2 links, 8 lanes each or is not bifurcated and is a single link with 16 lanes), execute the appropriate fru_bifurcate command shown below.
Page 30
To select 2 x8 mode: To select 1 x16 mode: ### Commit the change to Fuji’s persistent memory To select the different Ethernet speeds 50G FC FEC 100G RS FEC Here is a high-level description of these parameters: set-autoneg <Port> <TX> <RX> <FEC> <TECH> <Port>...