Broadcom 96 Series User Manual

Broadcom 96 Series User Manual

Pcie 4.0, 24g sas megaraid and ehba trimode storage adapters
Table of Contents

Advertisement

®
Broadcom
96xx PCIe 4.0, 24G SAS MegaRAID
and eHBA Tri-
Mode Storage Adapters
User Guide
Version 2.4
Broadcom
96xx-MR-HBA-Tri-Mode-UG108
January 17, 2024

Advertisement

Table of Contents
loading
Need help?

Need help?

Do you have a question about the 96 Series and is the answer not in the manual?

Questions and answers

Summary of Contents for Broadcom 96 Series

  • Page 1 ® ™ Broadcom 96xx PCIe 4.0, 24G SAS MegaRAID and eHBA Tri- Mode Storage Adapters User Guide Version 2.4 Broadcom 96xx-MR-HBA-Tri-Mode-UG108 January 17, 2024...
  • Page 2 Broadcom reserves the right to make changes without further notice to any products or data herein to improve reliability, function, or design. Information furnished by Broadcom is believed to be accurate and reliable. However, Broadcom does not assume any liability arising out of the application or use of this information, nor the application or use of any product or circuit described herein, neither does it convey any license under its patent rights nor the rights of others.
  • Page 3: Table Of Contents

    Device Certificate..............................26 Attestation Procedure..............................26 SPDM Capabilities................................26 Adapter Installation Instructions...................... 28 Broadcom MegaRAID and HBA Tri-Mode Storage Adapter Characteristics........ 31 MegaRAID 9670W-16i Adapter – Connector and LED Designations................ 31 MegaRAID 9670-24i Adapter – Connector and LED Designations................32 Auxiliary Power Connector............................34 MegaRAID 9660-16i Adapter –...
  • Page 4 ® ™ Broadcom 96xx User Guide Broadcom 96xx PCIe 4.0, 24G SAS MegaRAID and eHBA Tri-Mode Storage Adapters eHBA 9620-16i Adapter – Connector and LED Designations..................37 eHBA 9600-24i Adapter – Connector and LED Designations..................39 eHBA 9600-16i Adapter – Connector and LED Designations..................41 eHBA 9600-8i8e Adapter –...
  • Page 5: Overview

    This document is the primary reference and user guide for the Broadcom PCIe 4.0, 24G SAS MegaRAID storage adapters and enhanced HBA (eHBA) tri-mode storage adapters, based on the Broadcom PCIe 4.0, 24G SAS tri‑mode controllers. This document contains the complete installation instructions and specifications for the following tri‑mode storage adapters, referred to as adapters:...
  • Page 6 ® ™ Broadcom 96xx User Guide Broadcom 96xx PCIe 4.0, 24G SAS MegaRAID and eHBA Tri-Mode Storage Adapters Table 2: eHBA Tri-Mode Storage Adapter Features Adapter 9600-24i 9600-16i 9600-8i8e 9600W-16e 9600-16e Ports 24 internal 16 internal 8 internal 16 external...
  • Page 7: Features

    MegaRAID Tri-Mode Software User Guide, which can be found in the Support Documents and Downloads section of the Broadcom website. MegaRAID 9670W-16i, 9670-24i, and 9660-16i Adapter RAID Features The MegaRAID 9670W-16i, MegaRAID 9670-24i, and MegaRAID 9660-16i adapters support the following RAID features: •...
  • Page 8: Ehba 9600 Adapter Features

    ® ™ Broadcom 96xx User Guide Broadcom 96xx PCIe 4.0, 24G SAS MegaRAID and eHBA Tri-Mode Storage Adapters • RAID levels 0 and 1 • RAID span 10 • SAS/SATA drives: 32 • NVMe SSDs: 32 • JBOD PD state for SDS environments •...
  • Page 9: Pcie Host Interface

    • Fedora • FreeBSD The firmware and drivers are routinely updated and made available on the Broadcom Support and Download center. Visit http://www.broadcom.com/support/download-search and download the latest firmware and driver for the adapter. PCIe Host Interface The adapter’s PCIe 4.0 host interface provides maximum transmission and reception rates of up to 128 GT/s (16GB/s per lane).
  • Page 10 ® ™ Broadcom 96xx User Guide Broadcom 96xx PCIe 4.0, 24G SAS MegaRAID and eHBA Tri-Mode Storage Adapters – SAS data transfers at 22.5Gb/s, 12Gb/s, and 6Gb/s – DataBolt technology on all SAS phys to improve performance – Serial, point-to-point, enterprise-level storage interface –...
  • Page 11: Tri-Mode Storage Interface

    ® ™ Broadcom 96xx User Guide Broadcom 96xx PCIe 4.0, 24G SAS MegaRAID and eHBA Tri-Mode Storage Adapters Tri-Mode Storage Interface The tri-mode interface groups phys into two CSW blocks that contain 16 phys each. Depending on the adapter, either one or both CSW blocks are used for the storage interface.
  • Page 12: Pcie (Nvme) Support

    Tri-Mode Storage Adapters PCIe (NVMe) Support The following table shows how many NVMe drives or Broadcom PEX88000-series switches can directly attach to each adapter. The 9600W-16e adapter, 9600-16e adapter, and 9600-8i8e adapter (external ports) do not support direct attach to NVMe drives.
  • Page 13 ® ™ Broadcom 96xx User Guide Broadcom 96xx PCIe 4.0, 24G SAS MegaRAID and eHBA Tri-Mode Storage Adapters Connector 0 Connector 1 CSW1[0:7] CSW1[8:15] Table 6: 9660-16i Adapter PCIe Topology Configuration Combinations Connector 0 Connector 1 CSW0[0:7] CSW1[0:7] Table 7: 9670-24i Adapter PCIe Topology Configuration Combinations...
  • Page 14 ® ™ Broadcom 96xx User Guide Broadcom 96xx PCIe 4.0, 24G SAS MegaRAID and eHBA Tri-Mode Storage Adapters Connector 0 Connector 1 Connector 2 CSW1[0:7] CSW1[8:15] CSW0[8:15] x16 (paired with Connector 2) x16 (paired with Connector 0) Use any row from the previous 9660-16i Adapter PCIe Topology Configuration Combinations table.
  • Page 15 ® ™ Broadcom 96xx User Guide Broadcom 96xx PCIe 4.0, 24G SAS MegaRAID and eHBA Tri-Mode Storage Adapters Table 9: 9600-24i Adapter PCIe Topology Configuration Combinations Connector 0 Connector 1 Connector 2 CSW1[8:15] CSW0[8:15] CSW1[0:7] Combine with any of the first four configurations in this column.
  • Page 16: Common Refclk Support

    ® ™ Broadcom 96xx User Guide Broadcom 96xx PCIe 4.0, 24G SAS MegaRAID and eHBA Tri-Mode Storage Adapters Table 11: 9600-8i8e Adapter PCIe Topology Configuration Combinations Connector 0 Connector 1 Connector 2 CSW1[0:7] CSW0[12:15] CSW0[8:11] — — Common REFCLK Support The adapter uses x8 SFF-8654 (SlimSAS) connectors.
  • Page 17: Backplane Management

    (UBM) specification for backplane management. Virtual Pin Port Management Broadcom requires new designs to enable UBM for backplane management. The adapter maintains support for Virtual Pin Port (VPP) backplane management for legacy implementations. The adapters provide LED operation for NVMe devices based on the VPP over I C definition.
  • Page 18 ® ™ Broadcom 96xx User Guide Broadcom 96xx PCIe 4.0, 24G SAS MegaRAID and eHBA Tri-Mode Storage Adapters Table 12: Sideband Signal Pinout Connector Connector Sideband or Vendor UBM Assignments Direction Resistor Value A Side B Side Specific Pin Number...
  • Page 19 ® ™ Broadcom 96xx User Guide Broadcom 96xx PCIe 4.0, 24G SAS MegaRAID and eHBA Tri-Mode Storage Adapters Pin Name Settings Description C_TYPE, — Open collector/drain input or output signal. • D_INPL#, C_TYPE. If BP_TYPE is 0, the adapter drives this signal LOW. If BP_TYPE...
  • Page 20: Cables And Cabling Configurations

    ® ™ Broadcom 96xx User Guide Broadcom 96xx PCIe 4.0, 24G SAS MegaRAID and eHBA Tri-Mode Storage Adapters Cables and Cabling Configurations The internal adapter follows the SFF-9402 standard for connector signal assignments. Each x8 connector includes eight PCIe transmit and receive lanes and two sets of sidebands designated as A and B, in accordance with the SFF-9402 specification.
  • Page 21: Storage Interface Cabling

    NVMe drives. Broadcom provides the following cables to use for the adapter. Use the MPN listed to order a cable from Broadcom. If you source your own cables, use the Broadcom part number from the following table and the drawings and pinouts in...
  • Page 22: Backplane Connectors

    Broadcom 96xx User Guide Broadcom 96xx PCIe 4.0, 24G SAS MegaRAID and eHBA Tri-Mode Storage Adapters Table 16: Adapter Broadcom Cable Use Cases Type Description 05-60001-00 SAS/SATA and Use for backplanes with x4 SFF-8612 connectors with pinouts that follow the SFF-9402 specification.
  • Page 23 Vman port (management interface power) be toggled. The Broadcom adapter toggles the active cable’s Vact port (active cable power), but not the Vman port after an online firmware update or for other conditions that result in a soft reset. If your active cable requires that the Vman port be toggled, the cable must be removed and reinserted, or power to the card slot must be cycled.
  • Page 24: Cachevault Data Protection

    ® ™ Broadcom 96xx User Guide Broadcom 96xx PCIe 4.0, 24G SAS MegaRAID and eHBA Tri-Mode Storage Adapters CacheVault Data Protection The MegaRAID Tri-Mode storage adapters support data retention by using NAND flash memory down on the adapter, ™ backed up by a CacheVault Power Module 05 (CVPM05).
  • Page 25: Adapter Security

    Hardware secure boot requires that each image be signed with a valid digital signature; otherwise, the image is considered invalid and does not execute. The adapter ships with a valid signed firmware image. All Broadcom-supplied firmware includes a valid digital signature; therefore, the hardware secure boot process is transparent unless the adapter encounters a counterfeit image.
  • Page 26: Device Certificate

    The device certificate cannot be modified after manufacture. Broadcom manufacturing provisions each board with its device certificate. Every 96xx adapter manufactured is provisioned with a certificate chain. When the SPDM...
  • Page 27 ® ™ Broadcom 96xx User Guide Broadcom 96xx PCIe 4.0, 24G SAS MegaRAID and eHBA Tri-Mode Storage Adapters Table 19: SPDM CAPABILITIES Response Message Support CAPABILITIES Response Flags Field Value CACHE_CAP CERT_CAP CHAL_CAP MEAS_CAP MEAS_FRESH_CAP ENCRYPT_CAP MAC_CAP MUT_AUTH_CAP KEY_EX_CAP PSK_CAP...
  • Page 28: Adapter Installation Instructions

    Adapter Installation Instructions Use the following steps to install the adapter: 1. Unpack your adapter. Unpack and remove the adapter. Inspect the adapter for damage. If it appears damaged, contact Broadcom Technical Support. ATTENTION To avoid the risk of data loss, back up your data before you change your system configuration.
  • Page 29 ® ™ Broadcom 96xx User Guide Broadcom 96xx PCIe 4.0, 24G SAS MegaRAID and eHBA Tri-Mode Storage Adapters Figure 4: Installing an Adapter in a PCIe Slot 6. Configure and install the SAS, SATA, and PCIe (NVMe) devices in the host computer case.
  • Page 30 11. Install the operating system driver. The adapters can operate under various operating systems. To operate under these operating systems, you must install the software drivers. The firmware and drivers are routinely updated and made available on the Support and Download center. Visit http://www.broadcom.com/support/download-search, and download the latest firmware and driver for the adapter.
  • Page 31: Broadcom Megaraid And Hba Tri-Mode Storage Adapter Characteristics

    Broadcom 96xx PCIe 4.0, 24G SAS MegaRAID and eHBA Tri-Mode Storage Adapters Broadcom MegaRAID and HBA Tri-Mode Storage Adapter Characteristics This chapter presents the physical characteristics of each board, including the board size and the connector locations. MegaRAID 9670W-16i Adapter – Connector and LED Designations The adapter is a 167.52 (±0.13) mm ×...
  • Page 32: Megaraid 9670-24I Adapter - Connector And Led Designations

    I C interface connected to the I C bus for the Intelligent Platform Management Interface (IPMI). Default SBR header 2-pin connector. Reserved for Broadcom use. Advanced software options hardware key 2-pin connector. header Enables support for selected advanced features.
  • Page 33 I C interface connected to the I C bus for the Intelligent Platform Management Interface (IPMI). Default SBR header 2-pin connector. Reserved for Broadcom use. Advanced software options hardware key 2-pin connector. header Enables support for selected advanced features.
  • Page 34: Auxiliary Power Connector

    ® ™ Broadcom 96xx User Guide Broadcom 96xx PCIe 4.0, 24G SAS MegaRAID and eHBA Tri-Mode Storage Adapters Connector Type Description 9-pin connector. CacheVault power module interface Connects the adapter to a CacheVault power module. J17, J18, Storage interface connectors Three SFF-8654 8-port internal connectors.
  • Page 35: Megaraid 9660-16I Adapter - Connector And Led Designations

    ® ™ Broadcom 96xx User Guide Broadcom 96xx PCIe 4.0, 24G SAS MegaRAID and eHBA Tri-Mode Storage Adapters Table 24: Auxiliary Power Connector Pin Definition Signal +12V +12V +12V Ground Sense Ground MegaRAID 9660-16i Adapter – Connector and LED Designations The adapter is a 155.52 (±0.13) mm ×...
  • Page 36 With the PCIe interface, this connector provides power to the board and an C interface connected to the I C bus for the Intelligent Platform Management Interface (IPMI). Default SBR header 2-pin connector. Reserved for Broadcom use. Advanced software options hardware 2-pin connector. key header Enables support for selected advanced features.
  • Page 37: Ehba 9620-16I Adapter - Connector And Led Designations

    ® ™ Broadcom 96xx User Guide Broadcom 96xx PCIe 4.0, 24G SAS MegaRAID and eHBA Tri-Mode Storage Adapters Connector Type Description Global drive fault LED header 2-pin connector. Connects to an LED that indicates whether a drive is in a fault condition.
  • Page 38 C interface connected to the I C bus for the Intelligent Platform Management Interface (IPMI). Default SBR header 2-pin connector. Reserved for Broadcom use. Onboard serial UART connector 4-pin connector. Reserved for Broadcom use. Global HDD activity LED header 2-pin connector.
  • Page 39: Ehba 9600-24I Adapter - Connector And Led Designations

    ® ™ Broadcom 96xx User Guide Broadcom 96xx PCIe 4.0, 24G SAS MegaRAID and eHBA Tri-Mode Storage Adapters Connector Type Description J17, J18 Storage interface connectors Two SFF-8654 8-port internal connectors. Connect the adapter by cable to the storage devices.
  • Page 40 With the PCIe interface, this connector provides power to the board and an I interface connected to the I C bus for the IPMI. Default SBR header 2-pin connector. Reserved for Broadcom use. Onboard serial UART connector 4-pin connector. Reserved for Broadcom use. J17, J18, Storage interface connectors Three SFF-8654 8-port internal connectors.
  • Page 41: Ehba 9600-16I Adapter - Connector And Led Designations

    ® ™ Broadcom 96xx User Guide Broadcom 96xx PCIe 4.0, 24G SAS MegaRAID and eHBA Tri-Mode Storage Adapters Table 30: LED Designations Type Description LED2 Yellow IOC overtemperature Stays on solid to indicate that the SAS4024 IOC temperature sensor is over the temperature threshold.
  • Page 42: Ehba 9600-8I8E Adapter - Connector And Led Designations

    With the PCIe interface, this connector provides power to the board and an I interface connected to the I C bus for the IPMI. Default SBR header 2-pin connector. Reserved for Broadcom use. Onboard serial UART connector 4-pin connector. Reserved for Broadcom use. J17, J18 Storage interface connectors Two SFF-8654 8-port internal connectors.
  • Page 43 With the PCIe interface, this connector provides power to the board and an I interface connected to the I C bus for the IPMI. Default SBR header 2-pin connector. Reserved for Broadcom use. Onboard serial UART connector 4-pin connector. Reserved for Broadcom use. Storage interface connector One SFF-8654 8-port internal connector.
  • Page 44: Ehba 9600W-16E Adapter - Connector And Led Designations

    ® ™ Broadcom 96xx User Guide Broadcom 96xx PCIe 4.0, 24G SAS MegaRAID and eHBA Tri-Mode Storage Adapters Table 34: LED Designations Type Description LED2 Yellow IOC overtemperature Stays on solid to indicate that the SAS4016 IOC temperature sensor is over the temperature threshold.
  • Page 45: Ehba 9600-16E Adapter - Connector And Led Designations

    With the PCIe interface, this connector provides power to the board and an I interface connected to the I C bus for the IPMI. 2-pin connector. Reserved for Broadcom use. Default SBR header 4-pin connector. Reserved for Broadcom use. Onboard serial UART connector...
  • Page 46 With the PCIe interface, this connector provides power to the board and an I interface connected to the I C bus for the IPMI. 2-pin connector. Reserved for Broadcom use. Default SBR header 4-pin connector. Reserved for Broadcom use. Onboard serial UART connector...
  • Page 47 ® ™ Broadcom 96xx User Guide Broadcom 96xx PCIe 4.0, 24G SAS MegaRAID and eHBA Tri-Mode Storage Adapters Table 38: LED Designations Type Description LED2 Yellow IOC overtemperature Stays on solid to indicate that the SAS4016 IOC temperature sensor is over the temperature threshold.
  • Page 48: Tri-Mode Storage Adapter Technical Specifications

    ® ™ Broadcom 96xx User Guide Broadcom 96xx PCIe 4.0, 24G SAS MegaRAID and eHBA Tri-Mode Storage Adapters Tri-Mode Storage Adapter Technical Specifications This chapter presents the technical specifications of each board, including operating conditions and power supply requirements. Operating and Non-operating Conditions The following table lists the operating (thermal and atmospheric) conditions and non-operating (such as storage and transit) environment for the storage adapters.
  • Page 49: Megaraid Tri-Mode Storage Adapter Power Supply Requirements

    ® ™ Broadcom 96xx User Guide Broadcom 96xx PCIe 4.0, 24G SAS MegaRAID and eHBA Tri-Mode Storage Adapters used. For adapters with auxiliary power connectors, the slot power check is bypassed if the auxiliary power connector is connected. For more information on the auxiliary power connector, see Auxiliary Power Connector.
  • Page 50: Marks, Certifications, Compliance, And Safety Characteristics

    ® ™ Broadcom 96xx User Guide Broadcom 96xx PCIe 4.0, 24G SAS MegaRAID and eHBA Tri-Mode Storage Adapters Marks, Certifications, Compliance, and Safety Characteristics This chapter lists the adapter marks and certifications, FCC compliance statements, and safety characteristics. Marks, Certifications, and Compliance The design and implementation of the adapters minimize electromagnetic emissions, susceptibility to radio frequency energy, and the effects of electrostatic discharge.
  • Page 51 ® ™ Broadcom 96xx User Guide Broadcom 96xx PCIe 4.0, 24G SAS MegaRAID and eHBA Tri-Mode Storage Adapters Mark Symbol Description Korea (RRL) xxxxx = model number Meets the KN32/KN35 testing requirements. Taiwan Meets the following standards: • (BSMI) CNS 13438 •...
  • Page 52: Fcc Compliance

    ® ™ Broadcom 96xx User Guide Broadcom 96xx PCIe 4.0, 24G SAS MegaRAID and eHBA Tri-Mode Storage Adapters FCC Compliance This device complies with part 15 of the FCC rules. Operation is subject to the following two conditions: (1) this device may not cause harmful interference, and (2) this device must accept any interference received, including interference that may cause undesired operation.
  • Page 53: Appendix A: Cable Drawings And Pinouts

    Use the cable drawings and pinouts in this appendix if your design requires you to design your own cables. Cable 05-60001-00 The following figure shows the drawing and pinout for Broadcom cable 05-60001-00, a x8 SFF-8654 to 2 x4 SFF-8612 connection.
  • Page 54 Broadcom 96xx User Guide Broadcom ® 96xx PCIe 4.0, 24G SAS MegaRAID ™ and eHBA Tri-Mode Storage Adapters Figure 15: Cable 05-60001-00 Drawing and Pinout Broadcom 96xx-MR-HBA-Tri-Mode-UG108...
  • Page 55: Cable 05-60002-00

    Tri-Mode Storage Adapters Cable 05-60002-00 The following figure shows the drawing and pinout for Broadcom cable 05-60002-00, a x8 SFF-8654 to 2 x4 SFF-8643 connection. Use this cable for NVMe connections on SuperMicro Purley backplanes. Figure 16: Cable 05-60002-00 Drawing and Pinout...
  • Page 56: Cable 05-60003-00

    96xx PCIe 4.0, 24G SAS MegaRAID and eHBA Tri-Mode Storage Adapters Cable 05-60003-00 The following figure shows the drawing and pinout for Broadcom cable 05-60003-00, a x8 SFF-8654 to 2 x4 SFF-8643 connection. Figure 17: Cable 05-60003-00 Drawing and Pinout...
  • Page 57: Cable 05-60004-00

    96xx PCIe 4.0, 24G SAS MegaRAID and eHBA Tri-Mode Storage Adapters Cable 05-60004-00 The following figure shows the drawing and pinout for Broadcom cable 05-60004-00, a x8 SFF-8654 to 2 x4 SFF-8654 connection. Figure 18: Cable 05-60004-00 Drawing and Pinout...
  • Page 58: Cable 05-60005-00

    96xx PCIe 4.0, 24G SAS MegaRAID and eHBA Tri-Mode Storage Adapters Cable 05-60005-00 The following figure shows the drawing and pinout for Broadcom cable 05-60005-00, a x8 SFF-8654 to 2 U.2 SFF-8639 connection. Figure 19: Cable 05-60005-00 Drawing and Pinout...
  • Page 59: Cable 05-60006-00

    96xx PCIe 4.0, 24G SAS MegaRAID and eHBA Tri-Mode Storage Adapters Cable 05-60006-00 The following figure shows the drawing and pinout for Broadcom cable 05-60006-00, a x8 SFF-8654 to x8 U.3 SFF-8639 connection. Figure 20: Cable 05-60006-00 Drawing and Pinout...
  • Page 60 ® ™ Broadcom 96xx User Guide Broadcom 96xx PCIe 4.0, 24G SAS MegaRAID and eHBA Tri-Mode Storage Adapters Broadcom 96xx-MR-HBA-Tri-Mode-UG108...
  • Page 61: Cable 05-60007-00

    96xx PCIe 4.0, 24G SAS MegaRAID and eHBA Tri-Mode Storage Adapters Cable 05-60007-00 The following figure shows the drawing and pinout for Broadcom cable 05-60007-00, a x8 SFF-8654 to x8 SFF-8654 connection. Figure 21: Cable 05-60007-00 Drawing and Pinout Broadcom...
  • Page 62: Appendix B: Revision History

    Updated the 9670-24i adapter connector-to-CSW port associations in Tri-Mode Storage Interface PCIe (NVMe) Support. Version 2.1, August 1, 2023 • Updated the adapter dimensions in Broadcom MegaRAID and HBA Tri-Mode Storage Adapter Characteristics. • Updated External Adapter Connector Pinout. Version 2.0, August 30, 2022 •...
  • Page 63 ® ™ Broadcom 96xx User Guide Broadcom 96xx PCIe 4.0, 24G SAS MegaRAID and eHBA Tri-Mode Storage Adapters Advance, Version 0.2, March 19, 2021 • Changed the 9660-16i cache memory description in the MegaRAID Tri-Mode Storage Adapter Features table in Overview.

Table of Contents

Save PDF