Page 1 Reference COMPUTE PROCESSING MODULE ATCA-4616 ATCA-4618 ATCA-4648 March 2012 007-03446-0000...
Page 2: Revision History
Revision history Version Date Description -0000 March 2012 First edition. © 2012 by RadiSys Corporation. All rights reserved. Radisys is a registered trademark of RadiSys Corporation. AdvancedTCA, ATCA, and PICMG are registered trademarks of PCI Industrial Computer Manufacturers Group. All other trademarks, registered trademarks, service marks, and trade names are the property of their respective owners.
Page 3: Table Of Contents
Table of Contents Preface ..............................7 About this manual............................7 Where to get more product information .......................7 About related Radisys products........................8 Standards information ..........................8 Related documents............................8 Electrostatic discharge ..........................10 Notational conventions ..........................10 Chapter 1: Product Overview ......................11 Introduction..............................11 Major features............................11 Supported external interfaces........................12...
Page 4 Table of Contents Intel C600 series Platform Controller Hub (PCH) .................29 IPMI controller............................33 CPU Complex (CC) FPGA........................34 IPMI FPGA............................36 Intel I350 quad GbE Ethernet controller....................36 Mellanox dual 40GbE controller......................36 Clock synthesizer subsystem .......................37 Reset subsystems ..........................37 Watchdog timers...........................39 Power subsystems..........................40 Trusted Platform Module (TPM)......................42 MXM type A video module (optional) ....................43 1.8”...
Page 5 Table of Contents Chapter 5: Troubleshooting and Repair ..................69 Introduction..............................69 Field Replaceable Units (FRUs) ........................69 FRU information areas used.........................69 CPM and FRU device IDs........................70 CPM replacement procedures........................71 Removing the CPM..........................71 Removing the CPM board cover......................72 Installing the CPM board cover......................72 Installing the CPM..........................72 Memory module replacement procedures ....................73 Adding or replacing memory modules ....................73...
Page 6 Table of Contents Appendix B: IPMI Commands and Managed Sensors ..............86 IPMI command interfaces..........................86 IPMI commands............................86 OEM command descriptions.........................89 Managed sensors ............................95 Types of sensors ..........................95 IPMI Sensors ............................96 Appendix C: Pinouts and Mapping ....................108 Front panel connectors..........................108 COM serial connector .........................108 Dual USB connectors .........................108 Dual Ethernet connectors ........................109...
Page 7: Preface
Preface About this manual This manual describes the ATCA‐46xx, a compute processing module (CPM), which is fully compliant with AdvancedTCA (ATCA ). The CPM is designed to be incorporated into High ® ® Availability (HA) systems such as the Radisys platforms SYS‐6006 and SYS‐6010. Use this manual as a hardware reference for the operation and maintenance of the ATCA‐46xx CPM. The manual also provides information on the electrical, the mechanical, and the environmental aspects of the ATCA‐46xx CPM. The material presented here is not introductory; it is assumed that you are already familiar with the intended use of the ATCA‐ 46xx CPM in your organization’s ATCA platform. The simplified names “CPM” and “module” will be used in place of “ATCA‐46xx CPM” for the body text in the remainder of this manual. Note: The software material since software release 3.2.0 now resides in the Software Guide for Management Processors and General Computing Processors and the Command Line Interface Reference. These manuals consolidate the software information for the ATCA modules. Where to get more product information Visit the Radisys web site at www.radisys.com for product information and other resources. Downloads (manuals, release notes, software, etc.) are available at www.radisys.com/downloads. See the following resources for information on the CPM not described in this manual: • Installation and initial setup instructions. The ATCA‐4xxx Compute Processing Module Installation Guide provides the steps for installing the CPM into a shelf and completing the initial configuration. • Platform hardware information. The CPM is incorporated into platform systems such as the Radisys platforms SYS‐6002,SYS‐6014, and SYS‐6016. For information about the platforms and their shelf hardware, see the appropriate Platform Hardware Installation ...
Page 8: About Related Radisys Products
Preface • Software reference information. The Software Guide for Management Processors and General Computing Processors describes software concepts and serves as a reference for procedural and usage information. When referenced in this manual, the simplified name of Software Guide will be used. • Shelf Manager information. The Shelf Management Software Reference describes the architecture and the operation of the Shelf Manager. The Shelf Manager typically runs on the SCM and controls and monitors operations on the shelf. • Update information. Firmware and software updates may be available for the CPM components from time to time. For information on updating components on the CPM and other modules, see the Firmware and Software Update Instructions. About related Radisys products The ATCA‐46xx CPM is part of the following Radisys platforms: SYS‐6002 and SYS‐6014/6016. For information on the ATCA product family and other Radisys products, see the Radisys Web site at www.radisys.com. Standards information For information about the PCI Industrial Computer Manufacturers Group (PICMG ) and the ® AdvancedTCA standard, consult the PICMG Web site http://www.picmg.org). Related documents Advanced Switching Core Architecture Specification Revision 1.0, Advanced Switching Interconnect Special Interest Group, December 2003. IEC 60950‐1:2005 (Second Edition), International Electrotechnical Commission (www.iec.ch) Additionally evaluated to EN 60950‐1:2006/A11:2009; National. Differences specified in the CB Test Report. Information Technology ‐ Serial Attached SCSI ‐ 1.1 (SAS‐1.1), ANSI/INCITS 417‐2006 UL 60950‐1:2007, 2nd Edition, (Information Technology Equipment ‐ Safety ‐ Part 1: General Requirements) CSA C22.2 No. 60950:2007, 2nd Edition, (Information Technology Equipment ‐ Safety ‐ Part 1: General Requirements) DDR3 SDRAM VLP RDIMM MT36JBZS51272PY – 4GB, Micron.
Page 9 Preface IEEE Std 802.3‐2002 Telecommunications and information exchange between systems — Local and metropolitan area networks — Specific requirements, Part 3: Carrier sense multiple access with collision detection (CSMA/CD) access method and physical layer specifications, IEEE Computer Society, March 8, 2002. Intelligent Platform Management Interface Specification v1.5, Revision 2.0, Intel Corporation; Hewlett Packard Company, NEC Corporation, and Dell Computer Corporation. Linux PAM Modules web site. http://www.kernel.org/pub/linux/libs/pam/modules.html MSP20B01B Backplane External Product Specification Revision 1.0, Intel, August 31, 2001. PICMG 3.0 Advanced Telecommunications Computing Architecture R3.0, PCI Industrial Computer Manufacturers Group, March 24, 2008 PICMG 3.1 R1.0 Specification, Ethernet/Fibre Channel for AdvancedTCA Systems, PICMG, January 22, 2003 PICMG AMC.0 R2.0 Advanced Mezzanine Card Base Specification, PCI Industrial Computer Manufacturers Group, November 15, 2006 PM8380 QuadSMX 3G Quad SATA/SAS Mux/Demux for 3G, PMC‐2031101, Issue 8, PMC‐ Sierra, April 2005 Serial ATA Revision 2.5, Serial ATA International Organization, October 27, 2005 Platform Management FRU Information Storage Definition v1.0, Revision 1.13. Integrated RAID for SAS User’s Guide, http://www.lsi.com/DistributionSystem/AssetDocument/files/docs/techdocs/storage_stand_ prod/sas/ir_sas_ug.pdf from LSI Logic Corporation.
Page 10: Electrostatic Discharge
Preface Electrostatic discharge ! ARNING This product contains static‐sensitive components and should be handled with care. Failure to employ adequate anti‐static measures can cause irreparable damage to components. Electrostatic discharge (ESD) damage can result in partial or complete device failure, performance degradation, or reduced operating life. To avoid ESD damage, the following precautions are strongly recommended. • Keep each carrier in its ESD shielding bag until you are ready to install it. • Before touching a carrier, attach an ESD wrist strap to your wrist and connect its other end to a known ground. • Handle the carrier only in an area that has its working surfaces, floor coverings, and chairs connected to a known ground. • Hold carrier only by its edge and mounting hardware. Avoid touching PCB components and connector pins. For further information on ESD, visit www.esda.org. Notational conventions This manual uses the following conventions BoldText A keyword. ItalicText File, function, and utility names. MonoText Screen text and syntax strings. BoldMonoText A command to enter. Variable parameters. ItalicMonoText Brackets [ ] Command options.
Page 11: Chapter 1: Product Overview
Chapter Product Overview Introduction The ATCA‐46xx Compute Processing Module (CPM) is a high‐end, general‐purpose computing module that provides multi‐core processing power and multiple data‐storage options within a single Advanced Telecommunications Computing Architecture (ATCA) slot. The remaining sections in this chapter present the major features, specifications compliance, and product options that apply to the ATCA‐46xx CPM. Major features The following major features apply to the ATCA‐46xx CPM: ® ® • Dual Intel Xeon E5‐2400 family 64‐bit multi‐core processors using 32‐nm process technology • Intel C600 series Platform Controller Hub (PCH) • Multiple GbE interfaces • Fabric interface supporting 40 GbE connections • Intelligent Platform Management Controller (IPMC) • CPU Complex FPGA (CC FPGA) • Redundant 64 Mb SPI Boot Flash devices • eUSB Flash Memory modules (optional) • Trusted Platform Module (TPM) • MXM graphics module (optional) • Onboard SATA SSD drives (optional)
Page 12: Supported External Interfaces
Product Overview Supported external interfaces The ATCA‐46xx CPM supports both internal and external interfaces. Internal interfaces include the buses and communication protocols that are fully contained within the CPM blade or are included within the front panel, backplane, or RTM external interfaces. The following external interfaces are supported for the ATCA‐46xx CPM: • Front panel interfaces: • Dual USB connectors (Type A) • RJ‐45 serial port connector (RS‐232, COM1) • Dual RJ‐45 GbE connectors • Mini DisplayPort connector (when optional MXM video module is installed) • Reset push button • Hot Swap extraction switch (part of ejector handle) • LED indicators •Blue hot swap LED •Red or amber out of service (OOS) LED •Green/amber user‐defined (APP) LED •Green HDD activity LED •Green/amber Link/Speed/Activity LEDs (each Base/Fabric interface) • Backplane interfaces (ATCA Zone 2 connectors) • RTM interface (ATCA Zone 3 connectors)
Page 13: Specification Compliance
Product Overview Specification compliance The ATCA‐46xx CPM complies with the following specifications: • PCI Industrial Computers Manufacturers Group (PICMG) 3.0 R2.0 ECN0002 Advanced Telecommunications Computing Architecture (ATCA) specification • PICMG 3.1 R2.0 Ethernet over ATCA (Options 1 and 9) specification • European requirements for hazardous materials (ROHS 6/6) • Mechanical and environmental specifications: • CE • FCC Class A • VCCI‐A • cUL • NEBS Level 3 (designed to meet) • Safety specifications: • USA ‐ UL 60950‐1 • Canada ‐ CSA 22.2 #60950‐1 • EU ‐ EN 60950‐1 • Other ‐ IEC 60950‐1 • Electromagnetic Compatibility (EMC) specifications: • Emissions; radiated/conducted (FCC Part 15 and EN 55022:2006) • Immunity; ESD (EN 61000‐4‐2), radiated (EN 61000‐4‐3), fast transient/burst (EN 61000‐4‐4), surge voltages (EN 61000‐4‐5), conducted (EN 61000‐4‐6), magnetic field ...
Page 14: Product Options
Product Overview Product options The ATCA‐46xx CPM has the following product options: • A4616‐CPU‐Base ‐ The basic low‐power CPM with 10G Fabric interface, dual Intel Xeon six core processors, and no installed memory. • A4618‐CPU‐Base ‐ The basic high‐power CPM with 10G Fabric interface, dual Intel Xeon eight core processors, and no installed memory. • A4648‐CPU‐Base ‐ The basic high‐power CPM with 40G Fabric interface, dual Intel Xeon eight core processors and no installed memory. • A4600‐MEM‐xxGB ‐ A memory kit consisting of twelve yGB DDR3 VLP registered DIMM (RDIMM) modules that provides a total of xxGB of RDIMM memory for the CPM. • A4600‐eUSB‐32GB ‐ A memory expansion option with two eUSB NAND Flash modules with a capacity of 16GB per module. • A4600‐DSSDMXM‐M ‐ A mass storage option with two 1.8“ Solid State Disk (MLC SSD) modules of 64GB capacity each. The SSD modules mount on a dual drive carrier that connects through the MXM connector to the onboard SATA ports of the CPM. For a current listing of product options, visit the ATCA‐46xx product page at www.radisys.com.
Page 15: Chapter 2: Hardware Description
Chapter Hardware Description Introduction The ATCA‐46xx Compute Processing Module (CPM) uses a number of hardware components to implement the functions required of an ATCA node. The major hardware components in the ATCA‐46xx CPM are as follow: • Dual E5‐2400 family processors mounted in FCLGA1356 sockets • Intel C600 series Platform Controller Hub (PCH) chip that supports the major I/O functionality on the CPM. • Sockets for six registered DIMM (RDIMM) or Load Reduced DIMM (LRDIMM) memory modules per processor or twelve modules total. Supported RDIMMs and LRDIMMs include 800, 1066, or 1333 MHz DDR3 modules of 1, 2, 4, 8, or 16 GB capacity for a total of up to 96 GB per processor or 192 GB total for the module. • Intel I350 quad Gigabit Ethernet controller that supports two GbE Ethernet ports connected to the Base Interface channels and GbE Ethernet ports routed to the CPM front panel or the RTM interface. • A Mellanox dual 40 GbE controller that provides two 40 Gb Ethernet ports for the CPM fabric interface. • Heat sinks to dissipate heat generated by the E5‐2400 family processors, the PCH, and the 40 GbE controller. • Front panel connectors for serial, video, USB, and Ethernet. • Front panel LEDs that indicate CPM condition and status information. • Zone 1 (P10), Zone 2 (J20 and J23), and Zone 3 (J30 and J31) connectors to the backplane and RTM. • (Optional) MXM 3.0 type A video module • (Optional) One or two 1.8” microSATA SSD drives mounted in an MXM DSSD module tray • (Optional) One or two eUSB Flash modules...
Page 16: Functional Block Diagram
Hardware Description Functional block diagram Figure 1 is a functional block diagram that indicates major hardware components of the CPM. Figure 1. ATCA-46xx Functional Block Diagram DDR3 VLP RDIMMs DDR3 VLP RDIMMs CPU0 CPU1 CH. C CH. C Xeon® Xeon® Front CH. B CH. B E5-2400 Panel E5-2400 CH. A CH. A Family CPU Family CPU PCIe Gen 3 PCIe Gen 3...
Page 17: Atca-46Xx Front Panel Components
Hardware Description ATCA-46xx front panel components The following sections use text, figures, and lists to identify the physical features of the CPM. Figure 2 shows the CPM front panel and calls out the major features. Figure 2. Front Panel Components Thumbscrew Ejector Latch Video (MiniDP) Connector HDD LED Serial (COM) Port OOS LED PWR LED Ethernet Ports A,B APP LED (1000Base-T) USB Ports 0 & 1 Reset Button Base/Fabric Channel Status LEDs...
Page 18: Front Panel Connectors
Hardware Description Front panel connectors There are connectors for four separate interfaces on the front panel. Plugging into the connectors is straightforward, but the underlying interfaces need some explanation. The serial (COM) RJ45 connector is an RS232 serial interface with the pinout listed in Table 39 on page 108. The USB0 and USB1 connectors support USB 2.0 transactions. A USB cable connected to a single device up to 5 meters away can be used with the front panel USB connectors. Transfers at up to 480 Mbps and continuous load currents up to 500 mA are supported with these USB ports. Refer to Table 40 on page 108 for the USB connector pinout. The miniDisplayPort connector can be connected to a digital monitor if the MXM port has a video module installed. Refer to the video module documentation for operation details for this option. Refer to Table 42 on page 109 for the connector figure and pinout listing of the miniDisplayPort connector. During normal operation there is no need to attach a monitor to this connector. The two front panel Ethernet connectors provide interfaces to GbE ports with a peak available bandwidth of 100Mbps or 1Gbps. Figure 3 shows one of the front panel Ethernet connectors and calls out the Port Status and Link Status LEDs that are part of the connector. Figure 3. Front Panel Ethernet Connectors Port Status LED Link Status LED Table 1 lists the meanings assigned to the light color/activity of each LED on the Ethernet ...
Page 19: Front Panel Leds
Hardware Description Front panel LEDs The front panel LEDs can be separated into two major groups; the edge LEDs along the left (bottom) edge of the front panel and the Base/Fabric channel status LEDs. Table 2 describes the status and activity LEDs along the left (bottom) edge of the front panel. Table 2. Front Panel Edge Activity/Status LEDs LED Designation Indication Meaning ████ HDD (Hard Disk Short Green blink HDD being accessed Drive) LED ████ LED off No HDD activity, or not installed ████...
Page 20: Reset And Hot Swap Switches
Hardware Description Table 3 lists the meanings assigned to the light color/activity of each Base/Fabric LED. Table 3. Base/Fabric Channel LEDs Channel Indication Meaning ████ Steady Green Linked at 1 Gbps peak bandwidth Base Channel 1 ████ Blinking Green Activity at 1 Gbps peak bandwidth ████ Steady Amber Linked at 10/100 Mbps peak bandwidth Base Channel 2 ████...
Page 21: Rear Panel Connectors
Hardware Description Rear panel connectors The CPM supports E‐Key control by describing its backplane interfaces to the Shelf Manager. Alignment keys The CPM implements the K1 and K2 alignment blocks at the top of Zone 2 and Zone 3, as required by the PICMG 3.0 specification. The Zone 2 alignment block (K1) is assigned a keying value of 11. The Zone 3 alignment block (K2) is set to allow insertion of ATCA‐46xx‐compatible RTMs. Zone 1 connector The connectors in Zone 1 provide redundant ‐48VDC power and Shelf Management signals to the boards. The power control interface to the IPMC is through the backplane connector P10 (Zone 1 power distribution connector). Refer to Table 44 on page 111 for details. Zone 2 connectors The connectors in Zone 2 provide the connections to the Base Interface and Fabric Interface. All Fabric connections use point‐to‐point 100 Ω differential signals. Zone 2 is called "Fabric Agnostic" which means that any Fabric that can use 100 Ω differential signals can be used with an AdvancedTCA backplane. Backplane connector J20 provides Zone 2 connections to the synchronization clocks and to the AMC update channels. For detailed information about the connector itself, refer to Table 45 on page 112. Backplane connector J23 is the ATCA data transport connector, which provides Zone 2 connections for two 10/100/1000BASE‐T Ethernet Base channels and two 10GBASE‐BX4 Ethernet Fabric channels. For more information about the J23 backplane connector, refer to Table 46 on page 112.
Page 22: Zone 3 Connectors
Hardware Description Zone 3 connectors The CPM includes the standard Zone 3 backplane interface to provide connectivity to an optional RTM, such as the ATCA‐5400. This interface consists of two connectors: J30 for common and maintenance signals, and J31 for SerDes (serialization/deserialization connectivity). For details, refer to Table 47 on page 113 and Table 48 on page 113. The electrical connections between the CPM and the associated RTM include: • Switched +3.3V and +12V power, under the control of the IPMC • One configurable PCI Express port • One USB 2.0 Port • IPMB‐L I C Bus for an MMC on an RTM • IPMC Hot Swap control signals • RTM‐Link interface to CPU Complex FPGA • Serial Port (3.3V signals) • Two 1Gbps SerDes/SGMII ports • Two SFP I C ports for SGMII support • System reset signal • JTAG interface • PCH SMBus connection...
Page 23: Atca-46Xx Board Components
Hardware Description ATCA-46xx board components During normal operation the CPM board components are covered by the CPM cooling shroud sheet metal (side panel). This sheet metal must be removed to see and gain access to removable board components, jumper blocks, or headers. Figure 4 shows the CPM board layout and calls out the major components and other features. Figure 4. CPM Board Layout Customer header CPU1 RDIMM bank CPU1 (under heatsink) connector Front panel eUSB connectors Mellanox SuperCAP CPU0 (under heatsink) CPU0 RDIMM bank...
Page 24: Heatsinks
Hardware Description Heatsinks The CPU heatsinks are called out in Figure 4. In addition to the CPU heatsinks, the Intel C600 series PCH, and the Mellanox CX3 10/40 GbE controller each have individual heatsinks. The CPM incorporates a large heatsink covering each processor plus the additional heatsinks for other onboard high power devices to support a maximum CPM subsystem power dissipation of up to 50W. In addition, the installed RDIMM modules might also have their own heatsinks or other thermal solution. ARNING All heatsinks on the CPM are critical for proper board operation. Make sure all heatsinks have adequate mechanical and thermal contact with their associated components and ensure the sheet metal cooling shroud is secured in place before installing the CPM in its slot. Major components Figure 4 calls out the following major components on the CPM board: • Two Intel Xeon E5‐2400 family processors. Refer to Intel® Xeon® E5‐2400 family processor on page 25 for detailed information. • Sockets for twelve registered DIMM memory modules (six for each CPU). Refer to DIMM memory on page 28 for detailed information. • The C600 series PCH component. Refer to Intel C600 series Platform Controller Hub (PCH) on page 29 for detailed information. • Mellanox CX3 dual 10/40 GbE Ethernet controller. Refer to Mellanox dual 40GbE controller on page 36. In addition to the components called out in Figure 4, the following components (indicated in the Figure 1 block diagram) control significant portions of CPM operation: • The H8S/2400 series IPMI controller. Refer to IPMI controller on page 33 for more detailed information.
Page 25: Intel® Xeon® E5-2400 Family Processor
Hardware Description Intel® Xeon® E5-2400 family processor The CPM uses two Intel® Xeon E5‐2400 family multi‐core, 64‐bit processors built using a 32‐ nm process. The E5‐2400 processor includes a 3‐channel memory controller, QuickPath interconnect, and integrated I/O for PCI Express support. The processor cores share an up to 20MB cache and include support for the Execute Disable Bit, Speed‐Step, Virtualization, Streaming SIMD Extensions, Hyper‐Threading, Turbo Boost, and TXT. The ATCA‐46xx CPM E5‐ 2400 processors have a maximum thermal power dissipation (TPD) ranging from 50 W to 70 W depending on configuration. Table 4 and Table 5 provide the major processor specifications. Table 4. CPU bus specifications Number of Number of DDR3 Last Level Processor QPI Speed Cores Threads Speed Cache Size LV70W-8C 8.0 GT/s 1333 MHz 20 MB LV60W-6C...
Page 26: Hardware Description
Hardware Description Integrated Memory Controller (IMC) The integrated memory controller incorporated into each E5‐2400 family processor supports three channels of DDR3, each channel with 64 data bits and 8 ECC bits. The CPM supports up to six registered VLP RDIMMs per socket (two per channel) for a system total of twelve RDIMMs. Table 6 lists the supported RDIMM memory types of the CPM and provides the required specifications for the modules. Table 6. Supported Memory Types Mem Rank x Data Width Operating Voltage DIMMs per Channel Memory per DIMM Speed (MT/s) Registered DIMM (RDIMM) with ECC Single rank x 8 bits 1.5 v One or Two 1, 2, 4 GB...
Page 27: Memory
Hardware Description QuickPath Interconnect (QPI) The Quickpath interconnect (QPI) provides a point‐to‐point contact between the E5‐2400 family processors. The QPI interface is 20 lanes wide under full operation and is the communication path between the CPUs. Data of any width is converted to packets and then sent serially over the QPI link. The E5‐2400 supports QPI speeds of 6.4 GT/s to 8.0 GT/s depending on the installed processor. PCI Express The integrated I/O module on each E5‐2400 family processor provides 24 PCI Express lanes that are capable of Gen1 (2.5 GT/s), Gen2 (5.0 GT/s) and Gen3 (8.0 GT/s) speeds. The lanes are split into a x16 and a x8 port and both can be divided into x8, x4, x2 and x1 ports. CPU0 also uses a Gen 2 Direct Media Interface (DMI) port that can be configured for either DMI for PCH connectivity or used as a Gen2 x4 PCI Express port. Table 7 shows the PCI Express port mapping for each CPU on the CPM. Table 7. PCI Express Port Mapping Port# Port Width PCI Express Peripheral CPU0 PE1(A,B) Fabric Ethernet Controller CPU0 PE3(A,B,C,D) CPU0 DMI2 Patsburg DMI2 interface CPU1 PE1A...
Page 28 Hardware Description DIMM memory This memory is directly addressed by the internal memory controller of each E5‐2400 family processor. Due to the board height limitations posed by the ATCA PICMG specifications, only Very Low Profile (VLP) Dual In‐line Memory Module (DIMM) modules are supported with the CPM. The Registered DIMM (RDIMM) memory used on the CPM is buffered by integral registers and has built‐in Error Correcting Code (ECC) bits to support more reliable operation. As part of the installation process for the CPM, up to twelve RDIMM modules (six per processor, from 1GB to 16GB each) must be installed in the CPM DIMM sockets. Refer to Adding or replacing memory modules on page 73 for the installation procedures. Non-volatile on-board memory The CPM non‐volatile memory is mainly comprised of the following elements: • The two 64‐Mbit Flash BIOS boot and redundant boot devices. • The 64‐Mbit ME Flash holding the redundant ME firmware. • IPMI non‐volatile memory stores IPMC private data and board FRU information. • The optional eUSB NAND Flash memory modules are available for user applications and data. External memory options The following external memory options are available for the CPM: • Embedded NAND Flash modules. One or two eUSB NAND Flash modules. • 1.8” micro Solid State Disk (SSD) drives. One or two μSATA SSD modules. This option requires the Radisys Dual SSD (DSSD) MXM module for mounting the SSD modules. • RTM‐installed hard disk drives (HDDs). Number and capacity dependent on the RTM.
Page 29: Intel C600 Series Platform Controller Hub (Pch)
Hardware Description Intel C600 series Platform Controller Hub (PCH) The Intel C600 series Platform Controller Hub (PCH) provides a connection point between various I/O components and the E5‐2400 family Xeon processors used on the CPM. The PCH provides controllers for the following interfaces: • Direct Media Interface • Universal Serial Bus • Serial Addressed SCSI (SAS) • Serial ATA (SATA) • Low Pin Count (LPC) Bridge • Serial Peripheral Interface (SPI) • Real‐Time Clock (RTC) • SMBus/I C bus • Management Engine Refer to Figure 1 to identify the interfaces and the devices controlled by the PCH. The following sections provide details on the interfaces and associated controllers of the PCH. Direct Media Interface (DMI) There is a Direct Media Interface Gen 2 (DMI2) 4x link between the PCH and CPU0 that is capable of PCI Express Gen2 transfer rates (5 Gb/s) for a theoretical bandwidth of 20 Gb/s in each direction. This link is used to pass I/O from the SATA, SAS, LPC, and USB peripherals to ...
Page 30 Hardware Description Serial ATA (SATA) The PCH includes two SATA host controllers that provide six SATA 3.0 ports. These ports support data transfer rates of 3.0 Gb/s. The CPM connects two SATA ports (0 and 1) for use with the optional 1.8 inch micro SATA SSD drives that can be installed in an optional Radisys DSSD MXM module. Low Pin Count (LPC) Bridge The Low Pin Count (LPC) bridge of the PCH provides read/write cycles for memory, I/O, DMA, and Buss Master devices. The PCH implements the LPC Interface Specification, revision 1.1. The devices contacted over this LPC bridge include the IPMI controller, the port 80 debug header, the CPU complex FPGA, and the TPM. Serial Peripheral Interface (SPI) The PCH provides a 4‐pin SPI interface for connecting to and controlling the BIOS and ME Flash devices on the CPM. There are two 64MB Flash devices connected to the SPI bus that store BIOS boot and redundant BIOS boot code. Another 64MB Flash device contains redundant ME firmware images. On CPM power‐up, the primary BIOS Flash device is selected and used. If a corrupt BIOS is detected during operation, the IPMC forces a reboot and loads the redundant BIOS Flash image. The SPI bus allows the PCH to read and also program the primary and redundant BIOS boot Flash devices as well as the ME firmware Flash device. Controlling software as well as on‐ board jumpers provide Flash device write protection. Real-Time Clock (RTC) The PCH implements the CPM real‐time clock (RTC). Rather than a battery backup, the CPM uses a 1F “SuperCapacitor” to store and supply the minimum 2V backup RTC power. As a consequence, RTC power is available for at least two hours after a system power loss. The RTC is derived from a 32.768KHz crystal with the following specifications: • Frequency tolerance @ 25°C: ± 20 ppm •...
Page 31 Hardware Description SMBus/I C bus The PCH provides an SMBus host controller (SMBus 2.0 compliant) as well as an SMBus secondary interface. The host controller provides a mechanism for the CPU to initiate communications with SMBus peripherals (master/slave interface). The PCH also can operate in a mode that supports communication with I C compatible devices. The slave interface allows an external master to read from or write to the PCH. Write cycles can be used to cause certain events or pass messages and the read cycles can be used to determine the state of various status bits. The PCH internal Host Controller cannot access the internal slave Interface. Table 8 lists the SMBus and I C device addresses on the CPM. Table 8. SMBus and I C bus device addresses Device Bus Number Read Address Write Address IPMC FPGA IPMC I C bus 2 CPU Complex FPGA IPMC I C bus 2...
Page 32 Hardware Description Figure 5 shows the SMBus/I C bus mapping and lists important device addresses on the CPM. Figure 5. ATCA-46xx SMBus/I C bus mapping and addressing Update Channel RTM PCIe Hotswap PCIe Hotswap A0: 0xA8 Address Address A0: 0xA8 A1: 0xAA 0x40 0x42 A1: 0xAA 16-bit 16-bit Prog. Prog. I/O port I/O port Temp Sensor Temp Sensor Ref.
Page 33: Ipmi Controller
The CPM Management Engine (ME) uses Node Manager 2.0 firmware to perform power monitoring & alert, power limiting policies, thermal monitoring & alert, and power reduction tasks during boot. As described in Serial Peripheral Interface (SPI) on page 30 a redundant firmware image is stored in the ME Flash in case the ME firmware needs to be recovered. As shown in Figure 5, the IPMC I C Bus 5 is connected to SMLink0 on the PCH to allow the IPMC access to CPU and DIMM temperature information, PCH temperature information, CPU power information and other statistics. IPMI controller The IPMI controller (the IPMC) supports an “intelligent” hardware management system, based on the Intelligent Platform Management Interface Specification. The hardware management system can manage the power, cooling, and interconnect needs of intelligent devices, monitor events, and log events to a central repository. The IPMC is a Radisys‐designed reusable entity based on the Renesas H8S/2472 microcontroller and Lattice XP2 FPGA. The IPMC provides the following features: • An external 16‐bit address/data bus with internal 32‐bit configuration • Internal ROM (512KB of Flash ROM) and RAM (40KB) • 2 MB external SPIBus‐based Flash ROM • Control of front panel LEDs (H/S, OSS, PWR, and APP) • Monitoring of front panel Reset, and Hot Swap switching • Control of three backplane interfaces (redundant IPMB‐A/B buses and an x8 hardware address bus) • Controller for six I C buses (I C Bus 0 ‐ I C Bus 5) • Eight analog inputs • 8‐bit parallel data bus •...
Page 34: Cpu Complex (Cc) Fpga
Hardware Description CPU Complex (CC) FPGA The CC FPGA module performs a number of significant monitoring and interface functions on the CPM. Many of these functions are indicated in Figure 1 on page 16. The following sections provide more details on the CC FPGA functions. Power management and monitoring Most of the CPM payload power supplies are monitored by two ADM1066 power sequencers, but the CC FPGA controls the power sequencing for the CPM. On power‐up, it looks at the payload power enable from the IPMC to begin the power up sequence. When both power sequencers have indicated “powergood” and the external comparators indicate in‐tolerance, the CC FPGA asserts PWR_OK to the PCH. The CC FPGA de‐asserts PWR_OK and asserts fault signals when any monitored condition falls out‐of‐tolerance. System Reset monitoring The CC FPGA monitors the source of all Powergood, Platform, and MR‐Resets and provides a 16‐bit register for reset source monitoring. When the CC FPGA detects any monitored reset, it asserts an interrupt to the IPMC. The IPMC can read the reset source register and then clear the interrupt. Refer to Reset subsystems on page 37 for more information. Dual UARTs, COM port, and SPI mux Two 16550‐compatible UARTs are instantiated in the CC FPGA. COM1 and COM2 ports can operate up to a 115200 baud rate (default of 115,200 baud, 8‐bit, no parity, 1 stop bit). The CC FPGA is a bridge between the SPI flash programming header, the PCH and the redundant 64Mb flash devices. The FPGA includes functionality to multiplex the internal UART connections between the ...
Page 35 Hardware Description CC FPGA RTM-link The RTM‐Link relays the state of various control and status signals from a compatible programmable device on an RTM. The following signals are sent to the RTM for Ethernet LED control: • Front Ethernet port 0 status green • Front Ethernet port 0 status yellow • Front Ethernet port 1 status green • Front Ethernet port 1 status yellow The following signals are read in from the RTM: • RTM USB over‐current (sent to the PCH) • (Sent to I350 port 2) SFP0 RXLOS • (Sent to I350 port 2) SFP0 TX_FAULT • (Sent to I350 port 2) SFP0 MOD_DEF0 • (Sent to I350 port 3) SFP1 RXLOS • (Sent to I350 port 3) SFP1 TX_FAULT • (Sent to I350 port 3) SFP1 MOD_DEF0 Customer header configuration All control signals configured on the customer and debug headers are routed to the CC FPGA. ...
Page 36: Ipmi Fpga
Hardware Description IPMI FPGA The Intelligent Platform Management Interface (IPMI) Field Programmable Gate Array (FPGA) provides the specialized interfaces and glue logic needed between the H8 IPMC and rest of the CPM. The IPMC and the IPMI FPGA provide watchdog timers to help prevent the CPM from entering an unrecoverable state. Watchdog timer 2 on the IPMI FPGA is enabled by default and starts running as soon as +3_3V_SUS power is present and the FPGA has loaded its internal flash image into its internal SRAM. If a firmware or hardware problem on the IPMC causes it to stop strobing the watchdog timer in the IPMC FPGA, it isolates the IPMC from the IPMB‐A, IPMB‐B, and IPMB‐L buses so that they remain functional for the remaining blades in the chassis. Watchdog timer 2 then resets the IPMC. The interfaces supported by the IPMC FPGA include the SPI bus, the I C bus 2, and the JTAG chain. The IPMI FPGA is programmed via the same JTAG chain as is used for the CC FPGA and any programmable devices on the RTM. Intel I350 quad GbE Ethernet controller The I350 has four independent network interfaces that support SerDes/SGMII or MDI (copper) protocols. The I350 has a x4 PCI Express Gen2 host interface that operates at 5.0 GT/s and is capable of negotiating to x2 and x1 link widths. The I350 provides virtualization support including PCI‐SIG Single‐Root I/O Virtualization, VMDq2 support for up to eight virtual machines and eight TX and RX queues per port. The I350 has NC‐SI and SMBus interfaces that support pass‐through traffic for Serial‐Over‐LAN (SOL) support from the IPMC. The CPM has both interfaces connected to allow flexibility for the IPMI code development. Initial configuration information for the controller (including management setup and SMBus addresses) is provided via a 256Kb SPI EEPROM, which is read upon power‐up. Mellanox dual 40GbE controller The CPM provides two 40GbE ports using a Mellanox ConnectX‐3 (CX3) controller. The CX3 is connected to a x8 Gen 3 PCI Express port that provides a peak data bandwidth of 64Gb/s in each direction. The CX3 has two integrated Gigabit Ethernet MAC/PHY ports that support the following standard Ethernet interfaces for CPM: •...
Page 37: Clock Synthesizer Subsystem
Hardware Description The CX3 has iSCSI and PXE boot support enabled in the SPI flash firmware. iSCSI and PXE cannot be enabled at the same time; the desired function must be selected in the BIOS setup menu. Clock synthesizer subsystem CPM uses two major components, a CK420BQ and a DB1900Z to generate the clock signals used by the CPUs, by the PCH, and by other peripheral components. The CK420BQ clock generator provides host, chipset, PCI and LPC peripheral clocks on the CPM. The device uses a 25.0000 MHz reference crystal and receives control input from the BIOS over the SMBus. The clock signals are supplied to the CPUs, the PCH, the TPM, the IPMC, and the CC FPGA. The DB1900Z differential buffer distributes 100MHz clocks to the PCI Express and QPI devices. The clocks output by this device are based on a reference clock from the CK420BQ clock generator. In addition to the PCIe and QPI clocks, this device supplies reference clocks for the Mellanox CX2/CX3 and Intel I350 GbE controllers, the RTM PCIe interfaces, and the MXM connector. In addition to the above reference clocks, the following oscillators or crystals are provided on the CPM: • A 40.000 MHz 100ppm oscillator is provided for the CC FPGA. • The CC FPGA provides a 6.6 MHz clock for the RTM‐L interface to the RTM. • A 32.768 kHz crystal is provided for the PCH RTC. • A 32.768 kHz crystal is provided for the IPMC RTC. • A 32.768 kHz crystal is provided for the Trusted Platform Module. • A 25.000 MHz 30ppm crystal is provided for the CK420BQ • A 25.000 MHz 30ppm crystal is provided for the I350 quad GbE controller • A 32.000 MHz 50ppm oscillator is provided for the IPMC FPGA • A 156.25 MHz 50ppm oscillator is provided for the Mellanox CX3 40G Ethernet controller Reset subsystems CPM9 supports three types of reset: platform reset, powergood reset, and memory‐retained ...
Page 38 Hardware Description The platform reset signal originates in the PCH. The potential sources or triggers of a platform reset include: • The front panel Reset button • A reset assertion generated over the board debug header during a troubleshooting session • An IPMC reset command generated due to one of the following events: • Watchdog timer expiration • An MR reset is asserted (by either the PCH or the IPMC) but they are disabled in the CC FPGA. In this case it reverts to a platform reset. Powergood reset A powergood reset is similar to a platform reset, except that all sticky bits in the CPU and PCH are cleared. For the CPUs, a powergood reset will result in reset of all the states in the processor, including the sticky state that is preserved on the other resets. There is a “Powergood Reset” enable bit in the reset control register of the CC FPGA. When this bit is set, all platform resets are converted to Powergood resets. In other words, when this bit is set, any of the sources or triggers that would otherwise cause a platform reset will instead cause a powergood reset. There is a second enable bit in the CC FPGA, that when set also asserts Powergood Reset, normally from the BIOS. Memory-Retained (MR) reset A memory‐retained reset will preserve the contents of main memory while the rest of the system experiences a Platform Reset. The intended usage for MR‐Reset is to provide an operating system kernel crash dump location, so that debug information can be recovered after an OS crash. When an MR‐Reset is initiated from one of the sources, the result is an INIT* virtual legacy wire (VLW) message sent to the CPUs from the PCH. INIT* triggers the BIOS to run from the reset vector in the shadowed F000 segment, where the MR‐Reset code is stored. If a valid signature exists in this location (i.e. MR‐Reset has been enabled in the BIOS setup menu) then the MR‐Reset code is executed. The INIT* sequence has PCH drive the INIT* VLW message to the CPUs and the INIT_3_3V* signal to the CC FPGA. The MR‐Reset BIOS code puts the system into the S3 sleep state. In S3, the internal memory ...
Page 39: Watchdog Timers
Hardware Description Table 9 lists the potential sources/triggers for the MR‐Reset. Though similar, these sources are slightly different from those for a platform or powergood reset. Table 9. Memory-Retained Reset Sources Platform Reset Source Description Front panel Reset button Pushing the button triggers an RCIN* assertion to the PCH, thus beginning the INIT* (FP_RESET*) sequence. RTM reset push button The RTM MMC sends OEM command to IPMC. Upon receiving the “warm reset” OEM command, the IPMC asserts IPMC_MR_RESET* IPMC warm reset command One of the following two triggers are detected by the IPMC:...
Page 40: Power Subsystems
Hardware Description IPMC watchdog timer (Watchdog 2) The IPMC FPGA includes a hardware watchdog timer, Watchdog 2. This watchdog is enabled by default and will start running as soon as +3_3V_SUS power is present and the FPGA has loaded its internal flash image into its internal SRAM. The default timeout on power‐up is 10 seconds. After the IPMC boot‐loader is finished, the IPMC reprograms the timeout for 6 seconds and will continue to strobe every 2 seconds. If a firmware or hardware problem on the IPMC causes it to stop strobing the watchdog timer in the IPMC FPGA: 1. The IPMC is automatically isolated from the IPMB‐A, IPMB‐B and IPMB‐L buses so that they remain functional for the remaining blades in the chassis, 2. Watchdog 2 forces a reset of the IPMC. Power subsystems All CPM power is supplied through the P10 backplane connector as ‐48VDC, as specified by the PICMG 3.0 specification. The use of DC power minimizes the possibility of RFI and EMI interference for the on‐board and board‐to‐board signals in ATCA components. Figure 6 shows the power architecture and distribution for the CPM. Input protection and monitoring Each of the main power feeds (‐48 A/B) and return feeds (VRTN A/B) of the CPM is protected by a 10A fast‐blow fuse. This protects the CPM circuitry in the event power draw exceeds the rated 350W power by a significant amount. In addition, the ENABLE_A/B inputs to the power input module (PIM) are each protected by 100 mA fuses. The CPM does not monitor input voltage directly, but the IPMC does detect input voltage presence at the Zone 1 connector and compares it with the PIM_STATUS signal. Any status difference indicates one or more input fuses have blown.
Page 41 Hardware Description Figure 6. ATCA-46xx Power Subsystem ATCA Zone 1 -48V to (P10) 12V Brick 3.5A USB, MXM, PCH -48V A -48V A TPS54620 10A Fuse -48V -48V 4.5A -48V B -48V B 10A Fuse Misc TPS54620 +3_3V VRTN A 10A Fuse VRTN A 500mA...
Page 42: Trusted Platform Module (Tpm)
Hardware Description Power Input Module (PIM) A power input module (PIM) between the input fuses and main power brick (‐48V to 12V supply) provides input power conditioning and the following additional features: • Current handling up to 300W • Inrush current limit protection • Integrated filter designed to meet CISPR class B EMI limits • 11.88W of isolated auxiliary 3.3V power for IPMI circuitry. • 750mW of isolated auxiliary 5V power for IPMI circuitry. • ORing FETs on A and B –48V feeds • A/B feed loss alarm • Hot‐swap control • Glitch hold‐up circuit based on external capacitor and 72V charging circuit • Input under voltage and over voltage protection • Over current and thermal protection In addition to the above features, the PIM incorporates circuitry to charge external voltage‐ hold capacitors. These external capacitors (1100uF/100V) charge to a nominal 72V and provide short‐term transient and power loss protection in support of PICMG 3.0 requirements. Two internal PIM regulators (a 3.3V switching supply and a 5V linear supply) send power to IPMC components, the FPGAs, RTC circuitry, the RTM, H/S LEDs, and temperature sensors. Payload power supplies The main power supply (an isolated quarter brick DC‐DC convertor; the brick) is a ‐48V to 12V convertor. It supplies all of the +12VDC power for the CPM. With the exception of the +3.3V ...
Page 43: Mxm Type A Video Module (Optional)
Hardware Description • Public key signature, encryption, and decryption to enable secure storage of data and digital secrets • Storage of hashes (unique numbers calculated from pre‐runtime configuration information) that enable verifiable attestation of the machine configuration when booted • An endorsement key that can be used to anonymously establish that an identity key was generated in a TPM. • Initialization and management functions that allow the owner to turn TPM functionality on and off, reset the chip, and take ownership of its functions MXM type A video module (optional) The CPM includes support for an optional Type A MXM 3.0 module. Due to the limited board height available with ATCA CPM boards, the fan or heat sink for standard MXM video modules cannot be used with the ATCA‐46xx CPM. Contact Radisys Technical Support for help with a heat sink solution for any selected Type A MXM 3.0 video module. The MXM site supports DisplayPort digital video signaling to the front panel and also connects to the x16 PCI Express port from CPU0 (port 3). 1.8” Solid State Drive (SSD) module (optional) The CPM supports installation of one or two 6 Gbps microSATA SSD drives installed in an optional carrier module that mounts and connects to the board MXM port. Figure 7 shows the ATCA‐46xx CPM with a mounted DSSD MXM carrier module that has two microSATA SSD drives installed. The DSSD MXM carrier module usually ships separately, and includes a sheet with assembly and installation instructions.
Page 44: Eusb Embedded Flash Module (Optional)
Hardware Description eUSB Embedded Flash module (optional) The CPM supports up to two embedded USB (eUSB) NAND Flash modules. The modules are USB 2.0 compliant and currently provide sizes from 1GB to 32GB of SLC NAND Flash. The eUSB modules have read speed support of up to 35MB/s and write speed up to 17MB/s. Figure 8 shows an eUSB NAND Flash module ready for installation on the ATCA‐46xx CPM. Figure 8. eUSB NAND Flash module eUSB NAND Flash Module The modules are stackable with a “Bottom” version form factor that passes through a USB connection to a “Top” form factor module, providing redundant flash drive support in a small footprint. The CPM uses PCH USB ports 2 and 3 for the eUSB modules.
Page 45: Chapter 3: Software/Firmware Description
• The system BIOS • The IPMI firmware • Software/firmware update support software • Operating system support software The following sections provide more information. System BIOS he system BIOS is designed on a base source code licensed from American MegaTrends Inc. (AMIBIOS) and is adapted to meet the requirements of the ATCA‐46xx CPM. The system BIOS is an implementation of the Unified Extensible Firmware Interface (UEFI) specification 2.1 as published by the UEFI forum (www.uefi.com). System BIOS features The CPM system BIOS has the following features: • CPM boot time in less than 30 seconds excluding extended memory and option ROM initialization requirements • Source code based on UEFI specification • Support for the processor and PCH features of the CPM as well as all major board features • User configurable through the BIOS Setup utility BIOS setup menus The system BIOS contains a setup utility for modifying the system configuration. The system configuration information is maintained in the redundant boot flash devices and is used by the system BIOS to initialize the hardware. Access to the setup menus is available for only a short time during the boot process. Pressing F2 during boot enables access to the setup utility. The Delete key is an alternative that works in some situations when F2 does not. The BIOS setup menus include standard menus and additional custom menus from Radisys. A legend on the right side of each menu provides navigation and selection information. Table 10 gives a general overview of the BIOS setup menu hierarchy.
Page 46 User defined Note: The Radisys default setting for each of the above BIOS setup options is indicated with square brackets ([ ]) surrounding the option or the required entry. If multiple choices are available for a menu item, each choice is...
Page 47 Software/Firmware Description Table 12 lists the Radisys default settings for the Advanced menu of the CPM BIOS setup. Table 12. Radisys default BIOS Advanced menu setup options Advanced Menu Submenu Setup Item Values [Default] Top level: Front/Rear Ethernet [EEPROM Setting] Front/Copper Advanced Menu Rear/SGMII Rear/SERDES Spread Spectrum [Enabled] Disabled SOL Priority port [FRONT/Serial Port 0] REAR/Serial Port 1...
Page 48 Software/Firmware Description Table 12. Radisys default BIOS Advanced menu setup options (continued) Advanced Menu Submenu Setup Item Values [Default] PCI Express GEN2 Device Register Settings [Default] Shorter Longer Disabled Completion Timeout PCI Express GEN2 Device Register Settings Enabled [Disabled] ARI Forwarding...
Page 49 Software/Firmware Description Table 12. Radisys default BIOS Advanced menu setup options (continued) Advanced Menu Submenu Setup Item Values [Default] Security Device Support Enabled [Disabled] TPM State Enabled [Disabled] Pending operation [None] Enable Take Ownership Trusted Computing Settings: Disable Take Ownership TPM Clear...
Page 50 Software/Firmware Description Table 12. Radisys default BIOS Advanced menu setup options (continued) Advanced Menu Submenu Setup Item Values [Default] Power Technology [Disable] Energy Efficient Custom EIST Enabled [Disabled] Turbo Mode Enabled [Disabled] P-STATE Coordination [HW_ALL] SW_ALL SW_ANY CPU C3 Report...
Page 51 Software/Firmware Description Table 12. Radisys default BIOS Advanced menu setup options (continued) Advanced Menu Submenu Setup Item Values [Default] SATA Port[0 1] [Dynamic update] SATA Mode Disabled IDE Mode [ACHI Mode] RAID Mode Serial-ATA Controller 0 Disabled Enhanced [Compatible] SATA Configuration...
Page 52 Note: The Radisys default setting for each of the above BIOS setup options is indicated with square brackets ([ ]) surrounding the option or the required entry. If multiple choices are available for a menu item, each choice is indicated in italics. If there are no required settings for a menu item, that item does not appear in this table.
Page 53 Software/Firmware Description Table 13 lists the Radisys default settings for the Chipset menu of the CPM BIOS setup.. Table 13. Radisys default BIOS Chipset menu setup options Chipset Menu Submenu Setup Item Values [Default] Total Memory [Dynamic update] Current Memory Mode [Dynamic update] Current Memory Speed [Dynamic update] Mirroring [Dynamic update] Sparing [Dynamic update]...
Page 54 Software/Firmware Description Table 13. Radisys default BIOS Chipset menu setup options (continued) Chipset Menu Submenu Setup Item Values [Default] Intel(R) I/OAT Enabled [Disabled] DCA Support [Enabled] Disabled VGA Priority Onboard / [Offboard*] TargetVGA CPU 0 [Dynamic update] Gen3 Equalization WA's...
Page 55 Note: The Radisys default setting for each of the above BIOS setup options is indicated with square brackets ([ ]) surrounding the option or the required entry. If multiple choices are available for a menu item, each choice is indicated in italics. If...
Page 56 [Dynamic update] Note: The Radisys default setting for each of the above BIOS setup options is indicated with square brackets ([ ]) surrounding the option or the required entry. If multiple choices are available for a menu item, each choice is...
Page 57 Boot Override Note: The Radisys default setting for each of the above BIOS setup options is indicated with square brackets ([ ]) surrounding the option or the required entry. If multiple choices are available for a menu item, each choice is...
Page 58: Ras Support
Software/Firmware Description RAS support The CPM uses Reliability, Availability and Serviceability (RAS) features to support enhanced boot reliability and reduce system downtime. The following features promote RAS support: • BIOS protection and redundancy • Memory error handling • PCIe error handling • Processor and integrated memory controller error handling • POST error handling • Watchdog support • BIOS crisis recovery The system BIOS monitors errors and manages pre‐boot and boot events to enhance system uptime. These BIOS actions assist with the management activities performed by the IPMI subsystem and system OS to affect overall system RAS. BIOS protection and redundancy Prior to OS boot the BIOS checks the primary boot image checksum and notifies the IPMC if there is a problem. If there is a problem with the primary boot image, the IPMC switches to the standby version and resets the system. After boot and during normal operation, the IPMC Corrupt Flash Watchdog forces a switch to the secondary boot image by the IPMC if a corrupt boot image is detected that could prevent validation or cause a system hang. During normal operation, the BIOS program maintains an active write protection on the primary boot image stored in Flash. A jumper is also available at the Customer header to add a physical write protection for the BIOS image. Memory error handling Memory error detection and much of the memory error handling is performed by the integrated memory controllers (iMCs) in the CPUs, but the BIOS supports only a subset of the ...
Page 59 Software/Firmware Description At runtime, the CPU triggers a System Management Interrupt (SMI) when memory errors reach a preset threshold. If the runtime error logging is enabled. then SMI determines the cause, clears the error status, and reports the memory error to IPMC. Memory errors can be either correctable or uncorrectable. If the count of correctable memory errors goes above the BIOS "Max Mem Err Events" value, the SMI handler reports that the correctable error limit has been exceeded and disables further correctable error reporting (thus preventing performance degradation). Uncorrectable memory errors are also reported to IPMC, but error handling is determined by BIOS and OS settings. PCIe error handling The BIOS uses both legacy PCI error signaling (PERR/SERR) and PCI Express Advanced Error Reporting (AER). The AER mapping reports the error severity (correctable, uncorrectable/non‐ fatal, or uncorrectable/fatal) in addition to reporting the error. If the BIOS has been set up to enable PCI error logging support, the BIOS enumerates all PCI devices detected on the system at POST time, and enables the error reporting – PERR/SERR for legacy devices and AER reporting if the device supports it. The BIOS applies an error mask to all AER‐supported devices when errors are reported, and may trigger critical error action for detected AER errors of the proper severity. As with memory errors, at runtime PCI errors are signaled to SMI. The PCI device causing the error is next determined. The SMI routine then clears the error status and reports a platform event to IPMC. The SMI handler may then trigger critical error action depending on BIOS setup options. Processor and integrated controller error handling The CPUs as well as the integrated QuickPath Interconnect (QPI) and Integrated I/O (IIO) controllers implement various types of error detection, correction, containment, and reporting features. Processor core and uncore error reporting is performed via Machine Check Architecture (MCA). At startup or after a power‐good reset, BIOS initializes the machine check registers, clears the status registers by writing zeros into the registers, and writes all ones into the control registers to enable all MCA features. If the system is not coming up from a power‐good reset, it retains any error information by preserving the content of machine check status registers. The QPI protocol uses a CRC mechanism to ensure the data integrity of a serial stream. Unless ...
Page 60 Software/Firmware Description POST error handling The Power On Self Test (POST) carried out by BIOS after startup examines the functionality of the modules present on the system. It reports any errors to IPMC in the form of platform event messages. The BIOS continues the boot process as long as no errors are detected that might be essential to proper BIOS operation. Errors that might affect BIOS or system operation can cause the BIOS to halt the boot process. Due to the CPM’s design complexity, the initialize time of discovered sub‐FRUs can vary significantly from product to product. The BIOS cannot predict sub‐FRU initialization time, and some boot devices could be missed if allowance is not made for this variable initialization time. A BIOS setup option allows user specification of a sub‐FRU initialization time so the BIOS waits a specified amount of time before it proceeds with the boot device discovery. Optimization routines shorten the wait time if all the FRUs are either not installed, inactive, or become active before the specified initialization time expires. If communication with the IPMC fails, the BIOS unconditionally waits for the time configured. If any sub‐FRU is unable to reach a required FRU state within the time limit, the BIOS reports a system event to the IPMC. A progressive boot feature causes the BIOS to re‐attempt booting the boot devices even if it fails. The “round robin progressive boot” causes the BIOS to attempt booting from the next device in the device list until all boot order entries are tried. The process repeats indefinitely until a boot attempt succeeds, or the system is reset. Watchdog support In addition to the RAS features supporting CPM boot, the following watchdog timers support the CPM in the pre‐boot and OS runtime environments: • Corrupt Flash Detection (CFD) watchdog timer • IPMI Baseboard Management Controller (BMC) watchdog timer The Corrupt Flash Detection (CFD) watchdog timer is a software‐based watchdog implemented in the IPMC to recover the system at boot when the primary BIOS flash is either blank or corrupted. The IPMC starts the CFD watchdog timer at payload reset, and it will force a switch to the secondary boot Flash if it is not stopped within a specified time period. The Baseboard Management Controller (BMC) watchdog timer is a programmable watchdog timer controlled by the IPMC. The BMC watchdog timer triggers a configurable action (the default action is reset) if some fault condition in the system prevents a stop/strobe of the BMC watchdog. The BMC watchdog timer implementation is based on the IPMI specification and can be configured or disabled by standard IPMI v1.5 commands. Its programmable range ...
Page 61: Ipmc Functions
Software/Firmware Description BIOS recovery Refer to the Firmware and Software Update Instructions for the full BIOS image upload procedure. If the CPM BIOS fails boot due to Flash corruption of both the primary and secondary boot Flash images, the user can recover the Flash BIOS image using a BIOS crisis recovery procedure. When the “BIOS Force Recovery” jumper (pins 5‐6) is installed in the customer header (P2), the BIOS images can be reloaded. Refer to Onboard headers on page 114 for the full pinout of the CPM customer header. Once the recovery mode is entered, the system BIOS tries to find and load a BIOS image from either the front panel USB port or the front panel serial port. Once the BIOS image is loaded successfully, the system BIOS continues the subsequent boot phases from the loaded BIOS image until the BIOS image in the flash device is re‐programmed. The main and NVRAM BIOS blocks are updated, but not the boot block. Contact Radisys Technical Support for assistance or if the entire BIOS image (including the boot block) needs to be reloaded. IPMC functions The IPMC controls the hardware management subsystem by enabling payload power and ports, detecting component hardware states, initiating resets, and monitoring managed sensors. The CPM’s hardware management system complies with the PICMG 3.0 R3.0 base specification, the HPM.1 R1.0 firmware upgrade specification, and the Intelligent Platform Management Interface (IPMI) 1.5 specification, which defines a set of common interfaces for managing the system and for monitoring system health. Using serial‐over‐LAN and IPMI‐over‐LAN communications, the IPMC provides a remote user with access to the CPU serial console through the base interface. For more information, see Chapter 4, IPMI‐Over‐LAN, on page 63 and Serial‐Over‐LAN on page 65. The CPM's CPU communicates with the IPMC through a keyboard controller style (KCS) interface, which provides access to IPMI functions. This allows the CPU to send messages to the CPM's IPMC or to any other management controller in the shelf. For a list of the IPMI commands and details on the sensors used on the CPM, refer to Appendix B, IPMI Commands and Managed Sensors, on page 86. Software/Firmware Update Support Software or firmware update instructions are included in each update package. In the case of ...
Page 62: Operating System Support
Software/Firmware Description All upgrades can be performed from the CPM’s local CPU: • Through a serial console that is directly connected to the CPM or RTM serial port. • Through a remote serial console as described in Serial‐Over‐LAN on page 65. • By a remote login to the CPM over Ethernet. The session is lost when the CPM reboots. The BIOS setup menus cannot be accessed and the bootup messages cannot be viewed. Upgrades of the IPMC and IPMC EEPROM can alternatively be performed from a remote computer’s CPU: • By specifying the Shelf Manager IP address and the CPM’s IPMB address in the upgrade commands. • By specifying the CPM’s Base interface IP address in the upgrade commands. This method is available when access to the IPMC has been configured as described in IPMI‐Over‐LAN on page 63. Instructions for upgrading all of the components are included with the upgrade image that is available from the Radisys Web site or from Radisys Technical Support. In all cases, a link in the Release Notes points to a separate Firmware and Software Update Instructions manual that provides general update instructions. Contact Radisys Technical Support if there are problems or if you have questions about the update process. Operating System Support The following Linux operating systems are supported for the CPM: • Red Hat Enterprise Linux, 64‐bit • Monta Vista Carrier Grade Edition Linux, 64‐bit • Wind River Platform for Network Equipment, Linux Edition, version 4.0, 64‐bit Instructions for installing the supported operating systems are included in ATCA‐4xxx CPM Installation Instructions. Refer to Software Guide for Management Processors and General Purpose Computing Processors for more detailed information about installing OS packages. The distribution packages for each supported operating system include detailed installation ...
Page 63: Chapter 4: Operation And Maintenance
Chapter Operation and Maintenance Introduction This chapter presents the following operation and maintenance topics: • Hot swap process • IPMI over LAN • Serial over LAN • Firmware and software upgrades. Hot Swap of the CPM The CPM is hot swap capable and meets the hot swap requirements defined in the PICMG 3.0 Revision 2.0 AdvancedTCA Base Specification. The Shelf Manager controls the hot swap process, and the IPMC enables and disables payload power to the CPM when instructed by the Shelf Manager. Refer to Removing the CPM on page 71 and/or Installing the CPM on page 72 for detailed procedures on performing a hot swap of the CPM. IPMI-Over-LAN A remote management application can establish an IPMI‐over‐LAN session with the IPMC. The IPMC is remotely accessible through the Base Ethernet ports. The CPM implements IPMI‐ over‐LAN using RMCP and RMCP+ as described in the IPMI 2.0 specification. The IPMI‐over‐ LAN session can be used to enable the functionality described in Serial‐Over‐LAN on page 65. Configuring IPMI-over-LAN access The CPM must be initially configured to make IPMI‐over‐LAN access available. The IPMC saves the configuration changes in non‐volatile RAM so they are not lost during reboots, hot‐swaps, or firmware updates. Before you begin 1.
Page 64: Operation And Maintenance
Operation and Maintenance 2. Be prepared to fill in values for these variables: <ShMgr_IP> The IP address of the Shelf Manager. The IPMB address of the CPM in the shelf. <IPMB> <channel> The IPMI channel number representing the CPM Base interface channel. Channel 5 is base interface channel 2, and channel 6 is base interface channel 1. <IP> The static IP address to assign to each CPM Base interface channel. The protocol to use ( for RMCP or for RMCP+). The <protocol> lanplus configuration steps are done only once and apply to both protocols. IPMI-over-LAN basic configuration steps From the Linux prompt: 1. Set the IP address for a Base interface channel: rsys‐ipmitool ‐I lan ‐H <ShMgr_IP> ‐A none ‐t <IPMB> lan set <channel> ipaddr <IP> The option specifies the IPMI interface to use ( ), and the option specifies an ‐I ‐A authentication type to use during session activation ( ). The other command options none are defined in Step 2 on the previous page.
Page 65: Serial-Over-Lan
Operation and Maintenance rsys‐ipmitool ‐I lan ‐H <ShMgr_IP> ‐A none ‐t <IPMB> channel setaccess <channel> 01 ipmi=on link=on privilege=4 Repeat Step 2 to set access to the other channel. IPMI-over-LAN troubleshooting steps 1. View the current settings for a channel: rsys‐ipmitool ‐I <protocol> ‐H <ShMgr_IP> ‐A none ‐t <IPMB> lan print <channel> 2. Verify that the IP address is correct. 3. Verify that a non‐zero MAC address is set. A MAC address of 00:00:00:00:00:00 indicates the Ethernet controller is not responding to queries. This can occur if the Ethernet controller does not have the correct EEPROM file. Upgrade the Base EEPROM image and the front/rear Ethernet EEPROM image using the image files and instructions from the Radisys software distribution. 4. Repeat steps 1–3 to check the settings for the other channel. Serial-Over-LAN Serial‐over‐LAN (SOL) is the specification of packet formats and protocols for transmitting serial data over a LAN using IPMI‐over‐LAN packets. SOL operation is conceptually straightforward. A remote management application can establish an IPMI‐over‐LAN session with the IPMC. Once the session is established, the remote console can request SOL session activation. n SOL mode, any outgoing characters from the PCH are assembled into packets by the IPMC and sent to the remote console over one of the Base LAN interfaces. Conversely, inbound LAN packets carrying characters for the system serial controller have their character data extracted by the IPMC and delivered to the baseboard serial controller. The SOL character data is contained in SOL messages carried in UDP datagrams. The packet format is “IPMI v2.0 RMCP+” with the payload type set to “SOL”. Note: The CPM supports one SOL session at a time. The session must be closed before another session can be opened (through the other Base interface channel).
Page 66: Establishing A Sol Session
Operation and Maintenance Establishing a SOL session Prerequisite: This procedure assumes that the required setup for IPMI‐over‐LAN has been done once for this CPM. For details, see Configuring IPMI‐over‐LAN access on page 63. One-time SOL configuration steps The following two steps configure the CPM for SOL: 1. In the CPM BIOS Setup, change the active serial port to COM1, as follows: Advanced Settings > Serial Port 1 Configuration > Serial port: Enabled 2. Set the Linux console parameters to COM1 (ttyS1) to support SOL. To configure retry settings from a remote computer: Important: For these one‐time steps, use the Radisys‐supplied rsys‐ipmitool utility. 3. Configure the retries to keep SOL sessions open for a Base interface IP address. rsys‐ipmitool ‐I <protocol> ‐H <IP> ‐A none ‐C 0 sol set retry‐interval 0xF0 rsys‐ipmitool ‐I <protocol> ‐H <IP> ‐A none ‐C 0 sol set retry‐count 7 For an explanation of and , see Before you begin on page 63. <protocol> <IP> Repeat Step to configure retries for the other Base interface IP address. 5. View the retry settings for a Base interface IP address. rsys‐ipmitool ‐I <protocol> ‐H <IP> ‐A none ‐C 0 sol info The retry count should be 7, and the retry interval should be 2400 ms. Repeat Step 5 to verify access to the remaining interface IP addresses.
Page 67: Firmware And Software Upgrade
Operation and Maintenance SOL session open steps To open a SOL session from a remote computer: 1. Activate the SOL console window: rsys‐ipmitool ‐I <protocol> ‐H <IP> ‐A none ‐C 0 sol activate The SOL session is established. 2. Perform an action (such as pressing Enter) from the SOL console window. The SOL console window should respond. SOL session close step Note: Only one SOL session can be open at a time to a single CPM. To terminate a SOL session directly from the active SOL window, enter these characters: To terminate a SOL session from another window, enter: rsys‐ipmitool ‐I <protocol> ‐H <IP> ‐A none ‐C 0 sol deactivate Firmware and software upgrade The processes of updating onboard CPM firmware and associated software are covered in separate documents. Overall upgrade instructions for onboard firmware and software using tools are covered in detail in the ATCA Firmware and Software Update rsys_update Instructions document. If using Firmware Upgrade Management Instruments (FUMI) tools to perform the upgrade, refer to the ATCA Firmware and Software Update Instructions Using the Radisys Software Management Framework document for update instructions. If you have special update requirements, contact Radisys Technical Support for assistance.
Page 68: Overview Of Firmware Updates
CPM's CPU or remote CPU SAS EEPROM LSI SAS EEPROM CPM’s CPU Base NIC EEPROM image CPM’s CPU Fabric NIC Firmware CPM’s CPU All updates can be performed from the CPM’s local CPU: • Through a serial console that is directly connected to the CPM or RTM serial port. • Through a remote serial console as described in Chapter 4, Serial‐Over‐LAN, on page • By a remote login to the CPM over Ethernet. The session is lost when the CPM reboots. The BIOS setup menus cannot be accessed and the bootup messages cannot be viewed. Updates of the IPMC and IPMC EEPROM can alternatively be performed from a remote computer’s CPU: • By specifying the Shelf Manager IP address and the CPM’s IPMB address in the update commands. • By specifying the CPM’s Base interface IP address in the update commands. This method is available when access to the IPMC has been configured as described in Chapter 4, IPMI‐ Over‐LAN, on page Instructions for upgrading all necessary components are included with the update image, which is available from Radisys Technical Support.
Page 69: Chapter 5: Troubleshooting And Repair
Chapter Troubleshooting and Repair Introduction The procedures presented or referenced in this chapter detail removal and replacement of CPM Field Replaceable Units (FRUs) and provide troubleshooting procedures that can be used to discover FRU’s that need to be repaired or replaced. The following CPM‐related items can be installed or replaced: • The CPM itself. For installation instructions, see the ATCA‐4xxx Compute Processing Module Installation Guide. For replacement instructions, see CPM replacement procedures on page 71. • The DIMMs. For placement information and installation and removal instructions, see Memory module replacement procedures on page 73. • An MXM module. For installation or removal procedures, see MXM module installation/replacement procedures on page 75. • The eUSB modules. For installation or removal procedures, see eUSB module installation/replacement procedures on page 76. • A compatible RTM. For installation and removal instructions, see the Rear Transition Module Installation Guide. Refer to Troubleshooting Topics on page 77 for troubleshooting procedures to use on the CPM. Field Replaceable Units (FRUs) FRU information is stored in non‐volatile memory and is used by the IPMC to locate and communicate with the available FRUs. The CPM stores its FRU information in compliance with these specifications: • IPMI Platform Management FRU Information Storage Definition, v1.0, Revision 1.1 •...
Page 70: Cpm And Fru Device Ids
Troubleshooting and Repair Some of the multirecord area records used are: • Carrier information record • Carrier activation and current management record • Board point‐to‐point connectivity record • Carrier point‐to‐point connectivity record • Carrier clock point‐to‐point connectivity record CPM and FRU device IDs The CPM IPMC contains unique identification information. Table 17 describes those identifiers. Table 17. CPM ID information Field Value Device Name ATCA-46xx Device ID 012h Firmware Version <Current Firmware Version in format XX.YY> IPMI Version IPM Support 029h...
Page 71: Cpm Replacement Procedures
Troubleshooting and Repair CPM replacement procedures Removing the CPM The following steps explain the hot‐swap procedure for replacing the CPM: 1. Read Electrostatic discharge on page 10 and make sure you are adequately grounded before handling any of the modules. 2. Before replacing the CPM, disconnect all cables from the front panel. 3. Loosen the two thumbscrews securing the CPM. 4. Release the module locking ejector latch that contains the hot‐swap switch. This will be either the right or the Locking latch bottom ejector latch, depending on whether the module Lock is oriented vertically or horizontally. To release the locking ejector latch, first disengage the latch lock by sliding the lock toward the handle to release the catch from the shelf latch rail, then pull the handle Slide lock toward handle, then pull out. Refer to the illustration. handle out Do not release the other latch at this time. When the handle or ejector latch is opened, a signal is sent to the IPMC which causes the blue hot‐swap LED to begin blinking.
Page 72: Removing The Cpm Board Cover
Troubleshooting and Repair Removing the CPM board cover The CPM cooling shroud consists of the main board cover. The entire cover is attached to the board with a number of flat Phillips head screws. Perform the following steps to remove the CPM main board cover: 1. Power down the CPM or activate the hot swap switch so it can safely be uninstalled. 2. Remove the CPM from its slot and set it on an ESD‐safe work surface as described in Removing the CPM on page 71. 3. Remove the six screws securing the board cover to the board standoffs and set them aside. 4. Remove the board cover and set it aside. Installing the CPM board cover Perform the following steps to install the CPM main board cover: 1. Align the board cover so the countersunk screw holes are aligned with the board standoffs. 2. Attach the board cover using six flat‐head Phillips screws. Installing the CPM The replacement CPM should already be on a static‐free surface. Perform the following steps to install the replacement RTM or CPM. 1. Read Electrostatic discharge on page 10 and make sure you are adequately grounded before handling any of the modules.
Page 73: Memory Module Replacement Procedures
Troubleshooting and Repair Memory module replacement procedures Note: While the ATCA‐46xx CPM can operate using a wide range of DIMM memory combinations, at least one DIMM module should be installed in each CPU DIMM bank to ensure there are not boot initialization errors. Adding or replacing memory modules Replacing DIMMs consists of removing the CPM from the shelf, removing the main board cover, removing any DIMMs to be replaced, installing the new DIMMs, replacing the main board cover, and re‐installing the CPM in the shelf. The following sections detail these procedures. ! ARNING Observe appropriate ESD precautions when performing the procedures in this chapter (see Electrostatic discharge on page 10). Supported DIMM combinations The CPM normally uses registered DIMMs (RDIMMs) in full sets of twelve identical modules. The supported module kits are as follows: • A4600‐MEM‐48GB: This is a set of twelve modules of 4GB DDR3 VLP RDIMMs. • A4600‐MEM‐96GB: This is a set of twelve modules of 8GB DDR3 VLP RDIMMs. Note: The CPM can support memory module sets of DDR3 VLP RDIMMs in capacities of 1GB, 2GB, 4GB, 8GB, 16GB and 32GB per module as long as maximum memory temperature for the modules is not exceeded. Removing DIMMs These instructions assume the CPM has been removed from the shelf and the main board cover has been removed. Determine which DIMMs to remove (see previous discussion under Supported DIMM ...
Page 74 Troubleshooting and Repair Figure 9. DIMM insertion/removal 3. Hold the DIMM by the edges and remove it. 4. Remove the other DIMMs in the same manner. To replace the DIMM with a new card, follow the steps under Installing DIMMs. 5. Screw the main board cover back into place. 6. Power up the CPM and check the BIOS screens to make sure all memory is detected. Installing DIMMs These instructions assume that the CPM has been removed from the shelf and the main board cover has been removed. Open the DIMM ejector latches by pushing them outward, as shown in Figure 2. The DIMMs are keyed so they can be inserted in only one way. Hold the DIMM by the edges, align the slot (keyway) of the DIMM with the tab molded in the base of the socket, and push the card down firmly into the socket. The ejector latches close and click into place when the DIMM is firmly seated. 3. Insert the other DIMMs in the same manner. 4. Screw the main board cover back into place. 5. Power up the CPM and check the BIOS screens to make sure all memory is working. After the DIMMs are installed, verify that the expected memory is available and the memory is valid as described in the following sections. Verifying DIMM operation Verify that the BIOS detects and enables all installed memory. The total detected and enabled memory size is reported in the BIOS setup menu ( Main > Memory Information If the total amount of detected memory is less than the total physical DIMM memory that is ...
Page 75: Mxm Module Installation/Replacement Procedures
Troubleshooting and Repair • The DIMM is incorrectly installed • The DIMM is faulty • The DIMM installation violates the memory population rules A DIMM that is not detected or enabled properly can be identified through the BIOS setup menu ( ). Chipset > CPU Socket 0/1 DIMM Information MXM module installation/replacement procedures Installing an MXM module There are two types of MXM module supported for the CPM – an MXM video module or the Radisys DSSD MXM module. Note: Although any video module complying with the MXM specification will install in the CPM’s MXM footprint and connector, the constraints imposed by the ATCA board height requires a custom heatsink solution. Only Radisys‐qualified MXM video modules (with the Radisys‐designed heat sink solution) can be installed in the CPM. Contact Radisys Technical Support if MXM video is necessary. Figure 10 shows an MXM module being installed in the CPM. Figure 10. MXM module insertion/removal Connector DSSD MXM Module SSD Module...
Page 76: Removing An Mxm Module
Troubleshooting and Repair 4. Press the MXM module into the MXM connector until it is fully seated. 5. Align the MXM mounting holes with the board standoffs (nearer the CPM front panel) and secure the module with one (DSSD module) or two (MXM video module) screws. 6. Reinstall the CPM cover (see Installing the CPM board cover on page 72). 7. Reinstall the CPM in its slot (see Installing the CPM on page 72). Removing an MXM module Perform the following steps: 1. Remove the CPM from its slot (see Removing the CPM on page 71). 2. Remove the CPM cover (see Removing the CPM board cover on page 72). Refer to Figure 10 and locate the MXM module (near the top front of the CPM). 4. Remove the one (DSSD module) or two (MXM video module) screws securing the MXM module to the board standoffs. 5. Disconnect the MXM module from the MXM connector and set it aside. 6. Reinstall the CPM cover (see Installing the CPM board cover on page 72). 7. Reinstall the CPM in its slot (see Installing the CPM on page 72). eUSB module installation/replacement procedures One or two eUSB Flash memory modules can be installed on the ATCA‐46xx CPM board. ...
Page 77: Installing An Eusb Module
Removing an eUSB module Perform the following steps: 1. Remove the CPM from its slot (see Removing the CPM on page 71). 2. Remove the CPM cover (see Removing the CPM board cover on page 72). 3. Locate the eUSB module(s) (between the J20 and J23 connectors at the back of the CPM). 4. Disconnect the top eUSB module and set it aside. If there is a bottom eUSB module, disconnect it also and set it aside. 5. Reinstall the CPM cover (see Installing the CPM board cover on page 72). 6. Reinstall the CPM in its slot (see Installing the CPM on page 72). Troubleshooting Topics General troubleshooting tips When the CPM does not perform as expected, look for symptoms that might point to the cause. Performing the following actions can help diagnose the problem: • Check the state of the LEDs on the CPM and other modules in the platform, especially the power and out of service LEDs. • Check temperatures on the CPM. If the Radisys Shelf Manager is used, see the troubleshooting information in the Shelf Management Software Reference for details on how to perform these queries. • Verify the IP address and the subnet mask assignment. See the Software Guide for Management Processors and General Purpose Computing Processors for details on verifying these assignments. ...
Page 78: Symptoms And Recommended Actions
Use the information to determine whether the overheating might be caused by warm facility air, a module failure, or a failed fan module. If the Radisys Shelf Manager is used, see the troubleshooting information in the Shelf Management Software Reference for details on how to perform these que- ries.
Page 79: Sensor Alarm Troubleshooting
Troubleshooting and Repair Table 18. Troubleshooting actions based on symptoms (continued) Symptom Recommendation The login prompt does not appear • Verify the serial cable is plugged into both the CPM and the system with the serial connection. after BIOS has booted. •...
Page 80: Appendix A: Specifications
Appendix Specifications The Radisys ATCA‐46xx CPM complies with or meets the standards and specifications presented in the following sections. Standards and interfaces Table 19 lists the standards and interfaces that apply to the CPM. Table 19. CPM Standards and Interfaces Standards/Interfaces Description Standards • PICMG 3.0 R3.0.1 AdvancedTCA • PICMG 3.1 R2.0 AdvancedTCA Ethernet option 1/9 • European requirements for hazardous materials, ROHS 6/6 Networking •...
Page 81: Environmental Specifications
Specifications Environmental specifications Radisys does not provide environmental certification testing because any meaningful emissions agency certification must include the entire system. Thus, the CPM is designed and tested to pass the environmental specifications noted below, but it is not certified. ! ARNING This product contains static‐sensitive components and should be handled with care. Failure to employ adequate anti‐static measures can cause irreparable damage to components. The operating environment must provide sufficient airflow across the CPM to keep it within its temperature specification. Table 20. Environmental specifications Characteristic State Value Operating +5° C to +40° C 30° C/hr rate of change Temperature Short term operating –5°C to +55°C (ambient) 30° C/hr Rate of Change Storage –40°...
Page 82: Safety Specifications
Specifications Safety specifications The safety specifications are measured under laboratory ambient temperature and humidity (approximately 55C and humidity between 30% and 50%). Testing was performed in partnership with a Nationally Recognized Testing Laboratory (NRTL) accredited to provide the required certifications. Table 21. Safety specifications Characteristic Certification Standard and test criteria Product Safety–US Accessory Listing UL 60950-1 “Safety for Information Technology Equipment” Product Safety–Canada Approval CSA 22.2 #60950-1-03 “Safety for Information Technology Equipment” Product Safety–EU Conformance with the Low EN 60950-1 “Safety for Information Technology Voltage Directive...
Page 83: Electromagnetic Compatibility (Emc)
Specifications Electromagnetic compatibility (EMC) The ESD, EMC, and Immunity specifications are measured with ambient temperature between 20C and 30C and relative humidity between 30% and 50%. Table 23. Electromagnetic compatibility (EMC) Characteristic State Standard and criteria Radiated emissions Operating FCC Part 15, Class A requirement for chassis/system level, Class A objective for blade (See Note 1) Emissions EN 55022: 2006, Class A requirement for chassis/system level, Class A objective for blade (See Note 1) Conducted emissions...
Page 84: Network Equipment Building Standard (Nebs)
Specifications Network Equipment Building Standard (NEBS) The CPM is designed to meet the NEBS requirements listed in Table 24. Telecommunication specification (NEBS) requirements Standard Description GR-63-CORE, Issue 3 NEBS Requirements: Physical Protection GR-1089-CORE, Issue 4 Electromagnetic Compatibility and Electrical Safety SR-3580 Network Equipment – Building Systems (NEBS) Criteria Levels Level 3, indoor contaminants levels Note: Verification to be performed at system level by Customer Additional compliance In addition to the standards cited that the ATCA‐46xx has been designed to meet, the CPM ...
Page 85: Mean Time Between Failures (Mtbf)
Specifications Mean time between failures (MTBF) The board MTBF is designed to meet or exceed 150,000 hours @ +35C per Telcordia SR332 Issue 2, Method 1, Case 3. The calculation results in Table 25 were generated using the references and assumptions listed. This specification and its associated calculations supersede all other released mean time between failures (MTBF), annual failure rate (AFR), early return index (ERI), and dead‐on‐arrival (DOA) calculations with earlier dates. The reported failure rates do not represent catastrophic failure. Table 25. Reliability estimate data Product MTBF ATCA-4616 220,000 4.0% 19,992 PPM 3,320 PPM ATCA-4618 218,000 4.0% 20,105 PPM 3,351 PPM ATCA-4648 218,000 4.0% 20,105 PPM 3,351 PPM Note: Calculations based on the CPM only;...
Page 86: Appendix B: Ipmi Commands And Managed Sensors
Appendix IPMI Commands and Managed Sensors IPMI command interfaces The following interfaces use IPMI command support: • I C bus connections with the following links: • I C bus 0 ‐ IPMB0‐A • I C bus 1 ‐ IPMB0‐B • I C bus 2 ‐ access to IPMC FPGA, CC FPGA, and RTC circuitry • I C bus 3‐ access to voltage monitoring, MXM, and PCIe retimer • I C bus 4 ‐ SOL access to GbE controller and base/front panel Ethernet • I C bus 5 ‐ access to PCH (thermal monitoring), IPMB‐L, and RTM • Serial connections using the SCI interface, as follow: • SCI1 ‐ link to an external debug board via an onboard debug header • SCIF ‐ SOL interface via front panel serial (COM) port • SCI3 ‐ provides link to IPMC console while/if SCI1 link connected • Payload processor link implemented as a KCS port IPMI commands Table 26 presents the commands supported by the CPM in different interfaces. These ...
Page 87 IPMI Commands and Managed Sensors Table 26. Supported IPMI Commands (continued) IPMI Command name NetFn Code Command Code Send Message App (06h) Get Channel Authentication Capabilities App (06h) Get Session Challenge App (06h) Activate Session App (06h) Set Session Privilege Level App (06h) Close Session App (06h)
Page 88 IPMI Commands and Managed Sensors Table 26. Supported IPMI Commands (continued) IPMI Command name NetFn Code Command Code Add SEL Entry Storage (0Ah) Partial Add SEL Entry Storage (0Ah) Clear SEL Storage (0Ah) Get SEL Time Storage (0Ah) Set SEL Time Storage (0Ah) Set LAN Configuration Parameters Transport (0Ch)
Page 89: Oem Command Descriptions
IPMI Commands and Managed Sensors Table 26. Supported IPMI Commands (continued) IPMI Command name NetFn Code Command Code *Restore Factory Defaults OEM Group (2Eh) OEM 1(30h) *Disable CFD OEM Group (2Eh) OEM 1(30h) *Set WDT Reset Type OEM Group (2Eh) OEM 1(30h) *Switch Active Boot Flash OEM Group (2Eh) OEM 1(30h) *Get Active Boot Flash...
Page 90 IPMI Commands and Managed Sensors Get Control State command (for debug only) This command returns the current state of a control pin. Table 29 is the command description of the Get Control State command. Table 29. Get Control State OEM command Data Type Byte Data Field FRU ID Data Field Control number Completion Code Response Field Control State 00h = de-asserted 01h = asserted Disable CFD command This command indicates to the IPMC that the CFD Watchdog Timer needs to be disabled. No ...
Page 91 IPMI Commands and Managed Sensors Switch Active Boot Flash command This command sets the current Boot Flash and can cause a cold reset to the x86 Processor Complex portion of the board if the appropriate bits in the command data are set. The command can also set the primary Boot Flash, which is selected during a power‐on reset of the blade. Table 32 is the command description of the Switch Active Boot Flash command. Table 32. Switch Active Boot Flash OEM command Data Type Byte Data Field Data Field Bit 7 = Sets the primary boot flash 0b (Do not set) 1b (Set primary flash to Flash selected by Bit 0) Bit 6 = Selects whether or not a reset will occur...
Page 92 This command instructs the H8 IPMI Firmware to perform a COLD reset. Table 34 is the command description of the Get Active Boot Flash command. Table 34. RTM Reset Button OEM command Data Type Byte Data Field Radisys IANA PEN0: F1h Data Field Radisys IANA PEN0: 10h Radisys IANA PEN0: 00h Completion Code Response Field Radisys IANA PEN0: F1h Radisys IANA PEN0: 10h...
Page 93 35 is the command description of the Get Active Boot Flash command. Table 35. Set Payload Status OEM command Data Type Byte Data Field Radisys IANA PEN0: F1h Radisys IANA PEN0: 10h Data Field Radisys IANA PEN0: 00h Command Version: hard coded to 0x00 Status Byte 1 Bits 7:4 = Reserved...
Page 94 36 is the command description of the Get Active Boot Flash command. Table 36. Get Payload Status OEM command Data Type Byte Data Field Radisys IANA PEN0: F1h Radisys IANA PEN0: 10h Data Field Radisys IANA PEN0: 00h Command Version: hard coded to 0x00 Completion Code Radisys IANA PEN0: F1h...
Page 95: Managed Sensors
IPMI Commands and Managed Sensors Managed sensors On the CPM, the IPMC sensors monitor voltages, temperatures, control signals, and status events. For functional information, refer to IPMI controller on page 33. The sensors are described in Table 38. Types of sensors The CPM implements the following types of sensors. • Discrete — A discrete sensor can have up to 16 bitmapped states, with one state as true. • Digital — A digital sensor has two possible states, only one of which can be active at any given time. For example, a digital sensor monitoring the power may indicate whether the power is good or not good. • OEM — An OEM sensor has its states defined by the manufacturer. The reading types of these sensors are sometimes defined as “sensor‐specific.” • Threshold — A threshold sensor has a range of 256 values, which represent measurements on the CPM and its FRUs. Temperature, voltage, current, and fan speed sensors are examples of threshold sensors. Table 37 lists the possible threshold types for the managed sensors. Table 37. Threshold types Threshold type Description Upper non-recoverable thresholds generate a critical alarm on the high side. Upper critical thresholds generate a major alarm on the high side.
Page 96: Ipmi Sensors
IPMI Commands and Managed Sensors IPMI Sensors The CPM supports a variety of sensors, each with entries in the Sensor Data Records (SDR). Table 38 lists the IPMI sensors supported by the CPM. Table 38. ATCA-46xx IPMI Managed Sensors Sensor Reading Normal Name Type Notes Type Reading ATCA FRU Hot ATCA FRU Hot Sensor- 0x00 -0x07 This sensor returns the ATCA M0 through M7 hot swap states swap swap specific...
Page 97 IPMI Commands and Managed Sensors Table 38. ATCA-46xx IPMI Managed Sensors (continued) Sensor Reading Normal Name Type Notes Type Reading ENET Link 1 Slot or Digital 0 or 1 0 = E1_LINK* not asserted Connector 1 = E1_LINK* asserted ENET Link 2 Slot or Digital 0 or 1...
Page 98 IPMI Commands and Managed Sensors Table 38. ATCA-46xx IPMI Managed Sensors (continued) Sensor Reading Normal Name Type Notes Type Reading +12V Voltage Threshold 12.00 LNR = 0.00 LC = 10.8 LNC = 11.4 UNC = 12.6 UC = 13.2 UNR = 13.8 Voltage Threshold 5.00...
Page 99 IPMI Commands and Managed Sensors Table 38. ATCA-46xx IPMI Managed Sensors (continued) Sensor Reading Normal Name Type Notes Type Reading +1.5V PCH Voltage Threshold 1.50 LNR = 0.00 LC = 1.35 LNC = 1.43 UNC = 1.58 UC = 1.65 UNR = 1.73 +1.2V Voltage...
Page 100 IPMI Commands and Managed Sensors Table 38. ATCA-46xx IPMI Managed Sensors (continued) Sensor Reading Normal Name Type Notes Type Reading +VDDQ1 Voltage Threshold 1.50 LNR = 0.51 LC = 1.4 LNC = 1.46 UNC = 1.56 UC = 1.61 UNR = 1.66 +VTT0 Voltage Threshold...
Page 101 IPMI Commands and Managed Sensors Table 38. ATCA-46xx IPMI Managed Sensors (continued) Sensor Reading Normal Name Type Notes Type Reading +VCCPLL0 Voltage Threshold 1.8(SNB)/1. LNR = 0.00 7(IVB) LC = 1.68 LNC = 1.74 UNC = 1.89 UC = 1.95 UNR = 1.99 +VCCPLL1 Voltage...
Page 102 IPMI Commands and Managed Sensors Table 38. ATCA-46xx IPMI Managed Sensors (continued) Sensor Reading Normal Name Type Notes Type Reading +LVTT DDR0 Voltage Threshold 0.675 LNR = 0.50 (LVDIMM) LC = 0.63 LNC = 0.65 UNC = 0.70 UC = 0.73 UNR = 0.75 +LVTT DDR1 Voltage...
Page 103 IPMI Commands and Managed Sensors Table 38. ATCA-46xx IPMI Managed Sensors (continued) Sensor Reading Normal Name Type Notes Type Reading CPU0 Core DTS Temperature Threshold LNR = N/A LC = N/A LNC = N/A UNC = -20 UC = -13 UNR = -1 CPU1 Core DTS Temperature...
Page 104 IPMI Commands and Managed Sensors Table 38. ATCA-46xx IPMI Managed Sensors (continued) Sensor Reading Normal Name Type Notes Type Reading Processor Processor Sensor- Offset Description specific FRB2/ Hang in POST failure (used hang is believed to be due or related to a processor failure. Use System Firmware Progress sensor for other BIOS hangs.) System Firmware...
Page 105 IPMI Commands and Managed Sensors Table 38. ATCA-46xx IPMI Managed Sensors (continued) Sensor Reading Normal Name Type Notes Type Reading System Firmware System Firmware Progress (cont.) Progress (cont.) Event Data 2: 00h - Unspecified. 01h - Memory initialization. 02h - Hard-disk initialization 03h - Secondary processor(s) initialization 04h - User authentication 05h - User-initiated system setup...
Page 106 IPMI Commands and Managed Sensors Table 38. ATCA-46xx IPMI Managed Sensors (continued) Sensor Reading Normal Name Type Notes Type Reading Boot Error Boot Error Sensor- Offset Description specific Boot Record Corruption eventData1: 0xA0 - Boot record corruption eventData2: 0x00 - Currently executing Application image does not match non-volatile storage designation eventData3: 0x00 - Currently executing Application...
Page 107 1 & eventData3: FF' Failover Complete HPI Event OEM HPI Offset Description OEM HPI (Only applicable if Radisys Shelf Manager application is installed.) eventData1: 1 eventData2: 3:0 = channel 7:4 = LUN eventData3: 0 120-254 Open for statically added sensors from MXM and dynamically added sensors from RTM.
Page 108: Appendix C: Pinouts And Mapping
Appendix Pinouts and Mapping The ATCA‐46xx CPM pinout listings apply to front panel connectors and the Zone 1 and 2 backplane connectors. The onboard header pinouts apply to user‐accessible areas of the CPM board. Front panel connectors The following sections describe the pinouts for the front panel connectors. COM serial connector Table 39 lists the pinout for the front panel COM serial connector. Table 39. COM Serial Connector (J6) Pin 1 Pin# Signal Function Pin 2 Pin 3 Request to Send Data Terminal Ready Pin 6 Transmit Data Ground Ground Receive Data...
Page 109: Dual Ethernet Connectors
Pinouts and Mapping Dual Ethernet connectors Table 41 lists the pinout for each of the front panel RJ45 Ethernet GbE connectors. Note that power, ground, and the LED control signals are fed from the PCB and are not accessible at the connector pins. Refer to Table 1 on page 18 for detailed information on the connector LEDs. Table 41. Dual Ethernet GbE Connectors (J8) Pin# Signal Channel 0 Data + Channel 0 Data - Channel 1 Data + Channel 2Data + Channel 2 Data - Channel 1 Data - Channel 3Data + Channel 3Data -...
Page 110: Backplane Interfaces
Pinouts and Mapping Backplane interfaces Backplane connectivity summary This section describes the backplane interface connectivity. Table 43 lists the backplane connectors and summarizes their usage. Table 43. Backplane connectivity summary Connector Channel Port Board usage PICMG definition Comments Power Dual 48V DC power IPMB IPMB Dual IPMB Base 1 – 10/100/1000Base-T Port A Dual star Base interface Base 2 –...
Page 111: Zone 1 P10 Connector Pinout
Pinouts and Mapping Zone 1 P10 connector pinout Table 44 lists the P10 connector pinout. Table 44. Zone 1 contact assignments, P10 Contact Designation Description Mating sequence Reserved Reserved not applicable Reserved Reserved not applicable Reserved Reserved not applicable Reserved Reserved not applicable Hardware Address Bit 0 Third Hardware Address Bit 1 Third...
Page 112: Zone 2 J20 Connector Pinout
Pinouts and Mapping Zone 2 J20 connector pinout Table 45 lists the J20 connector pinout. Table 45. Backplane connector J20 signals Interface designation Clks CLK1A+ CLK1A– CLK1B+ CLK1B– CLK2A+ CLK2A– CLK2B+ CLK2B– Update Channel and Clks Tx4(UP)+ Tx4(UP)– Rx4(UP)+ Rx4(UP)– CLK3A+ CLK3A– CLK3B+ CLK3B– Tx0(UP)+ Tx0(UP)–...
Page 113: Rtm Interface Pinout
Pinouts and Mapping RTM interface pinout Zone 3 J30 connector pinout Table 47 lists the Zone 3 J30 connector pinout. Table 47. RTM connector J30 signals Row AB +12V_RTM +12V_RTM +12V_RTM +3.3V_IPMC RTM_PRSNT* RTM_HS_LED RTM_EN* +12V_RTM +12V_RTM +12V_RTM IPMC_I2C_CLK IPMC_I2C_DAT USB_D+ USB_D– SERIAL_0_TX SERIAL_0_RX JTAG_TDI JTAG_TDO JTAG_TMS JTAG_TCK JTAG_TRST INT_0...
Page 114: Onboard Switches, Headers, And Connectors
Pinouts and Mapping Onboard switches, headers, and connectors Onboard switches There are two switches on the CPM, both on the front panel, as follow: • The reset switch • The hot swap eject switch The recessed reset push button (SW1) is located in the lower half of the front panel, just above the Base/Fabric channel Status LEDs (see Figure 2 on page 17). The hot swap eject switch is connected to the lower blade latch assembly. When the latch is opened, it causes the switch to signal to the IPMC that a hot swap of the blade is desired. The IPMC monitors the process and controls the front panel H/S LED status. When the H/S LED turns steady blue, the top and bottom latches can then be used to eject the blade so it can be replaced. Refer to Table 2 on page 19 for details on the H/S LED status. Onboard headers There are several CPM onboard headers, but only the Customer header (P2) is available during normal operation. Refer to Figure 3 on page 18 for the location of the Customer header. In order to gain access to the Customer header, you will first need to remove the CPM from its slot (see Removing the CPM on page 71), and then remove the CPM cover (see Removing the CPM board cover on page 72). Table 49 lists the header pin pairs and the actions associated with an installed jumper. Table 49. Customer header (P2) Pin Pair No Jumper Jumper Installed...
Page 115: Onboard Connectors
Pinouts and Mapping BIOS Force Recovery 5‐6 The BIOS force recovery routine will execute on every boot when this jumper is installed. eUSB Flash Write Protect 7‐8 The write protect input to installed eUSB devices is asserted when this jumper is in place. Disable UNR Shutdown 9‐10 Prevents the IPMC from shutting down a blade when an upper non‐recoverable (UNR) threshold is exceeded. ME Firmware Recovery Mode 11‐12 Causes the ME firmware to stay in the recovery boot loader. Onboard connectors The CPM has a number of onboard connectors and other components that are used to expand onboard capabilities. The debug connectors along with the onboard power supply LED indicators are normally used only by development, manufacturing, and troubleshooting personnel. The remaining onboard connectors are useful to expand onboard memory and communications.The following onboard connectors are covered in more detail: • MXM connector • Dual micro SAS connector •...
Page 116 Pinouts and Mapping MXM connector The onboard MXM connector is used by two types of Mobile PCI eXpress Module (MXM) devices; a supported Type A MXM 3.0 video module or the Radisys Dual Solid State Drive (DSSD) MXM module. Both types of supported modules derive all input, output, and power resources from the MXM connector. The pinout for the MXM connector is defined in the MXM 3.0 specification and is also listed in Table Table 50. MXM 3.0 connector pinout Signal Signal Signal Signal +12V +12V PE3_CPU0_RX_C_DN<1> PE3_CPU0_TX_DN<1> PE3_CPU0_RX_C_DP<1> PE3_CPU0_TX_DP<1> +5V_MXM MXM_PRSNT_R* +5V_MXM PE3_CPU0_RX_C_DN<0> PE3_CPU0_TX_DN<0> +5V_MXM MXM_PWR_GOOD PE3_CPU0_RX_C_DP<0> PE3_CPU0_TX_DP<0> +5V_MXM MXM_PWR_EN +5V_MXM NC (Reserved)
Page 117 Pinouts and Mapping Table 50. MXM 3.0 connector pinout (continued) Signal Signal Signal Signal PE3_CPU0_TX_DN<11> PE3_CPU0_RX_C_DN<11> PE3_CPU0_TX_DP<11> PE3_CPU0_RX_C_DP<11> PE3_CPU0_TX_DN<10> PE3_CPU0_RX_C_DN<10> PE3_CPU0_TX_DP<10> PE3_CPU0_RX_C_DP<10> PE3_CPU0_TX_DN<9> NC (Reserved) PE3_CPU0_RX_C_DN<9> PE3_CPU0_TX_DP<9> NC (Reserved) PE3_CPU0_RX_C_DP<9> NC (Reserved) PE3_CPU0_TX_DN<8> NC (Reserved) PE3_CPU0_RX_C_DN<8> PE3_CPU0_TX_DP<8> NC (Reserved) PE3_CPU0_RX_C_DP<8> NC (Reserved) NC (Reserved) PE3_CPU0_TX_DN<7>...
Page 118 Pinouts and Mapping Dual micro SAS connector A dual stacked micro‐SAS connector is included on the SDDS MXM module to support the installation of two 1.8" micro SATA SSD drives. Table 51 lists the pinout for the dual micro SAS connectors. Table 51. Dual micro SAS connector pinout Lower Pins Signal Upper Pins Signal +3_3V +3_3V +3_3V +3_3V +5V (Stuffing option) +5V (Stuffing option) +5V (Stuffing option) +5V (Stuffing option) SATA_RX0+ SATA_RX1+ SATA_RX0-...
Page 119 Pinouts and Mapping eUSB connector There are two Embedded Universal Serial Bus (eUSB) connectors located between the J20 and J23 backplane connectors at the back edge of the CPM. These connectors can accept one or two dual stackable eUSB flash modules. Table 52 lists the pinout for each of the eUSB connectors. Table 52. eUSB connector pinout Signal Signal EUSB_RST* EUSB_WP* USB+ USB- +3.3V SATA connector The onboard SATA connector makes it possible to connect an external SATA device to the CPM and control it using the PCH SATA controller. The external SATA device will need to receive power externally as well. The onboard SATA connector is located near the upper front panel of the CPM between any installed MXM module and the front panel. Table 53 lists the pinout for the onboard SATA connector. Table 53. SATA connector pinout Signal Signal SATA_RXN SATA_TXP SATA_RXP SATA_TXN...

This manual is also suitable for:

Atca-4648 Atca-4618

RadiSys ATCA-4616 Reference

Preface

Chapter 1: Product Overview

Chapter 2: Hardware Description

Chapter 3: Software/Firmware Description

Chapter 4: Operation and Maintenance

Chapter 5: Troubleshooting and Repair

Appendix A: Specifications

Appendix B: IPMI Commands and Managed Sensors

Appendix C: Pinouts and Mapping

Quick Links

Troubleshooting

Related Manuals for RadiSys ATCA-4616

Summary of Contents for RadiSys ATCA-4616

This manual is also suitable for:

Table of Contents