ADLINK Technology aTCA-3710 User Manual

ADLINK Technology aTCA-3710 User Manual

40gbe advancedtca fabric interface switch
Table of Contents

Advertisement

Quick Links

Advance Technologies; Automate the World.
aTCA-3710
40GbE AdvancedTCA
Fabric Interface Switch
User's Manual
Manual Revision:
Revision Date:
Part Number:
1.01
March 18, 2015
50-1G032-1010
1

Advertisement

Table of Contents
loading
Need help?

Need help?

Do you have a question about the aTCA-3710 and is the answer not in the manual?

Questions and answers

Summary of Contents for ADLINK Technology aTCA-3710

  • Page 1 40GbE AdvancedTCA Fabric Interface Switch User's Manual Manual Revision: 1.01 Revision Date: March 18, 2015 Part Number: 50-1G032-1010 Advance Technologies; Automate the World.
  • Page 2: Revision History

    User's Manual Revision History Date Description 1.00 01/04/2014 Initial release 1.01 18/03/2015 Add hardware configuration settings...
  • Page 3: Preface

    User's Guide Preface Copyright 2014-15 ADLINK Technology, Inc. This document contains proprietary in formation protected by copyright. All rights are reserved. No part of this manual may be reproduced by any mechanical, electronic, or other means in any form without prior written permission of the manufacturer.
  • Page 4: Table Of Contents

    User's Guide Table of Contents Revision History ............................ ii Preface ..............................iii List of Tables............................vi List of Figures ............................vi Overview............................7 Introduction..........................7 Board Overview ........................10 Functional Block Diagram ...................... 11 Package Contents ........................11 Features Summary ........................12 Switch Board Specification ....................
  • Page 5 User's Guide U-Boot Installation ..........................60 BSP/Embedded Linux/BCM SDK Installation ................61 Redundant LMP Firmware & Boot....................63 IPMI Firmware Update Procedure ....................64 Safety ..............................66 Consignes de Sécurité ........................67 Getting Service ............................ 68...
  • Page 6 List of Figures Figure 1. aTCA-3710 Block Diagram......................11 Figure 2. IPMC UART Interface........................13 Figure 3. aTCA-3710 Mechanical Layout ....................16 Figure 4. aTCA-3710 Front Panel Layout....................16 Figure 5. aTCA-3710 Software Overview....................20 Figure 6. IPMI Function Blocks........................21 Figure 7.
  • Page 7: Overview

    Additionally, the aTCA-3710 also provides front I/O to handle high throughput requirements of telecom applications. In addition to a Fabric switch, the aTCA-3710 also incorporates a Base switch to provide PICMG 3.0 Gigabit Ethernet backplane connectivity. The Base switch provides up to 13 ports of node slot connectivity to support both 6 and 14-slot chassis in addition to 8 ports of front I/O, with 6 ports supporting 1GbE and 2 ports supporting 10GbE.
  • Page 8 User's Guide The selected set of abbreviations frequently used in this document are listed below. Term Description ASIC Application Specific Integrated Circuit ATCA Advanced Telecom Computing Architecture ATCA Base Interface on the Zone-2 connectors to connect to the dual hubs, 4 differential Base Interface pairs per Base Channel.
  • Page 9 User's Guide Term Description SDRAM Synchronous Dynamic Random Access Memory Small Form Pluggable (1G interface) SFP+ Small Form Pluggable Plus (10G interface) SGMII Serial Gigabit Medium Independent Interface Serial Over LAN QSFP+ Quad Small Form Pluggable Plus (40G interface)
  • Page 10: Board Overview

    User's Guide 1.2 Board Overview Table 1. Board Overview Feature Function Description LMP Subsystem Processor Freescale QorIQ P2041(Quad Core) @ 1.2GHz DRAM Memory 4GB DDR3 1333 Mini-RDIMM 1Rx8 VLP (512Mx8) Boot Flash Memory PCT25VF080B 2x 8 Mbit SPI Serial Flash...
  • Page 11: Functional Block Diagram

    Figure 1. aTCA-3710 Block Diagram 1.4 Package Contents The aTCA-3710 is shipped with the components listed below. If any of the items in the contents list are missing or damaged, retain the shipping carton and packing material and contact the dealer for inspection.
  • Page 12: Features Summary

    User's Guide 2 Features Summary 2.1 Switch Board Specification 2.1.1 Base Interface Broadcom BCM56334 24-port GbE switch with 4-port 10GbE HiGig2 (iPss) 13x 1000BASE-T ports to Base Interface (Zone2) and two dedicated 10/100BASE-T ports to shelf manager 2x 10GbE SFP+ egress ports to front panel...
  • Page 13: Figure 2. Ipmc Uart Interface

    User's Guide 2.1.3 Local Management/Control Plane Processor Freescale QorIQ P2041 (quad core) @ 1.2GHz Max. 4GB, DDR3 SDRAM on VLP mini-DIMM sockets 2x 2GB (Micro SSD card, redundant) 2x 1MB (SROM), optional: 1 x 2GB (Micro-SD card , SROM redundant) 1x GbE RJ-45 on front panel console port to LMP or IPMC (switched by IC), to LMP is default 1x USB 2.0 ports to front panel...
  • Page 14 User's Guide 2.1.5 Miscellaneous Out-of-Service (OOS) LED, 1x, Red Power Good LED, 1x, Green Health LED, 1x, Amber Hot Swap LED, 1x, Blue RJ-45 GbE port Link/Activity status, 9x, Left LED Green for Active, Right LED Yellow for 100/1000M...
  • Page 15 User's Guide VLAN, STP, link aggregation etc. Additional protocols are available upon request ADLINK Shelf Management in IPMI package can support the following functions: HW management per ATCA PICMG 3.0; includes IPMI management, FRU HW management, shelf environment management (power, cooling, watchdog timer, E-keying), and shelf sensor management IPMI over LAN (RMCP/IP) and Serial-Over-LAN per PICMG 3.0...
  • Page 16: Placement

    Base switch Link/Activity status LED LMP & IPMC 10GbE SFP+ of Fabric switch MGMT port 10GbE SFP+ of of LMP Base switch Reset LED Health LED Hot swap LED Power LED Out of Service LED Figure 4. aTCA-3710 Front Panel Layout...
  • Page 17: External Interfaces

    User's Guide 3 External Interfaces 3.1 Front Panel 3.1.1 Console Port One console serial port is provided in an RJ-45 shielded connector outlet, and is connected to the LMP, which enables access to all major devices using command line interface commands for configuration and monitoring.
  • Page 18: Table 4. Rj-45 Management Port Pin-Out

    User's Guide 3.1.2 USB Port A single USB port is available on the face plate, which is connected to the LMP. The P2041 supports 2x USB compliant to USB specification Rev 2.0. On the board, USB2 is routed to the faceplate USB connector.
  • Page 19: Zone 2 Backplane Interfaces

    Ethernet switch in the blade. Through the switch, the Base Interface traffic goes to the LMP for necessary processing. The Fabric Interface in aTCA-3710 blade can connect to up to thirteen boards in the chassis. The thirteen Fabric Interface ports are connected to the high capacity onboard Ethernet switch, where the port speed is configurable to 1Gbps/10Gbps/40Gbps in the form of one SerDes lane at 1Gbps or 4 SerDes lanes at 2.5Gbps each or 4 SerDes lanes at 10Gbps,...
  • Page 20: Software Component Overview

    This chapter describes the top level of software design of the aTCA-3710. The aTCA-3710 software can be logically divided into four subsystems as shown in Figure 5. This chapter will cover the IPMI subsystem, LMP subsystem, and switch subsystem. The remaining subsystem is the customer software, which can be a user add-on L2/L3 stack, or the upper level user interface/application.
  • Page 21: Lmp Subsystem

    The aTCA-3710 S/W package provides the user mode BCM-SDK only. The kernel mode BCM-SDK, where most of the functionalities are included in the...
  • Page 22: Figure 7. Broadcom Sdk Function Blocks

    User's Guide Figure 7. Broadcom SDK Function Blocks The ADLINK Software for Network (ADSN) is based on the Broadcom SDK and includes all Layer 2 and Layer 3 switching functions. The Fabric & Base Switches (BCM56842/56846/56334) can also be paired with ADSN. ADSN includes functions of port configuration, customized port mirror, VLAN setting, trunk group, ACL, static route, and management API with a more user-friendly CLI to enable more straightforward API development.
  • Page 23: Ipmi Subsystem

    IPMI session over the network with RMCP+ protocol. The aTCA-3710 supports SOL on the Base Interface which is powered by the BCM56334 Gigabit Ethernet Switch. The BCM56334 Gigabit Ethernet Switch is connected to the IPMC (Smart Fusion A2F500) via the NC-SI interface, which provides remote management capability before the payload power is authorized.
  • Page 24 User's Guide Item Sensor Name Sensor Address Description Upper Critical Threshold = 2.7 Volts Upper Non-Critical Threshold = 2.65 Volts Lower Non-Critical Threshold = 2.35 Volts Lower Critical Threshold = 2.3 Volts Lower Non-Recoverable Threshold = 2.25 Volts +3.3V...
  • Page 25 User's Guide Item Sensor Name Sensor Address Description Upper Non-Critical Threshold = 5.3 Volts Lower Non-Critical Threshold = 4.7 Volts Lower Critical Threshold = 4.6 Volts Lower Non-Recoverable Threshold = 4.5 Volts (13) DDR3 VTT +0.75V (0xC) Voltage Sensor Upper Non-Recoverable Threshold = 0.83 Volts...
  • Page 26 User's Guide Item Sensor Name Sensor Address Description Upper Critical Threshold = 64 Volts Upper Non-Critical Threshold = 55 Volts Lower Non-Critical Threshold = 43 Volts Lower Critical Threshold = 41 Volts Lower Non-Recoverable Threshold = 39 Volts (20)
  • Page 27 User's Guide Byte Data Field [0] – 1b = FRU Operational State M0 - FRU Not Installed [7:0] – Optional/Reserved. If provided, write as 80h (IPMI restriction). Ignore on read. 5.3.2 Get Sensor Reading (Physical IPMB-0 Sensor) Byte Data Field...
  • Page 28 User's Guide Byte Data Field Event Status” command for the sensor before the update has completed. This is most likely to be the case for sensors, such as fan RPM sensors, that may require seconds to accumulate the first reading after a re-arm.
  • Page 29 User's Guide 5.3.4 Version Change Sensor Sensor Type Sensor Sensor Event Type Code Specific Offset Version Change 00h Intelligent change detected with associated Entity. Informational. This offset does not imply whether the intelligent change was successful or not. Only that a change occurred.
  • Page 30 User's Guide Sensor Type Sensor Sensor Event Type Code Specific Offset component on the board/FRU that is not tracked as a FRU) 14h board/FRU replaced with equivalent version 15h board/FRU replaced with newer version 16h board/FRU replaced with older version 17h board/FRU intelligent configuration change (e.g.
  • Page 31: Fru Information

    User's Guide Byte Data field [5] - 1b = state 5 asserted [4] - 1b = state 4 asserted [3] - 1b = state 3 asserted [2] - 1b = state 2 asserted [1] - 1b = state 1 asserted [0] - 1b = state 0 asserted For discrete reading sensors only.
  • Page 32 User's Guide Product Serial Number Programmed with serial number FRU File ID FRU version 5.4.3 Board E-key information The board E-keying information is compatible with PICMG 3.0 R3.0 and PICMG 3.1 R2.0. Base Channel of Board Point-to-Point Connectivity: Field...
  • Page 33 User's Guide Field Field Data Link Descriptor 00102F41h ~ 00102F4Dh Link Grouping ID (Bits 31-24) 00h: Single-channel link Link Type Extension (Bits 23-20) 1h: Fixed 10GBASE-BX4 (XAUI) Link Class (bits 19-16) 0h: Basic signaling Link Class Link Type (Bits 15-12) 2h: PICMG 3.1 Ethernet Fabric Interface...
  • Page 34: Ipmi Commands

    User's Guide 5.5 IPMI Commands The following table presents all the commands which are supported by the aTCA-3710 through different interfaces and compatible with IPMI v1.5 and PICMG 3.0 R2.0 ECN001. There are two interfaces implemented with IPMI command support.
  • Page 35 User's Guide Sensor Device Commands (cont'd) Get Sensor Event Enable ● ● Rearm Sensor Events ● ● Get Sensor Event Status ● ● Get Sensor Reading ● ● FRU Device Commands Get FRU Inventory Area Info ● ● Read FRU Data ●...
  • Page 36: Lmp Subsystem

    LMP OS image redundancy 6.1.1 U-Boot Redundancy The U-boot image is stored in an SPI flash memory. On the aTCA-3710, the LMP has two such U-boot flash memories in a primary and a redundant fashion. To support redundancy, the IPMC is involved in...
  • Page 37: Table 5. Lmp U-Boot Boot Up Failover Actions

    User's Guide IPMC BOOTING_LMP LMP_INIT process process Start Change BootSel Setup BootSel Start CFD watchdog Start Reset LMP Init DRAM watchdog timeout? Init DRAM HANG watchdog clear? Clear CFD watchdog Figure 8. U-Boot Redundancy Flow The actions for each step are described in Table 5 .
  • Page 38: Lmp Sdk/Bsp Porting (U-Boot File System)

    User's Guide 6.1.2 Kernel/File System Redundancy The aTCA-3710 provides dual Chip SSD devices for primary storage and redundant storage of the OS image. 6.2 LMP SDK/BSP Porting (U-Boot file system) The LMP-SDK used is the Local Management Processor P2041 SDK provided by Freescale. As of December 2012, most up-to-date version is SDK 1.3, distributed in an ISO file.
  • Page 39: Table 6. Summary Of Devices And Access

    User's Guide Table 6. Summary of Devices and Access Device Description Host Program Remark Local Management (All LMP Linux LMP-SDK + P2041 Processor programs) ADLINK patch Mini VLP LMP-SDK + LMP_MEM DRAM for LMP LMP linux kernel DIMM ADLINK patch...
  • Page 40: Lmp State Machines

    Waits for a User command input. Disables the BMC watchdog. BOOTING_OS Triggers the OS booting. Performs the LMP services. aTCA-3710 board management services include the following: IN_SERVICE - ipmievd: IPMI message processing daemon Other services defines by a user can be included. Performs testing...
  • Page 41: Boot Sequence

    The boot sequence of devices with a CPU core is illustrated in Figure 11 . The IPMC is responsible for initiation of the booting sequence. The states in the boot sequence are summarized in Table 9 . Figure 11. aTCA-3710 Boot Sequence Table 9. Boot Sequence State Descriptions...
  • Page 42: Non-Volatile Memory Mapping

    User's Guide 6.6 Non-volatile Memory Mapping The aTCA-3710 provides various non-volatile memory (NVM) to store software images and log system events for different purposes. The memory map for the devices is shown in Table 10. Table 10. Non-volatile Memory Map...
  • Page 43: Switch Subsystem

    User's Guide 7 Switch Subsystem The switch subsystem includes Broadcom driver/SDK porting, and ADLINK switch software running over the Broadcom SDK. ADLINK Software for Network (ADSN) includes all Layer 2 and Layer 3 switching functions. The Fabric & Base Switch (BCM56842/56846/56334) can also be paired with ADSN. ADSN includes functions of port configuration, customized port mirror, VLAN setting, trunk group, ACL, static route, and management API with a more user-friendly CLI to enable more straightforward API development.
  • Page 44: Adlink Switch Software Overview

    User's Guide 7.2 ADLINK Switch Software Overview The aTCA-3710 switch ports support Layer 2 forwarding of frames according to IEEE 802.1D and 802.1 Q specifications. Layer 2 forwarding uses the media access control address (MAC address) from the host's network interface cards (NICs) to decide where to forward frames. Layer 2 switching is hardware based, which means the Broadcom switches use application-specific integrated circuits (ASICs) to build and maintain filter tables (also known as MAC address tables or CAM tables).
  • Page 45 User's Guide (3) Above the software stacks, the aTCA-3710 provides three kinds of user interface. The basic CLI interface can be used by the customer for quick access to the switch functions. The API and SNMP interfaces can be used by the customer for further customized development. Both API and SNMP interfaces support remote access, and the customer can link a local library on a remote client to invoke the switch APIs or send SNMP requests.
  • Page 46: Getting Started

    8.1 Safety Requirements The following safety precautions must be observed when installing or operating the aTCA-3710. ADLINK assumes no responsibility for any damage resulting from failure to comply with these requirements.
  • Page 47: Figure 14. Hardware Configuration Switches And Jumpers

    User's Guide Figure 14. Hardware Configuration Switches and Jumpers...
  • Page 48: Table 11. Ground Header Settings (Jp3)

    User's Guide Headers and Jumpers • Ground Connection (JP3): This header is used to short GND and GND_SHELF on the board. The default setting is Pin 1 and Pin 2 shorted. GND_SHELF BC892 C1000P2KV6MX7 PH-1*3-2D54-D-ST-ML-6/3 Figure 15. Ground Header Schematic (JP3)
  • Page 49: Table 12. Xaui_Sel Status Settings (Jp5)

    User's Guide Description BCM56842 connects to BCM56334 (HiGig) BCM56842 connects to aDB3710 (BCM8706) Table 12. XAUI_SEL Status Settings (JP5) • IPMC JTAGSEL Mode (JP8): This header is used to select either FPGA Fabric TAP or Cortex-M3 JTAG debug. The default setting is FPGA Fabric TAP.
  • Page 50: Figure 18. Ipmc Jtag Header Schematic (Jp9)

    User's Guide • IPMC JTAG (JP9): This header is used to program the IPMC bit file using a JTAG cable. P3V3_PRE R861 10KR2F R868 R873 R862 10KR2F IPMC_JTAG_TCK IPMC_JTAG_TDO IPMC_JTAG_TMS VJTAG VPUMP IPMC_JTAG_nTRST IPMC_JTAG_TDI PH-2*5-2D54-D-ST-ML-6/3-JVE 1KR2F Figure 18. IPMC JTAG Header Schematic (JP9) •...
  • Page 51: Table 14. Spi Flash Or Sd Card Selection (Sw1)

    User's Guide DIP Switches • SW1: SW1 is used to switch between SPI flash boot and SD card boot. The default setting is SPI flash boot. SW No. Mode Setting SPI Flash SD Card Table 14. SPI Flash or SD Card Selection (SW1) •...
  • Page 52: Table 16. Multi-Function Switch (Sw3)

    User's Guide • SW3: This is a multi-function switch. Pin 1 is used to select Shelf Mode or Stand Alone Mode. Pin 2 is used to select between boot from SPI Flash 0 or boot from SPI Flash 1. Pin 3 is used to select compliance to PICMG 3.0 or PICMG 3.1.
  • Page 53 User's Guide 8.2.2 Connecting to the Blade Connecting to the aTCA-3710 by COM port Step 1 Turn on the aTCA‐3710. Connect the USB port of a laptop/PC to the LMP via the front panel COM port (RJ-45) on the aTCA‐3710 using an RJ-45‐to‐USB adapter.
  • Page 54 172.20.5.162 netmask 255.255.255.0 Connecting to the aTCA-3710 remotely Before connecting to the aTCA-3710 remotely, you must configure the IP address of the LMP management port. If there is no IP address, please configure it first by following the instructions above in “Connecting to the aTCA-3710 by COM port”.
  • Page 55 Ethernet cable. LMP MGT port Step 3 Start a terminal emulator application (e.g. SecureCRT) on the laptop/PC with the IP of the aTCA-3710, for example (assuming IP address of LMP MGT port is 172.20.5.62): Step 4 Login to the aTCA‐3710.
  • Page 56 User's Guide 8.2.3 Boot into CLI for Base Switch Management BCM SDK will start automatically when the aTCA-3710 boots up. The user can boot into CLI for Base switch management as follows: Login to the LMP OS. Use “telnet localhost 9895” to access the BCM SMB diagnostic shell.
  • Page 57: Configuring And Managing The Fabric Switch

    User's Guide 9 Configuring and Managing the Fabric Switch Information on configuring and managing the Fabric switch is contained in the "Broadcom SMB Diagnostic Shell Quick Start Guide". To receive a copy of this document, please contact your ADLINK representative.
  • Page 58: Appendix I. Advanced Operation

    IPMC RJ-45 RJ-45 DIP SW Figure 20. Switching I/O between Front Panel and Base Switch Check the switch status. The BCM56842 connects to the front panel by default. root@atca-3710:~# ipmitool raw 0x2e 0x71 0x13 0x5f 0x00 13 5f 00 01...
  • Page 59: Switch Uart Serial Port Between Front Panel And Ipmc

    User's Guide Switch one 10GbE port of the BCM56842 to the Base Switch (BCM56334) and check the status. root@atca-3710:~# ipmitool raw 0x2e 0x70 0x13 0x5f 0x00 0x00 13 5f 00 root@atca-3710:~# ipmitool raw 0x2e 0x71 0x13 0x5f 0x00 13 5f 00 00 Set the BCM56842 to default status.
  • Page 60: Appendix Ii. Software Setup/Update/Recovery

    Appendix II. Software Setup/Update/Recovery U-Boot Installation Building U-boot When there is no U-boot on the aTCA-3710, perform the following actions 1. Boot the aTCA-3710 from an SD card, SW1-1 ON, SW1-4 ON. 2. Copy the U-boot firmware to SPI flash.
  • Page 61: Bsp/Embedded Linux/Bcm Sdk Installation

    User's Guide BSP/Embedded Linux/BCM SDK Installation The steps for BSP installation and updating are the same. Please perform the following actions: Enter U-boot and set MAC addresses => mac port 2 => mac 0 00:30:64:22:22:12 => mac 1 00:31:64:22:11:23 =>...
  • Page 62 Using FM1@DTSEC4 device host 172.20.5.198 is alive => saveenv => tftp 1000000 uImage; tftp c00000 atca-3710.dtb; tftp 2000000 rootfs.ext2.gz.uboot; => setenv bootargs root=/dev/ram rw console=ttyS0,115200 ramdisk_size=50000000; bootm 1000000 2000000 c00000 Update U-boot/BSP/BCM-SDK by executing /sbin/lmp-installer. Change the IP address and TFTP server IP those of your actual environment, and save the file.
  • Page 63: Redundant Lmp Firmware & Boot

    (main or backup). Redundant LMP Firmware & Boot The U-boot image is stored in an SPI flash memory. On the aTCA-3710, the LMP has two such U-boot flash memories in a primary and a redundant fashion. Boot Auto-Switch The user can enable the boot auto-switch function to U-boot to automatically switch to the redundant flash when the primary flash memory fails.
  • Page 64: Ipmi Firmware Update Procedure

    Perform the following steps to update the IPMC firmware: Step 1: Prepare an external host PC with Linux OS and connect it to the serial port on the aTCA-3710 via the COM port (USB Mini-B on the front panel). Put the IPMItool utility and new firmware image on the host PC.
  • Page 65 Perform the following steps to update firmware: Step1: Prepare an external host PC with Linux OS and connect it to the target aTCA-3710 via Base Interface. Put the IPMItool utility and the new firmware image on the host PC. Enter the following command: [root@localhost 3710]# ./ipmitool –I lan –H 172.17.172.134 hpm upgrade hpm1fw.img...
  • Page 66: Safety

    User's Guide Safety 1. Please read these safety instructions carefully. 2. Please keep this User‘s Manual for future reference. 3. When installing/mounting or uninstalling/removing equipment, turn off power and unplug any power cords/cables. Use a moist towel or cloth for cleaning.
  • Page 67: Consignes De Sécurité

    User's Guide Consignes de Sécurité 1. Veuillez lire attentivement ces consignes de sécurité. 2. Veuillez garder ce manuel de l’utilisateur pour référence ultérieure. 3. Une entrée de courant est fournie pour l’entretien de l’appareil débranché. Débrancher l’appareil des prises de courant et utiliser cette entrée de courant avant entretien ou nettoyage. Utilisez un tissu humide ou un chiffon pour le nettoyage.
  • Page 68: Getting Service

    Tel: +49-621-43214-0 Fax: +49-621 43214-30 Email: emea@adlinktech.com ADLINK Technology, Inc. (French Liaison Office) Address: 6 allée de Londres, Immeuble Ceylan 91940 Les Ulis, France Tel: +33 (0) 1 60 12 35 66 Fax: +33 (0) 1 60 12 35 66 Email: france@adlinktech.com...
  • Page 69 Singapore 349584 Tel: +65-6844-2261 Fax: +65-6844-2263 Email: singapore@adlinktech.com ADLINK Technology Singapore Pte. Ltd. (Indian Liaison Office) Address: #50-56, First Floor, Spearhead Towers Margosa Main Road (between 16th/17th Cross) Malleswaram, Bangalore - 560 055, India Tel: +91-80-65605817, +91-80-42246107 Fax: +91-80- 23464606 Email: india@adlinktech.com...

Table of Contents