Fw 08.01.00 mcdata sphereon 4500 fabric switch installation and service manual (620-000159-330, november 2005) (312 pages)
Summary of Contents for McDATA StorageWorks 64 - SAN Director Switch
Page 1
® McDATA Products in a SAN Environment Planning Manual P/N 620-000124-500 REV A...
Page 2
Record of Revisions and Updates Revision Date Description 620-000124-000 5/2002 Initial release of the manual. 620-000124-100 9/2002 Revision of the manual to describe the Intrepid 6140 Director, Sphereon 4500 Switch, and Release 6.3 of the Enterprise Fabric Connectivity Manager application. 620-000124-200 2/2003 Revision of the manual to include additional information and describe Release 7.1 of the...
Contents Preface .......................... xiii Chapter 1 Introduction to McDATA Multi-Protocol Products Product Overview................1-2 Multi-Protocol Hardware ............1-2 SAN Management Applications..........1-5 Directors .....................1-6 Director Performance ..............1-7 Intrepid 6064 Director ...............1-8 Intrepid 6140 Director .............1-10 Intrepid 10000 Director ............1-12 Fabric Switches................1-15 Fabric Switch Performance.............1-15 Sphereon 3232 Fabric Switch..........1-16 Sphereon 4300 Fabric Switch..........1-17 Sphereon 4500 Fabric Switch..........1-18...
Page 4
Contents Chapter 2 Product Management Product Management...............2-2 Out-of-Band Management ............2-2 Inband Management ..............2-5 Management Interface Summary..........2-6 Management Server Support ............2-7 Management Server Specifications .........2-8 Ethernet Hub................2-10 Remote User Workstations.............2-10 Product Firmware................2-11 Firmware Services ..............2-12 Backup and Restore Features ............2-13 SAN Management Applications...........2-15 SANavigator and EFCM Applications .........2-15 SANvergence Manager Application........2-21...
Page 5
Contents General Fabric Design Considerations ........3-29 Fabric Initialization ..............3-30 Fabric Performance ..............3-31 Fabric Availability ..............3-37 Fabric Scalability ..............3-39 Obtaining Professional Services..........3-40 Mixed Fabric Design Considerations .......... 3-40 FCP and FICON in a Single Fabric ........3-41 Multiple Data Transmission Speeds in a Single Fabric ..
Page 6
Contents Management Server, LAN, and Remote Access Support ...5-9 Management Server ..............5-9 Remote User Workstations............. 5-11 SNMP Management Workstations........5-13 SANpilot Interface..............5-14 Security Provisions .................5-15 Password Protection..............5-15 SANtegrity Authentication ............5-16 SANtegrity Binding..............5-19 PDCM Arrays................5-20 Preferred Path ................5-23 Zoning ..................5-25 Server and Storage-Level Access Control ......5-29 Security Best Practices ............5-30 Optional Feature Keys ..............5-33 Inband Management Access ..........5-35...
Page 7
Contents Task 12: Assign Port Names and Nicknames......6-13 Task 13: Complete the Planning Worksheet ....... 6-14 Task 14: Plan AC Power ..............6-28 Task 15: Plan a Multiswitch Fabric (Optional) ......6-29 Task 16: Plan Zone Sets for Multiple Products (Optional) ..6-30 Task 17: Plan SAN Routing (Optional)........
Page 8
Contents viii McDATA Products in a SAN Environment - Planning Manual...
Page 9
Tables Out-of-Band and Inband Product Support Summary ......2-6 ISL Transfer Rate Versus Fabric Port Availability (Two-Director Fabric) 3-22 mSAN Routing Domain ................4-18 mSAN Supported Limits ................4-21 mFCP Versus iFCP ..................4-28 Transport Technology Comparison ............4-48 Cable Type and Transmission Rate versus Distance and Link Budget 5-5 Types of User Rights ..................
Page 10
Tables McDATA Products in a SAN Environment - Planning Manual...
Page 11
Figures Cabinet-Mount McDATA Products ............1-4 Intrepid 6064 Director .................. 1-9 Intrepid 6140 Director ................1-11 Intrepid 10000 Director ................1-13 Sphereon 3232 Fabric Switch ..............1-16 Sphereon 4300 Fabric Switch ..............1-17 Sphereon 4500 Fabric Switch ..............1-19 Eclipse 1620 SAN Router ................1-22 Eclipse 2640 SAN Router ................
Page 12
Figures 3-10 2-by-14 Core-to-Edge Fabric ..............3-17 3-11 Example Multiswitch Fabric ..............3-19 3-12 ISL Oversubscription .................. 3-33 3-13 Device Locality .................... 3-34 3-14 Device Fan-Out Ratio ................. 3-35 3-15 Fabric Performance Tuning ............... 3-36 3-16 Redundant Fabrics ..................3-39 3-17 Intrepid 6140 Port Numbers and Logical Port Addresses (Front) ..
Preface This publication is part of a documentation suite that supports ® McDATA multi-protocol switching and routing products, including the: ® • Intrepid 6064 Director. • Intrepid 6140 Director. • Intrepid 10000 Director. • Sphereon™ 3232 Fabric Switch. • Sphereon 4300 Fabric Switch. •...
Page 14
Preface Chapter 1, Introduction to McDATA Multi-Protocol Products This chapter provides an overview of McDATA multi-protocol products, and describes product performance and connectivity, security, and serviceability features. Chapter 2, Product Management - This chapter describes out-of-band and inband product management; the management server;...
Preface Related Publications Other publications that provide additional information about McDATA products include: • Intrepid 6064 and 6140 Directors: — McDATA Intrepid 6064 and 6140 Directors Element Manager User Manual (620-000172). — McDATA Intrepid 6064 Director Installation and Service Manual (620-000108). —...
Preface • Eclipse 2640 SAN Router: — McDATA Eclipse 2640 SAN Router Administration and Configuration Manual (620-000203). — McDATA Eclipse 2640 SAN Router Installation and Service Manual (620-000202). • General Support Publications: — McDATA SANavigator Software User Manual (621-000013). — McDATA EFC Manager Software User Manual (620-000170). —...
Page 17
Preface Where to Get Help For technical support, contact the McDATA Solution Center. The center provides a single point of contact for assistance and is staffed 24 hours a day, seven days a week, including holidays. Contact the center at the phone number, fax number, or e-mail address listed below.
Page 18
Preface All other trademarked terms, indicated by a registered trademark symbol (®) or trademark symbol (™) on first use in this publication, are trademarks of their respective owners in the United States or other countries or both. Laser Compliance Product laser transceivers are tested and certified in the United States Statement to conform to Title 21 of the Code of Federal Regulations (CFR), Subchapter J, Parts 1040.10 and 1040.11 for Class 1 laser products.
Page 19
Preface International Safety A certification bodies (CB) test report supporting a product indicates Conformity safety compliance with the International Electrotechnical Declaration (CB Commission (IEC) system for conformity testing and certification of Scheme) electrical equipment (IECEE) CB scheme. The CB scheme is a multilateral agreement among participating countries and certification organizations that accepts test reports certifying the safety of electrical and electronic products.
Page 20
Preface • Products conform with all protection requirements of EU directive 73/23/EEC (Low-Voltage Directive) in accordance with the laws of the member countries relating to electrical safety. • Products conform with all protection requirements of EU directive 93/68/EEC (Machinery Directive) in accordance with the laws of the member countries relating to safe electrical and mechanical operation of the equipment.
Page 21
Preface People’s Republic of The China Compulsory Certification mark (CCC mark) on a product China CCC Mark indicates compliance with People’s Republic of China regulatory requirements for safety and EMC (for information technology equipment) as set forth by the National Regulatory Commission for Certification and Accreditation.
Page 22
Preface Korean MIC Mark The Korean Ministry of Information and Communications mark (MIC mark) on a product indicates compliance with regulatory requirements for safety and EMC (for information technology equipment) as authorized and certified by the Korean Radio Research Institute (RRI). Mexican NOM Mark The Official Mexican Standard (Normas Oficiales Mexicanas or NOM) mark on a product indicates compliance with regulatory...
Page 23
Preface DANGER Use the supplied power cords. Ensure the facility power receptacle is the correct type, supplies the required voltage, and is properly grounded. DANGER DANGER Utiliser les câbles d’alimentation fournis. S’assurer que la prise de courant du local est du type correct, délivre la tension requise et est correctement raccordée à...
Page 24
Preface PERICOLO Usare il cavo di alimentazione in dotazione. Assicurarsi che la presa di corrente a disposizione sia del tipo corretto, eroghi la tensione richiesta e sia dotata di messa a terra idonea. PERIGO Use os cordões elétricos fornecidos. Certifique-se de que o tipo de receptor de energia da facilidade é...
Page 25
Preface The following ATTENTION statements appear in this publication and describe practices that must be observed while installing or servicing a product. An ATTENTION statement provides essential information or instructions for which disregard or noncompliance may result in equipment damage or loss of data. ATTENTION ! Activating a preferred path can result in receipt of out-of- order frames if the preferred path differs from the current path, if input and output (I/O) is active from the source port, and if congestion is present on the...
Page 26
Preface McDATA Products in a SAN Environment - Planning Manual xxvi...
Introduction to McDATA Multi-Protocol Products The enterprise-level storage area network (SAN) of today is typically complex and managed at the device layer. These problems result in SANs that use storage assets inefficiently, and are complex, error prone, expensive, and time-consuming to manage. ®...
Introduction to McDATA Multi-Protocol Products Product Overview McDATA provides storage network solutions that are integrated across a variety of platforms, original equipment manufacturers (OEMs), and locations. Solutions are modular and support multiple technologies (current and future), protocols, and data transmission speeds.
Page 29
Introduction to McDATA Multi-Protocol Products — 32-port Sphereon 3232 Switch. — 12-port Sphereon 4300 Switch. The switch provides both switched fabric and Fibre Channel arbitrated loop (FC-AL) connectivity. — 24-port Sphereon 4500 Switch. The switch provides both switched fabric and FC-AL connectivity. Refer to Fabric Switches for detailed information about each...
Introduction to McDATA Multi-Protocol Products SAN Management McDATA offers the following SAN management applications Applications installed on the rack-mount management server: ® • SANavigator application - The SANavigator application (Version 4.2 or later) is an integrated software package that provides management of an enterprise-wide, heterogeneous SAN (with multiple vendor applications) from a single console.
Introduction to McDATA Multi-Protocol Products Refer to Chapter 2, Product Management for information about SAN management applications and the management server. Chapter 2 also describes switch management through: ® • The Internet using a product’s SANpilot interface. • Inband (Fibre Channel) application clients. •...
Introduction to McDATA Multi-Protocol Products Director Directors provide the following general performance features: Performance • High bandwidth - Ports on Intrepid-series directors provide full-duplex serial data transfer at a rate of 1.0625, 2.1250, or 10.2000 gigabits per second (Gbps). • Low communication overhead - Fibre Channel protocol provides efficient use of transmission bandwidth, reduces interlocked handshakes across the communication interface, and efficiently...
Introduction to McDATA Multi-Protocol Products — A multiswitch fabric topology provides the ability to connect directors (and other McDATA switch elements) through expansion ports (E_Ports) and interswitch links (ISLs) to form a Fibre Channel fabric. Director elements receive data from a device and, based on the destination N_Port address, route the data through the fabric (and possibly through multiple switch elements) to the destination device.
Page 35
Introduction to McDATA Multi-Protocol Products Intrepid 6064 Director Figure 1-2 The director supports McDATA’s non-blocking extendable open network (EON™) architecture and concurrent firmware downloads ® through hot code activation (HotCAT ) technology. The director also provides a modular design that enables quick removal and replacement of FRUs, including a: •...
Introduction to McDATA Multi-Protocol Products • A minimum of eight to a maximum of 16 Fibre Channel port cards as follows: — Fiber port module (FPM) cards. Each FPM card provides four 1.0625 Gbps Fibre Channel port connections through duplex small form factor pluggable (SFP) fiber-optic transceivers.
Page 37
Introduction to McDATA Multi-Protocol Products Figure 1-3 Intrepid 6140 Director The director supports McDATA’s non-blocking EON architecture and concurrent firmware downloads through HotCAT technology. The director also provides a modular design that enables quick removal and replacement of FRUs, including a: •...
Introduction to McDATA Multi-Protocol Products • Backplane. • A minimum of 16 to a maximum of 35 Fibre Channel port cards as follows: — UPM cards. Each UPM card provides four 2.1250 Gbps Fibre Channel port connections through duplex SFP fiber-optic transceivers.
Introduction to McDATA Multi-Protocol Products Intrepid 10000 Director Figure 1-4 The director supports McDATA’s non-blocking nScale™ architecture that allows the product to be flexibly partitioned into multiple (up to four) separate directors (FlexPar™ feature), each with its own management and Fibre Channel services subsystems. In addition, the director supports concurrent firmware downloads through HotCAT technology.
Page 40
Introduction to McDATA Multi-Protocol Products • Smaller fabrics or SAN islands built around the director (but separated through FlexPars) benefit from better resource utilization because port configurations are flexible to accommodate change and the hardware does not require over-provisioning for growth. The FlexPar feature enables additional fabric ports to be added to a partition on demand, without interrupting fabric traffic.
Introduction to McDATA Multi-Protocol Products Fabric Switches In similar fashion to directors, fabric switches also provide high- performance, dynamic connections between end devices in a Fibre Channel switched network. Fabric switches also support mainframe and OSI computing environments. Through non-blocking architecture and limited FRU redundancy, fabric switches also offer high availability and high-performance bandwidth.
Introduction to McDATA Multi-Protocol Products Sphereon 3232 The Sphereon 3232 Fabric Switch operates at 2.1250 Gbps, provides Fabric Switch fabric connectivity for to up to 32 Fibre Channel devices, and supports FICON, EON architecture, and HotCAT technology. Figure 1-5 illustrates the switch. Sphereon 3232 Fabric Switch Figure 1-5 The switch provides a modular design that enables quick removal...
Introduction to McDATA Multi-Protocol Products The switch rear panel provides a 9-pin DSUB maintenance port for connection to a local terminal or remote terminal. Although the port is typically used by authorized maintenance personnel, operations personnel can use the port to configure switch network addresses. Sphereon 4300 The Sphereon 4300 Fabric Switch operates at 1.0625 or 2.1250 Gbps, Fabric Switch...
Introduction to McDATA Multi-Protocol Products The switch provides a modular design that enables quick removal and replacement of FRUs, including up to 12 duplex SFP fiber-optic port transceivers. Shortwave laser transceivers are available for transferring data over multimode fiber-optic cable. Longwave laser transceivers are available for transferring data over singlemode fiber-optic cable.
Page 45
Introduction to McDATA Multi-Protocol Products Figure 1-7 illustrates the switch. Sphereon 4500 Fabric Switch Figure 1-7 The switch provides a modular design that enables quick removal and replacement of FRUs, including: • Redundant power supplies and cooling fans. The switch provides two power supplies, each with an AC power receptacle and three cooling fans (six fans total).
Introduction to McDATA Multi-Protocol Products The switch rear panel provides a 9-pin DSUB maintenance port for connection to a local terminal or remote terminal. Although the port is typically used by authorized maintenance personnel, operations personnel can use the port to configure switch network addresses. SAN Routers The Fibre Channel protocol was designed for high-performance channel and storage applications within the limited confines of a data...
Introduction to McDATA Multi-Protocol Products • Implement iSAN routing and BC/DR solutions - SAN routers provide TCP/IP-based (iFCP protocol) distance extension solutions that connect geographically-dispersed SANs into an internetworked storage area network (iSAN), perform iSAN routing, and run business continuance and disaster recovery applications over existing MAN or WAN infrastructures.
Introduction to McDATA Multi-Protocol Products • High-availability - To ensure an availability of 99.9%, multi- protocol SAN router design provides a redundant configuration of power supplies and cooling fans. High availability is also provided through concurrent firmware upgrades and spare or unused multi-protocol ports.
Page 49
Introduction to McDATA Multi-Protocol Products SAN router ports operate as follows: • Two user-configured FCP (FIBRE CHANNEL 1 and 2) ports provide 1.0625 Gbps Fibre Channel storage connectivity using SFP port connectors. FCP ports can be configured for: — Auto negotiation (FC-Auto) operation. This is the default selection.
Introduction to McDATA Multi-Protocol Products The SAN router front panel provides two AC power receptacles, an Ethernet LAN connector (MGMT), port status LEDs, green power supply status (PS 1 and PS 2) LEDs, and a green system status (SYS) LED. The panel also provides a 9-pin DSUB maintenance port (MGMT) for connection to a local terminal or remote terminal.
Introduction to McDATA Multi-Protocol Products — Fibre channel fabric loop port (FL_Port) for public loop device connectivity, fabric port (F_Port) for fabric device connectivity, loop port (L_Port) for private loop device connectivity, or router port (R_Port) for SAN routing operation. •...
Introduction to McDATA Multi-Protocol Products Connectivity McDATA directors, fabric switches, SAN routers, and their associated Features Element Manager applications support the following connectivity features. Products or product classes that do not support a connectivity feature are noted. • Any-to-any connectivity - Subject to user-defined restrictions such as zoning, directors, fabric switches, and FCP ports on SAN routers define the destination port with which a source port is allowed to communicate and provide any-to-any port...
Page 53
Introduction to McDATA Multi-Protocol Products • Extended distance support - Fibre Channel ports are configured for extended distance operation (using repeaters) by changing the port buffer-to-buffer credit (BB_Credit) setting to a higher value. Refer to Distance Extension Through BB_Credit for detailed information.
Introduction to McDATA Multi-Protocol Products — Eclipse-series SAN routers - Intelligent ports that support IP network connectivity are not assigned BB_Credits. However, the ports provide approximately 96 megabytes (MB) of Transmission Control Protocol (TCP) packet buffering per transmission direction. TCP output buffering absorbs fabric data to ensure Fibre Channel BB_Credits are not exhausted.
Introduction to McDATA Multi-Protocol Products • Audit log tracking - Configuration changes to a director or fabric switch are recorded in an audit log stored on the management server. Users can display the audit log through the Element Manager application. Log entries include the date and time of the configuration change, a description of the change, and the source of the change.
Page 56
Introduction to McDATA Multi-Protocol Products • LEDs provide visual indicators of hardware status or malfunctions. LEDs are provided on FRUs, operator panels, front panels, and bezels. • System alerts, event logs, audit logs, link incident logs, and hardware logs display the following director and fabric switch information at the management server or remote workstations: —...
Page 57
Introduction to McDATA Multi-Protocol Products • Automatic notification of significant system events (to support personnel or administrators) through e-mail messages or the call-home feature. NOTE: SAN routers do not support the e-mail message feature. The Sphereon 4300 Switch and SAN routers do not support the call-home feature.
Page 58
Introduction to McDATA Multi-Protocol Products • Status monitoring of redundant FRUs and alternate data paths to ensure continued product availability in case of failover. The SAN management application queries the status of each backup FRU daily. A backup FRU failure is indicated by an illuminated amber LED.
Page 59
Introduction to McDATA Multi-Protocol Products • SNMP management for SAN routers using the following MIBs as defined by IETF working documents, RFC memorandums, and McDATA. All listed MIBs run on each SAN router. Up to eight authorized management workstations can be configured through the Element Manager application to send SNMP trap messages that indicate product operational state changes and failure conditions.
Page 60
Introduction to McDATA Multi-Protocol Products • Advanced fabric diagnostic features that include: — ISL port fencing - Any ISL that bounces (repeatedly attempts to establish a connection) causes disruptive fabric rebuilds. ISL fencing establishes a user-defined bounce threshold that when reached, automatically blocks the disruptive E_Port. —...
Product Management This chapter describes the management of McDATA multi-protocol products, including Intrepid-series directors, Sphereon-series fabric switches, and Eclipse-series SAN routers. The chapter specifically describes: • Product management, including out-of-band (non-Fibre Channel) methods, inband (fibre connection (FICON) or Fibre Channel) methods, and a management interface summary.
Product Management Product Management Out-of-band (non-Fibre Channel) management server access to all McDATA products is provided through an Ethernet local area network (LAN) connection on a director control processor (CTP) card, fabric switch front panel, or SAN router front panel. As an optional feature, inband (Fibre Channel or FICON) management access to selected McDATA products is provided through a Fibre Channel port connection.
Page 63
Product Management • Management using simple network management protocol (SNMP). An SNMP agent is implemented through the Element Manager application that allows administrators on SNMP management workstations to access product management information using any standard network management tool. Administrators can assign Internet protocol (IP) addresses and corresponding community names as follows: —...
Product Management In contrast to the applications installed on the management server, EFCM Lite does not include support for the: — Call-home feature. — Ability to download remote clients from the server. Install clients on remote workstations from the software distribution disk provided with this management option.
Product Management Inband The following inband management access methods are provided for Management directors and fabric switches as options: • Management through the product’s open-system management server (OSMS) that communicates with an application client. The application resides on an open-systems interconnection (OSI) device attached to a director or switch port, and communicates using Fibre Channel common transport (FC-CT) protocol.
Product Management S/390 or zSeries 900 Parallel Enterprise Server Host-Attached Console FICON Channel OSI Server Fibre Channel Connection Intrepid 6064 Director Inband Product Management Figure 2-2 Management Table 2-1 summarizes McDATA products and the out-of-band or Interface Summary inband management interfaces available to support the products. For each table cell, a green indicates the management interface supports the product, and a red...
Product Management Out-of-Band and Inband Product Support Summary (continued) Table 2-1 SANvergence EFCM Product SANavigator EFCM Manager SNMP SANpilot Lite OSMS 4300 Fabric Switch 4500 Fabric Switch 1620 SAN Router 2640 SAN Router Management Server Support The management server is a one rack unit (1U) high, LAN-accessed, rack-mount unit that provides a central point of control for up to 48 connected directors, fabric switches, or SAN routers.
Product Management The server is dedicated to operation of the SAN management and associated Element Manager applications. These applications provide a GUI and implement web and other server functions. Refer SAN Management Applications for additional information. NOTE: The server and SAN management application provide a GUI to monitor and manage products and are a dedicated hardware and software solution that should not be used for other tasks.
Page 69
Product Management • 24X read speed slim-type compact disk-rewritable (CD-RW) and 8X read speed digital video disk (DVD) combination drive, data only. • 56K peripheral component interconnect (PCI) internal data and fax modem, using the V .92 dial-up specification. • 16 MB graphics card.
Product Management Ethernet Hub The management server and managed directors, fabric switches, and SAN routers connect through a 10/100 Base-T Ethernet hub. Figure 2-4 illustrates the 24-port hub. MDI X Po rt St at Gr ee n - 10 9 10 0M , Co llis Ba se...
Product Management • Unix workstation with color monitor, keyboard, and mouse, using a: — Linux-based system using an Intel Pentium III processor with ® one GHz or greater clock speed, using the Red Hat 7.3 or higher operating system. ® ®...
Product Management • E/OSi - The Enterprise Operating System (internetworking) performs system configuration, management, and Fibre Channel and IP-based routing functions for Eclipse-series SAN routers. Firmware Services Director and fabric switch firmware (E/OS and E/OSn) provides services that manage and maintain Fibre Channel connections between ports.
Product Management • Fabric services - This function supports the fabric controller (login server) and name server. For redundant directors, fabric services also implement a replication manager that synchronizes node port (N_Port) registration databases between redundant CTP cards and allows CTP failover. •...
Page 74
Product Management — Operating parameters, such as buffer-to-buffer credit (BB_credit), error detect timeout value (E_D_TOV), resource allocation timeout value (R_A_TOV), switch priority, switch speed (1.0625 or 2.1250 Gbps), and preferred Domain_ID. — Active zoning configuration. — SNMP configuration parameters, such as trap recipients, community names, and write authorizations.
Product Management SAN Management Applications This section describes SAN management applications that provide a GUI to monitor, manage, and control directors, fabric switches, and SAN routers. SAN management applications include SANavigator 4.2 (or later), EFCM 8.6 (or later), and SANvergence Manager 4.6 (or later).
Page 76
Product Management • Centralized configuration - Vendor-specific device management applications can be launched from the SAN management application, including McDATA Element Manager applications. The application also provides management of director and switch zoning across multiple vendors and product models. • Monitoring and notification - The application provides real- time monitoring and event notification for devices in the SAN.
Page 77
Product Management The main window provides a: • Menu bar - Commands at the top of the window provide drop-down menu selections to perform functions for SAN devices, including editing, viewing, planning, discovery, configuration, and monitoring. • Tool bar - The tool bar (below the menu bar) provides button selections to perform SAN management tasks, including opening a SAN configuration, configuring users, setting up and starting the device discovery process, configuring zoning, displaying a...
Page 78
Product Management • Master log - The master log at the lower left corner of the window displays a list of informational, warning, or fatal events. The log also includes the event source, type, description, time, and IP address of the device generating the event. •...
Product Management Element Manager An Element Manager application is provided for each managed Application product (Intrepid 6064 Director, Intrepid 6140 Director, Intrepid 10000 Director, Sphereon 3232 Switch, and Sphereon 4500 Switch). NOTE: An Element Manager application is not supported for the Sphereon 4300 Switch.
Page 80
Product Management Hardware View Figure 2-7 When the mouse cursor is moved over a FRU in the product graphic, the FRU border highlights in blue and a pop-up identification label appears. Mouse selections (right or left click) open dialog boxes or menus that display FRU properties or allow users to perform operations and maintenance tasks.
Product Management SANvergence This section describes the SANvergence Manager and Element Manager Manager applications that provide a GUI to monitor and manage Application SAN routers, attached Fibre Channel elements, and metropolitan storage area network (mSAN) connectivity. An Element Manager application is provided for Eclipse 1620 and 2640 SAN Routers. Application GUI The SANvergence Manager application is an intuitive GUI that communicates with an attached metropolitan simple name server...
Page 82
Product Management The main window provides a: • Menu bar - Commands at the top of the window provide drop- down menu selections to access Actions (add a SAN, remove a SAN, configure SAN properties), Tools (configure parameters, generate reports, perform backup and restore operations), and Options (set general, viewing, and zoning preferences).
Page 83
Product Management Device Window (Element Manager) Figure 2-9 The graphical representation of the product emulates the hardware configuration and operational status of the corresponding real product. For example, if all router ports are connected and functional, this configuration is reflected in the device window. Mouse selections (right or left click) open dialog boxes or menus that display FRU properties or allow users to perform operations and maintenance tasks.
Product Management SANpilot Interface With product firmware Version 1.2 (or later) installed, administrators or operators with a browser-capable PC and an Internet connection can monitor and manage the director or switch through the SANpilot interface. The interface provides a GUI similar to the Element Manager application and supports product configuration, statistics monitoring, and basic operation.
Page 85
Product Management — Enable port beaconing, reset ports, and perform port diagnostics. — Retrieve dump files, retrieve product information files, and perform product firmware upgrades. — Install optional feature keys. The SANpilot interface can be opened from a standard web browser running Netscape Navigator 4.6 or higher or Microsoft Internet Explorer 4.0 or higher.
Product Management • View - At the View panel, the Director or Switch (default), Port Properties, FRU Properties, Unit Properties, Operating Parameters, and Fabric task selection tabs appear. • Configure - At the Configure panel, the Ports (default), Director or Switch, Management, Zoning, Security, and Performance task selection tabs appear.
Planning Considerations for Fibre Channel Topologies A storage area network (SAN) is typically defined as a network of shared storage resources that can be allocated throughout a heterogeneous environment. This chapter describes planning considerations for incorporating McDATA switching products into Fibre Channel SAN topologies.
Planning Considerations for Fibre Channel Topologies • Arbitrated loop - This topology uses a Sphereon 4300 or 4500 Fabric Switch to connect multiple device node loop ports (NL_Ports) in a Fibre Channel arbitrated loop (FC-AL) or hub configuration without benefit of a multiswitch fabric. Both switches support a switched mode topology that provides a single, logical connection between two device NL_Ports.
Planning Considerations for Fibre Channel Topologies This section focuses on loop operation for Sphereon 4300 (12-port) and 4500 (24-port) Fabric Switches that operate at 1.0625 or 2.1250 gigabits per second (Gbps) and support FC-AL operation using FL_Ports and public and private device connectivity. Shared Mode Versus Legacy arbitrated loop switches (such as the McDATA ES-1000 Switched Mode...
Planning Considerations for Fibre Channel Topologies Part (A) of Figure 3-1 shows device D connected to server S through a pair of H_Ports. Although the remaining switch H_Ports (six ports) and device D are unavailable for connection, frame traffic between device D and server S travels through a loop that consists of all...
Page 91
Planning Considerations for Fibre Channel Topologies Part (A) of Figure 3-2 shows server S connected to device D through a switched pair of FL_Ports communicating at 1.0625 Gbps. Server S is connected to device D through a second switched pair of ports, also communicating at 1.0625 Gbps.
Planning Considerations for Fibre Channel Topologies Public Versus Private Sphereon support connection of public 4300 and 4500 Fabric Switches Devices and private arbitrated loop devices as follows: • Public device - A loop device that can transmit a fabric login (FLOGI) command to the switch, receive acknowledgement from the switch’s login server, register with the switch’s name server, and communicate with fabric-attached devices is a public device.
Planning Considerations for Fibre Channel Topologies • Private device - A loop device that cannot transmit an FLOGI command to the switch nor communicate with fabric-attached devices is a private device. As shown in Figure 3-4, device D is a private loop device connected to a Sphereon 4500 Switch and cannot communicate with any fabric-attached device.
Planning Considerations for Fibre Channel Topologies Private devices only communicate with other devices on the same arbitrated loop, and interconnected public and private devices can communicate with each other. Such intermixed devices establish operating parameters and loop topology configuration through a port login (PLOGI) command exchange, rather than through the switch’s name server.
Planning Considerations for Fibre Channel Topologies Figure 3-5 Public Loop Connectivity • Private loop - A private loop is not connected to a switched fabric and the switch’s embedded FL_Port is inactive. All devices attached to the loop can only communicate with each other. Private loop connectivity for a Sphereon 4500 Switch is illustrated Figure 3-6.
Planning Considerations for Fibre Channel Topologies FL_Port Connectivity Sphereon provide loop connectivity 4300 and 4500 Fabric Switches through GX ports that are active as FL_Ports. The ports provide port addressing, physical connectivity, and Fibre Channel frame routing. Each FL_Port (and the embedded FL_Port) has a 24-bit address identifier.
Planning Considerations for Fibre Channel Topologies Planning for Fabric-Attached Loop Connectivity Public arbitrated loop topology supports the connection of workgroup or departmental FC-AL devices to a switched fabric through any port active as an E_Port. This 4300 and 4500 Fabric Switch topology is well-suited to: •...
Planning Considerations for Fibre Channel Topologies • Improved resource manageability. Distributed resources are consolidated and managed through Fibre Channel connectivity instead of physical relocation. One management server manages the operation and connectivity of multiple fabric directors, fabric- attached devices, arbitrated loop switches, and FC-AL devices. •...
Planning Considerations for Fibre Channel Topologies Server Consolidation Figure 3-7 Tape Device Providing fabric connectivity for multiple FC-AL tape drives by Consolidation attaching them individually to a Fibre Channel director is likewise not a cost-effective solution. A practical solution is to consolidate the tape drives on an inexpensive loop switch, then connect the switch to a director E_Port.
Planning Considerations for Fibre Channel Topologies Tape Drive Consolidation Figure 3-8 Fabric Topologies Several topologies exist from which to build a Fibre Channel fabric infrastructure. This section describes the most effective fabric topologies and provides guidance on when to deploy each topology. The topologies are effective for a wide variety of applications, are extensively tested by McDATA, and are deployed in several customer environments.
Planning Considerations for Fibre Channel Topologies Interswitch Link Fabric Connection Full Mesh Fabric Figure 3-9 Full-mesh fabrics provide increased resiliency over cascaded or ring fabrics and are well suited for applications that require any-to-any connectivity. If a single ISL fails, traffic is automatically routed through an alternate path.
Planning Considerations for Fibre Channel Topologies A modified or partial-mesh fabric is similar to a full-mesh fabric, but each switch does not have to be directly connected to every other switch in the fabric. The fabric is still resilient to failure but does not carry a cost premium for unused or redundant ISLs.
Page 103
Planning Considerations for Fibre Channel Topologies Tier 2 Devices Edge Switches Tier 1 Tier 1 Device Core Core Device Director Director Edge Switches Tier 2 Devices Interswitch Link Fabric Connection Figure 3-10 2-by-14 Core-to-Edge Fabric Each edge switch connects (through at least one ISL) to each core switch but not to other edge switches.
Planning Considerations for Fibre Channel Topologies • Tier 2 - A Tier 2 device connects to an edge switch and Fibre Channel traffic from the device must traverse only one ISL (hop) to reach a device attached to a core director or switch. •...
Planning Considerations for Fibre Channel Topologies Multiswitch Fabric D = Device ISL = Interswitch Link S = Fabric Switch (Director) Figure 3-11 Example Multiswitch Fabric A multiswitch fabric is typically complex and provides the facilities to maintain routing to all device N_Ports attached to the fabric, handle flow control, and satisfy the requirements of the classes of Fibre Channel service that are supported.
Planning Considerations for Fibre Channel Topologies • Heterogeneous fabric - Vendor interoperability in the fabric environment is supported; therefore, fabric elements can include directors, fabric switches, and open-fabric compliant products supplied by original equipment manufacturers (OEMs). To determine if interoperability is supported for a product or if communication restrictions apply, refer to the supporting publications for the product or contact McDATA.
Page 107
Planning Considerations for Fibre Channel Topologies • Distance requirements - The distance between elements in a fabric affects the type of optical port transceiver and cabling required. In addition, variables such as the number of connections, grade of fiber-optic cable, device restrictions, application restrictions, buffer-to-buffer credit limits, and performance requirements can affect distance requirements.
Planning Considerations for Fibre Channel Topologies • Bandwidth - ISL connections can be used to increase the total bandwidth available for data transfer between two directors or switches in a fabric. Increasing the number of ISLs between elements increases the corresponding total ISL bandwidth but decreases the number of port connections available to devices.
Page 109
Planning Considerations for Fibre Channel Topologies • Preferred path - Preferred path is an option that allows a user to configure an ISL data path between multiple fabric elements (directors and fabric switches) by configuring the source and exit ports of the origination fabric element and the Domain_ID of the destination fabric element.
Page 110
Planning Considerations for Fibre Channel Topologies • Zoning - For multiswitch fabrics, zoning is configured on a fabric-wide basis. Changes to the zoning configuration apply to all directors and switches in the fabric. To ensure the zoning configuration is maintained, certain rules are enforced when two or more elements are connected through ISLs to form a fabric or when two or more fabrics are joined.
Page 111
Planning Considerations for Fibre Channel Topologies In the audit log, note the Principal setting maps to a number code of 1, Default maps to a number code of 254, and Never Principal maps to a number code of 255. Number codes 2 through 253 are not used.
Page 112
Planning Considerations for Fibre Channel Topologies — For the Intrepid 10000 Director and Eclipse-series SAN routers, the firmware does not add a base offset to the numerically-assigned preferred Domain_ID. — For non-McDATA directors and switches, the product firmware may not add a base offset to the numerical preferred Domain_ID or may add a different hexadecimal base offset (not 60).
Page 113
Planning Considerations for Fibre Channel Topologies When multiple minimum-hop paths (ISLs) between fabric elements are detected, firmware balances the data transfer load and assigns ISL as follows: — The director or switch assigns an equal number of device entry ports (F_Ports) to each E_Port connected to an ISL. For example, if a fabric element has two ISLs and six attached devices, the load from three devices is transferred through each ISL.
Page 114
Planning Considerations for Fibre Channel Topologies • E_Port segmentation - When an ISL activates, the two fabric elements exchange operating parameters to determine if they are compatible and can join to form a single fabric. If the elements are incompatible, the connecting E_Port at each director or switch segments to prevent the creation of a single fabric.
Planning Considerations for Fibre Channel Topologies RSCNs are transmitted to all registered device N_Ports attached to the fabric if either of the following occur: — A fabric-wide event occurs, such as a director logging in to the fabric, a director logging out of the fabric, or a reconfiguration because of a director or ISL failure.
Planning Considerations for Fibre Channel Topologies Fabric Initialization When multiple directors or switches are connected, E_Port (ISL) communication must be established between fabric elements and the fabric must be initialized. During fabric initialization, the fabric elements: • Establish the operating mode for connected E_Port pairs and exchange link parameters (E_Port names, timeout values, class- specific information, and flow control parameters).
Planning Considerations for Fibre Channel Topologies Fabric Performance During the design phase of a Fibre Channel fabric, performance requirements of the fabric and of component directors, fabric switches, and devices must be identified and incorporated. An effective fabric design can accommodate changes to performance requirements and incorporate additional directors, switches, devices, ISLs, and higher speed links with minimal impact to fabric operation.
Planning Considerations for Fibre Channel Topologies • Read/write mixture - Although application I/O is typically a mixture of read and write operations, some applications are very biased. For example, video server applications are almost 100% read intensive, while real-time video editing applications are mostly write intensive.
Page 119
Planning Considerations for Fibre Channel Topologies Storage 1 Gbps ISL NT Server 1 NT Server 2 1 Gbps ISL Time (Sec) NT Server 1 NT Server 2 (100 MBps Max) (100 MBps Max) ISL Oversubscription Figure 3-12 Two NT servers, each with maximum I/O of 100 MBps, are contending for the bandwidth of a single ISL operating at 1.0625 Gbps.
Planning Considerations for Fibre Channel Topologies • Upgrade the existing ISL - Fabric element software, firmware, and hardware can be upgraded to support a 2.1250 or 10.2000 Gbps bandwidth traffic load between fabric elements. A 2.1250 or 10.2000 Gbps ISL is sufficient to support the bandwidth of both NT servers operating at peak load.
Planning Considerations for Fibre Channel Topologies When designing a core-to-edge fabric, servers and storage devices that support such bandwidth-intensive applications should be attached to core directors as Tier 1 devices. As a best practices policy (assuming 1.0625 Gbps ISLs), devices that generate a sustained output of 35 MBps or higher are candidates for Tier 1 connectivity.
Planning Considerations for Fibre Channel Topologies • Device output in Gbps does not oversubscribe ISLs, leading to fabric congestion. • Device output in IOPS does not result in a connectivity scheme that exceeds fan-out ratios, leading to port congestion. Figure 3-15 illustrates performance tuning for a simple fabric using appropriate ISL connectivity, device locality, and fan-out regions for device connectivity.
Planning Considerations for Fibre Channel Topologies The fabric is divided into four performance regions as follows: • Local Tier 1 devices - A video server application with I/O capabilities of 40 MBps and 2,000 IOPS must be connected to the fabric.
Page 124
Planning Considerations for Fibre Channel Topologies Fibre Channel fabrics are classified by four levels of resiliency and redundancy. From least available to most available, the classification levels are: • Nonresilient single fabric - Directors and switches are connected to form a single fabric that contains at least one single point of failure (fabric element or ISL).
Planning Considerations for Fibre Channel Topologies Fabric Fabric “A” “B” Interswitch Link Fabric Connection Redundant Fabrics Figure 3-16 Some dual-attached devices support active-active paths, while others support only active-passive paths. Active-active devices use either output path equally, and thus use both fabrics and double the device bandwidth.
Planning Considerations for Fibre Channel Topologies A scalable fabric allows for nondisruptive addition of fabric elements (directors, fabric switches, and SAN routers) or ISLs to increase the size or performance of the fabric or SAN. Large, scalable fabrics and SANs are constructed by incorporating: •...
Planning Considerations for Fibre Channel Topologies FCP and FICON in a Fibre Channel Layer 4 (FC-4) describes the interface between Fibre Single Fabric Channel and various upper-level protocols. FCP and FICON are the major FC-4 protocols. FCP is the Fibre Channel protocol that supports the small computer system interface (SCSI) upper-level transport protocol.
Page 128
Planning Considerations for Fibre Channel Topologies • When a director or fabric switch is set to open systems management style, FCP connectivity is defined within a Fibre Channel fabric using WWNs of devices that are allowed to form connections. When connecting to the fabric, an FCP device queries the name server for a list of devices to which connectivity is allowed.
Planning Considerations for Fibre Channel Topologies • The Domain_ID and physical port number of the director or fabric switch port to which a device is attached. FICON configuration attributes are implemented through logical port addressing. This concept is consistent with the address-centric nature of FICON and allows ports to be swapped for maintenance operations without regenerating a host configuration.
Planning Considerations for Fibre Channel Topologies Figure 3-18 illustrates port numbering and logical port addressing for Intrepid 6140 Director ports accessed from the rear. UPM Cards UPM Cards SBAR - 1 Module SBAR - 0 Module Intrepid 6140 Port Numbers and Logical Port Addresses (Rear) Figure 3-18 The figure shows: •...
Page 131
Planning Considerations for Fibre Channel Topologies • FICON port-to-port connectivity is hardware enforced, while FCP port-to-port connectivity is software or hardware enforced (depending on the director or switch firmware release level). — FICON architecture controls connectivity through a host-based HCD program, the CUP, and a director or switch-resident PDCM array.
Page 132
Planning Considerations for Fibre Channel Topologies • When employing inband (Fibre Channel) director or switch management, the open-systems management server (OSMS) is associated with the FCP protocol, and the FICON management server (FMS) is associated with the FICON protocol. Management server differences tend to complicate security and control issues.
Page 133
Planning Considerations for Fibre Channel Topologies SANtegrity Binding McDATA offers a SANtegrity Binding feature (including both fabric binding and switch binding) that allows the creation of reliable SAN configurations and provides a mechanism for attached devices to query the user-configured security level employed in a SAN. The feature significantly reduces the impacts of accidental or operator- induced errors.
Page 134
Planning Considerations for Fibre Channel Topologies However, the firmware and SAN management applications do not prevent FCP and FICON device configurations that may interfere with each other. A successful intermix environment requires a set of best practice conventions as follows: 1.
Page 135
Planning Considerations for Fibre Channel Topologies — When using inband director or switch management, either (or both) of the FMS or OSMS features can be enabled. When either (or both) features are enabled, the director or switch can be set to open systems or FICON management style. 3.
Page 136
Planning Considerations for Fibre Channel Topologies 6. Configure PDCM arrays - For each director or switch managed by the FICON management style, define the allow and prohibit settings for FICON device connectivity. Use the Element Manager application’s Configure Allow/Prohibit Matrix - Active dialog box. Port connectivity assignment (step 4) should be reflected in...
Planning Considerations for Fibre Channel Topologies — All FICON devices must be included in the same zone to facilitate proper state change notification. This is achieved by creating a unique FICON zone or using the default zone. Disable the default zone and explicitly create a unique zone for all FICON devices.
Planning Considerations for Fibre Channel Topologies • Better fabric performance - As a connection between fabric switches, a 10.2000 Gbps ISL delivers significantly greater bandwidth. Fibre Channel devices that are not 10.2000 Gbps- capable benefit from a higher-speed ISL, because slower traffic is multiplexed and transmitted through the 10.2000 Gbps ISL.
Planning Considerations for Fibre Channel Topologies High-Integrity Cascaded FICON directors and switches must support high-integrity Fabrics fabrics. McDATA fabric elements must have the SANtegrity Binding feature installed and operational with Enterprise Fabric Mode enabled. High-integrity fabric architecture support includes: • Fabric binding - Only directors or switches with fabric binding installed are allowed to attach to specified fabrics in a SAN.
Planning Considerations for Fibre Channel Topologies Minimum The following are minimum hardware, firmware, and software Requirements requirements to configure and enable a FICON-cascaded SAN: • A single-vendor switching environment with two or more of the following McDATA directors or switches: —...
Planning Considerations for Fibre Channel Topologies FICON Cascading A successful FICON-cascaded SAN environment requires a set of Best Practices best practice conventions as follows: 1. Connect fabric elements - Establish one or more ISLs between cascaded directors of fabric switches as follows: a.
Page 142
Planning Considerations for Fibre Channel Topologies b. Ensure the Link Incident field displays None and the Reason field is blank. If an ISL segmentation or other problem is indicated, go to MAP 0000: Start MAP in the product-specific Installation and Service Manual. c.
Page 143
Planning Considerations for Fibre Channel Topologies 7. Verify FICON devices are still logged in - Maximize the Element Manager application. Inspect the Node List View and verify FICON devices (channels and control units inspected in step are still logged in to each director or switch. 8.
Page 144
Planning Considerations for Fibre Channel Topologies NOTE: An cascaded switch is a fabric director or switch connected to a destination control unit and an entry switch. c. Run the IOCP to create an input/output configuration data set (IOCDS). The switch ID (CHPID macroinstructions) and 2-byte link address (control unit macroinstructions) are updated in the IOCDS.
Implementing SAN Internetworking Solutions Enterprise-level information technology (IT) departments often deploy storage configurations that include direct-attached storage, network-attached storage (NAS), and small, isolated storage area networks (SANs). These configurations often result in: • Isolated and inefficiently-used applications, data storage, and computing resources.
Implementing SAN Internetworking Solutions SAN Island Consolidation SAN islands tend to be constructed along application (such as product test, finance, or engineering), operating system (OS), protocol, or geographical (site-based) boundaries. Because of application and OS segmentation, large data centers at single sites often consist of SAN islands constructed with relatively small Fibre Channel switches.
Page 147
Implementing SAN Internetworking Solutions • Inability to consistently schedule maintenance downtime for each SAN. • Stranded resources. Unused ports in one SAN cannot be used by applications in another (port limited) SAN, and expensive resources (such as tape backup elements) cannot be easily shared across SAN boundaries.
Implementing SAN Internetworking Solutions Another persistent problem associated with large Fibre Channel fabrics is multi-vendor incompatibility. Due to lack of common communications standards and fabric shortest path first (FSPF) protocol, switch vendors may have to support multiple (standards- compliant and proprietary) interoperability modes. In addition, protocol enhancements may force vendors to support multiple firmware versions for the same product.
Implementing SAN Internetworking Solutions This feature reduces unused ports and resources and consolidates the enterprise into a single infrastructure, while maintaining multiple independent application and fault isolation domains. Up to four partitions can be enabled for each director (0 though 3), where a partition consists of one or more line modules (LIMs).
Page 150
Implementing SAN Internetworking Solutions • Create up to three additional FlexPars and assign resources to those FlexPars. • Perform director firmware upgrades to all Flexpars. • Enable or disable the protocol subsystem for any Flexpar. • Enable or disable switch modules and control processor (CTP) cards.
Page 151
Implementing SAN Internetworking Solutions Zone FlexPars implement an RSCN zone isolation feature that prevents fabric-format RSCNs from propagating to devices in zones not impacted by the RSCN. With zone FlexPars enabled, zoning change RSCNs are handled like device availability change RSCNs. Because the feature is device centric, zone FlexPars work in loop environments and with node port ID virtualization (NPIV) enabled.
Implementing SAN Internetworking Solutions SAN Routing Connecting isolated, department-level, and application-specific Fibre Channel SANs is a requirement for most enterprises. Consolidating SAN islands: • Provides campus storage connectivity and interoperability between formerly-incompatible Fibre Channel fabrics (from the same or different vendors). •...
Implementing SAN Internetworking Solutions • Tier 2 - To connect SAN islands without physically merging the fabrics, the second tier consists of metropolitan storage area networks (mSANs). SAN routers connect fabrics within a data center or campus to form an mSAN and transmit data between fabrics through router ports (R_Ports).
Implementing SAN Internetworking Solutions SAN Routing Concepts Figure 4-3 The following sections discuss SAN routing concepts, including: • R_Port operation. • Routed SAN zoning. • mSAN routing. • iFCP operation. • iSAN routing. • Inter-FlexPar routing. • Best practices. McDATA Products in a SAN Environment - Planning Manual 4-10...
Implementing SAN Internetworking Solutions R_Port Operation To avoid building a large Fibre Channel fabric with its inherent reconfiguration issues, SAN Routing provides any-to-any connectivity (to maximize use of common assets across SAN islands), while retaining the fault isolation characteristics of smaller SANs. SAN routers also support multiple R_Port compatibility modes, making it possible to route OEM versions of a vendor switch, direct- marketed versions of a vendor switch, and switches produced by...
Page 156
Implementing SAN Internetworking Solutions Instead of Class F frame transmission, routing communication is provided by Fibre Channel network address translation (FC_NAT) technology. This is similar to the technology used by IP networks to convert private addresses to public addresses. The principal switch in each router-connected fabric assigns the Domain_ID to the associated R_Port acting as an edge switch.
Implementing SAN Internetworking Solutions NOTE: Proxy Domain_IDs 30 and 31 are reserved for routing domains and cannot be assigned to directors or switches in any router-attached fabric. The routing domain with proxy Domain_ID 30 represents Fibre Channel devices that are part of a router-attached fabric (part of a local mSAN).
Page 158
Implementing SAN Internetworking Solutions For attached fabrics in which participating element’s Interop Mode is set to McDATA Fabric 1.0, Domain_IDs of 30 and 31 are recognized by SAN management applications and all attached devices. For attached fabrics in which participating element’s Interop Mode is set to Open Fabric 1.0, Domain_IDs of 30 and 31 are recognized by SAN management applications.
Page 159
Implementing SAN Internetworking Solutions R_Port Domain_ID Assignment The default preferred Domain_ID for each SAN router R_Port is 1. However, each port should be assigned a preferred Domain_ID (set at the R_Ports tab of the Fabric Configuration dialog box) that is unique within the attached fabric.
Page 160
Implementing SAN Internetworking Solutions There is only one router fabric manager per fabric. If more than one R_Port (from the same or multiple routers) connects to a fabric, then the port with the lowest worldwide name (WWN) is elected router fabric manager for that fabric.
Page 161
Implementing SAN Internetworking Solutions A No Zone Synchronization policy is typically not suitable for larger SAN routing environments where many devices must be visible to numerous fabrics. Append IPS Zones When the zone policy is set to Append IPS Zones, Internet protocol storage (IPS) zone set information from the router is appended to the active zone set for every router-attached fabric in the mSAN.
Implementing SAN Internetworking Solutions mSAN Routing An mSAN consists of one or two SAN routers that interconnect up to six Fibre Channel fabrics. These fabrics are typically dispersed within a data center or metropolitan campus. If two SAN routers are used, they are connected with multiple (one to four) Gigabit Ethernet (GbE) bandwidth IRLs.
Page 163
Implementing SAN Internetworking Solutions During SAN router configuration, each R_Port is assigned (through the SANvergence manager application) a unique Fabric_ID between 1 and 12. Although the theoretical limit is 12 Fabric_IDs per mSAN, the supported limit is six. As shown in Table 4-1, four Area_IDs are available to each Fabric_ID.
Page 164
Implementing SAN Internetworking Solutions Router Connectivity through mFCP mFCP provides connectivity (through a GbE-bandwidth IRL) between two Eclipse 2640 SAN Routers. mFCP is similar to Fibre Channel protocol (FCP) but implements user datagram protocol (UDP) for open systems interconnection (OSI) Layer 4 transport. mFCP links are used for path failover in high-availability mSANs.
Implementing SAN Internetworking Solutions If a direct Fibre Channel connection exists between routed fabrics, storage traffic traverses the Fibre Channel ISL and not the router- to-router mFCP link. Only SNS traffic traverses the mFCP link. However, if a router-to-router mFCP link is the only path between two Fibre Channel devices, the link is traversed by storage traffic.
Page 166
Implementing SAN Internetworking Solutions iFCP Operation There are three protocols competing to transmit storage-related I/O traffic over long-distance transmission control protocol/Internet protocol (TCP/IP) links: • iSCSI is a TCP/IP-based protocol for establishing and managing connections between IP-based storage devices, hosts, and clients. iSCSI operates on top of TCP, moving block data (iSCSI packets) over an IP Ethernet network.
Page 167
Implementing SAN Internetworking Solutions From the standpoint of fabric build events, the only difference between a local and stretched E_Port connection is the latency introduced by the TCP/IP link and associated WDM or FCIP hardware. A disruptive build fabric event at one local site propagates to the connected site.
Implementing SAN Internetworking Solutions iFCP WAN Extension Figure 4-6 iSAN Routing An internetworked SAN (iSAN) is a network composed of multiple Fibre Channel fabrics or mSANs, connected by one or more SAN routers, where at least one fabric is remotely located and connected through a WAN.
Page 169
Implementing SAN Internetworking Solutions SAN Routing also streamlines SAN connectivity by eliminating network address issues associated with duplicate Domain_IDs. Because fabric elements and devices at either end of an iFCP connection remain in separate mSANs, address conflicts between the mSANs do not occur. SAN Routing provides address translation (through FC_NAT) for zoned devices with authorization to communicate across the network.
Page 170
Implementing SAN Internetworking Solutions • Rate limiting - If ingress traffic enters the SAN router faster than egress traffic leaves, port buffers fill and cause dropped data packets. Dropped packets cause TCP to resort to internal (and inefficient) flow control, causing dramatic link throughput decrease.
Page 171
Implementing SAN Internetworking Solutions and results in a lower compression bandwidth. The algorithm has an average compression ratio increase of approximately 30% over LZO. The algorithm is recommended when up to 8 TCP sessions are used and the available bandwidth is between 10 Mbps (thin Ethernet) and 45 Mbps (DS3 transport level).
Implementing SAN Internetworking Solutions mFCP to iFCP Table 4-3 compares mFCP to iFCP and summarizes the features of Comparison each protocol. mFCP Versus iFCP Table 4-3 Feature mFCP iFCP Purpose LAN protocol to support short-distance SAN WAN Protocol to support extended-distance router connectivity SAN router connectivity OSI Layer 4 protocol...
Implementing SAN Internetworking Solutions Figure 4-7 illustrates inter-FlexPar routing. Flexpar B (tape backup fabric) is isolated from Flexpar C (product development fabric) as normally desired. However, development personnel occasionally perform tape backups that require access to Flexpar B devices. An E_Port from each FlexPar is physically connected to a SAN router R_Port, and Flexpar C servers are zoned to communicate with Flexpar B tape devices.
Page 174
Implementing SAN Internetworking Solutions 2. Domain_ID assignment - Manually assign unique Domain_IDs to all Fibre Channel directors, fabric switches, and SAN router R_Ports. Ensure the Insistent Domain_ID option is enabled at the SAN management or SANvergence Manager application. Do not assign Domain_ID 30 or 31 to any fabric elements.
Page 175
Implementing SAN Internetworking Solutions 7. Use redundant mFCP connections - For high availability (not increased bandwidth), use multiple mFCP connections between SAN routers to ensure the mSAN does not partition and connectivity to routing domains 30 and 31 remains intact. 8.
Page 176
Implementing SAN Internetworking Solutions Local mSAN Name: Boston Local mSAN_ID: 20 Date: 1/12/05 Exported Zone_ID Range: 1 to 100 Local Zone_ID Range: 101 to 512 Local mSAN Zone Summary Exported mSAN ID mSAN Name Description (Y/N) DB_Replication_2 Remote site for disaster recovery. Tape_Library_3 Remote site for data center tape library access.
Page 177
Implementing SAN Internetworking Solutions Local mSAN Name: Chicago Local mSAN_ID: 30 Date: 1/12/05 Exported Zone_ID Range: 1 to 100 Local Zone_ID Range: 101 to 512 Local mSAN Zone Summary Exported mSAN ID mSAN Name Description (Y/N) DB_Replication_2 Replication site for disaster recovery. Web_Server Local web server and associated hardware.
Page 178
Implementing SAN Internetworking Solutions 12. Configure R_Ports - For all configured SAN router R_Ports in the same fabric: a. The R_Port interconnect modes (McDATA Fabric 1.0 or Open Fabric 1.0) must be identical. This parameter is set at the SANvergence Manager application. The corresponding E_Port interoperability mode must also be identical.
Page 179
Implementing SAN Internetworking Solutions c. Port zoning can be confusing in a multi-vendor environment because OEMs implement zoning in different ways. When zoning with vendor-specific SAN management applications, zone through port WWNs, not node (device) WWNs. d. Ensure devices are physically connected before importing their node (device) WWNs to a router.
Implementing SAN Internetworking Solutions Implementing BC/DR Solutions The post-9/11 business environment requires corporations to protect critical data by implementing cost-effective business continuity and disaster recovery (BC/DR) solutions. These BC/DR solutions drive the requirement to extend local data center SANs to geographically distant locations.
Implementing SAN Internetworking Solutions • Reliability - Local storage traffic requires high-reliability communication and is intolerant of data loss, out-of-order packet receipt, or data retransmission. WANs typically provide best- effort communication service and rely on upper-level protocols for end-to-end transport. Because of these differences, a protocol conversion approach is usually required to integrate Fibre Channel SAN traffic over a geographically-dispersed network.
Page 182
Implementing SAN Internetworking Solutions • Synchronous data replication (SDR) - This operational mode ensures a remote data copy (identical to the primary copy) is created at the time the primary data is created. An update operation does not complete until confirmed at both the primary and mirrored sites.
Implementing SAN Internetworking Solutions SAN Extension There are several extension transport technologies available to Transport connect geographically-dispersed SAN islands, all of which differ in Technologies performance, latency, and implementation cost. The primary technologies include: • Dark fiber (repeated or unrepeated). •...
Implementing SAN Internetworking Solutions • Creates one logical Fibre Channel fabric through a stretched E_Port connection. The connection is vulnerable to disruptions caused by events at each site or to disruptions caused by problems with the extended-distance dark fiber link. Dark Fiber Extended-Distance Connectivity Figure 4-8 Due to the high cost of burying cables, dark fiber has limited physical...
Implementing SAN Internetworking Solutions Light wavelengths used are typically around 1,550 nanometers (nm). Optical fiber performs well in this wavelength region, with very little attenuation. For CWDM, differing wavelengths are separated by multiples of 20.0 nm. For DWDM, differing wavelengths are separated by multiples of 0.8 nm.
Page 186
Implementing SAN Internetworking Solutions • Requires sufficient BB_Credits assigned to the link (such as credits available through the Intrepid 10000 Director buffer pool). Because WDM is a method to transmit multiple signals over the same fiber-optic cable, there is no BB_Credit limitation difference between WDM and dark fiber.
Implementing SAN Internetworking Solutions SONET and SDH are globally standardized technologies, more widely deployed than dark fiber or WDM, and provide a protected connection between two locations. SONET and SDH rings are also self-healing. This means a link is usually restored within 50 ms of break detection without user intervention.
Page 188
Implementing SAN Internetworking Solutions • Does not require Fibre Channel BB_Credits assigned to the link because buffering is built in to the GFP function to enable long-distance transmission of storage traffic. • Creates a routed iSAN or one logical Fibre Channel fabric through a stretched E_Port connection, depending upon the protocol and if one or more SAN routers are deployed in the link.
Page 189
Implementing SAN Internetworking Solutions SONET or SDH service can be purchased on a monthly basis in accordance with a negotiated SLA. However, the transport links may require sufficient BB _Credits to use the purchased bandwidth. Because of BB_Credit limitations, GFP equipment must provide buffering and flow control for native FCP or FICON storage data.
Implementing SAN Internetworking Solutions • Creates a routed iSAN through an extended-distance iFCP interface. Eclipse 1620 SAN routers at each end of the link provide intelligent port connectivity and iFCP protocol conversion. The routed SAN connection ensures disruptions at one site are isolated and not allowed to propagate to other locations.
Implementing SAN Internetworking Solutions Figure 4-12 SAN Extension Technology Comparison • WDM - This technology supports high-bandwidth, low-latency applications with short RTO and RPO requirements. Applications include peer-to-peer computer clustering (grid computing) and real-time disk mirroring (SDR or ADR operational mode) over short to medium metropolitan distances.
Implementing SAN Internetworking Solutions • IP - This technology supports low-bandwidth, high-latency applications with long RTO and RPO requirements. Applications include asynchronous disk backup or tape vaulting over metropolitan to extended (intercity) distances. SAN routers are included in the extended-distance link (iFCP only), so the technology isolates the connected SANS and prevents disruptions caused by fabric or link problems.
Implementing SAN Internetworking Solutions • SAN routing requirements - If a single logical Fibre Channel fabric (created through a stretched E_Port connection) is unacceptable because of the potential for disruptive fabric rebuilds, include one or more SAN routers in the extended- distance link.
Page 194
Implementing SAN Internetworking Solutions To support greater Fibre Channel transmission rates (long-link ports), the Intrepid 10000 Director provides a buffer pool that allocates user-defined BB_Credits to each port. This buffer pool is increased if the remote fabric PFE key is enabled (refer toRemote Fabric information).
Implementing SAN Internetworking Solutions • 10.2000 Gbps long link (mixed paddles) - A paddle pair with one 1.0625 or 2.1250 Gbps paddle and one 10.2000 Gbps paddle provides ten connections. 16 BB_Credits are assigned to the slowest eight short-link ports, and 96 BB_Credits are assigned to one 10.2000 Gbps short-link port (224 BB_Credits total).
Implementing SAN Internetworking Solutions WAN Link Performance (Rate Limiting Enabled) Figure 4-14 When configuring a SAN router for extended-distance operation over an IP WAN link, the peak available bandwidth must be determined or obtained from the network service provider, and storage traffic over the link must be rate-limited accordingly.
Page 197
Implementing SAN Internetworking Solutions • Digital Signal 1 (DS1) - A framing and formatting specification that transmits 24 digital data channels on a T1 synchronous line. Each channel transmits at 64 Kbps (full-duplex), providing an aggregate bandwidth of 1.544 Mbps. Typical T1 lines are long-distance, point-to-point connections used for private networks and corporate Internet communication.
Page 198
Implementing SAN Internetworking Solutions • Fast Ethernet - A transmission medium specified by IEEE 802.3 that carries information at 100 Mbps (full-duplex) in baseband form using Category-5 copper cable or fiber-optic cable. The specification was developed to enable faster communication of LAN-connected computers.
Implementing SAN Internetworking Solutions Distance Extension To implement a successful extended-distance BC/DR solution, follow Best Practices a set of best practice conventions as follows: 1. Use dedicated bandwidth and rate limiting - If possible, negotiate dedicated bandwidth as part of the SLA with the network service provider.
Page 200
Implementing SAN Internetworking Solutions — FastWrite technology - Enable FastWrite technology to reduce protocol overhead for extended-distance write transactions. The technology is very efficient over long distances with large write transactions (such as SDR applications). — Bandwidth management - Enable QoS processing to guarantee bandwidth over a shared link.
Page 201
Implementing SAN Internetworking Solutions 7. Zone controller port pairs - When implementing load sharing, create a separate zone for each pair of communicating ports (one initiator and one target per zone). Assign the zones to different intelligent (iFCP) ports on the SAN router. If the IP network correctly distributes the load across two paths, then load sharing is implemented.
Implementing SAN Internetworking Solutions 14. Explicitly assign unique mSAN_IDs - When iSAN routing is implemented, each Eclipse 1620 SAN router comprises an mSAN (not the case with an Eclipse 2640 SAN router). Each mSAN must be assigned a unique mSAN_ID. The ID ranges from 0 to 255, and is typically the last octet of the management port IP address (although not a requirement).
Implementing SAN Internetworking Solutions iSCSI Protocol iSCSI is based on SCSI protocol that enables hosts to perform block data I/O operations with a variety of target peripherals. Targets include disk drives, tape devices, optical storage devices, printers, and scanners. A standard host-to-peripheral SCSI connection is based on a parallel transport mechanism with inherent distance and device support limitations.
Implementing SAN Internetworking Solutions iSCSI Server Many enterprise-level IT departments have deployed decentralized Consolidation computing configurations that include low-end, iSCSI-enabled servers directly attached to storage. While server acquisition costs are typically low, licensing and maintenance costs are often very high in terms of dollars and personnel time.
Implementing SAN Internetworking Solutions As shown in the figure, server consolidation is enabled by installing an Eclipse 2640 SAN Router that provides iSCSI-to-native FCP connectivity. Stranded servers subject to consolidation do not require installation at one physical location. Servers may be located in a data center or remotely;...
Page 206
Implementing SAN Internetworking Solutions McDATA Products in a SAN Environment - Planning Manual 4-62...
Physical Planning Considerations This chapter describes physical planning considerations for incorporating McDATA directors and switches into storage area networks (SANs) and Fibre Channel fabric topologies. The chapter provides planning considerations and recommendations for: • Port connectivity and fiber-optic cabling. • Rack-mount management server, Ethernet local area network (LAN), and remote access support.
Physical Planning Considerations Port Requirements Plan for sufficient shortwave laser, longwave laser, 1.0625 gigabit per second (Gbps), 2.1250 Gbps, and 10.2000 Gbps Fibre Channel ports to meet the needs of the SAN configuration. The number of ports required is equal to the number of device connections (including redundant connections), plus the number of interswitch links (ISLs) between fabric elements, plus the total number of spare port connections.
Physical Planning Considerations — Optical paddles that operate at 1.0625 or 2.1250 Gbps provide eight Fibre Channel port connections. A fully-populated director supports up to 256 connections and can be configured with shortwave or longwave transceivers, or a combination of both.
Page 210
Physical Planning Considerations Consider the following when determining the number and type of transceivers to use: • Distance between a director or fabric switch and the attached Fibre Channel device or between fabric elements communicating through an ISL. • Cost effectiveness. •...
Physical Planning Considerations Cable Type and Transmission Rate versus Distance and Link Budget Table 5-1 Cable Type and Data Transmission Rate Unrepeated Distance Link Budget 62.5/125 micron multimode at 1.0625 Gbps 250 meters (820 feet) 2.8 dB 62.5/125 micron multimode at 2.1250 Gbps 120 meters (394 feet) 2.2 dB 62.5/125 micron multimode at 10.2000 Gbps...
Physical Planning Considerations Extended-Distance Through longwave laser transceivers and repeaters or wavelength Ports division multiplexing (WDM) equipment, directors and fabric switches (but not Sphereon 4300 or 4500 Switches) support Fibre Channel data transmission distances of over 100 km. The extended distance feature is enabled on a port-by-port basis by using entries in the RX BB Credit column for a specified port at the Element Manager application’s Configure Ports dialog box.
Physical Planning Considerations Cables Fiber-optic jumper cables are required to connect directors, fabric switches, and SAN routers ports to servers, devices, distribution panels, or other elements in a multiswitch fabric or routed SAN. Depending on the attached device and fabric element port, use one of the following types of cable: •...
Physical Planning Considerations Routing Fiber-Optic Follow a logical plan for routing fiber-optic cables to avoid confusing Cables connections during installation and operation. Route cables from the access holes at the bottom of the Fabricenter equipment cabinet to fabric element ports. When routing cables to ports be aware: •...
Physical Planning Considerations Management Server, LAN, and Remote Access Support Out-of-band (non-Fibre Channel) console access to directors, fabric switches, and SAN routers is provided to perform a variety of operations and management functions. These functions are performed from one or more of the following consoles: •...
Page 216
Physical Planning Considerations Management Server The management server provides an auto-detecting 10/100 Base-T Connectivity Ethernet interface that connects to the 24-port hub mounted at the top of the Fabricenter equipment cabinet. Each director CTP card, fabric switch front panel, or SAN router front panel also provides an auto-detecting 10/100 Base-T Ethernet interface that connects to the hub.
Physical Planning Considerations • Interconnecting Fabricenter cabinets - To increase the products managed by one management server, Ethernet hubs in one or more equipment cabinets must be connected. Plan for an Ethernet cable length that meets the distance requirement between cabinets.
Physical Planning Considerations • Part of the dedicated 10/100 megabit per second (Mbps) LAN segment that provides access to managed products. This Ethernet connection is part of the equipment cabinet installation and is required. Connection of remote workstations through the hub is optional.
Physical Planning Considerations Typical Network Configuration (Two Ethernet Connections) Figure 5-3 If only one management server connection is used and this connection is provided through the customer intranet, all functions provided by the server are available to users throughout the enterprise.
Physical Planning Considerations • Up to 12 authorized management workstations can be configured through the director or fabric switch SAN management application to receive unsolicited SNMP trap messages that indicate product operational state changes and failure conditions. • Up to eight authorized management workstations can be configured through a SAN router Element Manager application to send SNMP trap messages that indicate product operational state changes and failure conditions.
Physical Planning Considerations Security Provisions Security provisions are available to restrict unauthorized access to a director, switch, or attached Fibre Channel devices. Access to the director or switch (through the SAN management application, Element Manager application, or SANpilot interface) is restricted by implementing password protection.
Physical Planning Considerations System administrators can use the SAN management application to assign remote workstation access to directors and switches. Remote sessions are allowed for anyone on a customer intranet, disallowed completely, or restricted to specific workstations. Remote users must log into the SAN management application with a user name and password, just as when logging in to the local management server.
Page 223
Physical Planning Considerations The fabric element transmits a random value (used only once), an ID value (incremented at each login), and a shared CHAP secret (16-byte random value) to the server. The server concatenates the random value, ID value, and CHAP secret, and calculates a one- way message digest (also called a hash value).
Page 224
Physical Planning Considerations • RADIUS server support - Remote authentication dial-in user service (RADIUS) is a client-server, UDP-based protocol that supports storage and authentication of passwords and CHAP secrets. Directors, fabric switches, and SAN routers support a RADIUS client (LAN-connected to a primary or secondary RADIUS server) that authenticates CHAP responses and login passwords.
Physical Planning Considerations — Authorization errors. — Authentication errors. — Management application user connections. Use of the SANtegrity Authentication feature in conjunction with other security provisions must be carefully planned and coordinated. For additional information, refer to Security Best Practices. Obtain planning assistance from McDATA’s professional services organization before implementing the feature.
Physical Planning Considerations • Rerouting delay - If a fabric topology changes, directors and fabric switches calculate a new least-cost data transfer path through a fabric, and routing tables immediately implement that path. This may result in Fibre Channel frames being delivered to a destination device out of order, because frames transmitted over the new (shorter) path may arrive ahead of previously- transmitted frames that traverse the old (longer) path.
Physical Planning Considerations Configure Allow/Prohibit Matrix - Active Dialog Box Figure 5-4 To access the dialog box, ensure the FICON management style is enabled for the director or switch, then select the Allow/Prohibit and Active options from the Element Manager application’s Configure menu.
Physical Planning Considerations PDCM Array - Example Problem Figure 5-5 A PDCM array configured for Director A prohibits logical port address 05 from communicating with logical port addresses 0A, 0B, and 0C. No PDCM array is configured for Director B. The PDCM array configured for Director A prohibits the source server from transmitting or receiving data across ISL 2.
Physical Planning Considerations Preferred Path The preferred path option allows a user to specify and configure one or more ISL data paths between multiple directors or fabric switches in a fabric. At each fabric element, a preferred path consists of a source port on the director or switch being configured, an exit port on the director or switch, and the Domain_ID of the destination director or switch.
Page 230
Physical Planning Considerations A preferred path is configured between a source server and destination device (A or B), traversing Director 1, Director 2, and Director 3. To configure the preferred path through the first director: 1. Select the Preferred Path option from the Element Manager application’s Configure menu.
Physical Planning Considerations Zoning Directors and fabric switches support a user configuration that partitions attached devices into restricted-access groups called zones. Devices in the same zone can recognize and communicate with each other through switched port-to-port connections. Devices in separate zones cannot recognize name server or route table information and therefore cannot communicate with each other.
Page 232
Physical Planning Considerations Zones are configured through the SAN management application (SANavigator 4.2 or EFCM 8.6) by authorizing or restricting access to name server or route table information (depending on the firmware release level) associated with device N_Ports that attach to director or switch fabric ports (F_Ports).
Page 233
Physical Planning Considerations ATTENTION ! If zoning is implemented by WWN, removal and replacement of a device HBA or Fibre Channel interface (thereby changing the device WWN) disrupts zone operation and may incorrectly exclude a device from a zone. • The domain identification (Domain_ID) and physical port number of the director or fabric switch port to which the device is attached.
Page 234
Physical Planning Considerations — The default zone is enabled or disabled separately from the active zone set. — If the default zone is enabled, then all devices not in a specified zone are included in the default zone and can communicate with each other.
Physical Planning Considerations • Reasons for zone implementation - Determine if zoning is to be implemented for the enterprise. If so, evaluate if the purpose of zoning is to differentiate between operating systems, data sets, user groups, devices, processes, or some combination thereof. Plan the use of zone members, zones, and zone sets accordingly.
Physical Planning Considerations • Each server HBA is explicitly bound to a storage volume or LUN, and access is explicitly authorized (access is blocked by default). • The process is compatible with OSI standards. The following are transparently supported: — Different operating systems and applications. —...
Page 237
Physical Planning Considerations 2. SANtegrity Binding - The SANtegrity Binding feature is recommended for large and complex SANs with fabrics and devices provided by multiple OEMs or that intermix FCP and FICON protocols. The feature is required for FICON-cascaded high-integrity SANs. SANtegrity Binding includes: —...
Page 238
Physical Planning Considerations 5. Preferred path - A preferred path provides soft control of fabric routing decisions on a switch-by-switch or port-by-port basis. The path instructs a fabric to use a preferred exit port out of a director or fabric switch for a specified receive port and target domain. If a preferred path is prohibited by SANtegrity Binding, PDCM arrays, or hard zoning, the path is not programmed.
Physical Planning Considerations Follow best practices listed here in order of precedence. Logically work in sequence from the most restrictive method to the least restrictive method, ensuring the most restrictive connectivity or routing paths override all other paths. Optional Feature Keys McDATA offers several operating features that are available for the switch as customer-specified options.
Page 240
Physical Planning Considerations • OpenTrunking - Purchase and enablement of this feature provides dynamic load balancing of Fibre Channel traffic across multiple ISLs. NOTE: SAN routers do not support OpenTrunking. • Full volatility - Purchase and enablement of this feature ensures that no Fibre Channel frames are stored after a director or Fabric switch is powered off or fails, and a memory dump file (that possibly includes classified frames) is not included as part of the...
Physical Planning Considerations After purchasing a feature, obtain the required PFE key through your McDATA marketing representative. A PFE key is encoded to work with the serial number of a unique director or fabric switch and is an alphanumeric string consisting of both uppercase and lowercase characters.
Physical Planning Considerations • Connectivity to an IBM S/390 Parallel Enterprise Server (Generation 5 or Generation 6), with one or more FICON channel adapter cards installed, using System Automation for Operating System/390 (SA OS/390) for native FICON, Version 1.3 or later, plus service listed in the appropriate preventive service planning (PSP) bucket.
Physical Planning Considerations SANtegrity SANtegrity Authentication is a feature that significantly enhances Authentication and extends SAN data security by providing password safety; CHAP or DHCHAP verification for fabric elements, management servers, and devices; a PCP user database; CT authentication for the OSMS interface;...
Physical Planning Considerations Server 1 Storage 1 Trunk ISL 1 ISL 1 ISL 2 ISL 2 Server 2 Director A Director B Storage 2 Server 3 OpenTrunking Figure 5-8 The figure illustrates two Intrepid 6064 Directors connected by two ISLs. Three servers use the ISLs to communicate with two storage devices.
Physical Planning Considerations When a director or fabric switch (without the full volatility feature installed) powers off or fails, a dump file is written to non-volatile random-access memory (NV-RAM). This dump file retains the last 30 Fibre Channel frames transmitted from the embedded port and the last four frames transmitted to the embedded port.
Physical Planning Considerations CNT WAN Support Fibre Channel-based SANs are typically implemented as discrete islands - isolated networks accessible only from local servers connected through a Fibre Channel fabric. Many companies are striving to interconnect isolated SANs and consolidate computer resources through WAN extension technology.
Configuration Planning Tasks This chapter describes configuration planning best-practices tasks to be performed before installing one or more McDATA Fibre Channel switching products in a storage area network (SAN) configuration. Table 6-1 summarizes planning tasks described in the chapter. Table 6-1 Configuration Planning Tasks Task Page...
Configuration Planning Tasks Configuration Planning Tasks (continued) Table 6-1 Task Page Task 13: Complete the Planning Worksheet 6-14 Task 14: Plan AC Power 6-28 Task 15: Plan a Multiswitch Fabric (Optional) 6-29 Task 16: Plan Zone Sets for Multiple Products (Optional) 6-30 Task 17: Plan SAN Routing (Optional) 6-31...
Configuration Planning Tasks • Power requirements, including an optional uninterruptable power supply (UPS). • Lengths of power cables and location of electrical outlets (for directors, switches, and the management server) having the proper phase, voltage, amperage, and ground connection. DANGER Use the supplied power cords.
Configuration Planning Tasks In addition, consider the following when planning cable routing: • The need for additional fiber-optic cables could grow rapidly. Consider installing cable with extra fibers, especially in hard to reach places like underground trenches. Consider locating the equipment cabinet near a fiber-optic patch panel.
Configuration Planning Tasks NOTE: If the FICON management server feature is enabled, the default operating style is FICON. The open systems management style cannot be enabled. Consider purchasing and enabling the SANtegrity Authentication and SANtegrity Binding features to provide additional data security in a complex and multi-OEM environment.
Configuration Planning Tasks • Remote user workstations - If remote access to the management server is required, plan to install user workstations with the SAN management and Element Manager applications configured. Administrators can use these remote workstations to configure and monitor directors and fabric switches. Up to 25 sessions can be simultaneously active.
Configuration Planning Tasks • Connect equipment cabinets - Ethernet hubs in multiple equipment cabinets can be connected to provide management server access to up to 48 managed McDATA products. Cabinets can be placed at any distance up to the limit of the 10/100 megabit per second (Mbps) LAN segment.
Page 256
Configuration Planning Tasks • If installing products and the management server on a public LAN containing other devices, default network addresses may require change to avoid address conflicts with existing devices. For Intrepid-series directors, Sphereon-series fabric switches, and Eclipse-series SAN routers, the IP address, gateway address, and subnet mask are changed through a remote terminal connected to the product’s maintenance port.
Page 257
Configuration Planning Tasks • Eclipse 1620 SAN Router: — System addresses: • MAC address is unique for each product. • Default IP address is 192.168.111.100. • Subnet mask is 255.255.255.0. • Gateway address is 0.0.0.0. — 10/100 Base-T Ethernet management port addresses: •...
Configuration Planning Tasks — 10/100 Base-T Ethernet management port addresses: • Default IP address is 192.168.100.100. • Subnet mask is 255.255.255.0. • Gateway address is 0.0.0.0. — Intelligent port (13 through 16) addresses: • Default IP address is 0.0.0.0. • Subnet mask is 0.0.0.0. •...
Configuration Planning Tasks • Determine which (if any) management workstations can have write permission for SNMP variables. • Obtain product-specific trap information from McDATA to load onto SNMP management workstations. Task 8: Plan E-Mail Notification (Optional) As an option, network administrators can configure director and fabric switch e-mail support.
Configuration Planning Tasks • Remote access to products is possible through the maintenance port or an internal modem connection to the management server. These connections are for use by authorized service personnel only and should be carefully monitored. • The number of remote workstations with access to the management server and managed products can and should be restricted.
Configuration Planning Tasks Task 11: Diagram the Planned Configuration Determine peripheral devices that will connect to each director, switch, or SAN router and where connectivity should be limited (zoning). These devices may include servers, storage control devices, and other fabric elements in a multiswitch fabric. Part of this task may have been performed when the configuration was determined.
Configuration Planning Tasks Rules for Port Names Port names can be up to 24 alphanumeric characters in length. Spaces, hyphens, and underscores are allowed within the name. Each port name must be unique for a product. However, the same port name can be used on separate products.
Page 263
Configuration Planning Tasks Product Planning Worksheet (Page 1 of 13) Attached Devices Switch Name:____________________________ IP Address:____________________________ Unit Name:__________________________ Port Port Name Location Type Model IP Address Zone Configuration Planning Tasks 6-15...
Page 264
Configuration Planning Tasks Product Planning Worksheet (Page 2 of 13) Attached Devices Switch Name:____________________________ IP Address:____________________________ Unit Name:__________________________ Port Port Name Location Type Model IP Address Zone McDATA Products in a SAN Environment - Planning Manual 6-16...
Page 265
Configuration Planning Tasks Product Planning Worksheet (Page 3 of 13) Attached Devices Switch Name:____________________________ IP Address:____________________________ Unit Name:__________________________ Port Port Name Location Type Model IP Address Zone Configuration Planning Tasks 6-17...
Page 266
Configuration Planning Tasks Product Planning Worksheet (Page 4 of 13) Attached Devices Switch Name:____________________________ IP Address:____________________________ Unit Name:__________________________ Port Port Name Location Type Model IP Address Zone McDATA Products in a SAN Environment - Planning Manual 6-18...
Page 267
Configuration Planning Tasks Product Planning Worksheet (Page 5 of 13) Attached Devices Switch Name:____________________________ IP Address:____________________________ Unit Name:__________________________ Port Port Name Location Type Model IP Address Zone Configuration Planning Tasks 6-19...
Page 268
Configuration Planning Tasks Product Planning Worksheet (Page 6 of 13) Attached Devices Switch Name:____________________________ IP Address:____________________________ Unit Name:__________________________ Port Port Name Location Type Model IP Address Zone McDATA Products in a SAN Environment - Planning Manual 6-20...
Page 269
Configuration Planning Tasks Product Planning Worksheet (Page 7 of 13) Attached Devices Switch Name:____________________________ IP Address:____________________________ Unit Name:__________________________ Port Port Name Location Type Model IP Address Zone Configuration Planning Tasks 6-21...
Page 270
Configuration Planning Tasks Product Planning Worksheet (Page 8 of 13) Attached Devices Switch Name:____________________________ IP Address:____________________________ Unit Name:__________________________ Port Port Name Location Type Model IP Address Zone McDATA Products in a SAN Environment - Planning Manual 6-22...
Page 271
Configuration Planning Tasks Product Planning Worksheet (Page 9 of 13) Attached Devices Switch Name:____________________________ IP Address:____________________________ Unit Name:__________________________ Port Port Name Location Type Model IP Address Zone Configuration Planning Tasks 6-23...
Page 272
Configuration Planning Tasks Product Planning Worksheet (Page 10 of 13) Attached Devices Switch Name:____________________________ IP Address:____________________________ Unit Name:__________________________ Port Port Name Location Type Model IP Address Zone McDATA Products in a SAN Environment - Planning Manual 6-24...
Page 273
Configuration Planning Tasks Product Planning Worksheet (Page 11 of 13) Attached Devices Switch Name:____________________________ IP Address:____________________________ Unit Name:__________________________ Port Port Name Location Type Model IP Address Zone Configuration Planning Tasks 6-25...
Page 274
Configuration Planning Tasks Product Planning Worksheet (Page 12 of 13) Attached Devices Switch Name:____________________________ IP Address:____________________________ Unit Name:__________________________ Port Port Name Location Type Model IP Address Zone McDATA Products in a SAN Environment - Planning Manual 6-26...
Page 275
Configuration Planning Tasks Product Planning Worksheet (Page 13 of 13) Attached Devices Switch Name:____________________________ IP Address:____________________________ Unit Name:__________________________ Port Port Name Location Type Model IP Address Zone Configuration Planning Tasks 6-27...
Configuration Planning Tasks Task 14: Plan AC Power Plan for facility power sources for each Fabricenter equipment cabinet, director, fabric switch, or SAN router as follows: • The Fabricenter equipment cabinet operates at 47 to 63 Hertz (Hz), 200 to 240 volts alternating current (VAC), and requires a minimum dedicated 30-ampere service.
Configuration Planning Tasks Task 15: Plan a Multiswitch Fabric (Optional) If a multiswitch fabric topology is to be implemented, carefully plan the physical characteristics and performance objectives of the topology, including the proposed number of fabric elements, characteristics of attached devices, cost, nondisruptive growth requirements, and service requirements.
Configuration Planning Tasks Task 16: Plan Zone Sets for Multiple Products (Optional) If name server zoning is to be implemented, carefully plan the characteristics and security objectives (separation of operating systems, data sets user groups, devices, or processes) of zone members, zones and zone sets.
Configuration Planning Tasks Task 17: Plan SAN Routing (Optional) If a routed SAN is to be implemented, carefully plan metropolitan area network (MAN) or wide area network (WAN) connectivity and the integration of SAN routers with standard fibre channel fabric elements.
Page 280
Configuration Planning Tasks • IP network connections - At each location, cables with appropriate connectors must be routed between the IP transport network and the SAN router. The Eclipse 1620 SAN Router supports Ethernet RJ-45 connectors or SFP optical transceivers with LC duplex connectors.
Page 281
Configuration Planning Tasks • Negotiate SLA - Network service providers provide IP WAN transport services in accordance with a negotiated service level agreement (SLA). Ensure the SLA specifies the link availability, peak available bandwidth, latency, security level, monitoring level, packet loss, and mean time to repair (MTTR). •...
Configuration Planning Tasks Task 18: Complete Planning Checklists As a guide for planning tasks, complete the planning checklists under this task. Checklists provide detailed planning activities and provide space for a planned completion date for each activity. The customer’s management information system (MIS) project manager should examine the checklists and determine the personnel and resources required for completing planning and installation tasks.
Configuration Planning Tasks Physical Planning and Hardware Installation Tasks Table 6-2 Activity Task Owner Due Date Comments Locate the physical facilities. Connect the facility alternating current If more than one managed product, consider (AC) power circuits. separate power circuits for availability. Obtain an uninterruptable power supply Recommended.
Configuration Planning Tasks Physical Planning and Hardware Installation Tasks (continued) Table 6-2 Activity Task Owner Due Date Comments Set up LAN connections to corporate Remote workstation access is supported for intranet for remote workstation access directors and fabric switches only. (optional).
Page 285
Configuration Planning Tasks Operational Setup Tasks (continued) Table 6-3 Activity Task Owner Due Date Comments Determine if inband management of the Management server and Fibre-Channel-attached director or switch is to be used, and if so, server peripheral (optional). the type (FICON or open-systems). Determine if the call-home feature is to be used.
Page 286
Configuration Planning Tasks Operational Setup Tasks (continued) Table 6-3 Activity Task Owner Due Date Comments Configure extended distance ports. If SAN routing is supported, configure extended distance ports in accordance with IP WAN requirements. Enable and configure optional feature keys. Configure link incident alerts.
Product Specifications This appendix lists specifications for McDATA directors, fabric switches, storage area network (SAN) Routers, and the FC-512 Fabricenter equipment cabinet. Director, Fabric Switch, and SAN Router Specifications This section lists specifications (dimensions, weight, power requirements, heat dissipation requirements, cooling airflow clearances, acoustical noise generated, physical tolerances, storage and shipping environment requirements, and operating environment requirements) for directors, fabric switches, and SAN routers.
Product Specifications Power Requirements McDATA products have the following nominal power requirements: Intrepid 6064 Director: Input voltage: 100 to 240 VAC. Input current: 2.0 amps at 208 VAC. Input frequency: 47 to 63 Hz. Intrepid 6140 Director: Input voltage: 200 to 240 VAC. Input current: 4.2 amps at 208 VAC.
Product Specifications Eclipse 1620 SAN Router: Input voltage: 100 to 240 VAC. Input current: 0.35 amps at 208 VAC. Input frequency: 47 to 63 Hz. Eclipse 2640 SAN Router: Input voltage: 100 to 240 VAC. Input current: 0.95 amps at 208 VAC. Input frequency: 47 to 63 Hz.
Page 291
Product Specifications Intrepid 10000 Director: Right and left side: 5.1 centimeters (2.0 inches). Front and rear: 7.6 centimeters (3.0 inches). NOTE: If the Intrepid 10000 Director is installed in a non-McDATA equipment cabinet with a door that does not provide direct airflow over the full height of the unit, 17.8 centimeters (7.0 inches) of front clearance is required.
Product Specifications Acoustical Noise This section lists acoustical noise generated, shock and vibration and Physical tolerances, and inclination tolerances for McDATA directors, fabric Tolerances switches, and SAN routers. Acoustical noise generated: Intrepid 6064 Director: 55 dB “A” scale. Sphereon 4300 and 4500 Switches: 64 dB “A” scale. Intrepid 10000 Director: 75 dB “A”...
Product Specifications Operating This section specifies environmental requirements for operating Environment McDATA products. Temperature: F to 104 F (4 C to 40 Relative humidity: 8% to 80%. Maximum wet-bulb temperature: F (27 Altitude: 10,000 feet (3,048 meters). FC-512 Fabricenter Cabinet Specifications This section lists specifications (dimensions, weight, power requirements, cooling airflow clearances, and service clearances) for the FC-512 Fabricenter equipment cabinet.
Product Specifications Power Requirements The Fabricenter cabinet has the following power requirements: Input voltage: 200 to 240 VAC. Input current: 30.0 amps at 208 VAC. Input frequency: 47 to 63 Hz. Clearances The Fabricenter cabinet has the following cooling airflow and service clearances.
Firmware Summary This appendix summarizes differences and similarities between the Enterprise Operating System (E/OS) for Intrepid 6000-series directors and Sphereon-series fabric switches; Enterprise Operating System, nScale (E/OSn) for the Intrepid 10000 Director; and Enterprise Operating System, internetworking (E/OSi) for Eclipse-series SAN routers.
Firmware Summary E/OS versus E/OSn and E/OSi - System-Related Differences (Continued) Table B-4 Feature E/OS 7.0 E/OSn 6.0 E/OSi 4.6 Nondisruptive hot code Upgrade nondisruptive to Fibre Director must be set offline to Nondisruptive hot code activation (HotCAT) Channel traffic for single and upgrade a single CTP card.
Firmware Summary E/OS versus E/OSn and E/OSi - System-Related Differences (Continued) Table B-4 Feature E/OS 7.0 E/OSn 6.0 E/OSi 4.6 Misaligned word Misaligned words not generated. Misaligned word is generated Misaligned words not generated. generation when the port state machine (PSM) transitions from inactive to active state or from active to inactive state.
Page 300
Firmware Summary E/OS versus E/OSn and E/OSi - Fibre Channel Protocol-Related Differences (Continued) Table B-5 Feature E/OS 7.0 E/OSn 6.0 E/OSi 4.6 Expansion port (E_Port) E_Port staging not supported. The Intrepid 10000 Director E_Port staging not supported. staging supports E_Port staging. The director allows only one E_Port connection to a neighbor switch to take part in a fabric build...
Firmware Summary E/OS versus E/OSn and E/OSi - Fibre Channel Protocol-Related Differences (Continued) Table B-5 Feature E/OS 7.0 E/OSn 6.0 E/OSi 4.6 Reroute delay behavior With reroute delay enabled, the With reroute delay enabled, the Reroute delay not supported. destination route point is cleared Intrepid 10000 Director first and a delay equal to the error pauses incoming port traffic and...
Firmware Summary E/OS versus E/OSn and E/OSi - Management-Related Differences (Continued) Table B-6 Feature E/OS 7.0 E/OSn 6.0 E/OSi 4.6 General SNMP support SNMP interface supports SNMP interface supports SNMP interface supports Version 1, Version 2c, and a Version 1, Version 2c, and Version 1 and a configuration-related subset of Version 3.
Page 303
Firmware Summary E/OS versus E/OSn and E/OSi - Management-Related Differences (Continued) Table B-6 Feature E/OS 7.0 E/OSn 6.0 E/OSi 4.6 FICON Management FMS is auto-enabled upon PFE When PFE is installed, the user FMS PFE key not supported. Server (FMS) PFE key installation.
Page 304
Firmware Summary E/OS versus E/OSn and E/OSi - Management-Related Differences (Continued) Table B-6 Feature E/OS 7.0 E/OSn 6.0 E/OSi 4.6 Port group online Diagnostic errors for any single Diagnostic errors for any single Port grouping (port cards or diagnostics (directors) port in a group (port card) cause port in a group (LIM) are LIMs) not supported.
Page 305
Firmware Summary E/OS versus E/OSn and E/OSi - Management-Related Differences (Continued) Table B-6 Feature E/OS 7.0 E/OSn 6.0 E/OSi 4.6 Special port states and Port state Inactive with special Logical port addresses FE and At the CLI, port state Testing reason codes reason codes Reserved and FF forced offline and placed in...
Page 306
Firmware Summary E/OS versus E/OSn and E/OSi - Management-Related Differences (Continued) Table B-6 Feature E/OS 7.0 E/OSn 6.0 E/OSi 4.6 Diagnostics Port set to Failed state upon If external port diagnostics are If external port diagnostics or (port failure state) failing a user- invoked port performed without a loopback management port diagnostics...
Page 307
Firmware Summary E/OS versus E/OSn and E/OSi - Management-Related Differences (Continued) Table B-6 Feature E/OS 7.0 E/OSn 6.0 E/OSi 4.6 FICON management User-selectable option on a per User-selectable option on a per FICON management style not style support product basis. Setting is stored user (different for each login ID) supported through the on the product.
Page 308
Firmware Summary McDATA Products in a SAN Environment - Planning Manual B-12...
Page 309
Index asynchronous data replication description 4-38 access control list IP transport 4-48 inband 5-18 long-distance requirement 4-38 out-of-band 5-18 SONET/SDH transport 4-47 role-based access any-to-any connectivity 1-26 application I/O profiles 3-31 arbitrated loop switch backup features connectivity features 1-26 CD-RW drive 2-13 default network address director and fabric switch...
Page 310
Index best practices Ethernet cable routing 5-10 cabling 3-20 fiber-optic cable routing configuration planning planning considerations connectivity 3-20 port requirements distance extension 4-55 call-home support FCP and FICON intermix 3-47 description 1-31 FICON cascading 3-55 telephone connection 6-12 multiswitch fabric topology 3-20 CD-RW drive 2-13...
Page 311
Index Fabricenter cabinet SAN routers dark fiber distance extension director bandwidth 4-46 connectivity features 1-26 description 4-39 default network address illustration 4-40 description latency 4-46 FlexPars recovery point objective 4-46 performance features recovery time objective 4-46 product overview data compression security features 1-28 algorithm selections...
Page 313
Index transport technology FC-AL devices dark fiber 4-39 connecting to a multiswitch fabric 3-11 4-45 server consolidation 3-12 SONET/SDH 4-42 tape device consolidation 3-13 4-40 FCP and FICON intermix transport technology comparison 4-48 best practices 3-47 fabric element management 3-41 impacting features 3-46 management limitations...
Page 314
Index FICON management server Sphereon 3232 Fabric Switch 1-16 description 5-35 Sphereon 4300 Fabric Switch 1-18 introduction Sphereon 4500 Fabric Switch 1-19 plan console support full fabric feature firmware description 5-39 application services 2-12 Sphereon 4300 Fabric Switch 1-17 E/OS description 2-11 full volatility feature 5-38...
Page 315
Index IP address director or fabric switch I/O requirements Eclipse 1620 SAN Router application I/O profiles 3-31 Eclipse 2640 SAN Router device locality 3-34 management server ISL oversubscription 3-32 IP distance extension iFCP protocol bandwidth 4-48 build fabric events 4-23 description 4-45 comparison to mFCP...
Page 316
Index plan security measures 6-11 product overview large fabric recommended specifications fabric initialization 3-30 map, port card 3-43 fabric scalability 3-39 mesh fabric high-bandwidth ISLs 3-30 description 3-14 high-port count directors 3-30 illustration 3-15 problems suitability 3-16 laser transceiver mFCP protocol description comparison to iFCP 4-28...
Page 317
Index out-of-band product management command line interface N_Port DHCHAP authentication 5-17 EFCM name conventions, ports 6-14 EFCM Lite application name server zoning Element Manager application introduction 1-26 SAN management application planning requirements 6-30 SANavigator network addresses SANpilot interface default settings SANvergence Manager planning SNMP...
Page 318
Index plan SAN routing 6-31 Element Manager application plan SNMP support 6-10 FMS feature plan zone sets 6-30 inband methods prepare a site plan management interface summary planning worksheet 6-14 OSMS feature port out-of-band methods binding 1-28 SAN management application blocking 1-26 SANavigator...
Page 319
Index R_Port SAN island configuring 4-34 benefits Domain_ID assignment 4-15 characteristics 3-18 operation 4-11 consolidation RADIUS server support 5-18 FlexPar technology rate limiting SAN routing description 4-26 description 3-18 implementation 4-51 problems intelligent port speed selections 4-52 SAN management application recovery point objective EFCM application dark fiber transport...
Page 320
Index routing domain (iSAN) 4-25 security provisions routing domain (mSAN) 4-18 best practices 5-30 SAN island consolidation general description 5-15 Tier 1 (fabrics) password protection 5-15 Tier 2 (mSANs) PDCM arrays 5-20 Tier 3 (iSANs) persistent binding 5-29 zone policy 4-16 preferred path 5-23...
Page 321
Index SANavigator application 2-15 storage environment SANpilot interface 2-24 storage-level access control 5-30 SANvergence Manager application 2-21 subnet mask SONET/SDH distance extension director or fabric switch bandwidth 4-47 Eclipse 1620 SAN Router description 4-42 Eclipse 2640 SAN Router illustration 4-43 management server latency 4-47...
Page 322
Index WDM distance extension zone FlexPars bandwidth 4-47 zoned fabrics description 4-40 configuration rules 3-29 illustration 4-41 joining 5-28 latency 4-47 SAN router recovery point objective 4-47 append IPS zones 4-17 recovery time objective 4-47 no zone synchronization 4-16 weight SAN routing environment 4-34 directors...
Need help?
Do you have a question about the StorageWorks 64 - SAN Director Switch and is the answer not in the manual?
Questions and answers