Related Manuals for Qlogic Storage Networking (Unified Fabric Pilot
Summary of Contents for Qlogic Storage Networking (Unified Fabric Pilot
-
Page 1: Installation Guide
Installation Guide Storage Networking (Unified Fabric Pilot) QLogic 8100 Series Adapters, QLogic 5800 Series Switches, Cisco Nexus 5000 Series Switches, JDSU Xgig Platform 51031-00 A... - Page 2 QLogic Corporation reserves the right to change product specifications at any time without notice. Applications described in this document for any of these products are for illustrative purposes only. QLogic Corporation makes no representation nor warranty that such applications are suitable for the specified use without further testing or modification.
-
Page 3: Table Of Contents
Table of Contents Preface Intended Audience ..........Related Materials . - Page 4 QLogic QLE8142 Converged Network Adapter ..... . QLogic 5800V Series Fibre Channel Switch ......
- Page 5 Adapter, Switch, and Target ......... DCBX Exchanges Between QLogic QLE8152 Adapter and the Cisco Nexus 5000 FCoE Switch.
- Page 6 Installation Guide Storage Networking (Unified Fabric Pilot) 5-28 iSCSI Traffic Performance Validation Setup....... 5-26 5-29 Throughput Results Showing the Fibre Channel and iSCSI Application...
-
Page 7: Preface
Related Materials In addition to this guide, the FCoE Design Guide, developed jointly by Cisco and QLogic (Cisco document number C11-569320-01), is a valuable resource and is referenced throughout this guide. The following links provide more detailed information, and connect to the IEEE documents that define the Enhanced Ethernet functions: ... -
Page 8: Documentation Conventions
“Installation Checklist” on page 3-6. For more information, visit www.qlogic.com. Text in bold font indicates user interface elements such as a menu items, buttons, check boxes, or column headings. For example: ... - Page 9 Documentation Conventions Text in italics indicates terms, emphasis, variables, or document titles. For example: For a complete listing of license agreements, refer to the QLogic Software End User License Agreement. What are shortcut keys? To enter the date type mm/dd/yyyy (where mm is the month, dd is the day, and yyyy is the year).
- Page 10 Preface Documentation Conventions 51031-00 A...
-
Page 11: Overview
® disrupting operations, Cisco , JDSU™ (formerly Finisar ), and QLogic have collaborated to produce this guide to simplify the installation and certification of an FCoE pilot. The guide provides system engineers, architects, and end users with a step-by-step method to implement and validate a unified fabric and measure performance of a pilot operation. - Page 12 1–Overview 51031-00 A...
-
Page 13: Planning
The reference architecture described in this guide was assembled from equipment that was available in the QLogic NETtrack Developer Center. You will need to substitute your own equipment, and modify the installation and validation process accordingly. - Page 14 2–Planning Where and How to Deploy Currently, a Converged Network Adapter must always be connected to a switch that has DCB. There are two types of switches that have DCB: a DCB switch and an FCoE switch. The DCB switch has enhanced Ethernet support, but does not have Fibre Channel forwarder (FCF) capabilities, and does not support the conversion of Fibre Channel frames to FCoE frames.
-
Page 15: Architecture
Approach The test configuration described in this section was installed and validated at the QLogic NETtrack Developer Center (NDC) located in Shakopee, MN. The NDC provides QLogic customers and alliance partners with the test tracks and engineering staff to test interoperability and optimize performance with the latest server, networking, and storage technology. -
Page 16: Reference Architecture Diagram
3–Architecture Reference Architecture Description Figure 3-1. Reference Architecture Diagram 51031-00 A... -
Page 17: Equipment Details
Two Windows 2008, one VMware ESX™ with one Windows 2008 and Red Hat ® Enterprise Linux ® guest with Microsoft iSCSI initiator. All servers use the QLogic QLE8142 adapter. Table 3-2. JDSU Converged Network Validation Equipment Model/Part Product Number Xgig four-slot chassis... - Page 18 3–Architecture Reference Architecture Description 51031-00 A...
-
Page 19: Installation
Determine Configuration QLogic FCoE adapters are supported on multiple hardware platforms and operating systems. Generally, the following specifications apply, but you should always check the QLogic Web site for current information. This configuration uses a subset of the following equipment: ... -
Page 20: Equipment Installation And Configuration
Close the computer cover. Plug the appropriate Ethernet cable (either copper or optical) into the adapter. Optical models ship with optical transceivers installed. QLogic 8100 Series adapters used for this project operate only with optical transceivers sold by QLogic. ... -
Page 21: Install The Adapter Drivers
4–Installation Equipment Installation and Configuration Install the Adapter Drivers To install the FCoE and Ethernet drivers: Go to the QLogic Driver Downloads/Documentation page (Figure 4-1) at http://driverdownloads.qlogic.com/QLogicDriverDownloads_UI/default.aspx. Figure 4-1. Driver Download Page Model Selection In the table at the bottom of the page, select Converged Network Adapters, the adapter model, your operating system, and then click Go. -
Page 22: Install Sansurfer Fc Hba Manager
Equipment Installation and Configuration Install SANsurfer FC HBA Manager To install SANsurfer ® FC HBA Manager: Go to the QLogic Driver Downloads/Documentation page at http://driverdownloads.qlogic.com/QLogicDriverDownloads_UI/default.aspx. In the table at the bottom of the page (Figure 4-2), select Converged Network Adapters, the adapter model, your operating system, and then click Go. -
Page 23: Cabling
LUNs you are mapping. Refer to the Fibre Channel switch documentation for zoning instructions. FCoE Switches QLogic and Cisco have jointly developed the QLogic and Cisco FCoE Design Guide for implementing a unified data center using Cisco Nexus 5000 Series switches and QLogic second-generation Converged Network Adapters. Refer to... -
Page 24: Configuring Dcb On A Nexus Switch
4–Installation Equipment Installation and Configuration Configuring DCB on a Nexus Switch NOTE: In this procedure, you may need to adjust some of the parameters to suit your environment, such as VLAN IDs, Ethernet interfaces, and virtual Fibre Channel interfaces. In this example, the Cisco FCF uses NIC traffic on priority 2 and VLAN 2, and FCoE traffic on priority 3 and VLAN 1002. - Page 25 4–Installation Equipment Installation and Configuration Configure qos policy-maps. policy-map type qos policy1 class type qos class-nic set qos-group 2 Configure queuing policy-maps and assign network bandwidth. Divide the network bandwidth evenly between FcoE and NIC traffic. policy-map type queuing policy1 class type queuing class-nic bandwidth percent 50 class type queuing class-fcoe...
- Page 26 4–Installation Equipment Installation and Configuration Configure Ethernet port 1/3 to enable VLAN-1002-tagged FCoE traffic and VLAN-2-tagged NIC traffic. interface Ethernet1/3 switchport mode trunk switchport trunk allowed vlan 2,1002 spanning-tree port type edge trunk interface vfc3 bind interface Ethernet1/3 no shutdown Display the configuration to confirm that it is correct.
-
Page 27: Storage
4–Installation Equipment Installation and Configuration bandwidth percent 50 Class-map (queuing): class-fcoe (match-any) Match: qos-group 1 bandwidth percent 50 Class-map (queuing): class-default (match-any) Match: qos-group 0 bandwidth percent 0 Service-policy (queuing) output: policy1 policy statistics status: disabled Class-map (queuing): class-nic (match-any) Match: qos-group 2 bandwidth percent 50 Class-map (queuing):... -
Page 28: Verify Equipment Connectivity
4–Installation Verify Equipment Connectivity Verify Equipment Connectivity To verify that all equipment is logged in and operating properly: Verify LAN management capability on all devices through the associated device management application. Verify that servers and Converged Network Adapters are logged into an FCoE switch under both Ethernet (eth1/16) and Fibre Channel (vfc116) protocols. -
Page 29: Netapp Zone Validation
4–Installation Verify Equipment Connectivity When the LUNs have been created and all zoning is complete, use the management interface to add the Converged Network Adapter Fibre Channel WWNs and iSCSI initiators to your storage so that the servers can discover the LUNs. Figure 4-6 shows an example of the discovered FCoE Converged Network Adapters and the created initiator groups on a NetApp... -
Page 30: Sansurfer Management Validation Screen 1
4–Installation Verify Equipment Connectivity Figure 4-7. SANsurfer Management Validation Screen 1 Figure 4-8. SANsurfer Management Validation Screen 2 Use the operating system tools to create disk partitions and volumes on your servers using the assigned LUNs. Proceed to validation in Section 5, Validation Methods for DCB and FCoE. -
Page 31: Validation Methods For Dcb And Fcoe
Validation Step 1—Verify PFC and ETS Parameters Verify the interoperability of the unified fabric devices by capturing the DCBX exchanges between the QLogic initiator and each target. The key parameters to verify are: DCBX version used by each network component ... -
Page 32: Adapter, Switch, And Target
5–Validation Methods for DCB and FCoE Validation Step 1—Verify PFC and ETS Parameters Select the trigger mode. Set up the capture trigger condition to capture any LLDP frames. Set the trigger fill to capture 90 percent after the trigger. Figure 5-2. Analyzer TraceControl Configured to Capture LLDP Frames Only Between the Adapter, Switch, and Target Configure the FCoE switch with the following parameters: ... - Page 33 5–Validation Methods for DCB and FCoE Validation Step 1—Verify PFC and ETS Parameters pfc_enable value is the same for the adapter, switch, and targets. Application type-length-value (TLV) contains one entry for FCoE protocol and one entry for FCoE initialization protocol (FIP). The user_priority_map should be the same for the adapter, switch, and targets.
-
Page 34: Validation Results
Figure 5-3 shows one trace captured between the adapter and FCoE switch exchanging DCB parameters using the DCBX protocol. Both peers use version 1.01. Figure 5-3. DCBX Exchanges Between QLogic QLE8152 Adapter and the Cisco Nexus 5000 FCoE Switch 51031-00 A... -
Page 35: Fcoe Switch Validation With Emulator Setup
5–Validation Methods for DCB and FCoE Validation Step 1—Verify PFC and ETS Parameters From the trace, you can verify the following: DCBX Oper_Version is 0 for all devices. ER bit is OFF in all TLVs for all devices. ... -
Page 36: Validation Step 2-Fip Validation
5–Validation Methods for DCB and FCoE Validation Step 2—FIP Validation Validation Step 2—FIP Validation The next step is to verify FCoE initialization protocol (FIP). FIP performs the FCoE link initialization and maintenance to enable FCoE traffic in the unified fabric. FIP ensures that: ... -
Page 37: Validation Process
5–Validation Methods for DCB and FCoE Validation Step 2—FIP Validation The switch responds with the FIP Discovery Advertisement/Response To Solicitation message. This message is enlarged to the maximum FCoE frame size. The delivery of this message proves that the network is capable of delivering maximum-sized FCoE frames. -
Page 38: And Fcoe Switch
5–Validation Methods for DCB and FCoE Validation Step 2—FIP Validation Validation Results Figure 5-6 shows the FIP test results between the QLogic QLE8152 adapter and the Cisco Nexus 5000 FCoE switch. Figure 5-6. FIP Verification Results—Converged Network Adapter and FCoE Switch From the captured trace, you can verify the following: ... -
Page 39: Maintenance
5–Validation Methods for DCB and FCoE Validation Step 2—FIP Validation In summary, the FIP link initialization process is successful—the end device successfully connects to the switch, and the FCoE virtual link is set up. Similar results should be seen on the FCoE target side. When the FCoE link is up, the end device sends an FIP Keep Alive frame every eight seconds so that the switch knows that the device is present even when the link is idle. -
Page 40: Frames (To All_E_Nodes)
5–Validation Methods for DCB and FCoE Validation Step 2—FIP Validation Unlike end devices, the FCoE switches do not send FIP Keep Alive messages. Instead, FCoE switches send FIP Discovery Advertisement messages every eight seconds, which notifies end devices that the switches are still present. Figure 5-9 shows the time difference between two FIP Discovery Advertisements. -
Page 41: Virtual Link Message From The Switch
5–Validation Methods for DCB and FCoE Validation Step 2—FIP Validation Figure 5-11 shows the time difference between the last FIP Keep Alive Sent message and the FIP Clear Virtual Link message from the switch. Figure 5-11. Time Difference Between the Last FIP Keep Alive Sent and the FIP Clear Virtual Link Message from the Switch 51031-00 A 5-11... -
Page 42: Validation Step 3-Fcoe Function And I/O Tests
5–Validation Methods for DCB and FCoE Validation Step 3—FCoE Function and I/O Tests Validation Step 3—FCoE Function and I/O Tests Figure 5-12 shows the three test configurations that validate FCoE function and I/O in the FCoE fabric: Converged Network Adapter host > FCoE switch > FCoE storage ... -
Page 43: Validation Results
5–Validation Methods for DCB and FCoE Validation Step 3—FCoE Function and I/O Tests Configure the test as follows: MLTT test pattern: mixed read/write operations, 100 percent read and write traffic Data size: 512K Queue depth: eight Use MLTT to calculate IOPS and latency for the three test configurations, and verify data integrity. - Page 44 5–Validation Methods for DCB and FCoE Validation Step 3—FCoE Function and I/O Tests Similar to previous test results, the captured FCoE traces listed in Table 5-1 show no FCoE frame errors. Table 5-1. I/O Performance Comparison Among FCoE Storage Network Scenarios SCSI I/O Exchange Statistics Pending Exchange...
-
Page 45: Validation Step 4-Validate Pfc And Lossless Link
5–Validation Methods for DCB and FCoE Validation Step 4—Validate PFC and Lossless Link Validation Step 4—Validate PFC and Lossless Link The critical factor for a successful FCoE deployment is the ability of the virtual Ethernet link to carry lossless FCoE traffic. Figure 5-14 shows the test configuration to validate PFC function and to validate that the link is lossless. -
Page 46: Validation Results
5–Validation Methods for DCB and FCoE Validation Step 4—Validate PFC and Lossless Link To validate that PFC Pause frames are sent to reduce the traffic, configure TraceControl to capture a large buffer when it encounters a PFC Pause frame. To set up a capture: Select the trigger mode. -
Page 47: Expert Validating The Reaction To Pfc Pause Frames
5–Validation Methods for DCB and FCoE Validation Step 4—Validate PFC and Lossless Link As described earlier, the Expert software analyzes all the pauses in a capture and reports critical errors, such as Frame Received while PFC Class Paused 5-16). These errors are serious—it is equivalent to sending a Fibre (Figure Channel frame on a link without waiting for a credit. -
Page 48: Validation Step 5-Ets Verification
5–Validation Methods for DCB and FCoE Validation Step 5—ETS Verification Table 5-2 shows some of the statistics reported by Expert software for evaluating PFC functions and performance. The sample data indicates that there 33 frames were received during PFC pauses, and as a result, the link may not guarantee lossless traffic for the FCoE application. -
Page 49: Verifying Ets With Write Operations To The Storage
5–Validation Methods for DCB and FCoE Validation Step 5—ETS Verification Run all the tests with the Write configuration (Figure 5-18), and then repeat all the tests with the Read configuration (Figure 5-19). 4Gbps 10Gbps Figure 5-18. Verifying ETS with Write Operations to the Storage 4Gbps 10Gbps Figure 5-19. -
Page 50: Validation Results
5–Validation Methods for DCB and FCoE Validation Step 5—ETS Verification Use Expert to analyze the capture, and verify the following: No error occurs on FCoE traffic Link throughput per priority in MBps (Figure 5-20) Each traffic class can still generate I/O at its ETS setting when the link is saturated. -
Page 51: Fcoe Traffic In A Read Operation From Storage
5–Validation Methods for DCB and FCoE Validation Step 5—ETS Verification Figure 5-21 shows that the FCoE traffic is read at the rate of 837MBps. The throughput numbers on ports (1,1,1) and (1,2,2) are equal, indicating that all traffic from the FCoE storage is going to the server. The throughput of 837MBps represents the maximum rate for FCoE in this configuration. -
Page 52: Enabling Cross Port Analysis In Expert
5–Validation Methods for DCB and FCoE Validation Step 5—ETS Verification Figure 5-23 shows how to enable a cross-port analysis in Expert's Edit / Preferences dialog. In a cross-port analysis, Expert ensures that each frame is making its way across the switch, and reports frame losses and the latency through the switch. -
Page 53: Lan Traffic Only
5–Validation Methods for DCB and FCoE Validation Step 5—ETS Verification The next step is to verify that the switch allows more TCP traffic when more bandwidth is available. Figure 5-25 shows that the TCP traffic increases to 741MBps after stopping the FCoE traffic. Again, the switch is behaving properly. Figure 5-25. -
Page 54: Relationship Between Fcoe, Tcp Traffic, And Pause Frames
5–Validation Methods for DCB and FCoE Validation Step 5—ETS Verification Figure 5-26 shows the relationship between FCoE, TCP, and pause frames. Before the start of TCP traffic, server ingress is equal to the FCoE traffic. When the TCP traffic starts, server ingress is the sum of the FCoE and TCP traffic. The switch starts sending PFC pause requests to the storage when the TCP traffic starts to slow the FCoE traffic going to the server. -
Page 55: Exchange Completion Time (Latency Monitor)
5–Validation Methods for DCB and FCoE Validation Step 5—ETS Verification Exchange Completion Time (Latency Monitor) Latency is the measurement of how fast commands complete. The time between command issue and status received is measured in milliseconds (ms). Figure 5-27 shows the Fibre Channel read-response time in relationship to changes in traffic load. -
Page 56: Validation Step 6-Iscsi Function And I/O Test
Figure 5-28. iSCSI Traffic Performance Validation Setup Validation Process Enable the Converged Network Adapter to process FCoE and iSCSI data. Configure the Analyzer to capture any traffic. Use MLTT to map QLogic Fibre Channel storage devices. Configure the test as follows: Traffic: 100 percent read ... -
Page 57: Validation Results
5–Validation Methods for DCB and FCoE Validation Step 6—iSCSI Function and I/O Test Queue depth: eight. Capture a trace for the iSCSI storage. Use Expert to analyze the iSCSI storage trace and calculate the average, minimum, and maximum throughput. Capture a trace of the same read application for Fibre Channel and iSCSI storage together through the unified fabric. - Page 58 5–Validation Methods for DCB and FCoE Validation Step 6—iSCSI Function and I/O Test Table 5-5 lists the average, minimum, and maximum data. Table 5-5. Performance Comparison Between Separate and Combined Applications FCoE and iSCSI (MBps) FCoE only iSCSI Only Test option (MBps) (MBps) FCoE...
-
Page 59: Validation Step 7-Virtualization Verification
5–Validation Methods for DCB and FCoE Validation Step 7—Virtualization Verification Validation Step 7—Virtualization Verification This validation step compares the performance of virtual machines on a single host with performance on multiple hosts. The test runs one iSCSI initiator in one virtual machine, and one FCoE initiator in another virtual machine as shown in Figure 5-30. -
Page 60: Validation Results
5–Validation Methods for DCB and FCoE Validation Step 7—Virtualization Verification Configure the test as follows: Trigger: any PFC frame Post fill: 90 percent after the trigger Data size: 512K Traffic: 100 percent read on both virtual machines Use Analyzer to capture a trace. -
Page 61: Expert Shows Pfc Pause Request Released From The Target To The Switch Port
5–Validation Methods for DCB and FCoE Validation Step 7—Virtualization Verification Table 5-6 lists the average minimum and maximum throughput values. Table 5-6. FCoE and iSCSI Application Throughput when Running Separate Virtual Machines on One Physical Server Combined Test Option FCoE (MBps) iSCSI (MBps) (MBps) Average... - Page 62 5–Validation Methods for DCB and FCoE Validation Step 7—Virtualization Verification In summary, multiple factors may determine the behavior of virtual-server-based applications in the unified network environment. The internal application balancing algorithm (either among virtual servers or within the unified storage) plays the key role, and could interfere with the DCB-based bandwidth management during congestion.
-
Page 63: Hardware And Software
Hardware and Software Cisco Unified Fabric Switch The Cisco Nexus 5000 Series switch enables a high-performance, standards-based, Ethernet unified fabric. The platform consolidates separate LAN, SAN, and server cluster environments into a single physical fabric while preserving existing operational models. The Cisco Nexus 5000 Series switch provides an enhanced Ethernet topology by leveraging data center bridging features, which include priority flow control (PFC) for a lossless fabric and enhanced transmission selection (ETS) for bandwidth management. -
Page 64: Jdsu (Formerly Finisar) Equipment And Software
A–Hardware and Software JDSU (formerly Finisar) Equipment and Software JDSU (formerly Finisar) Equipment and Software JDSU Xgig JDSU Xgig is a unified, integrated platform employing a unique chassis and ® blade architecture to provide users with the utmost in scalability and flexibility. Various blades support a wide range of protocols and can be easily configured to act as a protocol Analyzer, Jammer, bit error rate tester (BERT), traffic generator, and Load Tester, all without changing hardware. -
Page 65: Qlogic Qle8142 Converged Network Adapter
QLogic 5800V Series Fibre Channel Switch The QLogic SB5800V Series Fibre Channel switch supports 2Gb, 4Gb, and 8Gb devices and optics, while providing 10Gb inter-switch links that are can be upgraded to 20Gb when workload demands increase. The QLogic 5800V Series switch provides intuitive installation and configuration wizards, stack and SNMP administration, standard adaptive trunking, and unparalleled upgrade flexibility. - Page 66 A–Hardware and Software QLogic 5800V Series Fibre Channel Switch 51031-00 A...
-
Page 67: Data Center Bridging Technology
Data Center Bridging Technology The following descriptions of Enhanced Ethernet were taken from Ethernet: The Converged Network Ethernet Alliance Demonstration at SC'09, published by the Ethernet Alliance November, 2009. Data Center Bridging (DCB) For Ethernet to carry LAN, SAN and IPC traffic together and achieve network convergence, some necessary enhancements are required. -
Page 68: Priority Flow Control
B–Data Center Bridging Technology Data Center Bridging (DCB) Enhanced Transmission Selection (ETS) protocol addresses the bandwidth allocation issues among various traffic classes to maximize bandwidth usage. The IEEE 802.1Qaz standard specifies the protocol to support allocation of bandwidth amongst priority groups. ETS allows each node to control bandwidth per priority group. -
Page 69: Fibre Channel Over Ethernet
B–Data Center Bridging Technology Data Center Bridging (DCB) Figure B-1. Priority Flow Control Fibre Channel Over Ethernet FCoE is an ANSI T11 standard for the encapsulation of a complete Fibre Channel frame into an Ethernet frame. The resulting Ethernet frame is transported over Enhanced Ethernet networks as shown in Figure B-2. -
Page 70: Iscsi
B–Data Center Bridging Technology Data Center Bridging (DCB) iSCSI The Internet Small Computer Systems Interface (iSCSI) is a SCSI mass storage transport that operates between the Transport Control Protocol (TCP) and the SCSI Protocol Layers. The iSCSI protocol is defined in RFC 3720 [iSCSI], which was finalized by the Internet Engineering Task Force (IETF) in April, 2004. - Page 71 References Fibre Channel over Ethernet Design Guide, Cisco and QLogic (2010), QLogic adapters and Cisco Nexus 5000 Series switches, Cisco document number C11-569320-01. Downloaded from http://www.qlogic.com/SiteCollectionDocuments/Education_and_Resource/white papers/whitepaper2/QLogic_Cisco_FCoE_Design_Guide.pdf Ethernet: The Converged Network Ethernet Alliance Demonstration at Sc'09, Ethernet Alliance. (2009), retrieved from http://www.ethernetalliance.org/files/static_page_files/281AD8C4-1D09-3519-AD...
- Page 72 C–References 51031-00 A...
-
Page 73: Index
Index Fibre Channel forwarder Fibre Channel switches adapters 4-2, FIP - see FCoE initialization protocol architecture 2-1, audience hardware platforms cabling 4-1, configuration connectivity 4-10 I/O test validation 5-12, 5-26 conventions viii installation converged network 1-1, iSCSI function validation 5-26 data center bridging exchange protocol latency monitor 5-25... - Page 74 5-29 PFC - see priority flow control planning priority flow control vii, 5-1, 5-15 process summary QLogic and Cisco FCoE Design Guide related materials SANsurfer FC HBA Manager server bus interface storage 4-1, switch FCoE 4-5, Fibre Channel...
- Page 76 Information supplied by QLogic Corporation is believed to be accurate and reliable. QLogic Corporation assumes no responsibility for any errors in this brochure. QLogic Corporation reserves the right, without notice, to make changes in product design...
Need help?
Do you have a question about the Storage Networking (Unified Fabric Pilot and is the answer not in the manual?
Questions and answers