Page 1 S e n d c o m m e n t s t o m d s f e e d b a c k - d o c @ c i s c o . c o m . Cisco MDS 9000 Family Troubleshooting Guide Cisco MDS SAN-OS Release 1.3...
Page 2 OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. Copyright © 2003 Cisco Systems, Inc. All rights reserved. CCIP, CCSP, the Cisco Arrow logo, the Cisco Powered Network mark, the Cisco Systems Verified logo, Cisco Unity, Follow Me Browsing, FormShare, iQ Net Readiness Scorecard, Networking Academy, and ScriptShare are trademarks of Cisco Systems, Inc.;...
Page 3: Table Of Contents
Basics Basic Connectivity Fibre Channel End-to-End Connectivity Fabric Issues Contacting Customer Support Using Host Diagnostic Tools Using Cisco MDS 9000 Family Tools Command-Line-Interface (CLI) CLI Debug FC Ping and FC Traceroute Cisco Fabric Manager Fabric Manager and Device Manager Analyzing Switch Device Health...
Page 4 SCSI Target Discovery SNMP and RMON Support Using RADIUS Using Syslog Using Fibre Channel SPAN Using Cisco Network Management Products Cisco MDS 9000 Port Analyzer Adapter Cisco Fabric Analyzer CiscoWorks RME Using Other Troubleshooting Products Fibre Channel Testers Fibre Channel Protocol Analyzers...
Page 5 Troubleshooting IP Connections Verifying Basic Connectivity OL-5183-02, Cisco MDS SAN-OS Release 1.3 3-19 3-20 3-21 3-23 3-25 3-25 3-26 3-27 4-11 4-13 4-14 3-19 3-19 3-22 3-24 3-26 4-10 4-14 4-16 4-17 4-19 Cisco MDS 9000 Family Troubleshooting Guide Contents...
Page 6 C H A P T E R Can I Set the Map Layout So It Stays After I Restart Fabric Manager? Two Switches Show on my Map, But I Only Have One Switch Cisco MDS 9000 Family Troubleshooting Guide 5-15 5-16...
Page 7 Specifying an Interface for Fabric Manager Client or Device Manager Configuring a Proxy Server Clearing Topology Maps Can I Use Fabric Manager in a Mixed Software Environment? N D E X OL-5183-02, Cisco MDS SAN-OS Release 1.3 Cisco MDS 9000 Family Troubleshooting Guide Contents...
Page 8 S e n d c o m m e n t s t o m d s f e e d b a c k - d o c @ c i s c o . c o m . Cisco MDS 9000 Family Troubleshooting Guide viii OL-5183-02, Cisco MDS SAN-OS Release 1.3...
Page 9: Document Organization
SAN environment, such as hubs, hosts, and storage arrays and Cisco MDS 9000 switches. It also covers basic storage commands, switch configurations, and common storage parameters. It introduces tools and methodologies to recognize a problem, determine its cause, and find possible solutions.
Page 10: Document Conventions
Means reader be careful. In this situation, you might do something that could result in equipment damage or loss of data. Related Documentation For Fabric Manager and Device Manager field descriptions, refer to the Cisco MDS 9000 Family Fabric Manager Online Help. For additional information, refer to the following documents: •...
Page 11 • Cisco MDS 9000 Family MIB Reference Guide • Cisco MDS 9000 Family CIM Programming Reference Guide Cisco MDS 9000 Family SAN Volume Controller Configuration Guide • OL-5183-02, Cisco MDS SAN-OS Release 1.3 Related Documentation Cisco MDS 9000 Family Troubleshooting Guide...
Page 12: Obtaining Documentation
VERITAS Storage Foundation for Networks README • For information on IBM TotalStorage SAN Volume Controller Storage Software for Cisco MDS 9000, refer to the following IBM documents available at IBM TotalStorage SAN Volume Controller Storage Software for Cisco MDS 9000 Getting Started •...
Page 13: Ordering Documentation
Nonregistered Cisco.com users can order documentation through a local account representative by • calling Cisco Systems Corporate Headquarters (California, U.S.A.) at 408 526-7208 or, elsewhere in North America, by calling 800 553-NETS (6387). Documentation Feedback You can submit comments electronically on Cisco.com. On the Cisco Documentation home page, click Feedback at the top of the page.
Page 14: Technical Support Center
Technical Support Website require a Cisco.com login ID and password. If you have a valid service contract but do not have a login ID or password, go to this URL to register: http://tools.cisco.com/RPF/register/register.do Cisco MDS 9000 Family Troubleshooting Guide Preface OL-5183-02, Cisco MDS SAN-OS Release 1.3...
Page 15: Obtaining Additional Publications And Information
Information about Cisco products, technologies, and network solutions is available from various online and printed sources. • The Cisco Product Catalog describes the networking products offered by Cisco Systems, as well as ordering and customer support services. Access the Cisco Product Catalog at this URL: http://www.cisco.com/en/US/products/products_catalog_links_launch.html •...
Page 16 S e n d c o m m e n t s t o m d s f e e d b a c k - d o c @ c i s c o . c o m . • Training—Cisco offers world-class networking training. Current offerings in network training are listed at this URL: http://www.cisco.com/en/US/learning/le31/learning_recommended_training_list.html Cisco MDS 9000 Family Troubleshooting Guide OL-5183-02, Cisco MDS SAN-OS Release 1.3...
Page 17: Introduction
This section provides a series of questions that may be useful when troubleshooting a problem with a Cisco MDS 9000 family switch or connected devices. Use the answers to these questions to plan a course of action and to determine the scope of the problem. For example, if a host can only see some of the OL-5183-02, Cisco MDS SAN-OS Release 1.3...
Page 18: C H A P T E R 1 Troubleshooting Overview
• Is there any SCSI exchange that takes place between the server and the disk array? • Is the HBA configured for NPort? • Cisco MDS 9000 Family Troubleshooting Guide Chapter 1 Troubleshooting Overview OL-5183-02, Cisco MDS SAN-OS Release 1.3...
Page 19: Fabric Issues
You can obtain Iometer from SourceForge.net at the following URL: OL-5183-02, Cisco MDS SAN-OS Release 1.3 Contacting Customer Support Obtaining Technical Assistance in the Preface. Cisco MDS 9000 Family Troubleshooting Guide...
Page 20: Using Cisco Mds 9000 Family Tools
Cisco MDS 9000 family of switches. This section introduces these tools and describes the kinds of problems for which you can use each tool. It...
Page 21: Cli Debug
S e n d c o m m e n t s t o m d s f e e d b a c k - d o c @ c i s c o . c o m . Command-Line-Interface (CLI) The Cisco MDS 9000 Family CLI lets you configure and monitor a Cisco MDS 9000 Family switch using a local console or remotely using a Telnet or SSH session. The CLI provides a command structure similar to Cisco IOS roles-based access control.
Page 22 Using Cisco MDS 9000 Family Tools S e n d c o m m e n t s t o m d s f e e d b a c k - d o c @ c i s c o . c o m .
Page 23: Fc Ping And Fc Traceroute
Enable vrrp debugging Enable VSAN manager debugging Enable vsh debugging Configure vshd debugging Configure WWN Manager Debugging Enable xbar debugging Enable Xbar Client debugging Zone server debug commands Cisco MDS 9000 Family Troubleshooting Guide Using Cisco MDS 9000 Family Tools...
Page 24 Using Cisco MDS 9000 Family Tools S e n d c o m m e n t s t o m d s f e e d b a c k - d o c @ c i s c o . c o m .
Page 25: Cisco Fabric Manager
Analyzing the Results of Merging Zones, page 1-12 • Alerts and Alarms, page 1-13 • For detailed information about using Cisco Fabric Manager, refer to the Cisco MDS 9000 Family Fabric Note Manager User Guide. Fabric Manager and Device Manager Fabric Manager provides a map of the discovered fabric and includes tables that display statistical information about the switches in the fabric.
Page 26: Analyzing Switch Device Health
Using Cisco MDS 9000 Family Tools S e n d c o m m e n t s t o m d s f e e d b a c k - d o c @ c i s c o . c o m .
Page 27: Analyzing End-To-End Connectivity
You use a policy file to define the rules to be applied when running the Fabric Checker. When you create a policy file, the system saves the rules selected for the selected switch. OL-5183-02, Cisco MDS SAN-OS Release 1.3 Using Cisco MDS 9000 Family Tools Cisco MDS 9000 Family Troubleshooting Guide 1-11...
Page 28: Analyzing The Results Of Merging Zones
Using Cisco MDS 9000 Family Tools S e n d c o m m e n t s t o m d s f e e d b a c k - d o c @ c i s c o . c o m .
Page 29: Alerts And Alarms
For the Cisco MDS 9000 Family of switches, the SCSI Target Discovery feature has been added to provide added insight into connected SCSI targets. This feature allows the switch to briefly log into connected SCSI target devices and issue a series of SCSI inquiry commands to help discover additional information.
Page 30: Snmp And Rmon Support
Note community strings and to create a SNMP v3 username and password. Cisco MDS 9000 Family switches support over 50 different MIBs, which can be divided into the following six categories: IETF Standards-based Entity MIBs (for example, RFC273 ENTITY-MIB) These MIBs are used to •...
Page 31: Using Radius
Authentication refers to the authentication of users for access to a specific device. You can use RADIUS to manage user accounts for access to Cisco MDS 9000 Family switches. When you try to log into a switch, the switch validates you with information from a central RADIUS server.
Page 32: Using Syslog
Authorization refers to the scope of access that you have once you have been authenticated. With Cisco MDS 9000 Family switches, assigned roles for users can be stored in a RADIUS server along with a list of actual devices that the user should have access to. Once the user has been authenticated, then switch can then refer to the RADIUS server to determine the extent of access the user will have within the switch network.
Page 33 In Ethernet networks, this problem can be solved using the SPAN utility, which is provided with the Cisco Catalyst Family of Ethernet switches. SPAN has also been implemented with the Cisco MDS 9000 Family of switches for use in Fibre Channel networks. SPAN lets you take a copy of all traffic and direct it to another port within the switch.
Page 34: Using Cisco Network Management Products
The Cisco MDS 9000 Port Analyzer Adapter is a stand-alone adapter card that converts Fibre Channel (FC) frames to Ethernet frames by encapsulating each Fibre Channel frame into an Ethernet frame. This product is meant to be used for analyzing SPAN traffic from a Fibre channel port on a Cisco MDS 9000 Family switch.
Page 35: Cisco Fabric Analyzer
Using the CLI-based interface, you can remotely access an Cisco MDS 9000 Family switch, using Telnet or a secure method such as Secure Shell (SSH). You can then capture and decode Fibre Channel control traffic, which offers a convenient method for conducting detailed, remote troubleshooting.
Page 36 Detection Function Fabric Detected (0x00000001) Num Entries 1 Device Entry 0 Port State 0x20 Port Id 7f.00.01 Port WWN 1000000530005f1f (000530) Node WWN 1000000530005f1f (000530) Cisco MDS 9000 Family Troubleshooting Guide 1-20 Chapter 1 SW_ILS 1 0x59b7 0xffff 0x7 -> 0x59b7 0x59c9 0xff ->...
Page 37: Ciscoworks Rme
Inventory Manager—Provides a facility to gather and audit a detailed hardware and software • inventory of all Cisco MDS 9000 Family devices deployed in the storage network. A reporting facility is included to generate inventory reports Configuration Manager—Maintains an active repository of device configuration files for devices •...
Page 38: Using Other Troubleshooting Products
Fibre Channel analyzer. These include the following: Credit starvation • • missing, malformed, or non-standard-compliant frames or primitives • protocol errors Cisco MDS 9000 Family Troubleshooting Guide 1-22 Chapter 1 Troubleshooting Overview OL-5183-02, Cisco MDS SAN-OS Release 1.3...
Page 39: Troubleshooting Switch System Issues
• Recovering the Administrator Password If you forget the administrator password for accessing a Cisco MDS 9000 Family switch, you can recover the password using a local console connection. For the latest instructions on password recovery, go to http://www.cisco.com/warp/public/474/ and click on “MDS 9000 Series Multilayer Directors and Fabric Switches”...
Page 40: C H A P T E R 2 Troubleshooting Switch System Issues
For information about configuring Call Home, refer to the Cisco MDS 9000 Family Configuration Guide or the Cisco MDS 9000 Family Fabric Manager User Guide.
Page 41 2abe333e httpd 2abfd33e xinetd 2ac1e33e sysmgr 2abe91a2 httpd 2abe91a2 httpd 2ac73419 2abe91a2 httpd 2abe91a2 httpd Normal-exit Stack-trace ------ ----------- ----------- Troubleshooting System Restarts Core Log-create-time ------- --------------- Jan 27 04:08 Jan 24 20:50 Cisco MDS 9000 Family Troubleshooting Guide...
Page 42 To copy the FSPF core dump in this example to a TFTP server with the IP address 1.1.1.1, enter the following command: switch# copy core://5/1524 tftp::/1.1.1.1/abcd The following command displays the file named switch# sh pro log pid 1473 ====================================================== Cisco MDS 9000 Family Troubleshooting Guide ECX 0804E994 EDX 00000008 EDI 7FFFFC9C EBP 7FFFFCAC XDS 0000002B...
Page 43 080FC060 - 080FCBA8 081795C0 - 081EC000 7FFFFCF0 20952 KB ECX 00000006 EDI 08109308 XDS 0000002B EIP 2AC8CC71 ESP 7FFFF2C0 Troubleshooting System Restarts EDX 2AD721E0 EBP 7FFFF2EC XES 0000002B XCS 00000023 XSS 0000002B ..*...*,... Cisco MDS 9000 Family Troubleshooting Guide...
Page 44: Working With Unrecoverable System Restarts
To respond to an unrecoverable restart, perform the steps listed in the Restarts” section on page Cisco MDS 9000 Family Troubleshooting Guide Chapter 2 2-2. Troubleshooting Switch System Issues “Working with Recoverable...
Page 45: Troubleshooting A Failed Supervisor
Never "init" the switch unless you understand that the switch will lose everything. Keep backup copies of running kickstart and system images on compact flash. OL-5183-02, Cisco MDS SAN-OS Release 1.3 Troubleshooting a Failed Supervisor Cisco MDS 9000 Family Troubleshooting Guide...
Page 46 S e n d c o m m e n t s t o m d s f e e d b a c k - d o c @ c i s c o . c o m . Cisco MDS 9000 Family Troubleshooting Guide...
Page 47: Troubleshooting Switch Level Issues And Interswitch Connectivity
Troubleshooting a ZONE Merge Failures, page 3-5 • • Troubleshooting a VSAN Configuration Conflict, page 3-6 Troubleshooting a Domain ID Conflict, page 3-7 • OL-5183-02, Cisco MDS SAN-OS Release 1.3 C H A P T E R Cisco MDS 9000 Family Troubleshooting Guide...
Page 48: Overview
• Zoning configuration VSAN configuration • Domain ID parameters • Fabric Parameters & Timers • Cisco MDS 9000 Family Troubleshooting Guide Chapter 3 Troubleshooting Switch Level Issues and Interswitch Connectivity -------------------------------- 20:01:00:05:30:00:13:9f [Local] 20:01:00:05:30:00:13:9e [Principal] ----------------------- 20:01:00:05:30:00:13:9f [Local] [Principal] -----------------------...
Page 49: Troubleshooting Steps
1445517676 packets input, 727667035658 bytes, 0 discards 0 input errors, 0 CRC, 0 invalid transmission words 0 address id, 0 delimiter OL-5183-02, Cisco MDS SAN-OS Release 1.3 0 unknown class Cisco MDS 9000 Family Troubleshooting Guide Troubleshooting E Port Connectivity - ISL Isolation...
Page 50 Max Rx Buffer Credit: 16 Max Private Devices: 63 Max Sourcable Pkt Size: 2112 Hw Capabilities: 0xb Connector Type: 0x0 Cisco MDS 9000 Family Troubleshooting Guide Chapter 3 Troubleshooting Switch Level Issues and Interswitch Connectivity 1080000 OL-5183-02, Cisco MDS SAN-OS Release 1.3...
Page 51: Troubleshooting A Zone Merge Failures
(allowed active) (1) In the above examples, the highligthed rxbuffsize is 2112 bytes. This represents the default settings on a Cisco MDS 9000 switch interface. Troubleshooting a ZONE Merge Failures In the example below the show interface command indicates that the E port did not come up due to a zone merge failure.
Page 52: Troubleshooting A Vsan Configuration Conflict
Using the Zone Merge Analysis tool in Fabric Manager, the compatibility of two active zonesets in two switches can be checked before actually merging the two zonesets. Refer to the Cisco MDS 9000 Fabric Manager User Guide for more information.
Page 53: Troubleshooting A Domain Id Conflict
To view which domain are currently in your fabric use the following command. This command can be issued to both fabrics to determine which domain IDs are overlapping. OL-5183-02, Cisco MDS SAN-OS Release 1.3 Troubleshooting E Port Connectivity - ISL Isolation Cisco MDS 9000 Family Troubleshooting Guide...
Page 54 The RCF functionality is disruptive and can cause end nodes to be assigned new domain IDs. By default, Cisco MDS 9000 Family switches accept disruptive restarts. You can configure the switch in order to reject incoming RCFs on a per-VSAN and port level basis (enabling the rcf-reject option).
Page 55 5 minutes input rate 0 bits/sec, 0 bytes/sec, 0 frames/sec 5 minutes output rate 0 bits/sec, 0 bytes/sec, 0 frames/sec OL-5183-02, Cisco MDS SAN-OS Release 1.3 0 unknown class End with CNTL/Z. Cisco MDS 9000 Family Troubleshooting Guide Troubleshooting E Port Connectivity - ISL Isolation...
Page 56: Troubleshooting Te Port Connectivity - Vsan Isolation
264 frames output, 16960 bytes, 0 discards Received 5 OLS, 3 LRR, 2 NOS, 11 loop inits Transmitted 8 OLS, 4 LRR, 2 NOS, 3 loop inits Cisco MDS 9000 Family Troubleshooting Guide 3-10 Chapter 3 Troubleshooting Switch Level Issues and Interswitch Connectivity...
Page 57 - if_index: 0x0109C000, phy_port_index: 0x3c Admin Config - state(up), mode(TE), speed(auto), trunk(on) OL-5183-02, Cisco MDS SAN-OS Release 1.3 Troubleshooting TE Port Connectivity - VSAN Isolation (1-3,5) (2-3,5) Cisco MDS 9000 Family Troubleshooting Guide 3-11...
Page 58 Vsan 8 - state(down), state reason(Isolation due to vsan not configure d on peer), fcid(0x000000) port init flag(0x0), current state [TE_FSM_ST_ISOLATED_VSAN_MISMATCH] switch# Cisco MDS 9000 Family Troubleshooting Guide 3-12 Chapter 3 Troubleshooting Switch Level Issues and Interswitch Connectivity OL-5183-02, Cisco MDS SAN-OS Release 1.3...
Page 59: Troubleshooting Fx Port Connectivity
0 runts, 0 jabber, 0 too long, 0 too short 0 input errors, 0 CRC, 0 invalid transmission words 0 address id, 0 delimiter 0 EOF abort, 0 fragmented, 0 unknown class OL-5183-02, Cisco MDS SAN-OS Release 1.3 Troubleshooting Fx Port Connectivity Cisco MDS 9000 Family Troubleshooting Guide 3-13...
Page 60: Link_Failure Or Not Connected
Tx path from switch to the Rx input on N port interface is properly connected. A faulty transmitter on the switch’s SFP or a faulty receiver on the N port’s SFP could also cause the issue. Cisco MDS 9000 Family Troubleshooting Guide 3-14 Chapter 3 Troubleshooting Switch Level Issues and Interswitch Connectivity OL-5183-02, Cisco MDS SAN-OS Release 1.3...
Page 61: Interface Bouncing Between Offline And Initializing
143274267 frames input, 182897329172 bytes, 0 discards 0 runts, 0 jabber, 0 too long, 0 too short 0 input errors, 0 CRC, 0 invalid transmission words OL-5183-02, Cisco MDS SAN-OS Release 1.3 Troubleshooting Fx Port Connectivity Cisco MDS 9000 Family Troubleshooting Guide 3-15...
Page 62 In the example below, the fcanalyzer tool has been used with the necessary filtering to capture the FLOGI sent by the N port. Refer to the Cisco MDS 9000 Family Configuartion Gude for more information. Frame 25 (176 bytes on wire, 176 bytes captured)
Page 63: Point-To-Point Link Comes Up As Fl_Port
- if_index: 0x 1304000, phy_port_index: 0x84 Admin Config - state(up), mode(auto), speed(auto), trunk(on) beacon(off), snmp trap(on), tem(false) rx bb_credit(default), rx bb_credit multiplier(default) rxbufsize(2112), encap(default), user_cfg_flag(0x1) OL-5183-02, Cisco MDS SAN-OS Release 1.3 0 unknown class Cisco MDS 9000 Family Troubleshooting Guide Troubleshooting Fx Port Connectivity 3-17...
Page 64: Interface Up And Connectivity Problems - Troubleshooting Vsans And Zones
CLI or the GUI. Therefore, checking the zone configuration is the first step to take when the host is not able to access the storage, and the port are all up along the path between the server HBA and the storage subsystem interface. Cisco MDS 9000 Family Troubleshooting Guide 3-18 Chapter 3 Troubleshooting Switch Level Issues and Interswitch Connectivity OL-5183-02, Cisco MDS SAN-OS Release 1.3...
Page 65: Troubleshooting Zones - Case Of End Devices Belonging To The Default Zone
Zone1 vsan 1 pwwn 50:06:0e:80:03:50:5c:03 pwwn 20:00:00:e0:69:a1:b9:fc zone name Zone2 vsan 1 pwwn 10:00:00:e0:02:21:df:ef pwwn 20:00:00:e0:69:a1:b9:fc zone name Zone3 vsan 1 fwwn 20:42:00:05:30:00:63:9e fwwn 20:43:00:05:30:00:63:9e OL-5183-02, Cisco MDS SAN-OS Release 1.3 Troubleshooting Fx Port Connectivity Cisco MDS 9000 Family Troubleshooting Guide 3-19...
Page 66: Verify Active Zoneset Membership
Zone3 vsan 1 fwwn 20:42:00:05:30:00:63:9e fwwn 20:43:00:05:30:00:63:9e zone name zone-cc vsan 2 pwwn 50:06:0e:80:03:50:5c:01 pwwn 20:00:00:e0:69:41:a0:12 Cisco MDS 9000 Family Troubleshooting Guide 3-20 Chapter 3 Troubleshooting Switch Level Issues and Interswitch Connectivity OL-5183-02, Cisco MDS SAN-OS Release 1.3...
Page 67: Other Useful Commands
Alias1 Use the show zone statistics command to display the number of control frames exchanged with other switches. switch# show zone statistics OL-5183-02, Cisco MDS SAN-OS Release 1.3 Troubleshooting Fx Port Connectivity Cisco MDS 9000 Family Troubleshooting Guide 3-21...
Page 68: Using The Gui To Troubleshoot Zoning Configuration Issues
Similarly, by expanding the active zoneset folder content (clicking on the ‘+’ next to the folder) the members of the active zoneset (the active zones) will be displayed as new folders. Cisco MDS 9000 Family Troubleshooting Guide 3-22 Chapter 3...
Page 69: Troubleshooting Vsans
(1-3,5) (2-3,5) Cisco MDS 9000 Family Troubleshooting Guide Troubleshooting Fx Port Connectivity fc2/12 fc2/13 fc2/14 fc7/4 fc7/5 fc7/6 fc7/12 fc7/13 fc7/14 fc7/20 fc7/21 fc7/22 fc7/30 fc7/31...
Page 70: Using The Gui To Troubleshoot Vsan Membership Problems
Fabric Analysis tool provided by Fabric Manager. The possible configuration consistency check tool is also provided by this application. Refer to the Cisco MDS 9000 Fabric Manager User Guide for further information about this tool. Cisco MDS 9000 Family Troubleshooting Guide 3-24...
Page 71: Troubleshooting Hardware Problems
Maintain a set of information for the module before these problems occur (if possible) and then gather another set of information after these problems occur. OL-5183-02, Cisco MDS SAN-OS Release 1.3 Troubleshooting Hardware Problems Cisco MDS 9000 Family Troubleshooting Guide 3-25...
Page 72: San-Os Version 1.3(4) And Above
<port-num> status-reg show hardware internal fc-mac2 port <port-num> state-info-log show hardware internal fc-mac2 port <port-num> gbic-info show hardware internal fc-mac2 port <port-num> statistics Cisco MDS 9000 Family Troubleshooting Guide 3-26 Chapter 3 Troubleshooting Switch Level Issues and Interswitch Connectivity...
Page 73: Software Vesrion 1.2(1A) And Below
<port-num> config-registers show hardware internal fc-mac port <port-num> maskreg show hardware internal packet-flow fc <port-num> show hardware internal packet-flow fc <port-num> show process link-events Cisco MDS 9000 Family Troubleshooting Guide 3-27 OL-5183-02, Cisco MDS SAN-OS Release 1.3...
Page 74 <port-num> flow-control show hardware internal fc-mac2 port <port-num> config-reg show hardware internal fc-mac2 port <port-num> mask-reg show hardware internal packet-flow fc <port-num> show process link-events show process port-cfg-error Cisco MDS 9000 Family Troubleshooting Guide 3-28 OL-5183-02, Cisco MDS SAN-OS Release 1.3...
Page 75: Chapter 4 Troubleshooting Switch Fabric Level Issues
N port at the well-known FCID 0xFFFFFC. One instance of the name server process runs on each Cisco MDS 9000 Family switch. In a multi-switch fabric configuration, instances running on different switches share information and create a distributed database of Nx port attributes.
Page 76: Nx Port Registration Problems
If the interface is working correctly, proceed to the next step. Verify that the device in question appears in the FLOGI database. To do this, enter the following Step 2 command: Cisco MDS 9000 Family Troubleshooting Guide /* This is the vsan */ 0 unknown class Chapter 4 Troubleshooting Switch Fabric Level Issues OL-5183-02, Cisco MDS SAN-OS Release 1.3...
Page 77 /* LineCard responds that it is done */ OL-5183-02, Cisco MDS SAN-OS Release 1.3 FCID PORT NAME 0x780200 21:00:00:e0:8b:07:a4:36 Step Cisco MDS 9000 Family Troubleshooting Guide Troubleshooting Name Server Issues NODE NAME 20:00:00:e0:8b:07:a4:36 8. If the device does not appear in the output,...
Page 78 /* The wwpn and FCID for the port, note that the bytes in the world wide name are reversed Feb 17 04:42:54 fcns: vsan 99: Registered cos 8 for port-id 780200 /* Class of Service */ Cisco MDS 9000 Family Troubleshooting Guide Chapter 4 Troubleshooting Switch Fabric Level Issues End with CNTL/Z.
Page 79 :ff ff ff ff ff ff ff ff /* Registered FC4 Types: example SCSI and /* Fibrechannel port type (F,FL) */ :0.0.0.0 :20:8e:00:05:30:00:86:9e /* wwn of the switch port */ :0x000000 Cisco MDS 9000 Family Troubleshooting Guide Troubleshooting Name Server Issues...
Page 80: Troubleshooting Fspf Issues
Hellos are sent with 0xFFFFFFFF in neighbor field until switch learns its neighbor’s domain The switch receives Hello with incorrect Hello and dead intervals. • Cisco MDS 9000 Family Troubleshooting Guide Chapter 4 Troubleshooting Switch Fabric Level Issues OL-5183-02, Cisco MDS SAN-OS Release 1.3...
Page 81: Loss Of Two-Way Communication
OL-5183-02, Cisco MDS SAN-OS Release 1.3 Dest Domain Route Cost 0xEF(239) 1000 0xED(238) 2000 0x01(1) 3000 Troubleshooting FSPF Issues error code = 4 Next hops fc1/1 -----1 fc1/1 fc1/1 -----2 Hello 2 Error packets 1 Hello 4 Retransmitted LSU Cisco MDS 9000 Family Troubleshooting Guide...
Page 82: Resolving The Wrong Hello Interval Problem
ML-88(config-if)# fspf hello-interval XX vsan 1 Wrong Dead Interval on an ISL Interface To identify a mismatch in dead intervals on the two sides of an ISL interface, follow these steps: Cisco MDS 9000 Family Troubleshooting Guide Chapter 4 Troubleshooting Switch Fabric Level Issues...
Page 83: Resolving A Wrong Dead Interval Problem
It is an intra-domain router Autonomous region is 0 OL-5183-02, Cisco MDS SAN-OS Release 1.3 Hello 2 Error packets 1 Hello 4 /* This is the region */ Cisco MDS 9000 Family Troubleshooting Guide Troubleshooting FSPF Issues error code = 4 Retransmitted LSU...
Page 84: Resolving A Region Mismatch Problem
The region must match on all switches in the VSAN. To correct a region mismatch, enter the following commands: switch1(config)# fspf config vsan 1 switch1(config-(fspf-config))# region 0 Cisco MDS 9000 Family Troubleshooting Guide 4-10 Chapter 4 Troubleshooting Switch Fabric Level Issues...
Page 85: Fspf Issues In A Single-Vsan Environment
= 237 -----7 Troubleshooting FSPF Issues Port 1 Index 0x00010000 Port 2 Metric 1000 Index 0x00010001 Domain_ID Switch3 Port 2 Index Metric 1000 0x00010001 Port 4 Index 0x00010003 Link Type 1000 -----5 1000 -----6 Cisco MDS 9000 Family Troubleshooting Guide Cost 4-11...
Page 86 Step 2 Verify that the FSPF parameters are correct for each interface by entering the following command at the exec prompt: switch1# show fspf vsan 1 interface fc1/2 Cisco MDS 9000 Family Troubleshooting Guide 4-12 Chapter 4 = 185 = 0x8000000c...
Page 87: Fspf Issues In A Multi-Vsan Environment
The next hop to (238) has two interfaces. This indicates that both paths will be used during load sharing. Up to sixteen paths can be used by FSPF with a Cisco MDS 9000 Family switch. FSPF Issues in a Multi-VSAN Environment With the implementation of VSANs used with Cisco MDS 9000 Family switches, a separate instance of FSPF runs within each VSAN, and each instance is independent of the others.
Page 88: Troubleshooting Zoning Issues
To identify this problem, follow these steps: Step 1 Enter the following command to display the active zoneset configuration of the first switch: Switch1# show zoneset active v 99 Cisco MDS 9000 Family Troubleshooting Guide 4-14 Chapter 4 Switch2 Port-channel 1...
Page 89 Vsan 1 is up, FCID is 0x650000 Vsan 99 is down OL-5183-02, Cisco MDS SAN-OS Release 1.3 (99) 0 unknown class Cisco MDS 9000 Family Troubleshooting Guide Troubleshooting Zoning Issues 4-15...
Page 90: Importing Or Exporting A Zoneset Between Switches
This method is destructive to one of the active zonesets. For instructions about changing the membership of a zone, refer to the Cisco MDS 9000 Family Configuration Guide. The following example shows the use of the export command:...
Page 91: Deactivating A Zoneset And Restarting The Zone Merge Process
To deactivate a zoneset and restart the zone merge process, follow these steps: Deactivate the zoneset configuration from the switch, as in the following example: Step 1 This will disrupt traffic and cause the MDS 9000 switch to lose connectivity with the network. Caution switch4(config)# no zoneset activate name excal2 vsan 1 The system output looks like this: Zoneset Dectivation initiated.
Page 92 * fcid 0x6201cd * fcid 0x6201ce * fcid 0x6201d1 * fcid 0x6201d2 * fcid 0x6201d3 * fcid 0x6201d4 * fcid 0x6201d5 switch4# Cisco MDS 9000 Family Troubleshooting Guide 4-18 Chapter 4 Troubleshooting Switch Fabric Level Issues OL-5183-02, Cisco MDS SAN-OS Release 1.3...
Page 93: Misconfigured Zones Within An Active Zoneset In The Same Vsan
For instructions about changing the membership of a zone, refer to the Cisco MDS 9000 Family Configuration Guide. Cisco MDS 9000 Family Troubleshooting Guide 4-19 OL-5183-02, Cisco MDS SAN-OS Release 1.3...
Page 94 S e n d c o m m e n t s t o m d s f e e d b a c k - d o c @ c i s c o . c o m . Cisco MDS 9000 Family Troubleshooting Guide 4-20 OL-5183-02, Cisco MDS SAN-OS Release 1.3...
Page 95: Troubleshooting Ip Storage Issues
VSANs, security, and traffic management. The IPS module can be used in any Cisco MDS 9000 Family switch and has eight Gigabit Ethernet ports. Each port can run the FCIP and iSCSI protocols simultaneously.
Page 96: C H A P T E R 5 Troubleshooting Ip Storage Issues
10.82.241.17 (10.82.241.17), 30 hops max, 38 byte packets 172.18.189.129 (172.18.189.129) 172.18.0.33 (172.18.0.33) 10.81.254.69 (10.81.254.69) 10.81.254.118 (10.81.254.118) 10.83.255.81 (10.83.255.81) 10.83.255.163 (10.83.255.163) 10.82.241.17 (10.82.241.17) Cisco MDS 9000 Family Troubleshooting Guide Chapter 5 0.411 ms 0.150 ms 0.146 ms 0.413 ms 0.257 ms 0.249 ms 0.296 ms...
Page 97 BW 1000000 Kbit Troubleshooting IP Connections Type Interface ARPA GigabitEthernet4/7 ARPA GigabitEthernet4/7 ARPA GigabitEthernet4/7 ARPA GigabitEthernet4/7 ARPA GigabitEthernet4/7 Type Interface ARPA GigabitEthernet4/7 ARPA GigabitEthernet4/7 Type Interface ARPA GigabitEthernet4/7 ARPA GigabitEthernet4/7 ARPA GigabitEthernet4/7 Type Interface ARPA mgmt0 Cisco MDS 9000 Family Troubleshooting Guide...
Page 98: Verifying Static Ip Routing
A sample output of the sh ip route config command follows. switch# sh ip route conf Destination default 11.2.36.0 11.2.56.0 11.3.36.0 11.3.56.0 172.17.8.0 switch# Cisco MDS 9000 Family Troubleshooting Guide Gateway Mask Metric 172.17.8.1 0.0.0.0 11.3.36.1 255.255.252.0 11.3.56.1 255.255.252.0 0.0.0.0 255.255.252.0...
Page 99: Troubleshooting Fcip Connections
GE 2/8 Network 10.10.10.2/24 10.10.11.2/24 FCIP Tunnel (VE Virtual E-Port) Troubleshooting FCIP Connections MDS2 GE 2/8 FC1/1 Fcip profile 28 IP adress 10.10.11.2 Interface fcip 28 use-profile 28 peer-info ipaddr 10.10.10.2 Cisco MDS 9000 Family Troubleshooting Guide...
Page 100 MDS2(config)# fcip profile 28 Mar 10 21:41:04 ips: Dequeued mts msg MTS_OPC_IPS_FCIP_CMI_REQUEST(mts opc 3321, msg id 32222) Mar 10 21:41:04 ips: Create Entity 28 Cisco MDS 9000 Family Troubleshooting Guide Chapter 5 Troubleshooting IP Storage Issues OL-5183-02, Cisco MDS SAN-OS Release 1.3...
Page 101 Mar 10 21:42:23 ips: (ips_demux) Mts Opcode is 1905, id is 7304 Mar 10 21:42:23 ips: FCIP28: Processing Pull Config Request Mar 10 21:42:23 ips: FCIP28: Bound to entity 28 port: 3225 ip: 10.10.11.2 Cisco MDS 9000 Family Troubleshooting Guide OL-5183-02, Cisco MDS SAN-OS Release 1.3...
Page 102 MTS_OPC_PM_LOGICAL_PORT_STATE_CHANGE_RANGE(msg_id 47540) dropped Mar 10 22:59:46 ips: fu_priority_select: - setting fd[3] for select call - found data in FU_PSEL_Q_CAT_MTS queue, fd(3), usr_q_info(1) Mar 10 22:59:46 ips: fu_priority_select_select_queue: round credit(6) Cisco MDS 9000 Family Troubleshooting Guide OL-5183-02, Cisco MDS SAN-OS Release 1.3...
Page 103 2717:FCIP28: Change the operational mode to TRUNK port8: 2718:FCIP28: Tunnel bringup debounce timer callbeck, try to bring port8: 2719:FCIP28: Tunnel is already in oper UP state, don't try to Cisco MDS 9000 Family Troubleshooting Guide Troubleshooting FCIP Connections - found data in...
Page 104 TCP Connection Information 2 Active TCP connections Control connection: Local 10.10.10.2:3225, Remote 10.10.11.2:65519 (The above is class F traffic.) Data connection: Local 10.10.10.2:3225, Remote 10.10.11.2:65521 (The above is class 2,3 traffic.) Cisco MDS 9000 Family Troubleshooting Guide 5-10 Ipaddr TcpPort 10.10.10.2 3225...
Page 105 Mode Trunk Mode Mode auto trunking Input (rate is 5 min avg) ----------------------------- Rate Total Mbits/s Frames Cisco MDS 9000 Family Troubleshooting Guide Troubleshooting FCIP Connections Profile Port-channel Output (rate is 5 min avg) ----------------------------- Rate Total Mbits/s Frames 5-11...
Page 106 FCIP tunnels. Note that FCIP tunnel activation is the same as an FC EISL becoming active (such as ELP, ESC, and EFP). The following traces were captured after configuration on both MDS 9000 Family switches, and the last “no shutdown” was entered on switch MDS1. All settings are default (for example, SACK is disabled, the TCP window is set to 64K).
Page 107 13 being the first FCIP frame. This frame carries the FC Standard ELP. Figure 5-4 Second Capture of TCP Connection OL-5183-02, Cisco MDS SAN-OS Release 1.3 Troubleshooting FCIP Connections Cisco MDS 9000 Family Troubleshooting Guide 5-13...
Page 108 S e n d c o m m e n t s t o m d s f e e d b a c k - d o c @ c i s c o . c o m . Figure 5-5 shows the FC portion of the EISL initialization over the FCIP tunnel. Figure 5-5 Third Capture of TCP Connection Cisco MDS 9000 Family Troubleshooting Guide 5-14 OL-5183-02, Cisco MDS SAN-OS Release 1.3...
Page 109: One To Three Fcip Tunnel Creation And Monitoring
Interface fcip 23 use-proficle 21 peer-info ipaddr 10.10.8.2 Network Interface fcip 28 use-proficle 21 peer-info ipaddr 10.10.7.2 Cisco MDS 9000 Family Troubleshooting Guide Troubleshooting FCIP Connections GE 2/1 10.10.10.2/24 Fcip profile 1 ip address 10.10.8.2 Interface fcip 1 use-proficle 1 peer-info ipaddr 10.10.10.2...
Page 110: Fcip Profile Misconfiguration Examples
PMTU discover is enabled, reset timeout is 3600 sec Keep alive is 60 sec Minimum retransmission timeout is 300 ms Maximum number of re-transmissions is 4 Advertised window size is 64 KB Cisco MDS 9000 Family Troubleshooting Guide 5-16 fcip profile 21 End with CNTL/Z. (interface GigabitEthernet2/5)
Page 111 MDS2(config)# int fcip 21 MDS2(config-if)# passive-mode (Put interface FCIP 21 in passive mode to guarantee MDS1 initiates a TCP connection.) OL-5183-02, Cisco MDS SAN-OS Release 1.3 Troubleshooting FCIP Connections (interface GigabitEthernet2/1) (interface GigabitEthernet2/1) Cisco MDS 9000 Family Troubleshooting Guide 5-17...
Page 112 (The port is 13 as configured.) Data connection: Local 10.10.10.2:65190, Remote 10.10.11.2:13 174 Attempts for active connections, 5 close of connections Cisco MDS 9000 Family Troubleshooting Guide 5-18 port1: 863:FCIP21: SUP-> Set Port mode 1 port1: 864:FCIP21: SUP-> Port VSAN (1) already set to same value port1: 865:FCIP21: SUP->...
Page 113: Interface Fcip Misconfiguration Examples
41:FCIP21: SUP-> Enable tunnel ADMIN UP port1: 42:FCIP21: Try to Bring UP the Tunnel port1: 43:FCIP21: Bring up tunnel Failed, peer-ip not set (interface GigabitEthernet2/1) Troubleshooting FCIP Connections State Send-Q Recv-Q ESTABLISH LISTEN Cisco MDS 9000 Family Troubleshooting Guide 5-19...
Page 114 1 Using Profile id 21 Peer Information Peer Internet address is 10.10.10.2 and port is 13 Cisco MDS 9000 Family Troubleshooting Guide 5-20 port1: 1:FCIP21: Create tunnel with ifindex: a000014 port1: 2:FCIP21: Get the peer info from the SUP-IPS-MGR port1: 3:FCIP21: SUP->...
Page 115 1357:FCIP21: Delete the DE object [2] 0xd802cdc0 port1: 1358:FCIP21: Try 8 to bring up the tunnel port1: 1359:FCIP21: Start the bringup tunnel timer, timeout: 128000 (1-2) (1-2) (interface GigabitEthernet2/1) Cisco MDS 9000 Family Troubleshooting Guide Troubleshooting FCIP Connections 5-21...
Page 116 (Change one or both sides to no passive-mode.) MDS2# show int fcip 21 fcip21 is trunking Cisco MDS 9000 Family Troubleshooting Guide 5-22 port1: 1870:FCIP21: SUP-> Set Port mode 1 port1: 1871:FCIP21: SUP-> Port VSAN (1) already set to same value port1: 1872:FCIP21: SUP->...
Page 117 The following example shows a time stamp acceptable difference failure, or no NTP server connected to synchronize clocks. When using time stamps, the MDS switch must be a synchronized clock. NTP is configurable on the MDS 9000 switch. MDS2(config)# int fcip 21...
Page 118 Time Stamp is enabled, acceptable time difference 2000 ms B-port mode disabled TCP Connection Information Cisco MDS 9000 Family Troubleshooting Guide 5-24 port1: 854886:FCIP21: IPS-> Enable timestamp acceptable difference 2000 port1: 854887:FCIP21: IPS-> acc diff in sec: 0x2 frac: 0x0...
Page 119 Figure 5-7 Trace of Timestamp Difference Failure Figure 5-8 shows a trace of timestamp difference accepted. Figure 5-8 Trace of Timestamp Difference Accepted OL-5183-02, Cisco MDS SAN-OS Release 1.3 Troubleshooting FCIP Connections Cisco MDS 9000 Family Troubleshooting Guide 5-25...
Page 120: Fcip Special Frame Tunnel Creation And Monitoring
Jan 14 15:25:38 (The Special Frame is sent.) Jan 14 15:25:38 Jan 14 15:25:38 (The Special Frame is correctly configured with the WWN of the remote MDS 9000 switch.) Jan 14 15:25:38 Jan 14 15:25:38 Jan 14 15:25:38 Jan 14 15:25:38...
Page 121 Peer receive window: Current: 64 KB, Maximum: 64 KB, Scale: 1 Congestion window: Current: 2 KB, Slow start threshold: 1048560 KB OL-5183-02, Cisco MDS SAN-OS Release 1.3 (1-2) (1-2) (interface GigabitEthernet2/1) Cisco MDS 9000 Family Troubleshooting Guide Troubleshooting FCIP Connections 5-27...
Page 122: Special Frame Misconfiguration Examples
Jan 14 15:14:30 Jan 14 15:14:30 Jan 14 15:14:30 Jan 14 15:14:30 Cisco MDS 9000 Family Troubleshooting Guide 5-28 port1: 855278:FCIP21: SUP-> Set Port mode 1 port1: 855279:FCIP21: SUP-> Port VSAN (1) already set to same value port1: 855280:FCIP21: SUP-> Trunk mode (1) already set to same port1: 855281:FCIP21: SUP->...
Page 123 855352:FCIP21: Bind the DE 0xd802cac0 [1] to tunnel LEP 0x80111570 port1: 855353:FCIP21: Bind DE 1 to TCP-hdl 0xd8071400 port1: 855354:FCIP21: Setup for Special Frame handling: I'm Responder port1: 855355:FCIP21: Setup timer to wait for SF Cisco MDS 9000 Family Troubleshooting Guide Troubleshooting FCIP Connections 5-29...
Page 124 Figure 5-10 shows a trace of an incorrect remote switch WWN using a Special Frame Figure 5-10 Trace of Incorrect Remote Switch WWN Using a Special Frame Cisco MDS 9000 Family Troubleshooting Guide 5-30 port1: 855356:FCIP21: processing SF frame, I'm Responder...
Page 125: Troubleshooting Iscsi Issues
Troubleshooting Dynamic iSCSI Configuration, page 5-36 Troubleshooting iSCSI Authentication iSCSI user login authentication is required with the Cisco MDS 9000 Family switch. There are two ways of the getting iSCSI users authenticated: either locally configured the in the switch’s configuration file, or using the Radius server database.
Page 126: Configuring Authentication
If iSCSI is configured for radius authentication, it should looks like this: switch# show authentication authentication method:none console:not enabled telnet/ssh:not enabled authentication method:radius console:not enabled Cisco MDS 9000 Family Troubleshooting Guide 5-32 Chapter 5 <<<<<<<<<<<<<<<<<<<< <<<<<<<<<<<<<<<<<<<< <<<<<<<<<<<<<<<<<<< OL-5183-02, Cisco MDS SAN-OS Release 1.3...
Page 127: Troubleshooting Username/Password Configuration
Adjust the radius timeout and retransmission accordingly, as they have default value of 1 sec and 1 time. Figure 5-13 shows a Windows-based Radius server configuration. OL-5183-02, Cisco MDS SAN-OS Release 1.3 <<<<<<<<<<<<<<<<<<< available for authentication on port:1812 available for accounting on port:1813 RADIUS shared secret:radius Cisco MDS 9000 Family Troubleshooting Guide Troubleshooting iSCSI Issues 5-33...
Page 128 4 23:16:25 securityd: reading RADIUS configuration 4 23:16:25 securityd: opening radius configuration for group:default 4 23:16:25 securityd: opened the configuration successfully 4 23:16:25 securityd: GET request for RADIUS global config Cisco MDS 9000 Family Troubleshooting Guide 5-34 Chapter 5 Troubleshooting IP Storage Issues...
Page 129 667 2003y3m14d 15h12m54s CHAP challenge=#ùÊÝü{_"__“´_Ux 668 2003y3m14d 15h12m54s NAS port=1426 669 2003y3m14d 15h12m54s NAS port type=5 670 2003y3m14d 15h12m54s Service type=8 671 2003y3m14d 15h12m54s User (user002) authenticate OK. OL-5183-02, Cisco MDS SAN-OS Release 1.3 Troubleshooting iSCSI Issues Cisco MDS 9000 Family Troubleshooting Guide 5-35...
Page 130: Troubleshooting Radius Routing Configuration
“partnered” with a virtual iSCSI interface. In order for iSCSI to operate on a particular Gigabit Ethernet, the virtual iSCSI interface for that port must be in a “no shutdown” state: interface Gigabit Ethernet 3/1 no shutdown Cisco MDS 9000 Family Troubleshooting Guide 5-36 5 00:51:13 securityd: received CHAP authentication request for user002 Chapter 5 Troubleshooting IP Storage Issues OL-5183-02, Cisco MDS SAN-OS Release 1.3...
Page 131: Performing Basic Dynamic Iscsi Troubleshooting
MaxBurstSize 0, MaxConn 0, DataPDUInOrder No DataSeqInOrder No, InitialR2T Yes, ImmediateData No Registered LUN 0, Mapped LUN 0 Stats: PDU: Command: 42, Response: 36 OL-5183-02, Cisco MDS SAN-OS Release 1.3 Troubleshooting iSCSI Issues Cisco MDS 9000 Family Troubleshooting Guide 5-37...
Page 132 1 switch# show flogi database vsan 1 ----------------------------------------------------------- INTERFACE VSAN ----------------------------------------------------------- fc1/1 fc1/6 fc1/8 fc1/9 fc1/10 iscsi2/7 Cisco MDS 9000 Family Troubleshooting Guide 5-38 TYPE PWWN (VENDOR) 20:23:00:a0:b8:0b:14:da (SymBios) 10:00:00:00:c9:30:ba:06 (Emulex) 20:0d:00:0b:be:77:72:42 50:08:05:f3:00:04:96:71 50:08:05:f3:00:04:96:79 20:00:00:02:3d:07:05:c0 (NuSpeed) FCID...
Page 133: Virtual Target Access Control
Session #2 (index 85) Target iqn.com.domainname.IPS-TEST.02-08.gw.2200002037c52e2e VSAN 5, ISID 00023d000055, TSID 135, Status active, no reservation Type Normal, ExpCmdSN 1356, MaxCmdSN 1366, Barrier 0 OL-5183-02, Cisco MDS SAN-OS Release 1.3 Troubleshooting iSCSI Issues Cisco MDS 9000 Family Troubleshooting Guide 5-39...
Page 134 AuthMethod none, HeaderDigest None (len 0), DataDigest None (len 0) Version Min: 0, Max: 0 FC target: Up, Reorder PDU: No, Marker send: No (int 0) Received MaxRecvDSLen key: Yes Cisco MDS 9000 Family Troubleshooting Guide 5-40 Chapter 5 Troubleshooting IP Storage Issues...
Page 135 Self orig command: 213, Rcvd: data: 142, resp: 213 Rcvd: PLOGI: 614, Sent: PLOGI_ACC: 490 LOGO: 197, Sent: LOGO_ACC: 111 PRLI: 0, Sent: PRLI_ACC: 0 ABTS: 183 iSCSI Drop: OL-5183-02, Cisco MDS SAN-OS Release 1.3 Troubleshooting iSCSI Issues Cisco MDS 9000 Family Troubleshooting Guide 5-41...
Page 136 No. of FC sessions: 1 No. of iSCSI sessions: 1 iSCSI session details Target node: Statistics: Cisco MDS 9000 Family Troubleshooting Guide 5-42 CmdSeqNo not in window 0, No Exchange ID 0, Reject 0 Persistent Resv 0 Data-Out: 0, TMF-Req: 0...
Page 137 PDU: Command: 13, Response: 13 Bytes: TX: 1344, RX: 0 Number of connection: 1 TCP parameters Connection Local 10.1.29.100:3260, Remote 10.1.29.101:1048 Path MTU 1500 bytes OL-5183-02, Cisco MDS SAN-OS Release 1.3 Troubleshooting iSCSI Issues Cisco MDS 9000 Family Troubleshooting Guide 5-43...
Page 138: Fine Tuning/Troubleshooting Ips Iscsi Tcp Performance
Gigabit Ethernet attached iSCSI clients. Each of the above-mentioned commands are taken from a scenario in sections of the displays are highlighted/italicized or bolded. Cisco MDS 9000 Family Troubleshooting Guide 5-44 Current retransmission timeout is 300 ms...
Page 139: Lab Setup
WWPNs for the IBM Shark storage subsytem. See (in VSAN 778) (in VSAN 777) Fine Tuning/Troubleshooting IPS iSCSI TCP Performance MDS 9216_Top 1.1(0.133c) FC 1/3 E-ISL ESS 2105/F20 FC 1/3 MDS 9216_Bottom 1.1(0.133c) Figure 5-14. Cisco MDS 9000 Family Troubleshooting Guide 5-45...
Page 140 Total number of entries = 2 MDS_BOTTOM# MDS_BOTTOM# show module Ports Module-Type ----- ------------------------------- ------------------ ------------ 1/2 Gbps FC/Supervisor IP Storage Module ----------- Cisco MDS 9000 Family Troubleshooting Guide 5-46 Remote Address 10.48.69.233:1026 10.48.69.233:1057 0.0.0.0:0 FCID PORT NAME 0x610000 50:05:07:63:00:c8:94:4c...
Page 141 Max-retransmissions 8 Window-size 1024000 Sack is enabled OL-5183-02, Cisco MDS SAN-OS Release 1.3 Fine Tuning/Troubleshooting IPS iSCSI TCP Performance 20:01:00:0c:30:57:5e:c0 to 20:10:00:0c:30:57:5e:c0 20:41:00:0c:30:57:5e:c0 to 20:48:00:0c:30:57:5e:c0 Serial-Num ---------- JAB070804Q3 JAB070205am Cisco MDS 9000 Family Troubleshooting Guide 5-47...
Page 142 You may need to adjust the Retransmission timeout.) (This is the window size set on the Windows Client. See Cisco MDS 9000 Family Troubleshooting Guide 5-48 PDU: Command: 0, Response: 0...
Page 143: Changing Tcp Parameters
MTU, provided the network in the middle supports jumbo frames.) Port mode is IPS Speed is 1 Gbps Beacon is turned off OL-5183-02, Cisco MDS SAN-OS Release 1.3 bytes, BW 1000000 Kbit Fine Tuning/Troubleshooting IPS iSCSI TCP Performance Figure 5-15.) Cisco MDS 9000 Family Troubleshooting Guide 5-49...
Page 144 0 packet after window, 0 bytes after window 0 packets after close 7067879 acks, 76746255713 ack bytes, 0 ack toomuch, 21 duplicate acks 0 ack packets left of snd_una, 0 non-4 byte aligned packets Cisco MDS 9000 Family Troubleshooting Guide 5-50 0 unknown class Chapter 5 Troubleshooting IP Storage Issues OL-5183-02, Cisco MDS SAN-OS Release 1.3...
Page 145 Target: RcvDataFieldSize 2048 our_RcvDataFieldSize 1392 MaxBurstSize 0, EMPD: FALSE Random Relative Offset: FALSE, Sequence-in-order: Yes PDU: Command: 0, Response: 0 Fine Tuning/Troubleshooting IPS iSCSI TCP Performance State Send-Q Recv-Q ESTABLISH ESTABLISH 29296 LISTEN Cisco MDS 9000 Family Troubleshooting Guide 5-51...
Page 146 TCP parameters VSAN ID 777, FCID 0x700003 No. of FC sessions: 1 No. of iSCSI sessions: 1 iSCSI session details Target node: shark_nas Cisco MDS 9000 Family Troubleshooting Guide 5-52 pWWN: 50:05:07:63:00:c8:94:4c nWWN: 50:05:07:63:00:c8:94:4c Target: shark_nas RcvDataFieldSize 2048 our_RcvDataFieldSize 1392...
Page 147 PDU: Command: 0, Response: 0 pWWN: 50:05:07:63:00:c4:94:4c nWWN: 50:05:07:63:00:c4:94:4c Target: shark_nas RcvDataFieldSize 2048 our_RcvDataFieldSize 2048 MaxBurstSize 0, EMPD: FALSE Random Relative Offset: FALSE, Sequence-in-order: Yes PDU: Command: 0, Response: 11 Fine Tuning/Troubleshooting IPS iSCSI TCP Performance Cisco MDS 9000 Family Troubleshooting Guide 5-53...
Page 148 712704 data, 2831 ack only packets 0 control (SYN/FIN/RST), 0 probes, 0 window updates 0 segments retransmitted, 0 bytes 0 retransmitted while on ethernet send queue, 345477 packets split ……… Cisco MDS 9000 Family Troubleshooting Guide 5-54 Chapter 5 Remote Address State 10.48.69.233:1026...
Page 149: Analyzing Switch Device Health
Click Clear to remove the contents of the Switch Health Analysis window. Step 3 Step 4 Click Close to close the window. OL-5183-02, Cisco MDS SAN-OS Release 1.3 C H A P T E R Cisco MDS 9000 Family Troubleshooting Guide 6-55...
Page 150: Analyzing End-To-End Connectivity
To use the Fabric Configuration option to analyze the configuration of a switch, follow these steps: OL-5183-02, Cisco MDS SAN-OS Release 1.3 Analyzing End-to-End Connectivity Cisco MDS 9000 Family Troubleshooting Guide 6-56...
Page 151: Analyzing The Results Of Merging Zones
Click Clear to remove the contents of the window. Step 5 Click Close to close the window. Step 6 OL-5183-02, Cisco MDS SAN-OS Release 1.3 Analyzing the Results of Merging Zones Cisco MDS 9000 Family Troubleshooting Guide 6-57...
Page 152: Issuing The Show Tech Support Command
Each file is named with the switch’s IP address and has a .TXT extension (for example, 111.22.33.444.txt). OL-5183-02, Cisco MDS SAN-OS Release 1.3 Issuing the Show Tech Support Command Cisco MDS 9000 Family Troubleshooting Guide 6-58...
Page 153: Using Traceroute And Other Troubleshooting Tools
Enter the appropriate read community string in the Read Community field. The default value for this string is “public.” Step 4 Click Display Cisco MDS 9000 Only to display only the Cisco MDS 9000 Family switches in your network fabric. Step 5 Click Search to discover switches and devices in your network fabric.
Page 154: Configuring An Oui
Restart Fabric Manager for your changes to take effect. Note This situation does not affect the availability or the functionality of the switch and/or fabric. Cisco MDS 9000 Family Troubleshooting Guide 6-60 Chapter 6 Troubleshooting the Fabric...
Page 155: Can I Set The Map Layout So It Stays After I Restart Fabric Manager
Right-click on a blank space in the map. A menu is displayed. Step 2 Select Layout -> Fix All Nodes from the menu. OL-5183-02, Cisco MDS SAN-OS Release 1.3 C H A P T E R Cisco MDS 9000 Family Troubleshooting Guide 7-61...
Page 156: Two Switches Show On My Map, But I Only Have One Switch
Even if the community string is incorrect, double-clicking on the Cisco SN5428 will launch the web tool. OL-5183-02, Cisco MDS SAN-OS Release 1.3 Two Switches Show on my Map, But I Only Have One Switch Cisco MDS 9000 Family Troubleshooting Guide 7-62...
Page 157: Running Cisco Fabric Manager With Multiple Interfaces
If the line is commented, remove the comment character. Set this value to the IP address or interface name of the NIC you want to use. Step 5 OL-5183-02, Cisco MDS SAN-OS Release 1.3 Running Cisco Fabric Manager with Multiple Interfaces Cisco MDS 9000 Family Troubleshooting Guide 7-63...
Page 158: Specifying An Interface For Fabric Manager Client Or Device Manager
(which will clear the information for all clients) without having to reboot the switch. To clear information from topology maps, follow these steps: Step 1 In the Map pane, click on the Refresh Map icon. Cisco MDS 9000 Family Troubleshooting Guide 7-64 Chapter 7 Troubleshooting Fabric Manager Issues...
Page 159: Can I Use Fabric Manager In A Mixed Software Environment
Can I Use Fabric Manager in a Mixed Software Environment? You can use Fabric Manager version 1.3(x) to manage a mixed fabric of Cisco MDS 9000 switches. Certain 1.3 feature tabs will be empty for any switches running a software version that does not support those features.
Page 160 S e n d c o m m e n t s t o m d s f e e d b a c k - d o c @ c i s c o . c o m . Cisco MDS 9000 Family Troubleshooting Guide 7-66 OL-5183-02, Cisco MDS SAN-OS Release 1.3...
Page 161: I N D E X
1-12 switch health 1-10 OL-5183-01, Cisco MDS SAN-OS Release 1.3 traceroute 1-12 troubleshooting with traceroute 1-12 verifying fabric configuration 1-11 zone configuration 1-12 zones merging 1-12 Cisco MDS 9000 Family Troubleshooting Guide I N D E X IN-67...

Cisco MDS 9000 Series Troubleshooting Manual

CHAPTER 1 Troubleshooting Overview

CHAPTER 2 Troubleshooting Switch System Issues

CHAPTER 3 Troubleshooting Switch Level Issues and Interswitch Connectivity

Chapter 4 Troubleshooting Switch Fabric Level Issues

CHAPTER 5 Troubleshooting IP Storage Issues

CHAPTER 6 Troubleshooting the Fabric

CHAPTER 7 Troubleshooting Fabric Manager Issues

I N D E X

Quick Links

Troubleshooting

Related Manuals for Cisco MDS 9000 Series

Summary of Contents for Cisco MDS 9000 Series

Table of Contents