Page 1 Sun StorEdge network FC switch-8 and switch-16 Field Troubleshooting Guide Sun Microsystems, Inc. 901 San Antonio Road Palo Alto, CA 94303 U.S.A. 650-960-1300 Part No.816-0252-10 April 2001, Revision A Send comments about this document to: docfeedback@sun.com...
Page 2 Sun, Sun Microsystems, the Sun logo, AnswerBook2, docs.sun.com, Sun StorEdge network FC switch-8, and Solaris are trademarks, registered trademarks, or service marks of Sun Microsystems, Inc. in the U.S. and other countries. All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc.
Page 3 Preface The Sun StorEdge network FC switch-8 and switch-16 Field Troubleshooting Guide describes how to diagnose and troubleshoot the Sun StorEdge network FC switch-8 and switch-16 hardware. It provides information and pointers to additional documentation you may need for installing, configuring, and using the configuration.
Page 4: Typographic Conventions
Typographic Conventions Typeface Meaning Examples The names of commands, files, Edit your .login file. AaBbCc123 and directories; on-screen Use ls -a to list all files. computer output % You have mail. What you type, when AaBbCc123 contrasted with on-screen Password: computer output AaBbCc123 Book titles, new words or terms,...
Page 5: Related Documentation
Related Documentation Application Title Part Number Installer’s information Sun StorEdge network FC switch-8 806-6922-10 and switch-16 Installation, and Configuration Guide 875-3060-10 Rev.X Installer/User’s SANbox-8/16 Segmented Loop Switch information Management and User’s Manual 875-3059-10 Rev.X GUI and User Sun SANbox 16 Segmented Loop Switch User’s Manual Late news Sun StorEdge network FC switch-8 and...
Page 6 Ordering Sun Documentation Fatbrain.com, an Internet professional bookstore, stocks select product documentation from Sun Microsystems, Inc. For a list of documents and how to order them, visit the Sun Documentation Center on Fatbrain.com at: http://www.fatbrain.com/documentation/sun Sun Welcomes Your Comments Sun is interested in improving its documentation and welcomes your comments and suggestions.
Page 7: Table Of Contents
Contents The Sun StorEdge Network FC Switch-8 and Switch-16 Troubleshooting Guide Introduction 1 Supported Configurations 2 Sun StorEdge network FC switch-8 and FC switch-16 Configuration 2 Zoning 3 Supported Hardware Configurations 4 Required Solaris Level 5 Guidelines for Configuration 5 Multi-Host 13 Diagnostic Tools 16 Hardware Tools 16...
Page 8 Heartbeat LED Blink Patterns 27 Cable Continuity Tests 32 Switch Counter Information 33 Counter Descriptions 35 Diagnostic Information and Isolation 41 Sun StorEdge StorTools 4.x qlctest 41 Sun StorEdge StorTools 4.x switchtest 42 Examples of Fault Isolation 46 Scenario 1a—Bad Cable Between Host and Switch (Using StorEdge Expert) Scenario 2—Bad GBIC in Switch 48 Scenario 1b—Bad Cable Between Host and Switch (Using Functional Test) 51 A Quick Functional Test (a5ksestest) to Test Full Loop 54...
Page 9 List of Figures Switch and Interconnections 1 FIGURE 1 Example: Single Host Connected to One Sun StorEdge A3500FC Controller Module Using FIGURE 2 Switches 7 Example: Single Host Connected to One Sun StorEdge A5200 Controller Module Using FIGURE 3 Switches 7 Example: Single Host Connected to One Sun StorEdge T3 Partner Pair Using Switches 8 FIGURE 4 Example: Single Host to Multiple A3500-FC Controller Modules Using switches 9...
Page 10 Web GUI 38 FIGURE 18 Sun StorEdge StorTools 4.x qlctest 41 FIGURE 19 Sun StorEdge StorTools 4.x Switch Test or SANSurfer GUI Start Test 42 FIGURE 20 Sun StorEdge StorTools 4.x Array Tests 43 FIGURE 21 Isolation in Areas 1, 2, and 3 44 FIGURE 22 Functional Test of Switch window 57 FIGURE 23...
Page 11 List of Tables Supported Hardware 4 TABLE 1 Arrays, Zones, and Initiators 6 TABLE 2 Dynamic Addition to a Zone* (without reboot of host) 6 TABLE 3 Port Display Window Counters 35 TABLE 4 Counter Names and Descriptions (Faceplate Window) 39 TABLE 5 List of Tables...
Page 12 Sun StorEdge Network FC Switch-8 and Switch-16 Field Troubleshooting Guide—April 2001...
Page 13: The Sun Storedge Network Fc Switch-8 And Switch-16 Troubleshooting Guide
The Sun StorEdge Network FC Switch-8 and Switch-16 Troubleshooting Guide Introduction The scope of this document includes the switch and the interconnections (HBA, GBIC, cables) on either side of the switch, as shown in the following diagram. Switch Storage Host Switch Switch and Interconnections FIGURE 1...
Page 14: Supported Configurations
This troubleshooting guide is intended to provide basic guidelines that can be used for isolating problems for the supported configurations identified in this document. It also assumes you have been trained on all the components that comprise storage and switch configurations. Sun StorEdge StorTools 4.01 or above is required to support the configurations in this document.
Page 15: Zoning
For more information on loop configurations and zoning, refer to the Sun StorEdge network FC switch-8 and switch-16 Installation and Configuration Guide and the SANbox 8/16 Segmented Loop Switch Management User’s Manual, which are shipped with your system. Note – No more than one adapter port from any given host should be connected to the same zone.
Page 16: Supported Hardware Configurations
GBIC Gigabit Interface Converter for the SBus FC-100 Host Adapter X973A 2M fiber optic cable X978A 15m fiber optic cable X6746A Sun StorEdge FC switch-8 Switch SG-XSW16- Sun StorEdge network FC switch-16 Switch Sun StorEdge Network FC Switch-8 and Switch-16 Field Troubleshooting Guide • April, 2001...
Page 17: Required Solaris Level
Required Solaris Level Be sure that all systems are running Solaris 8 (10/00 release and later) and that the necessary patches for switch support are installed. See http://www.sun.com/service/support/sunsolve/index.html for more information. Guidelines for Configuration Hosts Sun Enterprise™ 220, 250, 420, and 450 Sun Enterprise 3x00 through Enterprise 6x00 Sun Enterprise 10000 Arrays...
Page 18: Table 2 Arrays, Zones, And Initiators
Arrays, Zones, and Initiators TABLE 2 Array Maximum Arrays/Zone Maximum Initiators/Zone Sun StorEdge A3500- Sun StorEdge A5200 2 initiators per loop, or a maximum of four per array Sun StorEdge T3 Dynamic Addition to a Zone* (without reboot of host) TABLE 3 Array First / Additional...
Page 19: Figure 2 Example: Single Host Connected To One Sun Storedge A3500Fc Controller Module Using
Host Switches Sun StorEdge A3500FC controller module Controller A FC-AL port Host adapter Controller B FC-AL port SCSI x 5 Host adapter Fibre-optic cables Drive tray x 5 Example: Single Host Connected to One Sun StorEdge A3500FC Controller FIGURE 2 Module Using Switches Sun StorEdge A5200 controller module Host...
Page 20: Figure 4 Example: Single Host Connected To One Sun Storedge T3 Partner Pair Using Switches
Sun StorEdge T3 Partner Pair Host Switches Host adapter Host adapter Fiber-optic cables Example: Single Host Connected to One Sun StorEdge T3 Partner Pair Using FIGURE 4 Switches Sun StorEdge Network FC Switch-8 and Switch-16 Field Troubleshooting Guide • April, 2001...
Page 21: Figure 5 Example: Single Host To Multiple A3500-Fc Controller Modules Using Switches
Sun StorEdge A3500FC controller module 4 Controller A FC-AL port Controller B SCSI x 5 FC-AL port Host switches Host adapter Drive tray x 5 StorEdge A3500FC controller module Host adapter Controller A FC-AL port Controller B SCSI x 5 FC-AL port Drive tray x 5 StorEdge A3500FC controller module...
Page 22: Figure 6 Example: Single Host To Multiple A5200 Controller Modules Using Switches
Sun StorEdge A5200 controller modules - 3 Host switches Host adapter Host adapter Example: Single Host to Multiple A5200 Controller Modules Using switches FIGURE 6 Sun StorEdge Network FC Switch-8 and Switch-16 Field Troubleshooting Guide • April, 2001...
Page 23: Figure 7 Example: Single Host To Two Storedge T3 Partner Pairs Using Switches
Sun StorEdge T3 Partner Pairs - 2 Host switches Host adapter Host adapter Example: Single Host to Two StorEdge T3 Partner Pairs using switches FIGURE 7 Sun StorEdge Network FC Switch-8 and Switch-16 Field Troubleshooting Guide—April, 2001...
Page 24: Figure 8 Example: Single Host Connected To Multiple Storedge T3 Partner Pairs, Using Switches
Sun StorEdge T3 Partner Pairs (4) Host Switches Host adapter Host adapter Example: Single Host Connected to Multiple StorEdge T3 Partner Pairs, FIGURE 8 Using Switches Sun StorEdge Network FC Switch-8 and Switch-16 Field Troubleshooting Guide • April, 2001...
Page 25: Multi-Host
Multi-Host shows an example of a multi-host configuration: two hosts connected FIGURE 9 through fiber-optic cables to two Sun StorEdge A3500FC controller modules using switches. A3500FC controller modules -4 Controller A FC-AL port Host Controller B SCSI x 5 FC-AL port Host adapter switches Host adapter...
Page 26: Figure 10 Example: Two Hosts Connected To Three Sun Storedge A5200 Controller Modules Using
Sun StorEdge A5200 controller modules - 3 Host switches Host adapter Host adapter Host Host adapter Host adapter Example: Two Hosts Connected to Three Sun StorEdge A5200 Controller FIGURE 10 Modules using Switches Sun StorEdge Network FC Switch-8 and Switch-16 Field Troubleshooting Guide • April, 2001...
Page 27: Figure 11 Example: Two Hosts Connected To Four Sun Storedge T3 Partner Pairs Using Switches
Sun StorEdge T3 Partner Pairs (4) Host Switches Host adapter SL Zone 1 Host adapter Host SL Zone 2 Host adapter SL Zone 3 Host adapter SL Zone 4 Example: Two Hosts Connected to Four Sun StorEdge T3 Partner Pairs Using FIGURE 11 Switches Sun StorEdge Network FC Switch-8 and Switch-16 Field Troubleshooting Guide—April, 2001...
Page 28: Diagnostic Tools
Diagnostic Tools Note – Ensure that all the systems are running Solaris 8 (10/00 or later). The tools available for troubleshooting: Switch Sun StorEdge Network FC switch 2.0 GUI Host Sun StorEdge StorTools 4.x (offline/online) Sun StorEdge RASAgent 1.1 Explorer 3.4 Sun StorEdge T3 array extractor script Storage CM 2.1 - Sun StorEdge T3 array...
Page 29: Helpful Failure Information
Helpful Failure Information The following information should be gathered and reviewed before you start any troubleshooting effort. The information you gather may point you in the right direction or support other failure data. /var/adm/messages Sun StorEdge RASAgent 1.1 e-mail messages Weblog.file Explorer LED indicators...
Page 30: Fc Switch Leds And Back Panel Controls
FC Switch LEDs and Back Panel Controls identify the parts of the switch chassis back. Port numbers FIGURE 12 FIGURE 13 are marked on the chassis. Port Number Switch Management Connector (RJ45) Activity LED (Ethernet) Trafﬁc LED (Yellow) Logged-In LED Link Status LED (Green) (Ethernet)
Page 31: Figure 13 Chassis Back (16-Port Switch)
Port Number Switch Trafﬁc LED Management (Yellow) Connector (RJ45) Logged-In LED (Green) Fibre Channel Port AC Power MAC Address Plug Power Switch Label Over Heartbeat Temperature Force (Yellow) PROM (RED) Button Logged-In LED Fan Fail Switch Logic (Green) LED (RED) Power Good LED (Green) Trafﬁc LED (Yellow)
Page 32: Power Switch
Power Switch “Chassis Back (8-Port Switch)” on page 18 and “Chassis Back (16-Port Switch)” on page 19 shows the location of the power switch. The power switch is a rocker switch. Press the right side (labeled 1) to turn it ON; press the left side (labeled 0) to turn it OFF.
Page 33 Over Temperature LED (Red) This LED is normally OFF. The over temperature LED lights to indicate that the air temperature inside the switch has exceeded a certain limit. If this LED lights, inspect the following: Ambient air temperature: maximum 40°C (104°F) Proper clearance: 163 mm (6.5”) back, right side, and front Fan Operation Power supply operation...
Page 34: Ac Input Power Connector And Fuses
AC Input Power Connector and Fuses A standard 3-wire computer-type AC power cable (supplied with the switch) connects between the AC input power connector and an AC outlet. See FIGURE 12 FIGURE 13 An input fuse holder is incorporated into the AC input power connector assembly. It holds two input fuses.
Page 35: Diagnosing And Troubleshooting The Switch
Diagnosing and Troubleshooting the Switch This section provides information for diagnosing and troubleshooting problems with the switch. Power Checks and Troubleshooting help you solve AC power and Power Supply problems. Power-On-Self-Test (POST) checks the condition of the Switch, with the exception of the GBICs.
Page 36 During the POST, the switch logs any errors encountered. Some POST errors are fatal; others are non-fatal. A fatal error disables the switch so that it does not operate. A non-fatal error allows the switch to operate, but with some decrease in performance until the problem is corrected.
Page 37: Using The Test Mode Switch
The POST diagnostic program performs the following basic tests: Checksum tests on the Boot firmware located in a PROM and the main switch firmware located in FLASH memory. Functional hardware tests on internal switch memory. Various read/write register and loopback data-path tests on the switch logic board.
Page 38: Figure 14 Test Mode Switch Functions And Positions
Front Panel Switch Modes The following are the settings for the 10-position rotary switch: Normal operations Continuous test Test bypass Operator test Normal operation/initial test with force PROM mode Continuous test with force PROM mode Test bypass with force PROM Operator test with force PROM Normal operation/initial test with watchdog timer disabled Continuous test with watchdog timer disabled...
Page 39: Heartbeat Led Blink Patterns
Troubleshooting Test Mode Switch Functions 1. Use a small screwdriver to change the test mode switch positions. Use the normal position as reference and count the number of clicks (one click per position). These clicks are not audible and are best detected by touch. 2.
Page 40: Figure 16 Heartbeat Led-Failure Blink Patterns
Failure Blink Patterns The heartbeat LED indicates the error with a series of blinks, a three-second pause, and then the same series of blinks. The number of blinks between the three-second pause indicates the error. The blinks occur at about twice the speed of the normal heartbeat.
Page 41 You may load new flash control code via the Switch Management port. See the Switch Management manual for a description of how to load new flash code. Flash Checksum Failure/Switch Management port (Ethernet) Failure (Four Blinks) The switch is not operable. The flash checksum test verifies the integrity of the flash data.
Page 42 GBIC Bypass Port Loopback Test Failure (Seven Blinks) The switch is operable. The GBIC bypass port loopback test verifies (on a port-by-port basis) the ability of each switch ASIC to loop data out through the Serdes chip on a port and back to the ASIC control port (bypassing the GBIC).
Page 43 Switch Bus Test Failure (Nine Blinks) The switch is not operable. The switch bus test verifies the ability of the switch ASICs to communicate with each other via the buses that interconnect the ASICs. A failure indicates an inability of an ASIC pair to communicate over one or more buses.
Page 44: Cable Continuity Tests
NVRAM Test Failure (15 Blinks) The switch is not operable. The Non-Volatile Memory (NVRAM) test verifies the status of the NVRAM battery (not low), performs a checksum on any existing data, and performs a data write/read test on the unused areas of the NVRAM. A test failure in any the these three tests causes the heartbeat LED to blink 15 times between three-second pauses.
Page 45: Switch Counter Information
Switch Counter Information Sun Engineering is currently investigating how counters can be used to help isolate failure. At this time, counter data should be used only as supporting data. Do not use this data as the primary source in the troubleshooting process. General points to keep in mind when viewing counters follow.
Page 46: Figure 17 Port Display
Port Display FIGURE 17 on the following page describes the counters from the Port Display window. TABLE 4 Sun StorEdge Network FC Switch-8 and Switch-16 Field Troubleshooting Guide • April, 2001...
Page 47: Counter Descriptions
Counter Descriptions Port Display Window Counters TABLE 4 Counter Name (in port display) Description Address ID errors Number of address identifiers (S_ID, D_ID) found to be in error. AL Init Attempts Number of times the port entered the initialization state. AL Init Errors Number of times the port entered initialization and the initialization failed.
Page 48: Table 4 Port Display Window Counters
Port Display Window Counters TABLE 4 Counter Name (in port display) Description Link reset out Number of link reset primitives sent from this port to an attached port. LIP AL_PD AL_PS Number of F7, AL_PS LIPs, or AL_PD (vendor specific) resets performed.
Page 49 Port Display Window Counters TABLE 4 Counter Name (in port display) Description Reject Frames Number of frames, from devices, that have been rejected. Frames can be rejected for any of a large number of reasons. Reserved Retry LIPs Currently not used. Short Frame Errors Number of times a frame shorter than 36 bytes was received.
Page 50: Figure 18 Web Gui
Web GUI FIGURE 18 on the following page lists the counter names and briefly describes them. TABLE 5 Sun StorEdge Network FC Switch-8 and Switch-16 Field Troubleshooting Guide • April, 2001...
Page 51 Counter Names and Descriptions (Faceplate Window) TABLE 5 Counter Description COF CRC ASIC 0 Internal switch counter that tracks errors during frame COF CRC ASIC 1 outputs from the specified ASIC. A non-zero value may COF CRC ASIC 2* indicate an internal problem with the switch. COF CRC ASIC 3* COF Parity ASIC 0 Parity error detected curing reading of the frame in the...
Page 52: Table 5 Counter Names And Descriptions (Faceplate Window)
Counter Names and Descriptions (Faceplate Window) TABLE 5 Counter Description Intr low Bus ASIC 0 Number of times a low buffer condition has occurred on Intr low Bus ASIC 1 the specific ASIC. Intr low Bus ASIC 2* Intr low Bus ASIC 3* Out of buffers Number of large frames that have been sent by this switch.
Page 53: Diagnostic Information And Isolation
Diagnostic Information and Isolation Caution – When running in online mode, deselect system board and HBA tests. Sun StorEdge StorTools 4.x qlctest You can run the Sun StorEdge StorTools 4.x PCI FC-100 Board Test (qlctest) or SunVTS 4.1 qlctest to test the following portion of the SAN configuration: HBA to switch and return path FRUs tested: HBA, cable between HBA and switch, and Switch GBIC Caution –...
Page 54: Sun Storedge Stortools 4.X Switchtest
Sun StorEdge StorTools 4.x switchtest You can run Sun StorEdge StorTools 4.x (switchtest) or SANSurfer GUI Start Test to test the following portion of the SAN configuration. Both tests can be run online. Switch to HBA and return path when running on a selected port. See #1 in FIGURE 20 Switch to array and return path when running on a selected port.
Page 55: Figure 21 Sun Storedge Stortools 4.X Array Tests
Sun StorEdge StorTools 4.x Array Tests (t3test, a5ktest, a3500fctest) You can run Sun StorEdge StorTools 4.x Array Tests (t3test, a5ktest, a3500fctest) to test the following portion of the Sun StorEdge Network FC Switch-8 and Switch-16 configuration: Entire path This is online testing but may affect performance. Storage Switch Host...
Page 56: Figure 22 Isolation In Areas 1, 2, And 3
Diagnostic Isolation Use the following diagram and accompanying information to help you with the isolation process. See Appendix B, “Isolation of SAN Components.” This appendix contains a generic flowchart, which describes how to isolate Mamba phase faults. Caution – Be sure only the path under test is selected. For more information about Sun StorEdge StorTools 4.x, refer to the Sun StorEdge StorTools User’s Guide, Version 4.x, part number 806-6235-10.
Page 57 Area 1 If failure data indicate a problem in Area 1, execute Sun StorEdge StorTools 4.x and one of the following tests: switchtest for initiator port (online) Appropriate HBA test qlctest (offline) soctest (offline) These tests may indicate a failure and isolate to multiple FRUs (HBA, cable, switch GBIC or switch).
Page 58: Examples Of Fault Isolation
Examples of Fault Isolation This section contains examples of failures and subsequent isolation techniques. In general, the following items must be kept in mind before starting. A Snapshot Create must be taken after the installation is complete. Than a Snapshot Diff can be taken as part of the isolation process. Sun StorEdge StorTools 4.x must be kept up and running to maintain the path state.
Page 59 Functional a5ktest from Sun StorEdge StorTools 4.x GUI 02/08/01 15:54:12 diag233.Central.Sun.COM Sun VTS4.1: VTSID 1 a5ktest. VERBOSE :”Options: selftest=Enable,wrdevbuf=Enable,wrdevbufpasses=100,wrdevbufptn=Ox7e7e7e73,allwrd evbufptn=Enable,partition=0,rawsub=Enable,method=SyncIO+AsyncIO,rawcover=1,raw iosize=32KB,fssub=Disable,fssize-512KB,fsiosize=512B,fspattern=sequential,dev= c2t32d0-f0)” 02/08/01 15:54:12 diag233.Central.Sun.COM Sun VTS4.1: VTSID 8014 a5ktest. FATAL c2t32d0: “Couldn’t open /dev/rdsk/c2t32d0s0: No such device or address” Probable_Causes(s): (1) Cable loose or disconnected (2) Device off-line or missing...
Page 60: Scenario 2-Bad Gbic In Switch
Scenario 2—Bad GBIC in Switch In this example, the loss of a single A5200 loop was noted in format and /var/adm/messages. Sun StorEdge StorTools 4.x Functional tests were used to verify the loop quickly.The Sun StorEdge StorTools 4.x StorEdge Expert tests were used to isolate down to a single failed GBIC on the switch.
Page 61 Run GUI StorEdge Expert on Same Disk 02/08/01 15:01:55 diag233.Central.Sun.COM Sun VTS4.1: VTSID 2100 a5ktest.expert.INFO c2t0d0: “Expert Started.” 02/08/01 15:01:56 diag233.Central.Sun.COM Sun VTS4.1: VTSID 6100 a5ktest.expert. ERROR c2t02d0: “Expert error(s):reference Expert Log <<Feb082001_15:01:55>> STARTED:diagnosis expert session on /dev/rdsk/c2t32d0s2 <<Feb082001_15:01:56>> FAILED: for details see: /var/opt/SUNWvts/gogs/Feb082001_15:01:56_c2t0d0-f0.errlog <<Feb082001_15:01:56>>...
Page 62 Run StorEdge Expert from Command Line /opt/SUNWvts/bin/sparv9/stexpert -i -t /dev/rdsk/c2t0d)s2 stexpert: Diagnosis Begins <snip> stexpert: Remove fiber cable from DPORT GBIC in port 8 stexpert: Type ok to restart testing or exit to quit: ok Waiting 20 seconds for loopback to initialize <<Feb082001_15:05:19>>...
Page 63: Scenario 1B-Bad Cable Between Host And Switch (Using Functional Test)
Scenario 1b—Bad Cable Between Host and Switch (Using Functional Test) In this example, the loss of all storage connected to a switch was noted in /var/adm/messages and format (all disks labeled c2* were missing). A Snapshot diff was run to determine the extent of the problem. Functional tests were used to isolate individual subsection of the SAN to identify likely failed FRUs.
Page 64 Either the card was removed or we can no longer see storage attached to this card. Registername = qlc-0 LGroup = StorEdge-QLC-HostBusadapters Pgroup = /StorEdge Node WWN = 200000e08b026c2a Port WWN = 20000e08b026c2a DriverName = fp Detected missing device: Switch Switch ip address = 172.20.67.194 Switch port number = 5 Register Name...
Page 65 Detected Missing device: A5x00 Drive Box Name Logical Path -/dev/rdsk/c2t0d0s2 PhysPath /devices/pci@1f,4000/pci@4/SUNW,qlc@4/fp@0,0/ssd@w210000203719f7e0,0:c,raw Register Name =c2r0d0-f0 Logical Group =StorEdge-A5200-(qlc-0) Physical Group =/StorEdge/qlc-0/fc-8p-sw1-ip5(qlc-0)/fc-8p-sw1-dp8(qlc- 0)/qlc-0) NodeWWN =200000203719f7e0 PortWWN =210000203719f7e0 <snip> Sun StorEdge Network FC Switch-8 and Switch-16 Field Troubleshooting Guide—April, 2001...
Page 66: A Quick Functional Test (A5Ksestest) To Test Full Loop
A Quick Functional Test (a5ksestest) to Test Full Loop 02/09/01 13:05:46 diag233,Central.Sun.COM SunVTS4.1:VTSID 1012 a5ksestest,process_photest_argsVERBOSE SES:nws_enatest: called with options: disk_access=enable,delay=30,dev=a5k-ses11” 02/09/01 13:05:46 diag233,Central.Sun.COM SunVTS4.1:VTSID 0 a5ksestest.VERBOSE: “Started.” 02/09/01 13:05:46 diag233,Central.Sun.COM SunVTS4.1:VTSID 1000 a5ksestest.VERBOSE: “Started test on /dev/es/ses11” 02/09/01 13:05:46 diag233,Central.Sun.COM SunVTS4.1:VTSID 8005a5ksestest. FATAL:”Could not communicate with the enclosure”...
Page 67 A qlctest on the HBA in the path (qlc-0 in this example) can then be run to verify the HBA. (For this test, all Test Parameter Options for qlctest were disabled, except Online SelfTest and Firmware Checksum Test in the interest of test execution time. Further testing could be done, but the execution time would increase.) 02/09/01 13:38:59 diag233,Central.Sun.COM SunVTS4.1:VTSID 6qlctest.process_qlctest_args.VERBOSE qlc: “qlctest: called with options:...
Page 68: Scenario 3-Catastrophic Switch Failure
Another a5ksestest to Verify the Full Path—Successful 02/09/01 13:44:16 diag233.Central.Sun.COM SunVTS4.1: VTSID 1012 a5ksestest.process_photest_argsVERBOSE SES: “nws_enatest: called with options: disk_access=enable,delay=30,dev=a5k-ses11” 02/09/01 13:44:16 diag233.Central.Sun.COM SunVTS4.1: VTSID 0 a5ksestest.VERBOSE: “Started.” <snip> 02/09/01 13:44:59 diag233.Central.sun.COM SunVTS: VTSID0 a5ksestest.VERBOSE: “Stopped successfully.” Scenario 3—Catastrophic Switch Failure In this example, an entire switch has gone offline.
Page 69: Figure 23 Functional Test Of Switch Window
Functional Test of Switch (switchtest) Functional Test of Switch window FIGURE 23 02/09/01 10:19:55 diag233.Central.Sun.COM SunVTS4.1: VTSID 6031 switchtest FATAL switch0: “Switch not available on IP: 172.20.67.194 Pattern:.” Probable_Cause(s): (1) Wrong IP in /etc/hosts or /etc/fcswitch.conf (2) Network cable not attached to switch (3) Loss of power to switch Sun StorEdge Network FC Switch-8 and Switch-16 Field Troubleshooting Guide—April, 2001...
Page 70: Figure 24 Switch Gui Window
Look to Switch GUI No response from switch GUI, no connection. Switch GUI window FIGURE 24 Check Weblog.gui (/usr/opt/SUNWsmgr/Weblog.gui) A visual inspection of the switch revealed it was inadvertenly powered down, so the switch was repowered. 02/09/2001 10:23:47 <sysName undefined> timeout - No replay from Switch 02/09/2001 10:23:47 <sysName undefined>...
Page 71: Scenario 4-Bad Cable From Switch To Storage
Scenario 4—Bad Cable from Switch to Storage In this example, the loss of one path to an A5200 array was noted in format. A Snapshot Diff was run to determine the extent of the failure. Sun StorEdge StorTools 4.x Functional Tests were used to isolate various subsections of the SAN. Snapshot Diff shows loss of entire Sun StorEdge A5200 enclosure.
Page 72: Figure 25 Functional Test (Switchtest) On Initiator Port To Test Host-Switch Link Window
Run Functional Test (switchtest) on the Initiator Port to Test Host-Switch Link Functional Test (switchtest) on Initiator Port to Test Host-Switch Link FIGURE 25 window Sun StorEdge Network FC Switch-8 and Switch-16 Field Troubleshooting Guide • April, 2001...
Page 73 02/09/01 09:31:23 diag 233.Central.Sun.COM SunVTS4.1: VTSID 0 switchtest.VERBOSE switch0: “Started.” <snip> 02/09/01 09:31:59 diag 233.Central.Sun.COM SunVTS4.1: VTSID 0 switchtest.VERBOSE switch0: “Stopped successfully.” Sun StorEdge Network FC Switch-8 and Switch-16 Field Troubleshooting Guide—April, 2001...
Page 74: Figure 26 Functional Test (Switchtest) On Destination Port To Test Switch-Storage Link Window
Run Functional Test (switchtest on the Destination Port to Test Switch-Storage Link Functional Test (switchtest) on Destination Port to Test Switch-Storage Link FIGURE 26 window Sun StorEdge Network FC Switch-8 and Switch-16 Field Troubleshooting Guide • April, 2001...
Page 75 02/09/01 09:35:16 diag233.Central.Sun.COM Sun VTS4.1: VTSID 6 switchtest.process_args.VERBOSE switch0: “switchtest: called with options: xfer= 2000,passes=100000,pattern=0x7e7e7e7e,allpatterns=Disable,wait=2,dev=fc-8p-sw1- dp7(qlc-0)” 02/09/01 09:35:16 diag233.Central.Sun.COM Sun VTS4.1: VTSID 0 switchtest.VERBOSE switch0: “Started.” <snip> FATAL switch0: “Switch not Connected on Port: 7 Pattern: 0x7e7e7e7e.” Probable_Cause(s): (1) Fibre Channel cable disconnected (2) Bad GBIC or bad Fibre Channel cable (3) Loss of power to switch Sun StorEdge Network FC Switch-8 and Switch-16 Field Troubleshooting Guide—April, 2001...
Page 76: Figure 27 Insert Loopback In Destination Port To Test Switch's Gbic Window
Insert Loopback in Destination Port to Test Switch’s GBIC Insert Loopback in Destination Port to Test Switch’s GBIC window FIGURE 27 Sun StorEdge Network FC Switch-8 and Switch-16 Field Troubleshooting Guide • April, 2001...
Page 77 02/09/01 09:39:03 diag233.Central.Sun.COM Sun VTS4.1: VTSID 6 switchtest.process_args.VERBOSE switch0: “switchtest: called with options: xfer= 2000,passes=100000,pattern=0x7e7e7e7e,allpatterns=Disable,wait=2,dev=fc-8p-sw1- dp7(qlc-0)” 02/09/01 09:39:03 diag233.Central.Sun.COM Sun VTS4.1: VTSID 0 switchtest.VERBOSE switch0: “Started.” <snip> 02/09/01 09:39:03 diag233.Central.Sun.COM Sun VTS4.1: VTSID 0 switchtest.VERBOSE switch0: “Stopped successfully.” Problem is isolated to switch-to-storage cable or GBIC/connector on storage side. If the switch has empty ports, the storage-side GBIC could be temporarily placed in switch for loopback testing.
Page 78: Figure 28 Rerun A5Ksestest Window
In this instance, the cable was bad, and the replaced cable reran a5ksestest. Rerun a5ksesTest window FIGURE 28 Sun StorEdge Network FC Switch-8 and Switch-16 Field Troubleshooting Guide • April, 2001...
Page 79: Scenario 5-Bad Gbic In Storage (A5200)
Scenario 5—Bad GBIC in Storage (A5200) In this example, the loss of an A5200 loop was noted in /var/adm/messages and format. A Snapshot Diff was run to determine the extent of the failure. A Sun StorEdge StorTools 4.x Functional Test was run to do a quick loop test. StorEdge Expert was used to isolate down to a minimal number of suspect FRUs.
Page 80: Figure 29 Run Snapshot Diff Window
Run Snapshot DIFF Run Snapshot DIFF window FIGURE 29 Sun StorEdge Network FC Switch-8 and Switch-16 Field Troubleshooting Guide • April, 2001...
Page 81 Timestamp: Thu Feb 8 10:19:40 2001 Detected missing Host Bus Adapter Card. Either the card was removed or we can no longer see storage attached to this card. Registername=qlc-0 LGroup =StorEdge-QLC-HostBus adapters Pgroup =/StorEdge Node WWN =2000000e08b026c2a Port WWN =2100000e08b026c2a Driver Name Detected Missing device: A5x00 Enclosure...
Page 82 Run a5ktest on Drive in Failed Path 02/08/01 10:59:23 diag233.Central.Sun.COM SunVTS4.1:VTSID 8014 a5ktest. FATAL c2t32d0: “Couldn’t open /dev/rdsk/c2t32d0s0: No such device or address” Probable_Causes(s): (1) Cable loose or disconnected (2) Device off-line or missing (3) Device not configured (4) Device bypassed Recommended_Actions(s): (1) Check cable (2) Check device on-line...
Page 83 GBIC Replaced /var/adm/messages Feb 8 14:34:19 diag233.Central.Sun.COM qlc: [ID686697 kern.info] NOTICE: Qlogic qlc(0): Loop ONLINE Feb 8 14:34:19 diag233.Central.Sun.COM qlc: [ID799468 kern.info] ssd92 at fp0:name w2100002037450d3a,0, bus address bc Feb 8 14:34:19 diag233.Central.Sun.COM qlc: [ID936769 kern.info] ssd92 is /pci@1f,4000/pci@4/SUNW,qlc@4/fp@0,0/ssd@w2100002037450d3a,0 <snip> Verify with a GUI Functional Test (a5ktest) <snip>...
Page 84 Sun StorEdge Network FC Switch-8 and Switch-16 Field Troubleshooting Guide • April, 2001...
Page 85: Mamba Field Troubleshooting Guide Faq
A P P E N D I X Mamba Field Troubleshooting Guide FAQ Are 2x7 and 3x15 Sun StorEdge A3500-FC configurations supported in the Mamba phase? Yes. 1x5, 2x7, and 3x15 Sun StorEdge A3500-FC configurations are supported in the Mamba phase. What is the difference between “SL Zoning”...
Page 86 No. The current Sun switch GUI is installed with the SUNWsmgr package. The current version of this GUI is 2.07.54 (or 2.07.50, with patch 110696-xx — this patch can be found on Sunsolve). The syntax is as follows: java -jar /usr/opt/SUNWsmgr/bin/Sun.jar Refer to the installation guide for instructions on how to install the package.
Page 87 Yes. There is a file that should be saved, an Archive Fabric Config file. This file holds an archived copy of chassis configurable parameters, such as port modes, fabric name, SNMP settings, and zoning information (except zoning descriptions). After configuring the switch, create an archive file by clicking Special --> Archive Fabric from the topology view in the switch GUI.
Page 88 A Phillips-head screwdriver, size #0. Sun StorEdge StorTools 4.x is indicating a problem related to qlc0. What physical path is that? You can find the physical path by bringing up the Sun StorEdge StorTools 4.x GUI, right clicking on qlc0 (qlctest) and selecting Test Parameter Options. The physical path is indicated at the top of the screen.
Page 89 An example email of a Sun StorEdge RASAgent 1.1 Sun StorEdge T3 array LUN failover email is shown below. You requested the following events be forwarded to you. Message-Log Warnings: ** Identification: T300 - purple7 ** key=50020F23000003C5, ip=purple7, key_type=wwn, hostid=80b20f57, date=2001-03-17 16:00:18 ** New Information ** Warning : component='u2ctr', date='2001-03-17 15:54:10', name='purple7', text='u2ctr starting lun 0 failover',...
Page 90 # luxadm qlgc Found Path to 5 FC100/P, ISP2200 Devices Opening Device: /devices/pci@1f,4000/SUNW,ifp@5:devctl Detected FCode Version: FC100/P FC-AL Host Adapter Driver: 1.9 00/03/10 Opening Device: /devices/pci@1f,4000/pci@4/SUNW,qlc@4/fp@0,0:devctl Detected FCode Version: ISP2200 FC-AL Host Adapter Driver: 1.8 00/04/11 Opening Device: /devices/pci@1f,4000/pci@4/SUNW,qlc@5/fp@0,0:devctl Detected FCode Version: ISP2200 FC-AL Host Adapter Driver: 1.8 00/04/11 Opening Device: /devices/pci@1f,2000/pci@1/SUNW,qlc@4/fp@0,0:devctl Detected FCode Version: ISP2200 FC-AL Host Adapter Driver: 1.8 00/04/11 Opening Device: /devices/pci@1f,2000/pci@1/SUNW,qlc@5/fp@0,0:devctl...
Page 91 How can I force a LIP on a certain path, device, or HBA? There are multiple ways you can force an LIP on a system: 1. From the Faceplate Display screen on the switch GUI, double click the port from which you wish to send the LIP. Click the Send LIP button located on the right side of the screen.
Page 92 # /opt/SUNWvtsst/bin/sparcv9/discman (abbreviated) # /opt/SUNWvtsst/bin/sparcv9/discman Sun Microsystems, Inc. SunVTS FCAL StorEdge Discovery Version 1.000 Wed Mar 7 11:25:11 MST 2001 Copyright 2000 Sun Microsystems Inc. All rights reserved. Timestamp: Thu Mar 15 13:52:29 2001 Hostname: diag233.Central.Sun.COM Version: Detected 6 FCAL HBA port(s)
Page 93 < -- shows us the entire path to the T3 lun Device # 5: LogicalPath: /dev/rdsk/c5t1d1s2 PhysPath: /devices/pci@1f,4000/pci@4/SUNW,qlc@4/fp@0,0/ssd@w50020f23000003c5,1:c,raw RegisterName: c5t1d1 LGroup: StorEdge-T3-50020f20000003c5_qlc-0 PGroup: /StorEdge/qlc-0/fc-8p-sw0-ip3_qlc-0/fc-8p-sw0-dp2-qlc-0 NodeWWN: 50020f20000003c5 PortWWN: 50020f23000003c5 wNODEWWN: 00000000000000000 DualPort: Yes PortMode: Alternate Instance: 0 VendorID: SUN ProductID: T300 <...
Page 94 Using luxadm commands # luxadm -e port Found path to 4 HBA ports /devices/pci@1f,4000/pci@4/SUNW,qlc@4/fp@0,0:devctl NOT CONNECTED /devices/pci@1f,4000/pci@4/SUNW,qlc@5/fp@0,0:devctl CONNECTED /devices/pci@1f,2000/pci@1/SUNW,qlc@4/fp@0,0:devctl NOT CONNECTED /devices/pci@1f,2000/pci@1/SUNW,qlc@5/fp@0,0:devctl CONNECTED # luxadm -e dump_map /devices/pci@1f,4000/pci@4/SUNW,qlc@5/fp@0,0:devctl Pos AL_PA ID Hard_Addr Port WWN Node WWN Type e8 50020f23000003c5 50020f20000003c5 0x0 (Disk device) 0 210100e08b226c2a 200100e08b226c2a 0x1f (Unknown Type,Host Bus Adapter) I've heard about the sanbox command line and a utility called capture.
Page 95 Capture usage capture version 1.0.1.REV.2001.02.27.16.30 Usage: capture <ip_address> [-nvram] [Output filename] Example of capture output: # ./capture 172.20.67.194 capture.out # more capture.out Capture Version 1.0.1 ---------------------- IP Address: 172.20.67.194 ******************** Version Information ******************** PROM: 30200 FLASH: b30351 CHASSIS TYPE: CHASSIS NUMBER: 0 Fabric Id: WWN: 100000c0dd00562a...
Page 96 continued from previous page... ************ Port Status ************ Port # Port Type Admin State Oper State Status Loop Mode ------ --------- ----------- ---------- ------ --------- 1 SL_Port online offline Not-logged-in 2 SL_Port online online logged-in TargetDevices: 1 Address: 0x00 0xe8 3 SL_Port online online logged-in TargetDevices: 1...
Page 97 continued from previous page... Port Number: Inframes: 785611 Outframes: 4820054 LinkFails: SyncLosses: InvalidTxWds: 780498 Total LIP Rcvd: 69 LIP F7 F7: LIP F8 F7: AL Init Errs: AL Inits: 1060 loss_of_signal_cnt: 18113 lip_during_init: 1035 sync_loss: ------------------------- Port Number: Inframes: 9027777 Outframes: 1668118 LinkFails:...
Page 98 continued from previous page... ************ Name Server ************ Port Address Type PortWWN Node WWN FC-4 Types ---- ------- ---- ---------------- ---------------- ---------------------- Database is empty ********************* World-wide Name Zone ********************* WWN Zone total: 0 **************** NameServer Zone **************** NameServer Zone total : 0 *************** Broadcast Zone ***************...
Page 99 The sanbox API is a tool that can also be used to glean information from a switch. Use caution, as the sanbox API can be used to change state information on the switch. All documentation and source code for the API is included in the tarfile. The documentation is in html format and a example manpage is included as well.
Page 100 # vxdmpadm listctlr all CTLR-NAME DA-TYPE STATE DA-SNO ============================================== ctlr0 OTHER ENABLED OTHER_DISKS ctlr0=/pci@1f,4000/scsi@3 ctlr1 T300 ENABLED 60020f20000003c50000000000000000 ctlr1=/pci@1f,4000/pci@4/SUNW,qlc@5/fp@0,0 ctlr2 T300 ENABLED 60020f20000003c50000000000000000 ctlr2=/pci@1f,2000/pci@1/SUNW,qlc@5/fp@0,0 # vxdmpadm disable ctlr=/pci@1f,4000/pci@4/SUNW,qlc@5/fp@0,0 # vxdmpadm listctlr all CTLR-NAME DA-TYPE STATE DA-SNO ============================================== ctlr0 OTHER ENABLED OTHER_DISKS ctlr0=/pci@1f,4000/scsi@3 ctlr1...
Page 101: Isolation Of San Components Flowchart
A P P E N D I X Isolation of SAN Components Flowchart This appendix contains a generic flowchart, which describes how to isolate Mamba phase faults. The flowchart’s purpose is to help you use Stortools 4.x using a logical troubleshooting methodology.
Page 102 Start Run switchtest Isolation Run switchtest on replacement on suspect device DPORT GBIC GBIC/MIA Run path integrity test between host and suspect storage device Switchtest Isolated Switchtest on new DPORT on DPORT Dev GBIC/ DPORT loop GBIC (D) Loop passed? passed? Path integrity test...
Page 103 ...continued Run Device Test Device Disconnect daisy- Device Isolate chained devices from test device daisy-chained? suspect storage passed? array Reconnect Device is daisy-chained daisy- devices to suspect Verify that suspect chained? storage array storage device is available and powered-on Device Device missing/pulled available?
Page 104 ... continued Run Device Test Device Disconnect daisy- Device Isolate chained devices from test device daisy-chained? suspect storage passed? array Reconnect Device is daisy-chained daisy- devices to suspect Verify that suspect chained? storage array storage device is available and powered-on Device Device missing/pulled...
Page 105 ... continued Isolate Device (C) Run A5x00 Isolation Device (FiLTR) Test Isolate Failing LUN is A5x00? Failing Isolated Device Run A5x00 Isolation Failing Device Identiﬁed? (SCSI W/R Buffer) Test Reconnect daisy-chained devices to suspect storage array Failing Isolated Failing Device Identiﬁed? Device Systematic Isolation of the Various SAN Components (continued)
Page 106: Figure 30 Systematic Isolation Of The Various San Components
...continued Try new DPORT GBIC (D) Substitute new Replace original switch switch DPORT DPORT GBIC and GBIC and install reinstall original ﬁber Loopback connection Run switchtest Isolated failing on replacement switch DPORT GBIC Switchtest on DPORT Isolated DPORT GBIC Loop Passed? Systematic Isolation of the Various SAN Components (continued) Figure 30.
Page 107 ...continued IPORT Loop Test (E) Run switchtest Run switchtest between switch on replacement Reinstall ﬁber and suspect IPORT ﬁber HBA has into HBA host path removable GBIC. Suspect GBIC intermittent component Switch Switchtest test on IPORT Isolated IPORT on IPORT Loop passed? Fiber Loop passed?
Page 108 ...continued Try new IPORT GBIC (F) Substitute new switch IPORT GBIC and Replace original switch IPORT GBIC and install Loopback reinstall original ﬁber connection switchtest on Isolating replacement failing switch IPORT GBIC Switchtest Isolated on IPORT IPORT GBIC Loop passed? Try new HBA GBIC (G) Substitute new...
Page 109 ...continued Try Direct Connect Test (H) Remove GBIC(s) from Isolated External ports not associated Does HBA -- Hub Loopback Test with suspect loop HBA support GBIC passed? External Loopback Test? Substitute new ﬁber Restore original HBA between HBA and hub Reinstall GBIC(s) -- Hub GBIC and from ports not...
Page 110 ...continued Substitute new Isolated ﬁber between External Loopback External Isolated hub-->dev GBIC HBA and device Test device GBIC/ Loopback Test GBIC passed? passed? Restore original Run HBA Reinstall GBIC(s) Restore original hub-dev GBIC External from ports not GBIC/MIA at and substitute new Loopback associated with device...
Page 111: Brocade Troubleshooting
A P P E N D I X Brocade Troubleshooting Copyright 1998, 2000 Brocade Communications Systems, Incorporated. ALL RIGHTS RESERVED. BROCADE, SilkWorm, SilkWorm Express, Fabric OS, QuickLoop, and the BROCADE logo are trademarks or registered trademarks of Brocade Communications Systems, Inc., in the United States and/or in other countries.
Page 112: Introduction
Introduction This appendix provides basic guidelines that you can use to isolate problems found in a Brocade Silkworm® Mamba configuration. It assumes that you have been trained on all the components, such as storage and switch, that make up the configuration.
Page 113 Please refer to the Sun StorEdge FC switch-8 and switch-16 Installation and Configuration Guide, the Sun StorEdge FC switch-8 and switch-16 Release Notes or “Supported Configurations” on page 101 of this guide for details.
Page 114 Features Maximum of 126 devices within a single QL. Ports (looplets) of up to two switches can be included in a QL by Sun (not supported in Mamba phase). Each looplet supports transfer rates of up to 100 MB/sec and multiple, concurrent transfers can occur in multiple looplets.
Page 115 Diagnostic Tools The tools available for troubleshooting include most of the tools that are currently used for Sun StorEdge switch troubleshooting, except for the Sun StorEdge switch GUI (Brocade has its own GUI Interface called WebTools), Sun StorEdge StorTools 4.x and Sun StorEdge RASAgent 2.0.
Page 116 supportShow supportShow runs nearly all commands. Because the supportShow output can be quite lengthy, you should run supportShow and capture the output before you open a service call. Tip – When output is lengthy, as it can be with supportShow, simple cut-and-paste methods in a Solaris terminal window is difficult.
Page 117 switchShow example diag167:admin> switchshow switchName: diag167 switchType: switchState: Online switchRole: Principal switchDomain: switchId: fffc02 switchWwn: 10:00:00:60:69:20:1e:fc switchBeacon: port 0: -- No_Module port 1: -- No_Module port 2: -- No_Module port 3: sw Online L-Port 24 private, 2 phantom port 4: -- No_Module port 5: sw...
Page 118 diagShow example diag167:admin> diagshow Diagnostics Status: Thu Mar 29 14:04:00 2001 port#: diags: OK BAD state: pt3: 123904179 frTx 85600770 frRx LLI_errs. pt5: 1145104 frTx 1201 frRx 24399 LLI_errs. Central Memory OK Total Diag Frames Tx: 1279 Total Diag Frames Rx: 1877 Sun StorEdge Network FC Switch-8 and Switch-16 Field Troubleshooting Guide •...
Page 119 crossPortTest example diag167:admin> crossporttest Running Cross Port Test ..One moment please ... switchName: diag167 switchType: switchState: Testing switchRole: Disabled switchDomain: 2 (unconfirmed) switchId: fffc02 switchWwn: 10:00:00:60:69:20:1e:fc switchBeacon: port 0: -- No_Module Disabled port 1: -- No_Module Disabled port 2: -- No_Module Disabled port...
Page 120 loopPortTest example diag167:admin> loopporttest Configuring normal L-Ports ( pt3 pt5 ) to Cable Loopback L-ports..done. Running Loop Port Test ..Diags: (Q)uit, (C)ontinue, (S)tats, (L)og: s Diagnostics Status: Fri Mar 30 10:17:34 2001 port#: diags: state: pt3: 84 frTx 83 frRx LLI_errs.
Page 121 spinSilk example diag167:admin> spinsilk spinSilk: This command may not be executed on an operational switch. You must first disable the switch using the "switchDisable" command. diag167:admin> switchdisable diag167:admin> spinsilk Running Spin Silk ..... One moment please ... switchName: diag167 switchType: switchState: Testing switchRole:...
Page 122 Note – spinSilk is a test that requires you to disable the switch. In addition, you must insert a single cable that connects two ports together (that is, the cable goes from port 3 to port 7), and uncable the devices, which results in halted access to the devices via this path.
Page 123 Port Differences between Sun StorEdge Ports and Brocade Ports Port Differences TABLE C-1 Sun StorEdge Brocade Function T_Port E_Port Expansion Port. Used for interswitch connections SL_Port L-Port Loop Port. In Sun StorEdge switch, the SL_Port is (segmented loop) Private Loop only. TL_Port L-Port Loop Port.
Page 124 Accessing the Silkworm switch You can access the Silkworm switches in multiple ways: Telnet via a standard RJ-45 Ethernet port The front panel (2800 only) A serial connection (2400 only) The WebTools GUI The serial connection available on the 2400 switch is intended for initial IP address configuration only.
Page 125 Brocade Webtools GUI FIGURE C-1 See the Brocade Web Tools User’s Guide for more information on WebTools usage. Note – The rest of this guide will assume telnet usage. Appendix C Brocade Troubleshooting...
Page 126 Power On Self Tests (POST) When the switch is powered up, it runs a series of POST tests including: Dynamic RAM Test Port Register Test Central Memory Test CMI Connector Test CAM Test Port Loop Back Test POST behaves differently, depending on boot method. A power-cycle (power-off and power-on) is considered a cold boot.
Page 127: Removing Power
Removing Power Caution – Error messages are stored in RAM and are lost when power is removed from the switch. Capture and view the error log output and note any error messages before removing power. Status and Activity Indicators Front Panel LED Port Indicators Front Panel LEDs Definition No light showing...
Page 128 Initialization Steps: At power-on or reset, the following steps occur. 1. Preliminary POST diagnostics 2. VxWorks operating system initialization 3. Hardware initialization (resets, internal addresses assigned to ASICs, serial port initialized, front panel initialized) 4. Full POST 5. Universal Port Configuration 6.
Page 129: Troubleshooting Overview
Troubleshooting Overview This section highlights the troubleshooting methodology differences between the current Brocade switch in a Mamba configuration. Brocade and Sun StorEdge StorTools 4.x Note – The current version of Sun StorEdge StorTools ( 4.x) cannot recognize or utilize the Brocade switch in diagnostic routines. The features of the StorEdge switch and the Sun StorEdge StorTools test switchtest are not available in a configuration with a Brocade switch.
Page 130 Methodology In order to effectively isolate and diagnose a failing component in a Brocade Mamba configuration, certain broad steps can be outlined to assist you in pinpointing the source of the problem. In each step, tools or tests that may help you are noted. 1.
Page 131 Troubleshooting Case Study The following case study is included to illustrate a practical application of the steps outlined above. Note, however, that this application is not the only way to approach the problem. Knowledge and training on all the components in the SAN are a prerequisite before attempting the procedures below.
Page 132 In this diagram, Loop A is connected to one switch and Loop B is connected to the other switch. The server has two HBAs, with one port on each HBA connecting to each switch. Vxdmp is used to control the multi-pathing. Troubleshooting the Problem The path /pci@1f,2000/pci@1/SUNW,qlc@5/fp@0,0/ssd@w220000203719f7e0,0 are posting errors.
Page 133: Troubleshooting
# luxadm -e dump_map /devices/pci@1f,2000/pci@1/SUNW,qlc@5/fp@0,0:devctl Pos AL_PA ID Hard_Addr Port WWN Node WWN Type 22000020373cc1ac 20000020373cc1ac 0x0 (Disk device) 22000020374507de 20000020374507de 0x0 (Disk device) 22000020374504e2 20000020374504e2 0x0 (Disk device) 2200002037450d3a 2000002037450d3a 0x0 (Disk device) 22000020373cc091 20000020373cc091 0x0 (Disk device) 22000020373ccb07 20000020373ccb07 0x0 (Disk device) 220000203719f7e0 200000203719f7e0 0x0 (Disk device)
Page 134 # vxdmpadm listctlr all CTLR-NAME DA-TYPE STATE DA-SNO ============================================== ctlr0 OTHER ENABLED OTHER_DISKS ctlr0=/pci@1f,4000/scsi@3 ctlr1 SEAGATE ENABLED SEAGATE_DISKS ctlr1=/pci@1f,4000/pci@4/SUNW,qlc@4/fp@0,0 ctlr2 SEAGATE ENABLED SEAGATE_DISKS ctlr2=/pci@1f,2000/pci@1/SUNW,qlc@5/fp@0,0 # vxdmpadm disable ctlr=/pci@1f,2000/pci@1/SUNW,qlc@5/fp@0,0 5. Watch /var/adm/messages to verify that the path is disabled. Mar 28 12:18:23 diag233.Central.Sun.COM vxdmp: [ID 969440 kern.notice] NOTICE: vxvm:vxdmp: disabled controller /pci@1f,2000/pci@1/SUNW,qlc@5/fp@0,0 connected to disk array SEAGATE_DISKS # vxdmpadm listctlr all...
Page 135 7. If there is no customer documentation, or if you have no immediate access to the hardware, you can run the nsShow command on the Brocade switch. This command dumps the Name Server information with each device’s WWN noted, and to what port the device is connected. 021501;...
Page 136 Note – Brocade’s diagnostics mark a port BAD on error. 9. In order to continue running tests on Pt5, clear the current error condition with a diagClearError <port #>. Diags: (Q)uit, (C)ontinue, (S)tats, (L)og: q FAILED. Configuring Loopback L-port(s) back to normal L-port(s)..done. diag167:admin>...
Page 137 Note – For this test, a loopback connector is inserted into the HBA and the test is run with most of the options except External Loopback Test, which is turned off to speed up the execution time. You can also run this test from the Sun StorEdge StorTools GUI. # sparcv9/qlctest -v -o dev=qlc-3,run_connect=Yes,checksum=Disable,selftest= Disable,mbox=Disable,ilb_10=Disable,ilb=Disable,elb=Enable,icnt=1000,lbfpattern= 0x7e7e7e7e...
Page 138 diag167:admin> switchshow switchName: diag167 switchType: switchState: Online switchRole: Principal switchDomain: switchId: fffc02 switchWwn: 10:00:00:60:69:20:1e:fc switchBeacon: port 0: -- No_Module port 1: -- No_Module port 2: -- No_Module port 3: sw Online L-Port 24 private, 1 phantom port 4: -- No_Module port 5: sw Online...
Page 139 diag167:admin> diagclearerror 5 0x10f587a0 (tShell): Mar 28 14:46:10 Error DIAG-CLEAR_ERR, 3, Pt5 (Lm1) Diagnostics Error Cleared Err# 0001 diag167:admin> crossporttest 5,1 Running Cross Port Test ..passed. — The test now passed with a new GBIC. 16. Recable the link and retest the entire path. When recabling the HBA, you may need to send a LIP to force the HBA to "wake up"...
Page 140 diag167:admin> loopporttest 100000,5 Configuring L-port 5 to Cable Loopback Port..done. Running Loop Port Test ..Diags: (Q)uit, (C)ontinue, (S)tats, (L)og: s Diagnostics Status: Wed Mar 28 14:52:47 2001 port#: diags: state: pt3: 574893 frTx 15240 frRx LLI_errs. pt5: 160 frTx 160 frRx LLI_errs.
Page 141 Webtools Performance Page FIGURE C-3 Appendix C Brocade Troubleshooting...
Page 142 Sun StorEdge Network FC Switch-8 and Switch-16 Field Troubleshooting Guide • April, 2001...
Page 143: Glossary
Glossary This glossary contains a Fibre Channel reference model, definitions for terms, and examples of error messages used in Fibre Channel Arbitrated Loop (FC-AL). Fibre Channel Layers device drivers and applications upper level protocols, e.g. SCSI, IP FC-4 FC -3 common services FC-2 framing protocol and flow control...
Page 144 Cyclic Redundancy A method of detecting small changes in blocks of data. Check (CRC) E_Port An expansion port connecting two switches together. FL_Port On a Fibre Channel switch, a port that supports Arbitrated Loop devices. A fibre channel F_Port On a fibre channel switch, a port that supports an N_Port. port in a point-to-point or fabric connection.
Page 145 8b/10b encoding An encoding scheme that converts an 8-bit byte into one of two possible 10-bit characters (negative or positive). Glossary-133...
Page 146 Glossary-134 Sun StorEdge Network FC Switch-8 and Switch-16 Troubleshooting Guide—April 2001...
Page 147: Index
Index maximum length supported, 4 capture utility, 82 AC input power connector and fuses, 22 configuration multi-host, 13 adapter PIC single ﬁbre channel network, 4 configuration guidelines, 5 adapter ports configurations connection of, 2 hardware supported, 4 supported, 2 arrays conﬁguration guidelines, 5 connector maximum number possible per zone, 5...
Page 148 partner pairs, 12 firmware single host connected to one Sun StorEdge A5200 for Mamba conﬁguration, 74 controller module, 7 flowchart single host connected to one Sun StorEdge T3 isolation of SAN components, 89 partner pair, 8 frequently-asked questions (FAQ), 73 single host connection to one Sun StorEdge front panel A3500-FC controller module, 7...
Page 149 4.x, part number 806-6235-10, 41 Sun StorEdge T3 Disk Tray Administrator’s multi-host configuration, 13 Guide, v Sun StorEdge T3 Disk Tray Installations, Operations and Service Manual, v Sun Switch Management Installer’s/User’s part numbers Manual, 24 hardware supported, 4 patches for Mamba conﬁguration, 74 tools used to track, 76 SAN components patches necessary for switch support, 5...
Page 150 window functional test of switch, 57 table port display, 34 arrays, zones, and initiators, 6 switch GUI, 58 dynamic addition to a zone, 6 web gui, 38 test a5ksestest, 54, 59 functional a5ktest, 47 switchtest, 57, 60, 62 test mode switch zoning force PROM, 25 conﬁguration, 3, 5...

This manual is also suitable for:

Storedge fc switch-16

Sun Microsystems StorEdge FC Switch-8 Troubleshooting Manual

The Sun Storedge Network FC Switch-8 and Switch-16 Troubleshooting Guide

Mamba Field Troubleshooting Guide FAQ

Isolation of SAN Components Flowchart

Brocade Troubleshooting

Glossary

Index

Quick Links

Troubleshooting

Need help?

Questions and answers

Subscribe to Our Youtube Channel

Related Manuals for Sun Microsystems StorEdge FC Switch-8

Summary of Contents for Sun Microsystems StorEdge FC Switch-8

This manual is also suitable for:

Table of Contents