HP 3PAR StoreServ 7000 Troubleshooting Manual
  1. Manuals
  2. Brands
  3. HP Manuals
  4. Storage
  5. 3PAR StoreServ 7000
  6. Troubleshooting manual

HP 3PAR StoreServ 7000 Troubleshooting Manual

Hide thumbs Also See for 3PAR StoreServ 7000:
Table of Contents

Advertisement

Quick Links

See also: Administrator's Manual , Service Manual
HP 3PAR StoreServ 7000 Storage
Troubleshooting Guide
Abstract
This guide is for system administrators and experienced users who are familiar with HP 3PAR StoreServ 7000 Storage systems,
understand the operating systems they are using, and have a working knowledge of RAID. This guide provides information on
storage system alerts, components, LEDs, and power procedures.
HP Part Number: QL226-96512
Published: November 2012

Advertisement

Table of Contents
loading

Related Manuals for HP 3PAR StoreServ 7000

Summary of Contents for HP 3PAR StoreServ 7000

  • Page 1 Troubleshooting Guide Abstract This guide is for system administrators and experienced users who are familiar with HP 3PAR StoreServ 7000 Storage systems, understand the operating systems they are using, and have a working knowledge of RAID. This guide provides information on storage system alerts, components, LEDs, and power procedures.
  • Page 2 The information contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein.

  • Page 3: Table Of Contents

    Contents 1 Identifying Storage System Components............6 Understanding Component Numbering..................6 Drive Enclosures........................6 Disk Drive Numbering.....................6 Controller Nodes.........................7 PCIe Slots and Ports Numbering..................8 I/O Modules ........................9 Power Cooling Modules......................9 Power Distribution Units......................10 Service Processor Placement....................10 2 Understanding LED Indicator Status.............11 Enclosure LEDs........................11 Bezels LEDs........................11 Disk Drive LEDs........................12 Storage System Component LEDs....................13...
  • Page 4 Date..........................30 Format of Possible Date Exception Messages..............30 Date Example.......................30 Date Suggested Action....................30 LD............................31 Format of Possible LD Exception Messages................31 LD Example 1.......................31 LD Suggested Action 1....................31 LD Example 2.......................32 LD Suggested Action 2....................32 LD Example 3.......................32 LD Suggested Action 3....................32 LD Example 4.......................33 LD Suggested Action 4....................33 License..........................33...
  • Page 5 VV Suggested Action.....................48 Troubleshooting Storage System Setup..................48 Storage System Setup Wizard Errors..................48 Collecting SmartStart Log Files.....................54 Collecting Service Processor Log Files...................54 Contacting HP Support about System Setup................54 6 Support and Other Resources..............56 Contacting HP........................56 HP 3PAR documentation......................56 Typographic conventions......................59 HP 3PAR branding information....................59 7 Documentation feedback................60...

  • Page 6: Identifying Storage System Components

    The storage system can include two types of drive and node enclosures: The HP M6710 Drive Enclosure (2U24) holds up to 24, 2.5 inch small form factor (SFF) SAS disk drives arranged vertically in a single row at the front of the enclosure. The back of the enclosure includes two 580W PCMs and two I/O modules.

  • Page 7: Controller Nodes

    0 and 1 on the bottom and 2 and 3 on the top. The HP 3PAR StoreServ 7200 Storage system includes two nodes. The HP 3PAR StoreServ 7400 Storage system can include two nodes or four nodes.

  • Page 8: Pcie Slots And Ports Numbering

    Figure 4 HP 3PAR StoreServ Four-node Configuration Storage numbering PCIe Slots and Ports Numbering This section shows default configurations for the HP 3PAR StoreServ 7000 Storage systems: Table 1 Number of Cards per System Expansion cards Nodes 0 & 1 Nodes 2 &...

  • Page 9: I/O Modules

    PCMs per enclosure numbered 0 and 1, from left to right. Figure 6 PCM numbering NOTE: In the HP M6720 Drive Enclosure (4U24), there are two PCMs that are diagonal from each other; the remaining PCM slots are filled with blanks. Understanding Component Numbering...

  • Page 10: Power Distribution Units

    Depending on configuration, PDUs can also be mounted vertically. Service Processor Placement The HP 3PAR StoreServ 7000 Storage system may include a physical service processor (SP) or may use a virtual SP (VSP). If your configuration includes a physical SP, it will be located at the bottom of the rack under the enclosures and above the PDUs.

  • Page 11: Understanding Led Indicator Status

    2 Understanding LED Indicator Status Storage system components have LEDs to indicate status of the hardware and whether or not it is functioning properly. These indicators help diagnose basic hardware problems. You can quickly identify hardware problems by examining the LEDs on all components using the tables and illustrations in this chapter.

  • Page 12: Disk Drive Leds

    Disk Drive LEDs These LEDs are located on the front of the disk drives. Figure 9 Disk drive LEDs Figure 10 Disk drive LEDs Table 4 Disk drive LEDs Callout Appearance Indicates Activity Green On – Normal operation Flashing – Activity Fault Amber On –...

  • Page 13: Storage System Component Leds

    Table 4 Disk drive LEDs (continued) Callout Appearance Indicates Fault LEDs at the rear of the enclosure also blink). Fault LEDs for failed disk drives do not blink. Storage System Component LEDs The storage system includes the following components in the enclosure at the rear of the system. Power Cooling Module LEDs The PCM has four or six LEDs, depending on PCM;...

  • Page 14: I/O Modules Leds

    Table 5 PCM LEDs (continued) Callout Appearance Indicates Flashing – Soft (recoverable) Fault DC Output Fail Amber On – No AC Power or Fault or Out of Tolerance Flashing – Firmware Download Fan Fail Amber On – PCM Fail or PCM Fault Flashing –...

  • Page 15: Controller Node And Internal Component Leds

    Controller Node and Internal Component LEDs Controller nodes have the following LEDs: NOTE: Issue the locatenode command to flash the hotplug LED blue. Figure 14 Controller Node LEDs Table 7 Controller Node LEDs Callout Appearance Indicates Status Green Node status Good On –...

  • Page 16: Fc Port Leds

    Figure 15 Ethernet LEDs Table 8 Ethernet LEDs Callout Appearance Indicates Link Up Green On – 1 Gbe Link Speed Amber On – 100 Mbit Link Off – No link established or 10 Mbit Link Activity Green On – No Link activity Off –...

  • Page 17: Sas Port Leds

    Table 9 FC port LEDs Callout Appearance Indicates FP- 1 /FP-2 1 and 2 No light Wake up failure (dead device) or power is not applied Port speed (Amber) 1 Amber light off Not connected Amber (3 blinks Connected at 4 Gb/sec. per second) Amber (4 blinks Connected at 8 Gb/sec.

  • Page 18: Fibre Channel Adapter Port Leds

    Figure 18 Interconnect port LEDs Table 1 1 Interconnect port LEDs Callout Appearance Indicates Status Green On – Link established Off – Link not yet established Fault Amber On – Failed to establish link connection Off – No errors currently on link Flashing –...

  • Page 19: Converged Network Adapter Port Leds

    Table 12 Fibre Channel adapter port LEDs (continued) Callout Appearance Indicates 4 fast blinks – Connected at 8 GB/sec. Link status Green On – Normal/Connected - link up Flashing – Link down or not connected Converged Network Adapter Port LEDs The CNA in the controller node includes two ports;...

  • Page 20: Powering Off/On The Storage System

    The command blinks all node and drive enclosure LEDs. Before you begin, use either SPmaint or SPOCC to shut down and power off the system (see the section “Service Processor Onsite Customer Care” in the HP 3PAR StoreServ 7000 Storage Service Guide):...

  • Page 21: Alerts

    At the HP Storage Systems Guided Troubleshooting website, follow the link for your product. At the bottom of the HP 3PAR product page, click the link for HP 3PAR Alert Messages. At the bottom of the Alert Messages page, choose the correct message code series based on the first four characters of the alert.
  • Page 22 The next page shows the message type based on the message code selected and provides a link to the suggested action. Follow the link. On the suggested actions page, scroll through the list to find the message state listed in the alert message.

  • Page 23: Troubleshooting

    5 Troubleshooting The HP 3PAR OS CLI checkhealth command checks and displays the status of storage system hardware and software components. For example, the checkhealth command can check for unresolved system alerts, display issues with hardware components, or display information about virtual volumes that are not optimal.

  • Page 24: Troubleshooting Storage System Components

    The following information is included when you use the -detail option: Component ----Identifier---- -----------Description------- Alert sw_port:1:3:1 Port 1:3:1 Degraded (Target Mode Port Went Offline) Alert sw_port:0:3:1 Port 0:3:1 Degraded (Target Mode Port Went Offline) Alert sw_sysmgr Total available FC raw space has reached threshold of 800G (2G remaining out of 544G total) Alert sw_sysmgr Total FC raw space usage at 307G (above 50% of total 544G)

  • Page 25: Alert Suggested Action

    Alert Suggested Action View the full Alert output using the IMC (GUI) or the showalert -d CLI command. Cage Displays drive cage conditions that are not optimal and reports exceptions if any of the following do not have normal states: Ports Drive magazine states (DC1, DC2, &...

  • Page 26: Cage Example 2

    Link_Speed 0Gbps 4Gbps ----------------------------------SFP Info----------------------------------- FCAL SFP -State- --Manufacturer-- MaxSpeed(Gbps) TXDisable TXFault RXLoss DDM 0 OK FINISAR CORP. 4.1 No 1 OK FINISAR CORP. 4.1 No Interface Board Info FCAL0 FCAL1 Link A RXLEDs Link A TXLEDs Green Link B RXLEDs Green Link B TXLEDs Green...

  • Page 27: Cage Suggested Action 2

    Cage cage:1 Power supply 0's AC state is Failed Cage cage:1 Power supply 2 is Off Cage Suggested Action 2 A cage power supply or power supply fan is failed, is missing input AC power, or the switch is turned OFF. The showcage -d cageX and showalert commands provide more detail. cli% showcage -d cage1 Id Name LoopA Pos.A LoopB Pos.B Drives Temp...

  • Page 28: Cage Example 4

    Link B RXLEDs Green Link B TXLEDs Green LED(Loop_Split) LEDS(system,hotplug) Green,Off Green,Amber -----------Midplane Info----------- Firmware_status Current Product_Rev 2.37 State Normal Op Loop_Split VendorId,ProductId 3PARdata,DC2 Unique_ID 10320300000AD000 cli% showpd -s Id CagePos Type -State-- -----Detailed_State------ 20 1:0:0 degraded disabled_B_port,servicing 21 1:0:1 degraded disabled_B_port,servicing 22 1:0:2 degraded disabled_B_port,servicing...

  • Page 29: Cage Example 5

    Loop_Split VendorId,ProductId 3PARdata,DC2 Unique_ID 10320300000AD100 cli% showfirmwaredb Vendor Prod_rev Dev_Id Fw_status Cage_type Firmware_File 3PARDATA [2.37] Current /opt...dc2/lbod_fw.bin-2.37 Cage Example 5 Component -Identifier- ------------Description------------ Cage cage:4 Interface Card 0, SFP 0 is unqualified Cage Suggested Action 5 In this example, a 2 Gb/sec SFP was installed in a 4 Gb/sec drive cage (DC4), and the 2 Gb SFP is not qualified for use in this drive cage.

  • Page 30: Date

    --------Cage 4 FCAL 1 SFP 1-------- Cage ID Fcal ID SFP ID State Manufacturer FINISAR CORP. Part Number FTLF8524P2BNV Serial Number PF52GRF Revision MaxSpeed(Gbps) : Qualified TX Disable TX Fault RX Loss RX Power Low DDM Support Date Checks the date and time on all nodes and reports an error if they are not the same. Format of Possible Date Exception Messages Date -- "Date is not the same on all nodes"...

  • Page 31: Format Of Possible Ld Exception Messages

    Displays Logical Disks (LDs) that are not optimal: Checks for preserved LDs Checks that current and created availability are the same Checks for owner and backup Checks that preserved data space (pdsld's) is the same as total data cache Checks size and number of logging LDs Format of Possible LD Exception Messages LD ld:<ldname>...

  • Page 32: Ld Example 2

    LD Example 2 Component -------Description-------- Qty LDs in write through mode Component -Identifier-- --------Description--------- ld:Ten.usr.12 LD is in write-through mode LD Suggested Action 2 Examine the identified LDs using CLI commands such as showld, showld d, showldch, and showpd for any failed or missing disks. Write-through mode (WThru) indicates that host I/O operations must be written through to the disk before the host I/O command is acknowledged.

  • Page 33: Ld Example 4

    availability, but it currently has chunklet (disk) level availability (that is., the chunklets are on the same disk). cli% showld -d R1.usr.0 Id Name CPG RAID Own SizeMB RSizeMB RowSz StepKB SetSz Refcnt Avail CAvail 32 R1.usr.0 --- 1 0/1/3/2 0 cage cli% showldch R1.usr.0 Ldch Row Set PdPos Pdid Pdch...

  • Page 34: License Suggested Action

    License Suggested Action If desired, request a new or updated license from your Sales Engineer. Network Displays Ethernet issues for the Administrative and Remote Copy over IP (RCIP) networks that have been logged in the previous 24–hour sampling window. Reports if the storage system has fewer than two nodes with working admin Ethernet connections.

  • Page 35: Node

    NOTE: The error counters shown by shownet and shownet -d cannot be cleared except by rebooting a controller node. Because checkhealth is showing network counters from a history log, checkhealth stops reporting the issue if there is no increase in error in the next log entry. shownet -d IP Address: 192.168.56.209 Netmask 255.255.255.0...

  • Page 36: Node Suggested Action 1

    Node node:1 Power supply 0 AC state is Failed Node node:1 Power supply 0 DC state is Failed Node Suggested Action 1 Examine the states of the power supplies with commands such as shownode, shownode -s, shownode -ps, and the like. Turn on or replace the failed power supply. NOTE: In the example below, the battery state is considered Degraded because the power supply is Failed;...

  • Page 37: Node Example 3

    cli% showbattery Node PS Bat Serial -State-- ChrgLvl(%) -ExpDate-- Expired Testing 0 100A300B OK 100 07/01/2011 No 0 12345310 Failed 0 04/07/2011 No Node Example 3 Component -Identifier- --------------Description---------------- Node node:3 Node:3, Power Supply:1, Battery:0 has not been tested within the last 30 days Node Suggested Action 3 The indicated battery has not been tested in the past 30 days.

  • Page 38: Format Of Possible Pd Exception Messages

    Format of Possible PD Exception Messages PD disk:<pdid> "Degraded States: <showpd -s -degraded"> PD disk:<pdid> "Failed States: <showpd -s -failed"> PD -- "There is an imbalance of active PD ports" PD -- "Sparing algorithm is not set" PD disk:<pdid> "Disk is experiencing a high level of I/O per second: <iops>" PD -- There is at least one active servicemag operation in progress The following checks are performed when the -svc option is used, or on 7400/7200 hardware: PD File: <filename>...

  • Page 39: Pd Example 2

    Fibre Channel Info PortA0 PortB0 PortA1 PortB1 Link_Speed 2Gbps 0Gbps ----------------------------------SFP Info----------------------------------- FCAL SFP -State- --Manufacturer-- MaxSpeed(Gbps) TXDisable TXFault RXLoss DDM 0 OK SIGMA-LINKS 2.1 No 1 OK SIGMA-LINKS 2.1 No Interface Board Info FCAL0 FCAL1 Link A RXLEDs Green Link A TXLEDs Green Link B RXLEDs...

  • Page 40: Pd Example 3

    48 3:0:0 degraded 2:0:4 3:0:4\missing 2/- 49 3:0:1 degraded 2:0:4 3:0:4\missing 2/- 50 3:0:2 degraded 2:0:4 3:0:4\missing 2/- 51 3:0:3 degraded 2:0:4 3:0:4\missing 2/- cli% showcage -d cage3 Id Name LoopA Pos.A LoopB Pos.B Drives Temp RevA RevB Model Side 3 cage3 2:0:4 0 --- 32 29-41 2.37 2.37 DC2...

  • Page 41: Pd Example 4

    using statistical monitoring commands/utilities such as statpd, the OS IMC (GUI) and System Reporter. The following example reports disks whose total I/O is 150/sec or more. cli% statpd -filt curs,t,iops,150 14:51:49 11/03/09 r/w I/O per second KBytes per sec ... Idle % Port Max ...

  • Page 42: Pd Example 6

    PD Disk:32 ST3400755FC PD for cage type DC3 in cage position 2:0:0 is missing from the firmware database PD Suggested Action 6 Check the release notes for mandatory updates and patches to the HP 3PAR OS version that is installed and install as needed to support this PD in this cage. Port...
  • Page 43 or contaminated FC connection, such as a cable. An alert should identify the condition, such as the following: Port 0:0:2, SFP Degraded (Receiver Power Low: Check FC Cable) Check SFP statistics using CLI commands such as showport -sfp, showport -sfp -ddm, showcage, etc.

  • Page 44: Port Example 2

    Port Example 2 Component -Description- Qty Port Missing SFPs Component -Identifier- -Description-- Port port:0:3:1 SFP is missing Port Suggested Action 2 FC node-ports that normally contain SFPs will report an error if the SFP has been removed. The condition can be checked using the showport -sfp command. In this example, the SFP in 0:3:1 has been removed from the adapter: cli% showport -sfp N:S:P -State- -Manufacturer- MaxSpeed(Gbps) TXDisable TXFault RXLoss DDM...

  • Page 45: Port Example 5

    Port Example 5 Component ------------Description------------ Qty Port Ports with mismatched mode and type Component -Identifier- ------Description------- Port port:2:0:3 Mismatched mode and type Port Suggested Action 5 This output indicates that the port's mode, such as an initiator or target, is not correct for the connection type, such as disk, host, iscsi or rcfc.

  • Page 46: Rc Suggested Action

    RC Suggested Action Perform remote copy troubleshooting such as checking the physical links between the storage system, and using CLI commands such as showrcopy, showrcopy -d, showport -rcip, showport -rcfc, shownet -d, controlport rcip ping, etc. SNMP Displays issues with SNMP. Attempts the showsnmpmgr command and reports errors if the CLI returns an error.

  • Page 47: Vlun

    manually removed with the IMC (GUI) or CLI with removealert or setalert ack. To display system-initiated tasks, use showtask -all. cli% showtask -d 6313 Id Type Name Status Phase Step 6313 background_command upgradecage -a -f failed Detailed status is as follows: 2010-10-22 10:35:36 PDT Created task.

  • Page 48: Format Of Possible Vv Exception Messages

    “Collecting SmartStart Log Files” (page 54) Collect the SP log files. See “Collecting Service Processor Log Files” (page 54) Contact HP support and request support for your StoreServ 7000 Storage product. See “Contacting HP Support about System Setup” (page 54) Storage System Setup Wizard Errors You may see the following error messages in the Storage System Setup Wizard.
  • Page 49 54). Errors that appear on the Enter System to Setup page "Unable to execute the command. All required data was not sent to the SP server. Contact HP support for help." This message displays as an inline error on the bottom of the wizard page.
  • Page 50 {0} will be the version of the TPD package that the user must install so that the SP will work with the storage system. "The SP does not have an HP 3PAR OS version installed. Use SPOCC to install an HP 3PAR OS package."...
  • Page 51 “The storage system found an error while checking cage health. The firmware upgrade succeeded, but cage {0} has not come back. Contact HP support for help." This error message displays in a dialog box with Retry and Cancel buttons. This error might occur after the drive cages have had a firmware upgrade.
  • Page 52 This error message displays in a dialog box with Retry and Cancel buttons. This error might occur after the drive cages have had a firmware upgrade. {0} will be the name of the cage with the problem. Contact HP Support. For information about contacting HP Support, see “Contacting HP Support about System Setup” (page 54).
  • Page 53 This message displays in a dialog box. The error occurs if the storage system detects that the defined IPv4 gateway address could not be reached. Click Back and specify a valid IPv4 gateway address. If the error persists, contact HP Support. For information about contacting HP Support, see “Contacting HP Support about System Setup”...

  • Page 54: Collecting Smartstart Log Files

    Click Back and specify a valid time zone. Collecting SmartStart Log Files To collect the SmartStart log files for HP support, zip all the files in this folder:C:\Users\ <username>\SmartStart\log NOTE: You can continue to access the SmartStart log files in the Users folder after you have removed SmartStart from your system.
  • Page 55 Product model names and numbers Technical support registration number (if applicable) Product serial numbers Error messages Operating system type and revision level Detailed questions When you contact HP, specify that you are requesting support for your StoreServ 7000 Storage product. Troubleshooting Storage System Setup...

  • Page 56: Support And Other Resources

    6 Support and Other Resources Contacting HP For worldwide technical support information, see the HP support website: http://www.hp.com/support Before contacting HP, collect the following information: Product model names and numbers Technical support registration number (if applicable) Product serial numbers Error messages...
  • Page 57 Configuring the Secure Service Custodian server in order to HP 3PAR Secure Service Custodian Configuration Utility monitor and control HP 3PAR storage systems Reference Using the CLI to configure and manage HP 3PAR Remote HP 3PAR Remote Copy Software User’s Guide Copy Updating HP 3PAR operating systems...
  • Page 58 HP 3PAR StoreServ 10000 Storage Physical Planning Manual HP 3PAR StoreServ 10000 Storage Third-Party Rack Physical Planning Manual Installing and maintaining HP 3PAR 7200 and 7400 storage systems Installing 7200 and 7400 storage systems and initializing HP 3PAR StoreServ 7000 Storage Installation Guide the Service Processor HP 3PAR StoreServ 7000 Storage SmartStart Software User’s Guide...

  • Page 59: Typographic Conventions

    HP. HP 3PAR branding information The server previously referred to as the "InServ" is now referred to as the "HP 3PAR StoreServ Storage system." The operating system previously referred to as the "InForm OS" is now referred to as the "HP 3PAR OS."...

  • Page 60: Documentation Feedback

    7 Documentation feedback HP is committed to providing documentation that meets your needs. To help us improve the documentation, send any errors, suggestions, or comments to Documentation Feedback (docsfeedback@hp.com). Include the document title and part number, version number, or the URL when submitting your feedback.

Table of Contents