IBM Elastic Storage System 3500 Service Manual
  1. Manuals
  2. Brands
  3. IBM Manuals
  4. Storage
  5. Elastic Storage System 3500
  6. Service manual
IBM Elastic Storage System 3500 Service Manual

IBM Elastic Storage System 3500 Service Manual

Hide thumbs
Table of Contents

Advertisement

Quick Links

Download this manual
IBM Elastic Storage System 3500
6.1.4
Service Guide
IBM
SC27-9876-00

Advertisement

Table of Contents
loading

Related Manuals for IBM Elastic Storage System 3500

Summary of Contents for IBM Elastic Storage System 3500

  • Page 1 IBM Elastic Storage System 3500 6.1.4 Service Guide SC27-9876-00...
  • Page 2 IBM welcomes your comments; see the topic “How to submit your comments” on page x. When you send information to IBM, you grant IBM a nonexclusive right to use or distribute the information in any way it believes appropriate without incurring any obligation to you.

  • Page 3: Table Of Contents

    Contents Figures........................v Tables......................... vii About this information..................ix Who should read this information.......................ix IBM Elastic Storage System information units................... ix Related information.............................ix Conventions used in this information......................ix How to submit your comments........................x Chapter 1. Servicing (customer tasks)..............1 Removing and replacing an NVMe drive...................... 1 Removing and replacing a drive filler......................
  • Page 4 iv  ...

  • Page 5: Figures

    Figures 1. Displaying enclosure front view........................1 2. Displaying drive carrier assembly......................... 2 3. Displaying NVMe drive replacement part..................... 2 4. Displaying physical disks state........................3 5. Unlocking the drive and release latch......................4 6. Removing NVMe drive........................... 4 7. Releasing catch..............................5 8.
  • Page 6 24. Replacing right ear cap..........................16 25. Pressing release tab..........................16 26. Securing the enclosure..........................17 27. Displaying IBM brand label location......................17 28. 5141 - FN2 adapter slots..........................24 29. 5141 - FN2 adapter slot locations......................26 vi  ...
  • Page 7 Tables 1. Conventions..............................x 2. IBM Elastic Storage System 3500 power module status LEDs..............11 3. Adapter options............................23 4. Adapter placement priority......................... 24 5. CRU Part Numbers............................33 6. FRU Part Numbers............................33   vii...
  • Page 8 viii  ...

  • Page 9: About This Information

    • IBM POWER9 servers, see https://www.ibm.com/docs/en/ess/6.1.0_ent?topic=guide-5105-22e- reference-information. For the latest support information about IBM Spectrum Scale RAID, see the IBM Spectrum Scale RAID FAQ in IBM Documentation. Conventions used in this information Table 1 on page x describes the typographic conventions used in this information. UNIX file name conventions are used throughout this information.

  • Page 10: How To Submit Your Comments

    In the left margin of the document, vertical lines indicate technical changes to the information. How to submit your comments To contact the IBM Spectrum Scale development organization, send your comments to the following email address: scale@us.ibm.com x  IBM Elastic Storage System 3500: Service Guide...

  • Page 11: Chapter 1. Servicing (Customer Tasks)

    When you replace this part, you must follow recommended procedures for handling electrostatic discharge (ESD) sensitive devices. IBM Elastic Storage System 3500 enclosure supports up to 24 U.2 dual ported NVMe SSDs and the drive slots are numbered from 1 through 24 as shown in the following figure.

  • Page 12: Displaying Drive Carrier Assembly

    Alternatively, locate unhealthy drives in the management GUI, either from the Storage> Physical Disks page as shown in Figure 4 on page 3, or from the list of events that are available under the Monitoring > Events page. 2  IBM Elastic Storage System 3500: Service Guide...
  • Page 13 2. Prepare the failed drive for replacement. In an IBM Elastic Storage System 3500 enclosure, a faulty drive has amber fault LED ON (non-flashing) on the driver carrier. At the end of this step, confirm that the amber LED in the ON state for the drive to be removed.

  • Page 14: Unlocking The Drive And Release Latch

    Figure 5. Unlocking the drive and release latch b) Lift the handle and slide the drive out of the enclosure, as shown in the following figure. Figure 6. Removing NVMe drive 4. Insert the FRU replacement drive into the slot. 4  IBM Elastic Storage System 3500: Service Guide...

  • Page 15: Releasing Catch

    a) Press the release catch on the drive carrier in the direction of the arrow to open the handle as identified in Figure 7 on page 5. The cam releases from the locked position. Figure 7. Releasing catch b) Grab the frame beneath the touch point and gently push the drive carrier assembly into the drive bay until the carrier handle engages.

  • Page 16: Removing And Replacing A Drive Filler

    Use management GUI or the mmvdisk command to check and confirm that no failed disk is reported. Removing and replacing a drive filler Refer to the service procedure to remove a drive slot filler in an IBM Elastic Storage System 3500 enclosure and replace it with a new one from stock.

  • Page 17: Displaying Drive Carrier Assembly

    Figure 9. Displaying drive carrier assembly Service personnel need fully assembled drive carrier assembly housing a drive filler to complete this service procedure: • Part number - 01LL731 (Drive Filler) Figure 10. Displaying drive filler replacement kit Refer to the following steps to remove and replace a drive filler: 1.

  • Page 18: Releasing Catch

    Hold the drive blank the correct way up, as shown in Figure 14 on page 8. Figure 14. Holding the drive blank c) Ensure that the carrier handle is in the fully open position. d) Grab the drive filler as shown in Figure 14 on page 8. 8  IBM Elastic Storage System 3500: Service Guide...

  • Page 19: Removing And Replacing A Power Module

    Figure 15. Inserting the drive filler Removing and replacing a power module Refer to the service procedure to remove and replace a power module of an IBM Elastic Storage System 3500 enclosure. The following steps are the high-level flow of the procedure: 1.

  • Page 20: Displaying Power Modules

    Take precautions when performing all other types of maintenance. Use the following replacement parts to service a faulty power module in an IBM Elastic Storage System 3500 enclosure: •...
  • Page 21 2. Confirm the faulty power module location with the help of the power module LED. Each power module of an IBM Elastic Storage System 3500 enclosure has a bi-color LED as shown in Figure 17 on page 11. The LED indicates the status of the power module, as described in Table 2 on page 11.

  • Page 22: Displaying Power Module Jacks

    6. Reconnect power cable to the replaced power module. Insert the power cord into the power module jack. 7. Follow the appropriate steps to prepare for a service action to return the CMA arm. 12  IBM Elastic Storage System 3500: Service Guide...

  • Page 23: Removing And Replacing A Bezel Kit

    Installing chapter in ESS 3500 Hardware Planning and Installation Guide. Removing and replacing a bezel kit Refer to the service procedure to remove and replace both left and right ear caps in an IBM Elastic Storage System 3500 enclosure. The following steps are the high-level flow of the procedure: 1.
  • Page 24 Figure 21 on page 14, by using a Phillips driver. Figure 21. Displaying left ear cap front view b) Next, install the left ear cap part (01LL677) as shown in Figure 22 on page 15. 14  IBM Elastic Storage System 3500: Service Guide...
  • Page 25 Figure 22. Replacing left ear cap 4. Remove and replace the right ear cap. a) Remove the right ear cap by removing the two screws from the back side of the right flange by using a Phillips driver. Figure 23. Displaying right ear cap front view b) Next, install the right ear cap part (01LL678) as shown in Figure 24 on page 16.
  • Page 26 6. Secure the enclosure in the rack. Secure the enclosure in the rack. Tighten the left and right shipping screws on each side of the enclosure as demonstrated in Figure 26 on page 17. 16  IBM Elastic Storage System 3500: Service Guide...

  • Page 27: Miscellaneous Equipment Specification (Mes) Instructions

    Figure 26. Securing the enclosure 7. Place the IBM brand label on the right ear cap. Place the IBM brand label onto the right ear cap within the square area, as highlighted in Figure 27 on page 17. Figure 27. Displaying IBM brand label location If an MTM/SN label is needed to be placed on the left ear cap, contact IBM Quality Hotline for more information.
  • Page 28 • The system must be working correctly before the IBM Elastic Storage System 3500 storage MES upgrade. • The existing IBM Elastic Storage System 3500 must be a properly installed 12 NVMe system with 12 NVMe drives correctly located in slots 1 - 6 and 13 - 18.
  • Page 29 7. To check whether all 24 NVMe drives have the current firmware version, issue the following command from one of the canisters. For more information about the current firmware version, see the Drives section in IBM Elastic Storage System documentation. # mmlsfirmware --type <component type>...
  • Page 30 3.84 4 KiB + SN5ASN5A /dev/nvme8n1 S54KNE0NA00532 3.84TB NVMe Tier-1 Flash 3.84 TB / 3.84 4 KiB + SN5ASN5A /dev/nvme9n1 S54KNE0NA00246 3.84TB NVMe Tier-1 Flash 3.84 TB / 3.84 4 KiB + SN5ASN5A 20  IBM Elastic Storage System 3500: Service Guide...
  • Page 31 10. Issue the essinstallcheck command from both canisters and ensure that no error is shown before you proceed to the next step. # essinstallcheck -N <node list> 11. Issue the following command against the affected recovery group. This command adds the new capacity and begins re-balancing data onto the new disks.
  • Page 32 Because the decluster array size is doubled after new 12 drives, the set-size is set to 40% to keep the new VDisk size same as the original VDisk size. For information about non-default configurations, see IBM Spectrum Scale RAID: Administration in IBM Elastic Storage Server documentation.

  • Page 33: Ess 3500 Adapter Concurrent Mes Upgrade

    Administration Guide. ESS 3500 adapter concurrent MES upgrade This procedure is intended to add the third adapter to each of the server canisters of IBM Elastic Storage System 3500 during the concurrent (online) adapter MES installation. For this concurrent upgrade, it is suggested to complete this operation during a period of low workload stress.
  • Page 34 Table 4. Adapter placement priority Placement Feature Description Paired adapters to be installed in the same slot of BOTH Priority Code canisters pair pair pair pair AJQR 9500-16e Slot 4 Slot 2 12Gb SAS 24  IBM Elastic Storage System 3500: Service Guide...
  • Page 35 Table 4. Adapter placement priority (continued) Placement Feature Description Paired adapters to be installed in the same slot of BOTH Priority Code canisters pair pair pair pair AJZL Slot 1 Slot 3 Slot 2 Slot 4 InfiniBand/V AJZN CX6 DX Slot 1 Slot 3 Slot 2...
  • Page 36 A/1: # mmumount all -N <node name> 3. Customer task: Shut down GPFS only in canister A/1. # mmshutdown -N <node name> 4. Remove canister A/1 from the enclosure. (SSR task) 26  IBM Elastic Storage System 3500: Service Guide...
  • Page 37 Both options must be successful. Option 2 must show six network adapters. 7. Customer task: Incorporate new interfaces to the existing single master bond in canister A/1. To configure each of the two new ports to be either InfiniBand or Ethernet, see https://www.ibm.com/ docs/en/ess/6.1.3_lts?topic=guide-connectx-5-connectx-6-vpi-support. Contact technical support if required.
  • Page 38 Perform basic checks via the technician port in canister B/2 using the essserv1 service ID. e. Perform the following checks in the ecotels menu: i) Option 2. Quick storage configuration check ii) Option 3. Check enclosure cabling and paths to disks 28  IBM Elastic Storage System 3500: Service Guide...

  • Page 39: Formatting Drives Example

    Both options must be successful. Option 2 must show six network adapters. 16. Customer task: Incorporate new interfaces to the existing single master bond in canister B/2. To configure each of the two new ports to be either InfiniBand or Ethernet, see https://www.ibm.com/ docs/en/ess/6.1.3_lts?topic=guide-connectx-5-connectx-6-vpi-support. Contact technical support if required.
  • Page 40 4 KiB + SN1OSN1O /dev/nvme18n1 S43RNX0M903219 3.84TB NVMe G3 Tier-1 Flash 3.84 TB / 3.84 4 KiB + SN1OSN1O /dev/nvme19n1 S43RNX0M903031 3.84TB NVMe G3 Tier-1 Flash 3.84 TB / 3.84 4 KiB + SN1OSN1O 30  IBM Elastic Storage System 3500: Service Guide...

  • Page 41: Manually Restarting Gpfs On The Enterprise Storage Server 3500 Canisters Example

    /dev/nvme1n1 S43RNX0M903214 3.84TB NVMe G3 Tier-1 Flash 3.84 TB / 3.84 4 KiB + SN1OSN1O /dev/nvme20n1 S43RNX0M903136 3.84TB NVMe G3 Tier-1 Flash 3.84 TB / 3.84 4 KiB + SN1OSN1O /dev/nvme21n1 S43RNX0M903213 3.84TB NVMe G3 Tier-1 Flash 3.84 TB / 3.84 4 KiB + SN1OSN1O...
  • Page 42 6. Repeat the mmshutdown command and the mmstartup command on <node name>. 32  IBM Elastic Storage System 3500: Service Guide...

  • Page 43: Chapter 2. Part Listings

    Trusted Platform Module (TPM) FRU kit 01LL739 64GB memory DIMM 78P6815 960GB M.2 boot drive FRU kit 01LL663 PCIe adapter CX6 VPI 01LL648 PCIe adapter CX6 DX 01LL634 Enclosure chassis 01LL662 CR2032 coin cell battery 00RY543 © Copyright IBM Corp. 2022...
  • Page 44 Canister Screw Set/Kit 01LL788 Note: This part is required when servicing the PCIe riser, PCIe adapter, DIMM, coin battery, TPM, and boot drives. The part is already included in all canister part FRU kits. 34  IBM Elastic Storage System 3500: Service Guide...

  • Page 45: Accessibility Features For The System

    IBM Documentation, and its related publications, are accessibility-enabled. Keyboard navigation This product uses standard Microsoft Windows navigation keys. IBM and accessibility See the IBM Human Ability and Accessibility Center (www.ibm.com/able) for more information about the commitment that IBM has to accessibility. © Copyright IBM Corp. 2022...
  • Page 46 36  IBM Elastic Storage System 3500: Service Guide...

  • Page 47: Notices

    Consult your local IBM representative for information on the products and services currently available in your area. Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any IBM intellectual property right may be used instead.

  • Page 48: Trademarks

    IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corp., registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at www.ibm.com/legal/copytrade.shtml.
  • Page 49 IBM reserves the right to withdraw the permissions that are granted herein whenever, in its discretion, the use of the publications is detrimental to its interest or as determined by IBM, the above instructions are not being properly followed.
  • Page 50 40  IBM Elastic Storage System 3500: Service Guide...

  • Page 51: Glossary

    • See refers you from a non-preferred term to the preferred term or from an abbreviation to the spelled- out form. • See also refers you to a related or contrasting term. For other terms and definitions, see the IBM Terminology website (opens in new window): http://www.ibm.com/software/globalization/terminology building block A pair of servers with shared disk enclosures attached.
  • Page 52 A data structure containing key information about a file system. This information includes the disks assigned to the file system (stripe group), the current state of the file system, and pointers to key files such as quota files and log files. 42  IBM Elastic Storage System 3500: Service Guide...
  • Page 53 Hardware Management Console (HMC) Standard interface for configuring and operating partitioned (LPAR) and SMP systems. See Hardware Management Console (HMC). IBM Security Key Lifecycle Manager (ISKLM) For GPFS encryption, the ISKLM is used as an RKM server to store MEKs. independent fileset A fileset that has its own inode space.
  • Page 54 From a system management perspective, it is the central coordinator of the cluster. It also serves as a client node in an ESS building block. master encryption key (MEK) A key that is used to encrypt other keys. See also encryption key. 44  IBM Elastic Storage System 3500: Service Guide...
  • Page 55 maximum transmission unit (MTU) The largest packet or frame, specified in octets (eight-bit bytes), that can be sent in a packet- or frame-based network, such as the Internet. The TCP uses the MTU to determine the maximum size of each packet in any transmission. See master encryption key (MEK).
  • Page 56 Spanning Tree Protocol (STP) A network protocol that ensures a loop-free topology for any bridged Ethernet local-area network. The basic function of STP is to prevent bridge loops and the broadcast radiation that results from them. 46  IBM Elastic Storage System 3500: Service Guide...
  • Page 57 See secure shell (SSH). See Spanning Tree Protocol (STP). symmetric multiprocessing (SMP) A computer architecture that provides fast performance by making multiple processors available to complete individual processes simultaneously. See Transmission Control Protocol (TCP). Transmission Control Protocol (TCP) A core protocol of the Internet Protocol Suite that provides reliable, ordered, and error-checked delivery of a stream of octets between applications running on hosts communicating over an IP network.
  • Page 58 48  IBM Elastic Storage System 3500: Service Guide...

  • Page 59: Index

    Index accessibility features 35 audience ix comments x documentation on web ix information overview ix license inquiries 37 notices 37 overview of information ix patent information 37 preface ix resources on web ix submitting x trademarks 38 documentation ix resources ix Index  49...
  • Page 60 50  IBM Elastic Storage System 3500: Service Guide...
  • Page 61 Index accessibility features 35 audience ix comments x documentation on web ix information overview ix license inquiries 37 notices 37 overview of information ix patent information 37 preface ix resources on web ix submitting x trademarks 38 documentation ix resources ix Index  51...
  • Page 62 52  IBM Elastic Storage System 3500: Service Guide...
  • Page 64 IBM® Product Number: 5765-DME   5765-DAE SC27-9876-00...

Table of Contents