Lenovo ThinkSystem SD650-N V3 Neptune DWC Tray User Manual
  1. Manuals
  2. Brands
  3. Lenovo Manuals
  4. Server
  5. ThinkSystem SD650-N V3 Neptune DWC Tray
  6. User manual
Lenovo ThinkSystem SD650-N V3 Neptune DWC Tray User Manual

Lenovo ThinkSystem SD650-N V3 Neptune DWC Tray User Manual

Hide thumbs
1
2
Table Of Contents
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
Table of Contents

Advertisement

Quick Links

Download this manual
ThinkSystem SD650-N V3 Neptune DWC Tray
User Guide
Machine Types: 7D7N

Advertisement

Table of Contents
loading
Need help?

Need help?

Do you have a question about the ThinkSystem SD650-N V3 Neptune DWC Tray and is the answer not in the manual?

Questions and answers

Related Manuals for Lenovo ThinkSystem SD650-N V3 Neptune DWC Tray

Summary of Contents for Lenovo ThinkSystem SD650-N V3 Neptune DWC Tray

  • Page 1 ThinkSystem SD650-N V3 Neptune DWC Tray User Guide Machine Types: 7D7N...
  • Page 2 Before using this information and the product it supports, be sure to read and understand the safety information and the safety instructions, which are available at: https://pubs.lenovo.com/safety_documentation/ In addition, be sure that you are familiar with the terms and conditions of the Lenovo warranty for your server, which can be found at: http://datacentersupport.lenovo.com/warrantylookup First Edition (March 2024) ©...

  • Page 3: Table Of Contents

    Solution package contents ..GPU board (trained technician only) ..267 Identify the solution and access the Lenovo GPU board assembly replacement (trained XClarity Controller ....
  • Page 4 ....403 Troubleshooting by symptom ..372 GPU problems ... . . 372 ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...

  • Page 5: Safety

    Vor der Installation dieses Produkts die Sicherheitshinweise lesen. Prima di installare questo prodotto, leggere le Informazioni sulla Sicurezza. Les sikkerhetsinformasjonen (Safety Information) før du installerer dette produktet. Antes de instalar este produto, leia as Informações sobre Segurança. © Copyright Lenovo 2024...

  • Page 6: Safety Inspection Checklist

    1 & IEC 60950-1, the standard for Safety of Electronic Equipment within the Field of Audio/Video, Information Technology and Communication Technology. Lenovo assumes you are qualified in the servicing of equipment and trained in recognizing hazards energy levels in products. Access to the equipment is by the use of a tool, lock and key, or other means of security, and is controlled by the authority responsible for the location.
  • Page 7 Click Power ➙ Power Cables to see all line cords. • Make sure that the insulation is not frayed or worn. 3. Check for any obvious non-Lenovo alterations. Use good judgment as to the safety of any non-Lenovo alterations.
  • Page 8 ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...

  • Page 9: Chapter 1. Thinksystem Sd650-N V3, Types 7D7N

    If a Features on Demand feature is integrated in the solution or in an optional device that is installed in the solution, you can purchase an activation key to activate the feature. For information about Features on Demand, see: https://fod.lenovo.com/lkms © Copyright Lenovo 2024...
  • Page 10 The Lenovo XClarity Controller consolidates multiple management functions in a single chip on the server system board. Some of the features that are unique to the Lenovo XClarity Controller are enhanced performance, higher-resolution remote video, and expanded security options.

  • Page 11: Tech Tips

    The ThinkSystem RAID provides software RAID support for RAID level 0, 1, and 10. Tech Tips Lenovo continually updates the support website with the latest tips and techniques that you can use to solve issues that your server might encounter. These Tech Tips (also called retain tips or service bulletins) provide procedures to work around issues or solve problems related to the operation of your server.

  • Page 12: Technical Specifications

    • New Socket Technology (Socket E with PCIe 5.0) LGA4677. Notes: 1. Use the Setup Utility to determine the type and speed of the processors in the node. 2. For a list of supported processors, see https://serverproven.lenovo.com ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 13 ® Xeon ® Scalable processors (Sapphire Rapids, SPR) – Lenovo DDR5 at 4800 MT/s – 3DS RDIMM at 4800 MT/s – Capacity: • 32 GB and 64 GB ECC RDIMM • 48GB, and 96 GB ECC RDIMM (supported by XCC processors) •...
  • Page 14 “KVM breakout cable” on page • External LCD diagnostics handset connector • One Gigabit Ethernet port with RJ45 connector, shared between operating system and Lenovo XClarity Controller. • Two 25Gb SFP28 ports. One port is shared between operating system and Lenovo XClarity Controller.
  • Page 15 • An OSFP module with either two 400Gb or two 800Gb OSFP ports, connecting to four ConnectX-7 chip sets on the network board. • One Gigabit Ethernet port with RJ45 connector, shared between operating system and Lenovo XClarity Controller. • Two 25Gb SFP28 ports. One port is shared between operating system and Lenovo XClarity Controller.

  • Page 16: Mechanical Specifications

    • Width: 437.5 mm (17.22 inches) Weight SD650-N V3 Tray: • Weight estimation: 22.7 kg (50.05 lbs) • Enclosure with six SD650-N V3 trays and three DWC PSU: approximately 183.3 kg (404.11 lbs) ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...

  • Page 17: Environmental Specifications

    Lenovo recommends that you consult with qualified experts in this field to determine whether you are in compliance with the applicable regulations.
  • Page 18 – Water flow rate for 35°C (95°F): 21 liters per minute (lpm) per enclosure, assuming 3.5 liters per tray with 6 trays per enclosure. 1 tray consists of 1 compute node and 1 GPU node. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...

  • Page 19: Particulate Contamination

    If Lenovo determines that the levels of particulates or gases in your environment have caused damage to the device, Lenovo may condition provision of repair or replacement of devices or parts on implementation of appropriate remedial measures to mitigate such environmental contamination.

  • Page 20: Management Options

    Management options The XClarity portfolio and other system management options described in this section are available to help you manage the servers more conveniently and efficiently. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 21 • Mobile application • Redfish API Usage and downloads https://pubs.lenovo.com/lxcc-overview/ Application that reports the XCC events to local OS system log. Interface • CLI application Lenovo XCC Logger Utility Usage and downloads • https://pubs.lenovo.com/lxcc-logger-linux/ • https://pubs.lenovo.com/lxcc-logger-windows/ Centralized interface for multi-server management.
  • Page 22 Important: Lenovo XClarity Provisioning Manager (LXPM) supported version varies by product. All versions of Lenovo XClarity Provisioning Manager are referred to as Lenovo XClarity Provisioning Manager and LXPM in this document, unless specified otherwise. To see the LXPM version supported by your server, go to https:// pubs.lenovo.com/lxpm-overview/...
  • Page 23 1. Most options can be updated through the Lenovo tools. Some options, such as GPU firmware or Omni- Path firmware require the use of supplier tools. 2. The server UEFI settings for option ROM must be set to Auto or UEFI to update firmware using Lenovo XClarity Administrator, Lenovo XClarity Essentials, or Lenovo XClarity Controller.
  • Page 24 ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...

  • Page 25: Chapter 2. Gpu Power And Maximum Number Of Trays In The Enclosure

    • TGP User selected optimal: 600W (maximum 5 trays in the enclosure) • TGP User selected minimum: 500W (maximum 6 trays in the enclosure) Note: Mismatching GPU power capping value and maximum tray number may cause GPU performance impact. © Copyright Lenovo 2024...
  • Page 26 ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...

  • Page 27: Chapter 3. Server Components

    You can also use it to force a blue-screen memory dump. Use this button only when you are directed to do so by Lenovo Support. Drive Bay 2/3 1 Gb RJ45 Ethernet port with share-NIC feature for Lenovo XClarity Controller. See “Front LEDs” on page 357 for more information.

  • Page 28: Top View

    25 Gb SFP28 Ethernet port (Port 2). See Pull-out information tab with Lenovo XClarity Controller network “Front LEDs” on page 357 for more access label. See Identify the solution and access the Lenovo information. XClarity Controller for more information. Top view This section contains information on the top view of the solution.

  • Page 29: Kvm Breakout Cable

    Figure 4. GPU numbering GPU node top view Figure 5. GPU node top view Table 4. Components on the GPU node top view Water loop quick connect Carrier board (for network board) Network board GPU node power distribution board Carrier board retention clips GPU board GPU board handles GPUs...

  • Page 30: System-Board Layout (Compute Node)

    System-board connectors The following illustrations show the internal connectors on the system board. The following illustration shows the location of the DIMM slots numbering on the system board of the compute node. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 31 Figure 7. DIMM slots numbering on compute node system board Figure 8. Processor locations Figure 9. Internal connectors on compute node system board Table 6. Internal connectors on compute node system board Trusted cryptographic module (TCM) connector Drive power connector 1 and 2 PCIe x 16 MCIO 1 connector CMOS battery (CR2032) connector M.2 connector...

  • Page 32: System-Board Switches

    Switch name number BMC reset Force hot reset BMC chip (Default) Normal. Machine Engine (ME) ME update by jumper (Default) Normal. firmware security override Password override Overrides the power-on password (Default) Normal. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...

  • Page 33: Gpu Node Connectors

    Table 9. SW13 switch block definition Usage description Switch Switch name number Clear CMOS Clear CMOS data (Default) Normal. Serial select Sends the XCC to the serial port (Default) Send the serial input output (SIO) to the front serial port. Table 10.

  • Page 34: System Leds And Diagnostics Display

    System LEDs and diagnostics display See the following section for information on available enclosure LEDs and diagnostics display. For more information, refer to “Troubleshooting by system LEDs and diagnostics display” on page 357. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...

  • Page 35: Chapter 4. Parts List

    Tier 1 CRU at your request with no service agreement, you will be charged for the installation. • T2: Tier 2 customer replaceable unit (CRU). You may install a Tier 2 CRU yourself or request Lenovo to install it, at no additional charge, under the type of warranty service that is designated for your server.
  • Page 36 3. Enter the serial number to view a listing of parts for your solution. Tray cover Screw Rear cross brace Power distribution board (GPU node) Tray Water loop Power distribution board (compute node) DIMM comb DIMM cover ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 37 Table 12. SD650-N V3 parts list (continued) Index Description Type Memory module Memory module filler DIMM tool Conduction plate kit OSFP module VR water clamp plate Venting hole block VR water loop trough M.2 drive Front cross brace Cross brace–cable cover Processor System board GPU board assembly (GPU board installed with four GPUs)

  • Page 38: Power Cords

    The cord set should have the appropriate safety approvals for the country in which the equipment will be installed. • Power cords for a specific country or region are usually available only in that country or region. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...

  • Page 39: Chapter 5. Unboxing And Setup

    They might be required to receive warranty service. Identify the solution and access the Lenovo XClarity Controller This section contains instruction on how to identify your solution and where to find the Lenovo XClarity Controller access information.
  • Page 40 Lenovo XClarity Controller network access label In addition, the Lenovo XClarity Controller network access label for both nodes can be found on the pull out information tag located at the front of the tray. You can use the information on the pull out tag to access the XCC MAC address and LLA for each node.
  • Page 41 Figure 14. Lenovo XClarity Controller network access label on the pull-out information tab Service Label and QR code In addition, the system Service Label is located on the inside surface of the tray cover, provides a quick response (QR) code for mobile access to service information. You can scan the QR code with a mobile device using a QR code reader application and get quick access to the Service Information web page.

  • Page 42: Solution Setup Checklist

    • You can press the power button. • The solution can restart automatically after a power interruption. • The solution can respond to remote power-on requests sent to the Lenovo XClarity Controller. Note: You can access the management processor interface to configure the system without powering on the solution.
  • Page 43 1. Set the network connection for theLenovo XClarity Controller to the management network. 2. Update the firmware for the solution, if necessary. 3. Configure the firmware for the solution. The following information is available for RAID configuration: • https://lenovopress.lenovo.com/lp0578-lenovo-raid-introduction • https://lenovopress.lenovo.com/lp0579-lenovo-raid-management-tools-and-resources 4. Install the operating system. 5. Back up the solution configuration.
  • Page 44 ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...

  • Page 45: Chapter 6. Hardware Replacement Procedures

    • If you replace a part, such as an adapter, that contains firmware, you might also need to update the firmware for that part. For more information about updating firmware, See “Update the firmware” on page 341. © Copyright Lenovo 2024...

  • Page 46: Safety Inspection Checklist

    This equipment must be installed or serviced by trained personnel, as defined by the NEC, IEC 62368- 1 & IEC 60950-1, the standard for Safety of Electronic Equipment within the Field of Audio/Video, ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...

  • Page 47: System Reliability Guidelines

    Click Power ➙ Power Cables to see all line cords. • Make sure that the insulation is not frayed or worn. 3. Check for any obvious non-Lenovo alterations. Use good judgment as to the safety of any non-Lenovo alterations.

  • Page 48: Handling Static-Sensitive Devices

    ® 1. Flat mode 2. Cache mode Information about optimizing memory performance and configuring memory is available at the Lenovo Press website: https://lenovopress.lenovo.com/servers/options/memory In addition, you can take advantage of a memory configurator, which is available at the following site:...
  • Page 49 Specific information about the required installation order of memory modules in your solution based on the system configuration and memory mode that you are implementing is shown below. Memory modules and processors layout Figure 16. Memory modules and processors layout The memory-channel configuration table below shows the relationship between the processors, memory channels, and memory module slot numbers.

  • Page 50: Dram Dimms Installation Order

    • Sub NUMA Clustering 2 (SNC2) is supported and can be enabled in UEFI. • Software Guard Extensions (SGX) is supported. See “Enable Software Guard Extensions (SGX)” on page to enable this feature. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...

  • Page 51: Dram Dimms Installation Order For Intel Xeon Cpu Max Processor

    DRAM DIMMs installation order for Intel ® Xeon ® CPU Max processor This section contains information of how to install DRAM DIMMs properly when the system is installed with Intel Xeon CPU Max processors. ® ® Intel ® Xeon ® CPU Max operating systems support Intel Xeon...

  • Page 52: Gap Pad/Putty Pad Identification And Location

    See the following for gap pads, putty pads, and PCMs used in SD650-N V3: • “Water loop and GPU node gap pads and putty pads” on page 45 • “Compute node components gap pads and putty pads” on page 47 ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 53 Water loop and GPU node gap pads and putty pads Figure 17. Water loop and GPU node gap pads and putty pads Chapter 6 Hardware replacement procedures...
  • Page 54 OSFP module conduction Putty Pad shown in: plate • “Install the OSFP module” on page 179 • “Install the water loop” on page 78 ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 55 Compute node components gap pads and putty pads Chapter 6 Hardware replacement procedures...
  • Page 56 148. E3.S drive conduction plate Align pad to the conduction Putty Pad plate as shown in “Install Putty Pad an E3.s drive” on page 143. Both sides can be facing outward ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...

  • Page 57: Power On And Power Off The Solution

    “Power off the solution” on page Power off the solution The solution remains in a standby state when it is connected to a power source, allowing the Lenovo XClarity Controller to respond to remote power-on requests. To remove all power from the solution (power status LED off), you must disconnect all power cables.

  • Page 58: Replace Components In The Tray

    When in a standby state, the solution can respond to remote power-on requests sent to the Lenovo XClarity Controller. For information about powering on the solution, see “Power on the solution” on page Replace components in the tray Use the following information to remove and install tray components.
  • Page 59 Figure 18. Tray removal Attention: • To maintain proper system cooling, do not operate the enclosure without a DWC tray or tray bay filler installed in each tray bay. For more information, see “Install a DWC tray in the enclosure” on page •...
  • Page 60 Connect all required external cables to the enclosure. Note: Use extra force to connect QSFP cables to the solution. Step 4. Press the power buttons to turn on both nodes in the DWC tray. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...

  • Page 61: Tray Cover Replacement

    Attention: Shared I/O configuration requires specific power on and power off sequence for the two nodes, see below: • Power on sequence: first, power on the node with main adapter (right node); then, power on the node with auxiliary adapter (left node). •...
  • Page 62 A video of this procedure is available at YouTube: https://www.youtube.com/playlist?list=PLYV5R7hVcs- DCqagDqjI0xAqs12RCavnk Procedure Step 1. Position the cover on top of the tray and slide the cover towards the front of the tray. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...

  • Page 63: Cross Braces Replacement

    Figure 21. Tray cover installation Important: Before you slide the cover forward, make sure that all the tabs on the front, rear, and side of the cover are engaged with the enclosure correctly. If all the tabs do not engage the enclosure correctly, it will be very difficult to remove the cover later.
  • Page 64 Removing the front cable cover (x4 M3.5 screws). Figure 22. Front cable cover removal Step 3. Remove the front and the rear cross braces (x14 Phillips #1 screws, x7 screws for each cross brace). ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 65 Figure 23. Front and rear cross braces removal After you finish If you are instructed to return the component or optional device, follow all packaging instructions, and use any packaging materials for shipping that are supplied to you. Install the cross braces Use this information to install the cross braces.
  • Page 66 Install the front and the rear cross braces (x14 Phillips #1 screws, x7 screws for each cross brace). Figure 24. Front and rear cross braces installation Step 2. Installing the front cable cover (x4 M3.5 screws). ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...

  • Page 67: Water Loop Replacement

    Figure 25. Front cable cover installation After you finish 1. Install the tray cover. See “Install the tray cover” on page 2. Install the tray into the enclosure. See “Install a DWC tray in the enclosure” on page 3. Connect all required external cables to the solution. Note: Use extra force to connect QSFP cables to the solution.
  • Page 68 Remove the drive cage. Depending the system configurations, see “Remove a drive cage assembly” on page 107, “Remove a 7mm NVMe drive cage assembly” on page 124, or “Remove the E3.S drive cage assembly” on page 136. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 69 Remove the OSFP module. See “Remove the OSFP module” on page 172. Separate the water loop from the GPU node. Step 2. Remove the carrier board power cable from the GPU node. Figure 26. Carrier board power cable removal From (carrier board) To (GPU node power distribution board) Power and side band connector Power connector...
  • Page 70 Remove the quick connect screws (x4 Torx T10) with a torque screwdriver set to the proper torque. Note: For reference, the torque required for the screws to be fully tightened/removed is 5.0+/- 0.5 lbf-in, 0.55+/- 0.05 N-M. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 71 Figure 30. Quick connect screw removal (GPU node) Step 7. Follow the screw removal sequence specified on the network board label, and remove network cold plate screws (x8 Torx T10 screws) with a torque screwdriver set to the proper torque. Note: For reference, the torque required for the screws to be fully tightened/removed is 5.0+/- 0.5 lbf-in, 0.55+/- 0.05 N-M.
  • Page 72 Step 10. Release the GPU cold plates from the GPUs. There are notches on the sides of the GPU cold plates for inserting a flat head screwdriver. The locations of the notches are shown in the illustration below. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 73 Attention: Inserting the flat head screwdriver ONLY to the notches circled in the illustration below. Otherwise, the screwdriver may damage the GPUs. Figure 34. Notches for releasing GPU cold plates Insert a flat screwdriver into all the notches shown in the illustration; then, slightly rotate the screwdriver to release the GPU cold plates from the GPUs.
  • Page 74 Step 12. Tighten water loop carrier screws (x20 Phillips #2 screws) with a torque screwdriver set to the proper torque. Note: For reference, the torque required for the screws to be fully tightened/removed is 5.0+/- 0.5 lbf-in, 0.55+/- 0.05 N-M. Figure 37. Tightening water loop carrier screws (GPU node) ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 75 Step 13. Carefully rotate the GPU node side water loop so that it is sitting on top of the Compute node side water loop. Figure 38. Folding the GPU node side water loop onto the Compute node side water loop Step 14.
  • Page 76 Figure 40. Cleaning putty pads from water loop Figure 41. Cleaning putty pads and PCM from the components in the GPU node Step 16. Flip the GPU node water loop back to the GPU node. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 77 Figure 42. Flipping GPU node water loop to the GPU node Separate the water loop from the compute node. Step 17. Remove the two Hex screws from the OSFP module with a 4.5 mm hex head screwdriver. Figure 43. OSFP module conduction plate Hex screws removal Step 18.
  • Page 78 M3 screw (x2) T10 screwdriver Figure 44. OSFP module conduction plate removal Step 19. Remove two Torx T10 screws (per node); then, remove the VR (voltage regulator) clamp plate out of the node. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 79 Figure 45. VR clamp plate removal Step 20. Remove water loop screws (x9 Torx T10 screws for two nodes) with a torque screwdriver set to the proper torque. Note: For reference, the torque required for the screws to be fully tightened/removed is 5.0+/- 0.5 lbf-in, 0.55+/- 0.05 N-M.
  • Page 80 Step 22. Orient two water loop carriers with the guide pins; then, gently put the two water loop carriers down and ensure they are seated firmly on the water loop. Figure 48. Water loop carrier installation ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 81 Step 23. Tighten water loop carrier screws (x12 Phillips #2 screws). Figure 49. Water loop carrier screws installation Step 24. Loosen processors properly. Fully loosen all Torx T30 captive screws (x8 Torx T30 captive screws) on cold plates with a general screwdriver, following the removal sequence shown on the cold plate label.
  • Page 82 Step 25. Carefully rotate the water loop so one half is sitting on top of the other half. Figure 51. Folding the water loop Step 26. Fasten two captive thumbscrews to secure water loop carriers to each other. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 83 Figure 52. Tightening captive thumbscrews Step 27. Remove the processor from the retainer. This process differs by processor SKU. Check the processor SKU and follow the applicable procedure. Note: Do not touch the contacts on the processor. For non Intel ®...
  • Page 84 Step 29. Remove the processor retainer from the underside of the cold plate. Note: The processor retainer will be discarded and replaced with a new one. Carefully release the retaining clips from the cold plate. Lift the retainer from the cold plate. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 85 Figure 55. Processor retainer removal Step 30. Wipe the thermal grease from the bottom of the cold plate with an alcohol cleaning pad. Step 31. Carefully lift the water loop up off the system board and out of the node. Chapter 6 Hardware replacement procedures...
  • Page 86 • Drive gap pad or putty pad kits according to the drives installed in the tray. See their respective replacement procedures for more information. • Screws and screwdrivers Prepare the following screwdrivers to ensure you can install and remove corresponding screws properly. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 87 Screw Type Screwdriver Type Hex screw (GPU node water loop) 6 mm hex head screwdriver Hex screw (OSFP module conduction plate) 4.5 mm hex head screwdriver Torx T10 screw Torx T10 head screwdriver Phillips #1 screw Phillips #1 head screwdriver Phillips #2 screw Phillips #2 head screwdriver Important: Gap pad/putty pad replacement guidelines...
  • Page 88 Figure 58. Installing a processor retainer CPU Max only) After installing the retainer onto processor, make sure the slot (Intel ® Xeon ® on the TIM breaking cam is vertical. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 89 TIM breaking cam Figure 60. TIM breaking cam on processor retainer Step 3. When removing the water loop from the package box, make sure to hold the touch points marked in grey in the illustration below. Attention: Holding the water loop anywhere other than the touch points may cause damage to it. Chapter 6 Hardware replacement procedures...
  • Page 90 Top view of the water loop Bottom image Figure 61. Touching points when removing water loop from package box Step 4. If needed, remove plastic grease covers from the underside of processor cold plates. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 91 Figure 62. Plastic grease covers removal Step 5. Remove two plastic grease covers if needed. Use a pair of scissors to cut off the tape. Remove plastic grease covers from the underside of water loop cold plates. Note: The right node is shown as example. Remove plastic grease covers if needed when installing processor in either node.
  • Page 92 Rotate all anti-tilt wire bails (8x anti-tilt wire bails per node) outwards to the unlocked position. Figure 65. Processor unlocked position Step 8. Check the gap pads on the water loop, if any of them are damaged or detached, replace them with the new ones. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 93 Figure 66. Water loop gap pads GPU node water loop gap pad/putty pad replacement. Make sure to follow Gap pad/putty pad replacement guidelines. Step 9. Replace the Phase Change Material (PCM) and putty pads on the GPU node water loop with new ones.
  • Page 94 GPU. Repeat to replace all putty pads on the four GPUs. GPU VR (Cover the GPU VR with putty pad) Figure 69. GPU putty pads replacement Make sure to follow Gap pad/putty pad replacement guidelines. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 95 Step 10. Fully loosen two captive thumbscrews located at each end of the water loop carrier. Figure 70. Loosening water loop captive thumbscrews Step 11. Install the compute node side of the water loop into the compute node of the tray. Carefully hold the water loop with both hands and flip it.
  • Page 96 Rotate 8x anti-tilt wire bails outwards to the locked position. Fully tighten 8x Torx T30 captive screws on cold plates with a general screwdriver, following the installation sequence shown on the cold plate label. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 97 Note: For reference, the torque required for the screws to be fully tightened/removed is 10+/- 2.0 lbf-in, 1.1+/- 0.2 N-m. Attention: To prevent damage to components, make sure that you follow the indicated tightening sequence. Figure 73. Securing Torx T30 captive screws Step 14.
  • Page 98 Note: For reference, the torque required for the screws to be fully tightened/removed is 5.0+/- 0.5 lbf-in, 0.55+/- 0.05 N-M. Figure 76. Water loop screws installation Step 17. Install the Torx T10 screws (x7 screws) to secure the quick connect. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 99 Figure 77. Quick connect screw installation Step 18. Install two VR clamp plates into the nodes and install Torx T10 screws (4x Torx T10 screws for two nodes). Figure 78. VR clamp plate installation Step 19. Replace the putty pads on the top side and the bottom side of the OSFP module conduction plate. Chapter 6 Hardware replacement procedures...
  • Page 100 Gap pad/putty pad replacement guidelines. Step 20. Install the OSFP module conduction plate onto the water loop. Screw Type Screwdriver Type Phillips #1 head screwdriver M3x5 screw (x3) M3 screw (x2) T10 screwdriver ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 101 Figure 80. Installing the OSFP module conduction plate Step 21. Install the two Hex screws to the OSFP module with a 4.5 mm hex head screwdriver. Figure 81. OSFP module conduction plate Hex screws installation Install water loop to the GPU node Step 22.
  • Page 102 Figure 82. Water loop screws and quick connect screws installation (GPU node) Step 23. Remove the water loop carrier from the GPU node. Figure 83. Water loop carrier removal (GPU node) ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 103 Step 24. Install GPU cold plate screws (x16 PH2 screws). Install the GPU cold plates in diagonal pattern. Complete installing screws of GPU 2, then proceed to GPU 4 ➙ GPU 1 ➙ GPU 3. Set the torque screwdriver to 3.5 lb-In, 0.39 N-m. →...
  • Page 104 Step 27. Install water loop screws and quick connect screws (x11 Torx T10 screws) with a torque screwdriver set to the proper torque. Note: For reference, the torque required for the screws to be fully tightened/removed is 5.0+/- 0.5 lbf-in, 0.55+/- 0.05 N-M. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 105 Figure 87. Water loop Torx T10 screws installation (GPU node) Step 28. Install the Hex screw (x1) and the PH1 screws (x3). Note: For reference, the torque required for the screws to be fully tightened/removed is 5.0+/- 0.5 lbf-in, 0.55+/- 0.05 N-M. Figure 88.
  • Page 106 2. Install the drive cage. Depending on system configurations, see “Install a drive cage assembly” on page 118, “Install a 7mm NVMe drive cage assembly” on page 131, or “Install the E3.S drive cage assembly” on page 148. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...

  • Page 107: Replace Components In The Compute Node

    • S004 CAUTION: When replacing the lithium battery, use only Lenovo specified part number or an equivalent type of battery recommended by the manufacturer. If your system has a module containing a lithium battery, replace it only with the same module type made by the same manufacturer. The battery contains lithium and can explode if not properly used, handled, or disposed of.
  • Page 108 • The following notes describe information that you must consider when replacing the battery. – Lenovo has designed this product with your safety in mind. The lithium battery must be handled correctly to avoid possible danger. If you replace the battery, you must adhere to the following instructions.
  • Page 109 Watch the procedure A video of this procedure is available at YouTube: https://www.youtube.com/playlist?list=PLYV5R7hVcs- DCqagDqjI0xAqs12RCavnk Procedure Step 1. Make preparations for this task. Remove the tray from the enclosure. See “Remove a DWC tray from the enclosure” on page Remove the tray cover. See “Remove the tray cover”...
  • Page 110 CAUTION: When replacing the lithium battery, use only Lenovo specified part number or an equivalent type of battery recommended by the manufacturer. If your system has a module containing a lithium battery, replace it only with the same module type made by the same manufacturer. The battery contains lithium and can explode if not properly used, handled, or disposed of.

  • Page 111: Dimm Comb Replacement

    Table 23. CMOS battery location CMOS battery Watch the procedure A video of this procedure is available at YouTube: https://www.youtube.com/playlist?list=PLYV5R7hVcs- DCqagDqjI0xAqs12RCavnk Procedure Step 1. Follow any special handling and installation instructions that come with the CMOS battery. Step 2. Insert the new CMOS battery: Face the positive (+) symbol to the center of the node;...
  • Page 112 Note: Memory module tool is recommended due to space limitations caused by location of water loop tubes through the memory section. Step 3. Gently hold the DIMM comb and removal it out of the chassis. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 113 Figure 96. DIMM comb removal After you finish If you are instructed to return the component or optional device, follow all packaging instructions, and use any packaging materials for shipping that are supplied to you. Install a DIMM comb Use this information to install a DIMM comb. About this task Attention: •...
  • Page 114 Note: Use extra force to connect QSFP cables to the solution. 4. Check the power LED on each node to make sure it changes from fast blink to slow blink to indicate all nodes are ready to be powered on. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...

  • Page 115: Drive And Drive Cage Assembly

    Drive and drive cage assembly replacement Use the following procedures to remove and install a drive and drive cage assembly. Remove a drive cage assembly Use this information to remove a drive cage assembly. About this task Required tools You may need the SD650-N V3 Water Loop Gap Pad Kit to properly replace the component. Attention: •...
  • Page 116 Figure 99. Drive assembly cable routing SATA 2-3 connector NVMe 2-3 connector Drive power 1 connector Step 3. Remove the drive cage assembly. Remove the 3 screws. Remove the drive assembly out of the node. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 117 Figure 100. One 15 mm drive cage assembly removal Chapter 6 Hardware replacement procedures...
  • Page 118 Figure 101. One 7 mm drive cage assembly removal ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 119 Figure 102. Two 7 mm drive cage assembly removal Step 4. If necessary, remove the blank bezel fillers. Figure 103. Blank bezel filler removal If you are instructed to return the component or optional device, follow all packaging instructions, and use any packaging materials for shipping that are supplied to you.
  • Page 120 For removing one 7 mm or 15 mm drive only, complete the following steps. Remove the drive out of the bottom of the drive cage. Disconnect the cable from the drive. Figure 104. 7 mm drive removal ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 121 Figure 105. 15 mm drive removal For removing two 7 mm drives, complete the following steps. 1. Remove the upper drive. Push and hold the release latch. Slide the metal tab to the unlocked position. Pivot the drive out of the top of the drive cage as shown. Disconnect the cable from the drive.
  • Page 122 Figure 108. Conduction plate removal After you finish If you are instructed to return the component or optional device, follow all packaging instructions, and use any packaging materials for shipping that are supplied to you. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 123 Install a drive Use this information to install a drive. About this task Required tools You may need the following kits and to properly replace the component. • SD650-N V3 Water Loop Gap Pad Kit • Conduction Plate Parts U2 Important: Gap pad/putty pad replacement guidelines •...
  • Page 124 ) is damaged or detached, discard the conduction plate. Then, install a new conduction plate already installed with the two gap pads. Note: Make sure the metal tab is in the unlocked position. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 125 Figure 111. Conduction plate installation 2. Install the lower drive. Find the cable with its plugger marked as SSD 0/2, and connect it to the drive. Flip the drive over. Install the drive into the bottom of the drive cage and make sure it is secured in place. Figure 112.
  • Page 126 “Safety inspection checklist” on page 38 to ensure that you work safely. • Ensure the drives are installed correctly into the drive cage assembly. See “Install a drive” on page 115. Watch the procedure ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 127 A video of this procedure is available at YouTube: https://www.youtube.com/playlist?list=PLYV5R7hVcs- DCqagDqjI0xAqs12RCavnk Procedure Step 1. Make sure the blank bezel fillers are installed. If not, install them to the node. Figure 114. Blank bezel filler installation Step 2. If the gap pad located on the reverse side of the drive cage is damaged or detached, replace it with a new one.
  • Page 128 Figure 116. Markings on water loop for gap pad alignment Figure 117. Replacing gap pad Step 4. Install the drive cage assembly. Install the drive assembly into the node. Secure the 3 screws. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 129 Figure 118. One 15 mm drive cage assembly installation Chapter 6 Hardware replacement procedures...
  • Page 130 Figure 119. One 7 mm drive cage assembly installation ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 131 Figure 120. Two 7 mm drive cage assembly installation Step 5. Connect the drive assembly cable to the system board and route the cable as the following illustration. Figure 121. Drive assembly cable routing Chapter 6 Hardware replacement procedures...

  • Page 132: 7Mm Nvme Drive And Drive Cage Assembly

    Remove the tray from the enclosure. See “Remove a DWC tray from the enclosure” on page Remove the tray cover. See “Remove the tray cover” on page Remove the cross braces. See “Remove the cross braces” on page ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 133 Remove the M.2 backplane assembly. See “Remove the M.2 backplane assembly” on page 152. Step 2. Disconnect the cables from the system board. Figure 122. 7mm NVMe drive assembly cable routing SATA 2-3 connector NVMe 2-3 connector Drive power 1 connector Step 3.
  • Page 134 Figure 124. Blank bezel filler removal After you finish If you are instructed to return the component or optional device, follow all packaging instructions, and use any packaging materials for shipping that are supplied to you. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 135 Remove a 7mm NVMe drive Use this information to remove a 7mm NVMe drive. About this task Required tools Make sure you have the required tools listed below in hand to properly replace the component. • SD650-N V3 Water Loop Gap Pad Kit •...
  • Page 136 Make sure you have the required tools listed below in hand to properly replace the component. • SD650-N V3 Water Loop Gap Pad Kit • Conduction Plate Parts U3 7mm Dual • U3 7mm Dual Putty Pad Kit Attention: ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 137 • Read “Installation Guidelines” on page 37 “Safety inspection checklist” on page 38 to ensure that you work safely. Important: Gap pad/putty pad replacement guidelines • To identify the gap pad/putty pad location and orientation, see “Gap pad/Putty pad identification and location”...
  • Page 138 Rotate the drive into the drive cage. Press the drive into the drive cage. While pressing down the drive, push and hold the release latch. Slide the metal tab to the lock position. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 139 Figure 130. Upper drive installation Step 5. Replace the putty pad on the conduction plate. Make sure to follow Gap pad/putty pad replacement guidelines. After you finish 1. Install the 7mm NVMe drive cage. See “Install a 7mm NVMe drive cage assembly” on page 131.
  • Page 140 Watch the procedure A video of this procedure is available at YouTube: https://www.youtube.com/playlist?list=PLYV5R7hVcs- DCqagDqjI0xAqs12RCavnk Procedure Step 1. Make sure the blank bezel fillers are installed. If not, install them to the node. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 141 Figure 131. Blank bezel filler installation Step 2. If the drive cage gap pad is damaged or missing, replace it with a new one. See the following for the gap pad location for each drive cage. Make sure to follow Gap pad/putty pad replacement guidelines.
  • Page 142 Install the drive cage assembly. Install the drive assembly into the node. Secure the 3 screws. Note: The following illustration might differ slightly from your hardware, but the installation method is the same. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 143 Figure 134. 7mm NVMe drive cage assembly installation Step 4. Connect the drive assembly cable to the system board. Figure 135. 7mm NVMe drive assembly cable routing Chapter 6 Hardware replacement procedures...

  • Page 144: E3.S Drive And Drive Cage Assembly

    Make preparations for this task. Remove the tray from the enclosure. See “Remove a DWC tray from the enclosure” on page Remove the tray cover. See “Remove the tray cover” on page ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 145 Remove the cross braces. See “Remove the cross braces” on page Remove the M.2 backplane assembly. See “Remove the M.2 backplane assembly” on page 152. Step 2. Disconnect the cables from the system board. Figure 136. E3.S drive cable routing SATA 2-3 connector NVMe 2-3 connector Drive power 1 connector...
  • Page 146 If you are instructed to return the component or optional device, follow all packaging instructions, and use any packaging materials for shipping that are supplied to you. Remove an E3.s drive Use this information to remove an E3.s drive. About this task Required tools ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 147 Make sure you have the required tools listed below in hand to properly replace the component. • E3.S Drive Conduction Plate Assembly • E3S Putty Pad Kit • E3S Gap Pad Kit Attention: • Read “Installation Guidelines” on page 37 “Safety inspection checklist”...
  • Page 148 Figure 141. E3.S drive cable-ended backplane removal Step 5. Remove the bottom E3.S drive from the drive cage. Remove the four screws. Remove the bottom E3.S drive from the E3.S drive cage. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 149 Figure 142. Bottom E3.S drive removal Step 6. Remove the conduction plate. Remove the two screws. Insert a flat head screwdriver into the notch on the corner of the drive cage; then, slightly rotate the screwdriver to release the conduction plate from the E3.S drive cage. Remove the conduction plate from the E3.S drive cage.
  • Page 150 Figure 143. Conduction plate removal Step 7. Remove the top E3.S drive. Remove the two screws. Remove the top E3.S drive from the drive cage. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 151 Figure 144. Top E3.S drive removal After you finish If you are instructed to return the component or optional device, follow all packaging instructions, and use any packaging materials for shipping that are supplied to you. Install an E3.s drive Use this information to install an E3.s drive.
  • Page 152 Figure 145. Top E3.S drive installation Step 2. If the gap pad located on the conduction plate is damaged or detached, discard the conduction plate. Then, install a new conduction plate already attached with the gap pad. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 153 Figure 146. Replacing the conduction plate gap pad Step 3. If the putty pads on the reverse side of the conduction plate are damaged or detached, replace them with new ones. Make sure to follow Gap pad/putty pad replacement guidelines. Figure 147.
  • Page 154 Orient the E3.S drive so that the connector is on the left side as shown. Then, place the bottom E3.S drive into the E3.S drive cage. Fasten the four screws to secure the bottom E3.S drive to the E3.S drive cage. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 155 Figure 149. Bottom E3.S drive installation Step 6. Install the E3.S drive cable-ended backplane. Align the backplane to the screw hole on the E3.S drive cage; then, insert the backplane to the E3.S drive cage. Fasten the screw to secure the backplane to the E3.S drive cage. Figure 150.
  • Page 156 6. Check the power LED on each node to make sure it changes from fast blink to slow blink to indicate all nodes are ready to be powered on. Install the E3.S drive cage assembly Use this information to install the E3.S drive cage assembly. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 157 About this task Required tools Make sure you have the required tools listed below in hand to properly replace the component. • E3.S Drive Conduction Plate Assembly • E3S Putty Pad Kit • E3S Gap Pad Kit Important: Gap pad/putty pad replacement guidelines •...
  • Page 158 Install the E3.S drive cage assembly. Install the E3.S drive cage assembly into the node. Secure the three screws. Note: The following illustration might differ slightly from your hardware, but the installation method is the same. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 159 Figure 156. E3.S drive cage assembly installation Step 4. Connect the drive assembly cable to the system board and route the cable as the following illustration. For more information, see “System-board connectors” on page Figure 157. E3.S drive cable routing SATA 2-3 connector NVMe 2-3 connector Drive power 1 connector...

  • Page 160: Drive And Backplane Assembly

    “Remove a DWC tray from the enclosure” on page Remove the tray cover. See “Remove the tray cover” on page Remove the cross braces. See “Remove the cross braces” on page Step 2. Disconnect the cable from the system board. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 161 Figure 158. M.2 backplane assembly cable removal Step 3. Remove the M.2 backplane assembly. Remove the six screws. Remove the M.2 backplane assembly out from the node. Disconnect the cable from the M.2 backplane assembly. Figure 159. M.2 backplane assembly removal Step 4.
  • Page 162 Make sure the M.2 drive is removed from the M.2 backplane. To remove the M.2 drive, see “Remove an M.2 drive” on page 155. Loosen the two screws. Carefully lift the M.2 board out of the M.2 bracket. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 163 Figure 161. Disassembling M.2 backplane assembly 3. Recycle the unit in compliance with local regulations. Remove an M.2 drive Use this information to remove an M.2 drive. About this task Required tools Make sure you have the M.2 Putty Pad Kit and SD650-N V3 Miscellaneous Parts Kit to properly replace the component.
  • Page 164 Make sure you have the M.2 Putty Pad Kit and SD650-N V3 Miscellaneous Parts Kit to properly replace the component. Important: Gap pad/putty pad replacement guidelines • To identify the gap pad/putty pad location and orientation, see “Gap pad/Putty pad identification and location” on page ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 165 • Before replacing the gap pad/putty pad, gently clean the interface plate or the hardware surface with an alcohol cleaning pad. • Hold the gap pad/putty pad carefully to avoid deformation. Make sure no screw hole or opening is blocked by the gap pad/putty pad material. •...
  • Page 166 Make sure you have the following kits to properly replace the component: • SD650-N V3 Miscellaneous Parts Kit • SD650-N V3 Conduction Plate Parts • M.2 Putty Pad Kit Important: Gap pad/putty pad replacement guidelines ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 167 • Turn off the corresponding DWC tray that you are going to perform the task on. Firmware and driver download: You might need to update the firmware or driver after replacing a component. • Go to https://datacentersupport.lenovo.com/products/servers/thinksystem/sd650nv3/7d7n/downloads/driver- to see the latest firmware and driver updates for your server. list/ • Go to “Update the firmware”...
  • Page 168 Step 3. Replace the putty pad on the processor (Processor 1) cold plate and the M.2 backplane cold plate with a new ones. Make sure to follow Gap pad/putty pad replacement guidelines. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 169 Figure 166. Gap pad on processor cold plate and M.2 backplane cold plate Step 4. Install the M.2 backplane assembly. Connect the cable to the M.2 backplane assembly. Install the M.2 backplane assembly into the node. Secure the six screws. Figure 167.

  • Page 170: Memory Module Replacement

    • Memory modules are sensitive to static discharge and require special handling. In addition to the standard guidelines for “Handling static-sensitive devices” on page – Always wear an electrostatic-discharge strap when removing or installing memory modules. Electrostatic-discharge gloves can also be used. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 171 – Never hold two or more memory modules together so that they touch. Do not stack memory modules directly on top of each other during storage. – Never touch the gold memory module connector contacts or allow these contacts to touch the outside of the memory-module connector housing.
  • Page 172 Unlock the retaining clips on each end of the memory module connector with the memory module tool. Place the tip of the tool on the top of the retaining clip. Press the tool to rotate and unlock the retaining clip. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 173 Figure 171. Pressing retaining clips on memory module connector Attention: • Memory modules are static-sensitive devices. The package must be grounded before it is opened. • To avoid breaking the retaining clips or damaging the memory module connectors, open and close the clips gently.
  • Page 174 Figure 172. Memory module removal Note: Memory module should be in horizontal position during removal. Step 5. Install the memory module tool to the DIMM comb. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 175 Figure 173. Memory module tool installation After you finish If you are instructed to return the component or optional device, follow all packaging instructions, and use any packaging materials for shipping that are supplied to you. Install a memory module Use this information to install a memory module.
  • Page 176 Attention: Make sure to remove or install memory module 20 seconds after disconnecting power cords from the system. It allows the system to be completely discharged of electricity and safe for handling memory module. Step 1. Remove the memory module tool attached to the DIMM comb. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 177 Figure 175. Memory module tool removal Note: Memory module tool is recommended due to space limitations caused by location of water loop tubes through the memory section. Step 2. Unlock the retaining clips on each end of the memory module connector with the memory module tool.
  • Page 178 Align the memory module with the connector while gently placing it onto the connector. Then, firmly press both ends of the memory module straight down into the connector until the retaining clips snap into the locked position. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 179 Figure 177. Memory module installation Note: Memory module should be in horizontal position during installation. Step 6. Make sure the retaining clips are properly locked. If the retaining clips are open, firmly press right and left ends of the memory module until the clips are locked. Chapter 6 Hardware replacement procedures...

  • Page 180: Osfp Module Replacement

    OSFP module replacement Use the following procedures to remove and install the OSFP module. Remove the OSFP module Use this information to remove the OSFP module. About this task ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 181 Required tools Make sure that you have the SD650-N V3 OSFP Putty Pad Kit to properly replace the component. Prepare the following screwdrivers to ensure you can install and remove corresponding screws properly. Screw Type/Usage Screwdriver Type Hex screw 4.5 mm hex head screwdriver Releasing OSFP module from conduction plate Flat head screwdriver Attention:...
  • Page 182 Figure 181. 7mm NVMe drive assembly cable routing Figure 182. E3.S drive cable routing Step 3. Disconnect MCIO 2, MCIO 3, and MCIO 4 cables from carrier board. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 183 Figure 183. Disconnecting MCIO 2, MCIO 3, and MCIO 4 cables From (carrier board in GPU node) To (system board in compute node) MCIO 2 PCIe x 16 MCIO 4 connector MCIO 3 PCIe x 16 MCIO 1 connector MCIO 4 PCIe x 16 MCIO 2 connector Stud (in between the nodes) Make sure the cables are routed around the stud as shown in the illustration.
  • Page 184 Network connector (near Compute node) Port 1 Stud (in between the nodes) Make sure the cables are routed around the stud as shown in the illustration. Step 5. Remove the three screws from the OSFP module. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 185 Figure 185. Removing the OSFP module Step 6. Remove the OSFP module. Insert a flat head screwdriver into the gap between the OSFP module and the OSFP module conduction plate; then, rotate the flat head screwdriver to release the OSFP module from the conduction plate.
  • Page 186 Figure 186. Removing the OSFP module Step 7. Disconnect the power cables from the OSFP module. Figure 187. OSFP module power cables removal ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 187 • Turn off the corresponding DWC tray that you are going to perform the task on. Firmware and driver download: You might need to update the firmware or driver after replacing a component. • Go to https://datacentersupport.lenovo.com/products/servers/thinksystem/sd650nv3/7d7n/downloads/driver- list/ to see the latest firmware and driver updates for your server. • Go to “Update the firmware”...
  • Page 188 Install the OSFP module. Align the OSFP module to the guide pins, and install the OSFP module to the compute node. Tighten the three screws to secure the OSFP module to the compute node. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 189 Figure 190. OSFP module installation Step 4. Connect the OSFP module cables to the network board on the GPU node. Note: Hold the cable connector at 45 degree angle when inserting it into the port. Chapter 6 Hardware replacement procedures...
  • Page 190 Stud (in between the nodes) Make sure the cables are routed around the stud as shown in the illustration. Figure 191. Connecting the OSFP module cables to the network board Step 5. Connect MCIO 1, MCIO 2, and MCIO 3 cables to the carrier board. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 191 → → Make sure to follow the sequence Figure 192. Two-processor configuration—Connecting MCIO 2, MCIO 3, and MCIO 4 cable to the GPU node From (carrier board in GPU node) To (system board in compute node) MCIO 2 PCIe x 16 MCIO 4 connector MCIO 3 PCIe x 16 MCIO 1 connector MCIO 4...
  • Page 192 Figure 193. Drive assembly cable routing Figure 194. 7mm NVMe drive assembly cable routing Figure 195. E3.S drive cable routing ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...

  • Page 193: Power Distribution Board Replacement (Trained Technician Only)

    After you finish 1. Install the M.2 backplane assembly. See “Install the M.2 backplane assembly” on page 158. 2. Install the cross braces. See “Install the cross braces” on page 3. Install the tray cover. See “Install the tray cover” on page 4.
  • Page 194 “Remove the E3.S drive cage assembly” on page 136. Remove the OSFP module. See “Remove the OSFP module” on page 172. Step 2. Remove the two Hex screws from the OSFP module with a 4.5 mm hex head screwdriver. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 195 Figure 196. OSFP module conduction plate Hex screws removal Step 3. Remove the OSFP module conduction plate. With alcohol cleaning pads, wipe of any remaining putty pads from the conduction plate. Screw Type Screwdriver Type Phillips #1 head screwdriver M3x5 screw (x3) M3 screw (x2) T10 screwdriver Chapter 6...
  • Page 196 Figure 197. OSFP module conduction plate removal Step 4. Remove two Torx T10 screws (per node); then, remove the VR (voltage regulator) clamp plate out of the node. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 197 Figure 198. VR clamp plate removal Step 5. Remove water loop screws (x9 Torx T10 screws for two nodes) with a torque screwdriver set to the proper torque. Note: For reference, the torque required for the screws to be fully tightened/removed is 5.0+/- 0.5 lbf-in, 0.55+/- 0.05 N-M.
  • Page 198 Orient two water loop carriers with the guide pins; then, gently put the two water loop carriers down and ensure they are seated firmly on the water loop. Figure 201. Water loop carrier installation ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 199 Step 8. Tighten water loop carrier screws (x12 Phillips #2 screws). Figure 202. Water loop carrier screws installation Step 9. Loosen processors properly. Fully loosen all Torx T30 captive screws (x8 Torx T30 captive screws) on cold plates with a general screwdriver, following the removal sequence shown on the cold plate label.
  • Page 200 Gently pull the power distribution board connector to disconnect it from the system board. Carefully pull the power distribution board inwards to disengage it from the node. Note: Use a 3/16" hex head screwdriver to ensure the proper removal and installation. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 201 Figure 205. Power distribution board removal After you finish If you are instructed to return the component or optional device, follow all packaging instructions, and use any packaging materials for shipping that are supplied to you. Install the power distribution board Use this information to install the power distribution board.
  • Page 202 Gently push the power distribution board connector to ensure it is connected to the system board. Fasten the five Phillips #1 screws. Note: Use a 3/16" hex head screwdriver to ensure the proper removal and installation. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 203 Figure 206. Power distribution board installation Step 2. Check the gap pads on the water loop, if any of them are damaged or detached, replace them with the new ones. Figure 207. Water loop gap pads Make sure to follow Gap pad/putty pad replacement guidelines.
  • Page 204 Note: For reference, the torque required for the screws to be fully tightened/removed is 10+/- 2.0 lbf-in, 1.1+/- 0.2 N-m. Attention: To prevent damage to components, make sure that you follow the indicated tightening sequence. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 205 Figure 209. Securing Torx T30 captive screws Step 5. Loosen water loop carrier screws (x12 Phillips #2 screws for two nodes). Figure 210. Loosening water loop carrier screws Step 6. Carefully lift each water loop carrier up and away from the water loop one at a time. Chapter 6 Hardware replacement procedures...
  • Page 206 Note: For reference, the torque required for the screws to be fully tightened/removed is 5.0+/- 0.5 lbf-in, 0.55+/- 0.05 N-M. Figure 212. Water loop screws installation Step 8. Install the Torx T10 screws (x7 screws) to secure the quick connect. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 207 Figure 213. Quick connect screw installation Step 9. Install two VR clamp plates into the nodes and install Torx T10 screws (4x Torx T10 screws for two nodes). Figure 214. VR clamp plate installation Step 10. Replace the putty pads on the top side and the bottom side of the OSFP module conduction plate. Chapter 6 Hardware replacement procedures...
  • Page 208 Gap pad/putty pad replacement guidelines. Step 11. Install the OSFP module conduction plate onto the water loop. Screw Type Screwdriver Type Phillips #1 head screwdriver M3x5 screw (x3) M3 screw (x2) T10 screwdriver ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 209 Figure 216. Installing the OSFP module conduction plate After you finish 1. Install the OSFP module. See “Install the OSFP module” on page 179. 2. Install the drive cage. Depending on system configurations, see “Install a drive cage assembly” on page 118, “Install a 7mm NVMe drive cage assembly”...

  • Page 210: Processor Replacement (Trained Technician Only)

    Processor replacement (trained technician only) Use the following procedures to replace an assembled processor. Attention: Before you begin replacing a processor, make sure that you use Lenovo proven alcohol cleaning pad and thermal grease. Important: The processor in your solution can throttle in response to thermal conditions, temporarily lowering its speed to reduce heat output.
  • Page 211 • Do not touch the processor socket or processor contacts. Processor-socket contacts are very fragile and easily damaged. Contaminants on the processor contacts, such as oil from your skin, can cause connection failures. • Do not allow the thermal grease on the processor or water loop to come in contact with anything. Contact with any surface can compromise the thermal grease, rendering it ineffective.
  • Page 212 Remove the OSFP module conduction plate. With alcohol cleaning pads, wipe of any remaining putty pads from the conduction plate. Screw Type Screwdriver Type M3x5 screw (x3) Phillips #1 head screwdriver M3 screw (x2) T10 screwdriver ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 213 Figure 219. OSFP module conduction plate removal Step 4. Remove two Torx T10 screws (per node); then, remove the VR (voltage regulator) clamp plate out of the node. Chapter 6 Hardware replacement procedures...
  • Page 214 Remove water loop screws (x9 Torx T10 screws for two nodes) with a torque screwdriver set to the proper torque. Note: For reference, the torque required for the screws to be fully tightened/removed is 5.0+/- 0.5 lbf-in, 0.55+/- 0.05 N-M. Figure 221. Water loop screw removal ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 215 Step 6. Remove Torx T10 screws (x7 screws) to loosen the quick connect. Note: For reference, the torque required for the screws to be fully tightened/removed is 5.0+/- 0.5 lbf-in, 0.55+/- 0.05 N-M. Figure 222. Quick connect screw removal (Compute node) Step 7.
  • Page 216 Attention: To prevent damage to components, make sure that you follow the indicated loosening sequence. Rotate all anti-tilt wire bails (16x anti-tilt wire bails for two nodes) inwards to the unlocked position. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 217 Figure 225. Loosening Torx T30 captive screws Step 10. Carefully rotate the water loop so one half is sitting on top of the other half. Figure 226. Folding the water loop Step 11. Remove the processor from the retainer. This process differs by processor SKU. Check the processor SKU and follow the applicable procedure.
  • Page 218 Step 13. Remove the processor retainer from the underside of the cold plate. Note: The processor retainer will be discarded and replaced with a new one. Carefully release the retaining clips from the cold plate. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 219 Lift the retainer from the cold plate. Figure 229. Processor retainer removal Step 14. Wipe the thermal grease from the bottom of the cold plate with an alcohol cleaning pad. After you finish • If you are instructed to return the component or optional device, follow all packaging instructions, and use any packaging materials for shipping that are supplied to you.
  • Page 220 • Use extra force to disconnect QSFP cables if they are connected to the solution. • Each processor socket must always contain a cover. When removing or installing a processor, protect empty processor sockets with a cover. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 221 • See for a list of processors supported for your system. All processors on https://serverproven.lenovo.com the system board must have the same speed, number of cores, and frequency. • Optional devices available for your system might have specific processor requirements. See the documentation that comes with the optional device for information.
  • Page 222 CPU Max only) Make sure the slot on the TIM breaking cam is vertical. See (Intel ® Xeon ® Figure 4. Gently place the processor retainer on the processor; then, carefully press the four sides of the processor retainer to secure the processor. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 223 TIM breaking cam Figure 233. Installing a Intel ® Xeon ® CPU Max processor retainer Figure 232. Installing a processor retainer CPU Max only) After installing the retainer onto processor, make sure the slot (Intel ® Xeon ® on the TIM breaking cam is vertical. TIM breaking cam Figure 234.
  • Page 224 Figure 236. Processor installation Step 5. Rotate all anti-tilt wire bails (8x anti-tilt wire bails per node) outwards to the unlocked position. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 225 Figure 237. Processor unlocked position Step 6. Check the gap pads on the water loop, if any of them are damaged or detached, replace them with the new ones. Figure 238. Water loop gap pads Make sure to follow Gap pad/putty pad replacement guidelines.
  • Page 226 Note: For reference, the torque required for the screws to be fully tightened/removed is 10+/- 2.0 lbf-in, 1.1+/- 0.2 N-m. Attention: To prevent damage to components, make sure that you follow the indicated tightening sequence. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 227 Figure 240. Securing Torx T30 captive screws Step 9. Loosen water loop carrier screws (x12 Phillips #2 screws for two nodes). Figure 241. Loosening water loop carrier screws Step 10. Carefully lift each water loop carrier up and away from the water loop one at a time. Chapter 6 Hardware replacement procedures...
  • Page 228 Note: For reference, the torque required for the screws to be fully tightened/removed is 5.0+/- 0.5 lbf-in, 0.55+/- 0.05 N-M. Figure 243. Water loop screws installation Step 12. Install the Torx T10 screws (x7 screws) to secure the quick connect. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 229 Figure 244. Quick connect screw installation Step 13. Install two VR clamp plates into the nodes and install Torx T10 screws (4x Torx T10 screws for two nodes). Figure 245. VR clamp plate installation Step 14. Replace the putty pads on the top side and the bottom side of the OSFP module conduction plate. Chapter 6 Hardware replacement procedures...
  • Page 230 Gap pad/putty pad replacement guidelines. Step 15. Install the OSFP module conduction plate onto the water loop. Screw Type Screwdriver Type Phillips #1 head screwdriver M3x5 screw (x3) M3 screw (x2) T10 screwdriver ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...

  • Page 231: Only)

    Figure 247. Installing the OSFP module conduction plate After you finish 1. Install the OSFP module. See “Install the OSFP module” on page 179. 2. Install the drive cage. Depending on system configurations, see “Install a drive cage assembly” on page 118, “Install a 7mm NVMe drive cage assembly”...
  • Page 232 • To avoid damaging the water loop, always use the water loop carrier when removing, installing or folding the water loop. Watch the procedure A video of this procedure is available at YouTube: https://www.youtube.com/playlist?list=PLYV5R7hVcs- DCqagDqjI0xAqs12RCavnk Procedure Step 1. Make preparations for this task. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 233 Remove the tray from the enclosure. See “Remove a DWC tray from the enclosure” on page Remove the tray cover. See “Remove the tray cover” on page Remove the cross braces. See “Remove the cross braces” on page Remove the DIMM comb. See “Remove a DIMM comb”...
  • Page 234 Note: Avoid touching the connectors on the system board. Be careful not to damage any surrounding components inside the node. Figure 250. System board removal ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 235 If you are instructed to return the component or optional device, follow all packaging instructions, and use any packaging materials for shipping that are supplied to you. Important: Before you return the system board, make sure that you install the processor socket dust covers from the new system board.
  • Page 236 Watch the procedure A video of this procedure is available at YouTube: https://www.youtube.com/playlist?list=PLYV5R7hVcs- DCqagDqjI0xAqs12RCavnk Procedure Note: Depending on the model, your solution might look slightly different from the illustration. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 237 Step 1. Install the system board. Carefully hold the front guide pin and tilt the system board at an angle. Align the connectors with the corresponding holes on the front of the node; then, gently slide the system board forward. Orient the system board on the two guide pins near the rear of the node to insure proper alignment;...
  • Page 238 Make sure to follow Gap pad/putty pad replacement guidelines. Step 4. Align the VR water loop trough with the holes on the system board; then, fasten the two Torx T10 screws (per node). ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 239 “Update the Vital Product Data (VPD)” on page 232. Machine type number and serial number can be found on the ID label, see “Identify the solution and access the Lenovo XClarity Controller” on page 15. Enable TPM/TCM. See “Enable TPM” on page 233.
  • Page 240 Using Lenovo XClarity Provisioning Manager Steps: 1. Start the server and press the key according to the on-screen instructions. The Lenovo XClarity Provisioning Manager interface is displayed by default. 2. Choose System Summary. The “System Summary” tab page is displayed.
  • Page 241 To set the TPM policy from Lenovo XClarity Provisioning Manager: 1. Start the server and press the key specified in the on-screen instructions to display the Lenovo XClarity Provisioning Manager interface. (For more information, see the “Startup” section in the LXPM documentation compatible with your server at https://pubs.lenovo.com/lxpm-overview/...
  • Page 242 Note: Although the setting undefined is available as a policy setting, it should not be used. • From Lenovo XClarity Essentials OneCLI Note: Please note that a Local IPMI user and password must be setup in Lenovo XClarity Controller for remote accessing to the target system.
  • Page 243 OneCli.exe config set TrustedComputingGroup.HideTPMfromOS "Yes" --imm <userid>:<password>@<ip_address> --override where: • <userid>:<password> are the credentials used to access the BMC (Lenovo XClarity Controller interface) of your server. The default user ID is USERID, and the default password is PASSW0RD (zero, not an uppercase o) •...
  • Page 244 Follow the procedure below to see the TPM firmware version: From Lenovo XClarity Provisioning Manager 1. Start the server and press the key specified in the on-screen instructions to display the Lenovo XClarity Provisioning Manager interface. (For more information, see the “Startup” section in the LXPM documentation compatible with your server at https://pubs.lenovo.com/lxpm-overview/...

  • Page 245: Replace Components In The Gpu Node

    To enable UEFI Secure Boot from Lenovo XClarity Provisioning Manager: 1. Start the server and press the key specified in the on-screen instructions to display the Lenovo XClarity Provisioning Manager interface. (For more information, see the “Startup” section in the LXPM documentation compatible with your server at https://pubs.lenovo.com/lxpm-overview/...
  • Page 246 Install the bus bar Use this information to install the bus bar. About this task Attention: • Read “Installation Guidelines” on page 37 “Safety inspection checklist” on page 38 to ensure that you work safely. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 247 • Turn off the corresponding DWC tray that you are going to perform the task on. • Disconnect all external cables from the enclosure. • Use extra force to disconnect QSFP cables if they are connected to the solution. • To avoid damaging the water loop, always use the water loop carrier when removing, installing or folding the water loop.

  • Page 248: Carrier Board Replacement

    – SD650-N V3 Water Loop Putty Pad Kit – SD650-N V3 SXM5 PCM Fixture – SXM5 PCM Kit • Screws and screwdrivers Prepare the following screwdrivers to ensure you can install and remove corresponding screws properly. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 249 Screw Type Screwdriver Type Torx T10 screw Torx T10 head screwdriver Torx T15 screw Torx T15 head screwdriver Phillips #1 screw Phillips #1 head screwdriver Phillips #2 screw Phillips #2 head screwdriver Attention: • Read “Installation Guidelines” on page 37 “Safety inspection checklist”...
  • Page 250 Release the GPU board retention clips and rotate them towards the GPU board. Pull the carrier board towards the front of the server to disconnect it from the GPU board. Hold the carrier board at an angle and remove it from the tray. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 251 Figure 261. Carrier board removal After you finish If you are instructed to return the component or optional device, follow all packaging instructions, and use any packaging materials for shipping that are supplied to you. Install the carrier board Use this information to install the carrier board. About this task Required tools Make sure you have the required tools listed below in hand to properly replace the component.
  • Page 252 Watch the procedure A video of this procedure is available at YouTube: https://www.youtube.com/playlist?list=PLYV5R7hVcs- DCqagDqjI0xAqs12RCavnk Procedure Step 1. Rotate the GPU board retention clips and handle towards the GPU board. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 253 Figure 262. GPU board retention clips and handles rotation Step 2. Install the carrier board. Hold the carrier board at an angle and align it with the guide pins on the tray; then, install the carrier board to the tray. Push the carrier board towards the GPU board to connect it to the GPU board.
  • Page 254 Figure 264. Securing the carrier board to the GPU board Step 4. Install the two standoffs to the carrier board. Figure 265. Installing carrier board standoffs Step 5. Connect the carrier board power cable. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...

  • Page 255: Gpu Replacement (Trained Technician Only)

    Figure 266. Connecting carrier board power cable From (carrier board) To (GPU node power distribution board) Power and side band connector Power connector After you finish 1. Install the network board. See “Install the network board (trained technician only)” on page 319.
  • Page 256 237. Remove the MCIO cables. Follow the guidance and routing information in Chapter 7 “Internal cable routing” on page 336. Step 2. Remove the carrier board power cable from the GPU node. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 257 Figure 267. Carrier board power cable removal From (carrier board) To (GPU node power distribution board) Power and side band connector Power connector Step 3. Remove cable tie from the GPU board. Figure 268. Cable tie removal Step 4. Remove the Hex screw (x1) and the PH1 screws (x3) from the water loop with a torque screwdriver set to the proper torque.
  • Page 258 Remove the quick connect screws (x4 Torx T10) with a torque screwdriver set to the proper torque. Note: For reference, the torque required for the screws to be fully tightened/removed is 5.0+/- 0.5 lbf-in, 0.55+/- 0.05 N-M. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 259 Figure 271. Quick connect screw removal (GPU node) Step 7. Follow the screw removal sequence specified on the network board label, and remove network cold plate screws (x8 Torx T10 screws) with a torque screwdriver set to the proper torque. Note: For reference, the torque required for the screws to be fully tightened/removed is 5.0+/- 0.5 lbf-in, 0.55+/- 0.05 N-M.
  • Page 260 Step 10. Release the GPU cold plates from the GPUs. There are notches on the sides of the GPU cold plates for inserting a flat head screwdriver. The locations of the notches are shown in the illustration below. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 261 Attention: Inserting the flat head screwdriver ONLY to the notches circled in the illustration below. Otherwise, the screwdriver may damage the GPUs. Figure 275. Notches for releasing GPU cold plates Insert a flat screwdriver into all the notches shown in the illustration; then, slightly rotate the screwdriver to release the GPU cold plates from the GPUs.
  • Page 262 Step 12. Tighten water loop carrier screws (x20 Phillips #2 screws) with a torque screwdriver set to the proper torque. Note: For reference, the torque required for the screws to be fully tightened/removed is 5.0+/- 0.5 lbf-in, 0.55+/- 0.05 N-M. Figure 278. Tightening water loop carrier screws (GPU node) ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 263 Step 13. Carefully rotate the GPU node side water loop so that it is sitting on top of the Compute node side water loop. Figure 279. Folding the GPU node side water loop onto the Compute node side water loop Step 14.
  • Page 264 Then, carefully remove the GPU out of the GPU board. Note: For reference, the torque required for the screws to be fully tightened/removed is 0.45-0.56 N-m, 4.0-5.0 in-lbf. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 265 → → → Screw installation sequence: Figure 283. GPU removal After you finish If you are instructed to return the component or optional device, follow all packaging instructions, and use any packaging materials for shipping that are supplied to you. Install a GPU (trained technician only) Use this information to install a GPU.
  • Page 266 • Go to “Update the firmware” on page 341 for more information on firmware updating tools. Watch the procedure A video of this procedure is available at YouTube: https://www.youtube.com/playlist?list=PLYV5R7hVcs- DCqagDqjI0xAqs12RCavnk Procedure ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 267 Note: Make sure to inspect the connectors and sockets on the GPU and the GPU board. Do not use the GPU or the GPU board if its connectors are damaged or missing, or if there are debris in the sockets. Replace the GPU or the GPU board with a new one before continuing the installation procedure.
  • Page 268 Replace the putty pads (x5) on the GPU. Make sure to align the putty pads to the GPU VR ( and the markings on GPU. Repeat to replace all putty pads on the four GPUs. GPU VR (Cover the GPU VR with putty pad) ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 269 Figure 289. GPU putty pads replacement Make sure to follow Gap pad/putty pad replacement guidelines. Step 3. Flip the GPU node water loop back to the GPU node. Figure 290. Flipping GPU node water loop to the GPU node Step 4. Loosen water loop carrier screws (x20 Phillips #2 screws).
  • Page 270 Figure 291. Water loop screws and quick connect screws installation (GPU node) Step 5. Remove the water loop carrier from the GPU node. Figure 292. Water loop carrier removal (GPU node) ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 271 Step 6. Install GPU cold plate screws (x16 PH2 screws). Install the GPU cold plates in diagonal pattern. Complete installing screws of GPU 2, then proceed to GPU 4 ➙ GPU 1 ➙ GPU 3. Set the torque screwdriver to 3.5 lb-In, 0.39 N-m. →...
  • Page 272 Install water loop screws and quick connect screws (x11 Torx T10 screws) with a torque screwdriver set to the proper torque. Note: For reference, the torque required for the screws to be fully tightened/removed is 5.0+/- 0.5 lbf-in, 0.55+/- 0.05 N-M. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 273 Figure 296. Water loop Torx T10 screws installation (GPU node) Step 10. Install the Hex screw (x1) and the PH1 screws (x3). Note: For reference, the torque required for the screws to be fully tightened/removed is 5.0+/- 0.5 lbf-in, 0.55+/- 0.05 N-M. Figure 297.
  • Page 274 1. Install the MCIO cables. Follow the guidance and routing information in Chapter 7 “Internal cable routing” on page 336. 2. Install the bus bar. See “Install the bus bar” on page 238. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...

  • Page 275: Gpu Board (Trained Technician Only)

    3. Install the cross braces. See “Install the cross braces” on page 4. Install the tray cover. See “Install the tray cover” on page 5. Install the tray into the enclosure. See “Install a DWC tray in the enclosure” on page 6.
  • Page 276 Step 3. Release the carrier board from the GPU board. Rotate the GPU board handle towards the carrier board. Release the GPU board retention clips and rotate them towards the GPU board. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 277 Figure 301. Releasing the carrier Step 4. Disconnect the carrier board from the GPU board. Figure 302. Disconnecting the carrier board from the GPU board Step 5. Remove M3 screws (x7) from the GPU board. Note: For reference, the torque required for the screws to be fully tightened/removed is 5.0+/- 0.5 lbf-in, 0.55+/- 0.05 N-M.
  • Page 278 Note: For reference, the torque required for the screws to be fully tightened/removed is 0.6 N-m, 5.3 in-lbf, torque tolerance is +/- 4%. Figure 304. GPU board T15 screws removal Step 7. Remove the GPU board from the tray. Figure 305. GPU board removal ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 279 After you finish If you are instructed to return the component or optional device, follow all packaging instructions, and use any packaging materials for shipping that are supplied to you. Install a GPU board (trained technician only) Use this information to install a GPU board. About this task Required tools Make sure you have the required tools listed below in hand to properly replace the component.
  • Page 280 • Once the GPU board is removed from the plastic protective bag, hold two handles with both hands to move the GPU board. Figure 307. Removing GPU board from the package box ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 281 Firmware and driver download: You might need to update the firmware or driver after replacing a component. • Go to https://datacentersupport.lenovo.com/products/servers/thinksystem/sd650nv3/7d7n/downloads/driver- to see the latest firmware and driver updates for your server. list/ • Go to “Update the firmware”...
  • Page 282 Install the T15 screws (x2) on the GPU board. Note: For reference, the torque required for the screws to be fully tightened/removed is 0.6 N-m, 5.3 in-lbf, torque tolerance is +/- 4%. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 283 Figure 310. Installing the T15 screw on the GPU board Step 4. Push the carrier board to connect it to the GPU board. Figure 311. Connecting the carrier board to the GPU board. Step 5. Secure the carrier board to the GPU board. Insert the GPU board retention clips into the slots on the carrier board.
  • Page 284 7. Install the tray into the enclosure. See “Install a DWC tray in the enclosure” on page 8. Connect all required external cables to the solution. Note: Use extra force to connect QSFP cables to the solution. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...

  • Page 285: Gpu Board Assembly Replacement

    9. Check the power LED on each node to make sure it changes from fast blink to slow blink to indicate all nodes are ready to be powered on. GPU board assembly replacement (trained technician only) Use the following procedures to remove and install the GPU board assembly. Remove a GPU board assembly (trained technician only) Use this information to remove a GPU board assembly.
  • Page 286 Rotate the GPU board handle towards the carrier board. Release the GPU board retention clips and rotate them towards the GPU board. Figure 315. Releasing the carrier Step 4. Disconnect the carrier board from the GPU board. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 287 Figure 316. Disconnecting the carrier board from the GPU board Step 5. Remove M3 screws (x7) from the GPU board assembly. Note: For reference, the torque required for the screws to be fully tightened/removed is 5.0+/- 0.5 lbf-in, 0.55+/- 0.05 N-M. Figure 317.
  • Page 288 Remove the GPU board assembly from the node. After you finish If you are instructed to return the component or optional device, follow all packaging instructions, and use any packaging materials for shipping that are supplied to you. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 289 Install a GPU board assembly(trained technician only) Use this information to install a GPU board assembly. About this task Required tools Make sure you have the required tools listed below in hand to properly replace the component. • Screws and screwdrivers Prepare the following screwdrivers to ensure you can install and remove corresponding screws properly.
  • Page 290 • To identify the gap pad/putty pad location and orientation, see “Gap pad/Putty pad identification and location” on page • Before replacing the gap pad/putty pad, gently clean the interface plate or the hardware surface with an alcohol cleaning pad. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 291 Firmware and driver download: You might need to update the firmware or driver after replacing a component. • Go to https://datacentersupport.lenovo.com/products/servers/thinksystem/sd650nv3/7d7n/downloads/driver- to see the latest firmware and driver updates for your server. list/ • Go to “Update the firmware”...
  • Page 292 Note: For reference, the torque required for the screws to be fully tightened/removed is 0.6 N-m, 5.3 in-lbf, torque tolerance is +/- 4%. Figure 323. Installing the T15 screws on the GPU board assembly Step 4. Push the carrier board to connect it to the GPU board. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 293 Figure 324. Connecting the carrier board to the GPU board. Step 5. Secure the carrier board to the GPU board. Insert the GPU board retention clips into the slots on the carrier board. Rotate the GPU board handle toward the GPU board, and press it down to lock the carrier board in place.

  • Page 294: Gpu Power Distribution Board Replacement (Trained Technician Only)

    – SD650-N V3 Water Loop Putty Pad Kit – SD650-N V3 SXM5 PCM Fixture – SXM5 PCM Kit • Screws and screwdrivers ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 295 Prepare the following screwdrivers to ensure you can install and remove corresponding screws properly. Screw Type Screwdriver Type 6 mm hex head screwdriver Hex screw (GPU node water loop) Hex screw (OSFP module conduction plate) 4.5 mm hex head screwdriver Torx T10 screw Torx T10 head screwdriver Phillips #1 screw...
  • Page 296 Figure 328. Carrier board power cable removal From (carrier board) To (GPU node power distribution board) Power and side band connector Power connector Step 3. Remove cable tie from the GPU board. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 297 Figure 329. Cable tie removal Step 4. Remove the Hex screw (x1) and the PH1 screws (x3) from the water loop with a torque screwdriver set to the proper torque. Note: For reference, the torque required for the screws to be fully tightened/removed is 5.0+/- 0.5 lbf-in, 0.55+/- 0.05 N-M.
  • Page 298 (x8 Torx T10 screws) with a torque screwdriver set to the proper torque. Note: For reference, the torque required for the screws to be fully tightened/removed is 5.0+/- 0.5 lbf-in, 0.55+/- 0.05 N-M. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 299 Figure 333. Network card screw removal Step 8. Loosen GPU cold plate screws (x16 Torx T10 screws) in the diagonal pattern with a torque screwdriver set to the proper torque. Note: For reference, the torque required for the screws to be fully tightened/loosened is 0.4+/-0.05 N-m, 3.5+/-0.5 lb-In.
  • Page 300 Figure 336. Notches for releasing GPU cold plates Insert a flat screwdriver into all the notches shown in the illustration; then, slightly rotate the screwdriver to release the GPU cold plates from the GPUs. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 301 Figure 337. Releasing the GPU cold plate from the GPU Step 11. Gently put the water loop carrier down onto the water loop and ensure it is seated firmly on the water loop. Figure 338. Water loop carrier installation (GPU node) Step 12.
  • Page 302 Step 14. Immediately clean the PCM off all the GPUs with alcohol cleaning pads. Gently clean the PCM to avoid GPU damages. Attention: • It is recommended to clean the PCM while it is in liquid state. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 303 • The electrical components around the die on the GPUs are extremely delicate. When removing the PCM and cleaning the GPU die, avoid touching the electrical components to prevent damages. Figure 341. Cleaning PCM off from all GPUs Step 15. With alcohol cleaning pads, wipe off any remaining putty pad and PCMs from the water loop and components in the GPU node.
  • Page 304 Figure 344. GPU power distribution board screw removal Step 17. Remove the GPU power distribution board out of the node. Note: When handling the GPU power distribution board, make sure to follow GPU power distribution board handling guidelines. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 305 Figure 345. Removing the GPU power distribution board After you finish If you are instructed to return the component or optional device, follow all packaging instructions, and use any packaging materials for shipping that are supplied to you. Install a GPU power distribution board (trained technician only) Use this information to install a GPU power distribution board.
  • Page 306 • Turn off the corresponding DWC tray that you are going to perform the task on. • Disconnect all external cables from the enclosure. • Use extra force to disconnect QSFP cables if they are connected to the solution. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 307 • A torque screwdriver is available for request if you do not have one at hand. Firmware and driver download: You might need to update the firmware or driver after replacing a component. • Go to https://datacentersupport.lenovo.com/products/servers/thinksystem/sd650nv3/7d7n/downloads/driver- to see the latest firmware and driver updates for your server. list/ • Go to “Update the firmware”...
  • Page 308 Repeat to replace the PCM of all four GPU cold plates. Figure 349. Water loop GPU cold plate PCM replacement (GPU node) Replace the putty pads on the GPU node water loop. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 309 Figure 350. Water loop putty pads replacement (GPU node) Replace the putty pads (x5) on the GPU. Make sure to align the putty pads to the GPU VR ( and the markings on GPU. Repeat to replace all putty pads on the four GPUs. GPU VR (Cover the GPU VR with putty pad) Figure 351.
  • Page 310 Loosen water loop carrier screws (x20 Phillips #2 screws). Figure 353. Water loop screws and quick connect screws installation (GPU node) Step 6. Remove the water loop carrier from the GPU node. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 311 Figure 354. Water loop carrier removal (GPU node) Step 7. Install GPU cold plate screws (x16 PH2 screws). Install the GPU cold plates in diagonal pattern. Complete installing screws of GPU 2, then proceed to GPU 4 ➙ GPU 1 ➙ GPU 3. Set the torque screwdriver to 3.5 lb-In, 0.39 N-m.
  • Page 312 Install the quick connect screws (x4 Torx T10) with a torque screwdriver set to the proper torque. Note: For reference, the torque required for the screws to be fully tightened/removed is 5.0+/- 0.5 lbf-in, 0.55+/- 0.05 N-M. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 313 Figure 357. Quick connect screw installation (GPU node) Step 10. Install water loop screws and quick connect screws (x11 Torx T10 screws) with a torque screwdriver set to the proper torque. Note: For reference, the torque required for the screws to be fully tightened/removed is 5.0+/- 0.5 lbf-in, 0.55+/- 0.05 N-M.
  • Page 314 Figure 359. Water loop Hex and PH1 screws installation (GPU node) Step 12. Install the cable tie to the GPU board. Figure 360. Installing the cable tie Step 13. Connect the carrier board power cable. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...

  • Page 315: Network Board (Trained Technician Only)

    Figure 361. Connecting carrier board power cable From (carrier board) To (GPU node power distribution board) Power and side band connector Power connector After you finish 1. Install the MCIO cables. Follow the guidance and routing information in Chapter 7 “Internal cable routing”...
  • Page 316 Remove the MCIO cables. Follow the guidance and routing information in Chapter 7 “Internal cable routing” on page 336. Step 2. Disconnect the OSFP module cables from the network board on the GPU node. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 317 Figure 362. Disconnecting OSFP module cables from the network board From OSFP module connectors Network board connectors Network connector (near GPU node) Port 0 Power connector P1 OOB port 0 Power connector P2 OOB port 1 (on network board) and power connector (on carrier board) Network connector (near Compute node) Port 1...
  • Page 318 Remove the Hex screw (x1) and the PH1 screws (x3) from the water loop with a torque screwdriver set to the proper torque. Note: For reference, the torque required for the screws to be fully tightened/removed is 5.0+/- 0.5 lbf-in, 0.55+/- 0.05 N-M. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 319 Figure 365. Water loop Hex and PH1 screws removal (GPU node) Step 6. Remove water loop screws and quick connect screws (x11 Torx T10 screws) with a torque screwdriver set to the proper torque. Note: For reference, the torque required for the screws to be fully tightened/removed is 5.0+/- 0.5 lbf-in, 0.55+/- 0.05 N-M.
  • Page 320 Loosen GPU cold plate screws (x16 Torx T10 screws) in the diagonal pattern with a torque screwdriver set to the proper torque. Note: For reference, the torque required for the screws to be fully tightened/loosened is 0.4+/-0.05 N-m, 3.5+/-0.5 lb-In. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 321 Figure 369. Loosening GPU cold plate screw Step 10. Release the front and rear MISC conduction plates of the water loop from the GPU board. Insert a flat head screwdriver into the gaps between the MISC conduction plates (front and rear) and the GPU board.
  • Page 322 Figure 372. Releasing the GPU cold plate from the GPU Step 12. Gently put the water loop carrier down onto the water loop and ensure it is seated firmly on the water loop. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 323 Figure 373. Water loop carrier installation (GPU node) Step 13. Tighten water loop carrier screws (x20 Phillips #2 screws) with a torque screwdriver set to the proper torque. Note: For reference, the torque required for the screws to be fully tightened/removed is 5.0+/- 0.5 lbf-in, 0.55+/- 0.05 N-M.
  • Page 324 Figure 376. Cleaning PCM off from all GPUs Step 16. With alcohol cleaning pads, wipe off any remaining putty pad and PCMs from the water loop and components in the GPU node. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 325 Figure 377. Cleaning putty pads from water loop Figure 378. Cleaning putty pads and PCM from the components in the GPU node Step 17. Follow the sequence shown in the illustration below to remove the network board screws (x4 Torx T15 screws) with a torque screwdriver set to the proper torque.
  • Page 326 Figure 379. Network board screws removal Step 18. Remove the network board from the GPU node. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 327 Figure 380. Removing the network board After you finish If you are instructed to return the component or optional device, follow all packaging instructions, and use any packaging materials for shipping that are supplied to you. Install the network board (trained technician only) Use this information to install the network board.
  • Page 328 A video of this procedure is available at YouTube: https://www.youtube.com/playlist?list=PLYV5R7hVcs- DCqagDqjI0xAqs12RCavnk Procedure Step 1. Align the network board to the guide holes on the carrier board; then, place the network board onto the carrier board. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 329 Figure 381. Installing the network board Step 2. Follow the sequence shown in the illustration below to fasten the network board screws (x4 Torx T15 screws) to the carrier board. Note: For reference, the torque required for the screws to be fully tightened/removed is 0.9 newton-meters, 7.96 inch-pounds.
  • Page 330 Replace the Phase Change Material (PCM) and putty pads on the GPU node water loop with new ones. Install the PCM jig on the GPU cold plate. Attach the PCM to the square opening of the jig. Repeat to replace the PCM of all four GPU cold plates. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 331 Figure 383. Water loop GPU cold plate PCM replacement (GPU node) Replace the putty pads on the GPU node water loop. Figure 384. Water loop putty pads replacement (GPU node) Replace the putty pads (x5) on the GPU. Make sure to align the putty pads to the GPU VR ( and the markings on GPU.
  • Page 332 Flip the GPU node water loop back to the GPU node. Figure 386. Flipping GPU node water loop to the GPU node Step 5. Loosen water loop carrier screws (x20 Phillips #2 screws). ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 333 Figure 387. Water loop screws and quick connect screws installation (GPU node) Step 6. Remove the water loop carrier from the GPU node. Figure 388. Water loop carrier removal (GPU node) Chapter 6 Hardware replacement procedures...
  • Page 334 (x8 Torx T10 screws) with a torque screwdriver set to the proper torque. Note: For reference, the torque required for the screws to be fully tightened/removed is 5.0+/- 0.5 lbf-in, 0.55+/- 0.05 N-M. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 335 Figure 390. Network card screw installation Step 9. Install the quick connect screws (x4 Torx T10) with a torque screwdriver set to the proper torque. Note: For reference, the torque required for the screws to be fully tightened/removed is 5.0+/- 0.5 lbf-in, 0.55+/- 0.05 N-M.
  • Page 336 Note: For reference, the torque required for the screws to be fully tightened/removed is 5.0+/- 0.5 lbf-in, 0.55+/- 0.05 N-M. Figure 393. Water loop Hex and PH1 screws installation (GPU node) Step 12. Install the cable tie to the GPU board. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 337 Figure 394. Installing the cable tie Step 13. Connect the carrier board power cable. Figure 395. Connecting carrier board power cable From (carrier board) To (GPU node power distribution board) Power and side band connector Power connector Step 14. Connect the OSFP module cables to the network board on the GPU node. Note: Hold the cable connector at 45 degree angle when inserting it into the port.
  • Page 338 “Install the tray cover” on page 5. Install the tray into the enclosure. See “Install a DWC tray in the enclosure” on page 6. Connect all required external cables to the solution. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...

  • Page 339: Venting Block Replacement

    Note: Use extra force to connect QSFP cables to the solution. 7. Check the power LED on each node to make sure it changes from fast blink to slow blink to indicate all nodes are ready to be powered on. Venting block replacement Use the following procedures to remove and install the venting block.
  • Page 340 – SD650-N V3 Water Loop Putty Pad Kit – SD650-N V3 SXM5 PCM Fixture – SXM5 PCM Kit • Screws and screwdrivers Prepare the following screwdrivers to ensure you can install and remove corresponding screws properly. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 341 Screw Type Screwdriver Type Hex screwdriver 6 mm hex head screwdriver Torx T10 screw Torx T10 head screwdriver Torx T15 screw Torx T15 head screwdriver Phillips #1 screw Phillips #1 head screwdriver Phillips #2 screw Phillips #2 head screwdriver Attention: •...

  • Page 342: Complete The Parts Replacement

    • Reconfigure the disk arrays if you have installed or removed a hot-swap drive or a RAID adapter. See for the LXPM documentation compatible with your solution. https://pubs.lenovo.com/lxpm-overview/ For DW612S Enclosure User Guide, see https://pubs.lenovo.com/dw612s_neptune_enclosure/ ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 343 © Copyright Lenovo 2024...

  • Page 344: Chapter 7. Internal Cable Routing

    SD650-N V3 cable routing Follow the information below for SD650-N V3 cable routing. 1. Connect the carrier board power and side band cable. Figure 401. Carrier board power and side band cable routing ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 345 From Power and side band Power connector (on GPU node power distribution board) connector (on carrier board) 2. Connect the MCIO cables from the GPU node to the Compute node. → → → Attention: Make sure to follow the cable installation sequence: Table 24.
  • Page 346 Figure 403. Secure the cables with cable ties ( ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...

  • Page 347: Chapter 8. System Configuration

    The following methods are available to set the network connection for the Lenovo XClarity Controller if you are not using DHCP: • If a monitor is attached to the server, you can use Lenovo XClarity Provisioning Manager to set the network connection.

  • Page 348: Set Usb Port For Lenovo Xclarity Controller

    For more information about using the Lenovo XClarity Administrator Mobile app, see: http://sysmgt.lenovofiles.com/help/topic/com.lenovo.lxca.doc/lxca_usemobileapp.html Set USB port for Lenovo XClarity Controller connection Before you can access the Lenovo XClarity Controller through the USB port, you need to configure the USB port for Lenovo XClarity Controller connection. Server support...

  • Page 349: Update The Firmware

    Enterprise Server (SLES) operating system distributions. Machine-type-specific firmware-only Update Bundles (Service Packs) are also available. Firmware updating tools See the following table to determine the best Lenovo tool to use for installing and setting up the firmware: Chapter 8 System configuration...
  • Page 350 √ Lenovo XClarity band O devices Integrator (LXCI) for VMware vCenter Off-Target In-band All I/O √ √ √ Lenovo XClarity devices Integrator (LXCI) for Out-of- Microsoft Windows band Admin Center On-Target Off-Target ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 351 • Lenovo XClarity Controller If you need to install a specific update, you can use the Lenovo XClarity Controller interface for a specific server. Notes: – To perform an in-band update through Windows or Linux, the operating system driver must be installed and the Ethernet-over-USB (sometimes called LAN over USB) interface must be enabled.

  • Page 352: Configure The Firmware

    Several options are available to install and set up the firmware for the server. Important: Lenovo does not recommend setting option ROMs to Legacy, but you can conduct this setting if necessary. Note that this setting prevents UEFI drivers for the slot devices from loading, which may cause negative side effects to Lenovo software, such as LXCA, OneCLI, and XCC.

  • Page 353: Memory Module Configuration

    In addition, you can choose to make the text-based interface the default interface that is displayed when you start LXPM. To do this, go to Lenovo XClarity Provisioning Manager ➙ UEFI Setup ➙ System Settings ➙ <F1>Start Control ➙ Text Setup. To start the server with Graphic User Interface, select Auto or Tool Suite.

  • Page 354: Enable Software Guard Extensions (Sgx)

    OS logical drives or volumes. An introduction to RAID is available at the following Lenovo Press website: https://lenovopress.lenovo.com/lp0578-lenovo-raid-introduction Detailed information about RAID management tools and resources is available at the following Lenovo Press website: https://lenovopress.lenovo.com/lp0579-lenovo-raid-management-tools-and-resources Intel VROC...

  • Page 355: Psu Configuration

    Setup Utility. (For more information, see the “Startup” section in the LXPM documentation compatible with your server at https://pubs.lenovo.com/lxpm-overview/ 2. Go to System settings ➙ Devices and I/O Ports ➙ Intel® VMD technology ➙ Enable/Disable Intel®...

  • Page 356: Gpu Power Capping Settings

    Use IPMI command to read the GPU power capping value. See following for more details. Steps: 1. Read user-defined GPU power capping value with command line below: ipmitool raw 0x3a 0x6 0xc0 [Slot] The return value would be as below: ipmitool raw 0x3a 0x6 0xc0 [Slot] ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 357 [x] [y] where • [Slot] is GPU numbering—GPU 1: [Slot]= [3], GPU 2: [Slot]= [4], GPU 3: [Slot]= [5], GPU 4: [Slot]= [6] • [x] is the first digit and [y] is the second and third digits of a three-digit hexadecimal number. Convert the hexadecimal number to decimal number.

  • Page 358: Deploy The Operating System

    Available tools: – Lenovo XClarity Administrator http://sysmgt.lenovofiles.com/help/topic/com.lenovo.lxca.doc/compute_node_image_deployment.html – Lenovo XClarity Essentials OneCLI https://pubs.lenovo.com/lxce-onecli/onecli_r_uxspi_proxy_tool – Lenovo XClarity Integrator deployment pack for SCCM (for Windows operating system only) https://pubs.lenovo.com/lxci-deploypack-sccm/dpsccm_c_endtoend_deploy_scenario • Single-server Available tools: – Lenovo XClarity Provisioning Manager “OS Installation” section in the LXPM documentation compatible with your server at https:// pubs.lenovo.com/lxpm-overview/...

  • Page 359: Back Up The Server Configuration

    DW612S enclosure allows mixing trays configuration, which is installing different supporting trays in the same DW612S enclosure. Mixing trays configuration must meet the following requirements. Mixing tray requirements 1. Enclosure total power consumption is reviewed and approved in Lenovo Capacity Planner (LCP): https:// datacentersupport.lenovo.com/solutions/lnvo-lcp 2.
  • Page 360 • Power on sequence: first, power on the right node; then, power on the left node. • Power off sequence: first, power off the left node; then, power off the right node. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 361 Figure 405. Left: shared I/O with auxiliary adapter kit, right: shared I/O with cable Chapter 8 System configuration...
  • Page 362 ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...

  • Page 363: Chapter 9. Problem Determination

    An alert is a message or other indication that signals an event or an impending event. Alerts are generated by the Lenovo XClarity Controller or by UEFI in the servers. These alerts are stored in the Lenovo XClarity Controller Event Log. If the server is managed by the Chassis Management Module 2 or by the Lenovo XClarity Administrator, alerts are automatically forwarded to those management applications.
  • Page 364 Figure 407. SMM2 event log Lenovo XClarity Controller event log The Lenovo XClarity Controller monitors the physical state of the server and its components using sensors that measure internal physical variables such as temperature, power-supply voltages, fan speeds, and component status. The Lenovo XClarity Controller provides various interfaces to systems management software and to system administrators and users to enable remote management and control of a server.

  • Page 365: Troubleshooting By System Leds And Diagnostics

    The Lenovo XClarity Controller monitors all components of the server and posts events in the Lenovo XClarity Controller event log. Figure 408. Lenovo XClarity Controller event log For more information about accessing the Lenovo XClarity Controller event log, see: “Viewing Event Logs” section in the XCC documentation compatible with your server at https:// pubs.lenovo.com/lxcc-overview/...
  • Page 366 XCC event log for additional information. Identification LED (blue): Use this blue LED to visually locate the node among other nodes. This LED is also used as a presence detection button. You can use Lenovo XClarity Administrator to light this LED remotely.

  • Page 367: External Diagnostics Handset

    Flashing rapidly (4 times per second): The node is turned off and is not ready to be turned on. The power button is disabled. This will last approximately 5 to 10 seconds. Flashing slowly (once per second): The node is turned off and is ready to be turned on. You can press the power button to turn on the node.
  • Page 368 Option flow diagram The LCD panel displays various system information. Navigate through the options with the scroll keys. Depending on the model, the options and entries on the LCD display might be different. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 369 Full menu list Following is the list of available options. Switch between an option and the subordinate information entries with the select button, and switch among options or information entries with the scroll buttons. Depending on the model, the options and entries on the LCD display might be different. Chapter 9 Problem determination...
  • Page 370 CPU 1 Status: Configuration Error System VPD Information Example Sub Menu Machine Type: xxxx • Machine type and serial number Serial Num: xxxxxx • Universal Unique ID (UUID) Universal Unique ID: xxxxxxxxxxxxxxxxxxxxxxxxxxxx ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 371 System Firmware Sub Menu Example XCC Primary XCC Primary (Active) • Firmware level (status) Build: DVI399T • Build ID Version: 4.07 • Version number Date: 2020-04-07 • Release date XCC Backup XCC Backup (Active) • Firmware level (status) Build: D8BT05I •...

  • Page 372: Leds

    • Modify XCC Static IPv4 Address/Net mask/Gateway • Modify System Name • Generate/Download FFDC Service Data System Management Module 2 (SMM 2) LEDS The following illustration shows the LEDs on the SMM2 module. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 373 Figure 410. SMM2 LEDs Table 26. SMM2 connectors and LEDs Identification LED (blue) Ethernet port 2 link (RJ-45) LED (green) Power LED (green) Ethernet port 2 activity (RJ-45) LED (green) Status LED (green) Ethernet port 1 link (RJ-45) LED (green) Check log LED (yellow) Ethernet port 1 activity (RJ-45) LED (green) Identification LED: When this LED is lit (blue), it indicates the enclosure location in a rack.

  • Page 374: Power Supply Leds

    The following illustration shows the LEDs on the power supply: Figure 411. AC power supply LEDs Input (AC) power LED (green) Power supply error LED (yellow) Output (DC) power LED (green) ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...

  • Page 375: Dwc Power Supply Leds

    Power supply error LED (yellow): When this LED is lit (yellow), it indicates that there is a fault with the power supply. Dump the FFDC log from the system and contact Lenovo back end support team for PSU data log reviewing.

  • Page 376: Drip Sensor Led

    Power supply error LED (yellow): When this LED is lit (yellow), it indicates that there is a fault with the corresponding power supply bay. Dump the FFDC log from the system and contact Lenovo back end support team for PSU data log reviewing.

  • Page 377: System-Board Leds

    Figure 413. Drip sensor LED Table 27. Drip sensor LED Drip sensor LED (yellow) Drip sensor LED: When this LED is lit (yellow), it indicates that the drip sensor detects water in its respective catch basin. System-board LEDs The following illustrations show the light-emitting diodes (LEDs) on the system board. Figure 414.

  • Page 378: General Problem Determination Procedures

    Power problems can be difficult to solve. For example, a short circuit can exist anywhere on any of the power distribution buses. Usually, a short circuit will cause the power subsystem to shut down because of an overcurrent condition. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...

  • Page 379: Resolving Suspected Ethernet Controller Problems

    Complete the following steps to diagnose and resolve a suspected power problem. Step 1. Check the event log and resolve any errors related to the power. Note: Start with the event log of the application that is managing the server. For more information about event logs, see “Event logs”...

  • Page 380: Troubleshooting By Symptom

    1. Check the event log of the application that is managing the server and follow the suggested actions to resolve any event codes. • If you are managing the server from the Lenovo XClarity Administrator, begin with the Lenovo XClarity Administrator event log.
  • Page 381 Collect XCC service data (see “Collecting service data” on page 396). b. Contact Lenovo Service. System fails to detect a specific GPU When the event Sensor GPU CPUs has transitioned to critical from a less severe state appears in the XCC web event log, it indicates the system fails to detect one or more specific GPUs.
  • Page 382 When seeing an event in XCC web event log, refer to the following table to identify defective unit and problem type. For example: 6 | 01/08/2021 | 14:34:53 | 0x0020 | Add-in Card GPU Board | Trasition to Critical from less severe | Asserted |0xA2F60F ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...

  • Page 383: Intermittent Problems

    Table 29. XCC GPU sensor specifications 1/2 Sensor Sensor Sensor Sensor Entity Instance/ Reading Mask (data set to sensor) name Number Type Reading Type Type 00h –Transition to OK board 02h –Transition to Critical from less severe • Evt Data: 2 –...
  • Page 384 (Windows and Linux base operating systems will be using different method). Enter the UEFI setup menus and disable the feature, or disable it with the following OneCli command. OneCli.exe config set SystemRecovery.RebootSystemOnNMI Disable --bmcxcc_userid PASSW0RD@xcc_ipaddress ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...

  • Page 385: Problems

    355 for information about viewing the event log. If you are using Linux base operating system, then capture all logs back to Lenovo support for further investigation. Keyboard, mouse, KVM switch or USB-device problems Use this information to solve problems related to a keyboard, mouse, KVM switch or USB-device problems.

  • Page 386: Memory Problems

    • Memory mirrored channel does not account for the discrepancy. • The memory modules are seated correctly. • You have installed the correct type of memory module (see “Memory module installation rules and order” on page 40 for requirements). ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...

  • Page 387: Monitor And Video Problems

    LXPM interface is displayed by default. (For more information, see the “Startup” section in the LXPM documentation compatible with your server at .) You https://pubs.lenovo.com/lxpm-overview/ can perform memory diagnostics with this interface. From the Diagnostic page, go to Run Diagnostic ➙ Memory test ➙ Advanced Memory Test.

  • Page 388: Network Problems

    3. If the server is installed with the graphical adapters while turning on the server, the Lenovo logo is displayed on the screen after approximately 3 minutes. This is normal operation while the system loads. 4. Make sure that: • The server is turned on and there is power supplied to the server.

  • Page 389: Observable Problems

    If the system hangs during the UEFI boot process with the message UEFI: DXE INIT on the display, make sure that Option ROMs were not configured with a setting of Legacy. You can remotely view the current settings for Option ROMs by running the following command using the Lenovo XClarity Essentials OneCLI: onecli config show EnableDisableAdapterOptionROMSupport --bmc xcc_userid:xcc_password@xcc_ipaddress...
  • Page 390 Configuration changes, such as added devices or adapter firmware updates, and firmware or application code problems can cause the server to fail POST (the power-on self-test). If this occurs, the server responds in either of the following ways: ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 391 Complete the following steps until the problem is solved. 1. An unusual smell might be coming from newly installed equipment. 2. If the problem remains, contact Lenovo Support. Server seems to be running hot Complete the following steps until the problem is solved.

  • Page 392: Optional-Device Problems

    • “Insufficient PCIe resources are detected” on page 384 • “A Lenovo optional device that was just installed does not work” on page 384 • “A Lenovo optional device that worked previously does not work now” on page 384 Insufficient PCIe resources are detected If you see an error message stating “Insufficient PCI Resources Detected,”...

  • Page 393: Power On And Power Off Problems

    • Any external SCSI device is turned on. You must turn on an external SCSI device before you turn on the server. 4. Reseat the failing device. 5. Replace the failing device. Power on and power off problems Use this information to resolve issues when powering on or powering off the server. •...

  • Page 394: Power Problems

    8. Implement the minimum configuration (one processor and one DIMM) to check whether any specific components lock the power permission. 9. Collect the failure information by capturing the system logs and provide it to Lenovo support. 10. See “Power supply LEDs” on page 366 “DWC power supply LEDs”...

  • Page 395: Software Problems

    1. Verify that the drive is supported for the server. See for a list of https://serverproven.lenovo.com supported hard drives. 2. Make sure that the drive is seated in the drive bay properly and that there is no physical damage to the drive connectors.

  • Page 396: System Management Module 2 Problems

    System Management Module 2 ping failure Complete the following steps until the problem is solved. 1. Check the SMM2 IP address and network status via Lenovo XClarity Controller. 2. Alternatively, you can check SMM2 LEDs to diagnose the SMM2 status (see “System Management...
  • Page 397 1. Check enclosure SMM2 messages to see if any leakage warnings have been reported. See https:// for more information. pubs.lenovo.com/dw612s_neptune_enclosure/messages_introduction 2. Walk to the rear of the rack and visually check the status of each enclosure's left and right side drip sensor LEDs.
  • Page 398 Table 31. Drip sensor LED Drip sensor LED (yellow) 3. Visually check the drip sensor catch basin for any moisture. a. Remove the lower and upper EMC shields in the front of the left drip sensor. ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 399 Note: If there is a manifold vertical pipe in front of the EMC shield, you need to slide it sideways out from underneath the pipe. Figure 417. Lower EMC shield removal Chapter 9 Problem determination...
  • Page 400 Figure 418. Upper EMC shield removal b. Use a flashlight to visually inspect the plastic catch basin for any moisture. c. Re-install the upper and lower EMC shields. Figure 419. Upper EMC shield installation ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...
  • Page 401 Figure 420. Lower EMC shield installation d. Repeat steps for the right side drip sensor catch basin. 4. Power down all nodes either through the OS or by pressing and holding the power button for five seconds. Note: Green Power LED for each node (two per compute nodes) should be blinking to indicate the nodes are in standby state.
  • Page 402 59). Contact Product Engineer for the further assistance. Important: The procedures above only cover the Lenovo DWC solution (from the Eaton Ball valves up through the manifold and into the enclosures and compute trays). If your datacenter Cooling Distribution Unit experiences repeated or frequent low water alerts then you should inspect the data center plumbing between the cooling distribution unit and the Lenovo DWC solution (rack).

  • Page 403: Appendix A. Getting Help And Technical Assistance

    Appendix A. Getting help and technical assistance If you need help, service, or technical assistance or just want more information about Lenovo products, you will find a wide variety of sources available from Lenovo to assist you. On the World Wide Web, up-to-date information about Lenovo systems, optional devices, services, and support are available at: http://datacentersupport.lenovo.com...

  • Page 404: Collecting Service Data

    Collecting service data To clearly identify the root cause of a server issue or at the request of Lenovo Support, you might need collect service data that can be used for further analysis. Service data includes information such as event logs and hardware inventory.

  • Page 405: Contacting Support

    Support when certain serviceable events occur in Lenovo XClarity Administrator and the managed endpoints. You can choose to send diagnostic files to Lenovo Support using Call Home or to another service provider using SFTP. You can also manually collect diagnostic files, open a problem record, and send diagnostic files to the Lenovo Support.
  • Page 406 ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...

  • Page 407: Appendix B. Documents And Supports

    • ThinkSystem Heavy Duty Full Depth Rack Cabinets User Guide Support websites This section provides driver and firmware downloads and support resources. • Drivers and Software download website – https://datacentersupport.lenovo.com/products/servers/thinksystem/sd650nv3/7d7n/downloads/driver-list/ • https://forums.lenovo.com/t5/Datacenter-Systems/ct-p/sv_eg • Lenovo Data Center Support – https://datacentersupport.lenovo.com/products/servers/thinksystem/sd650nv3/7d7n • https://datacentersupport.lenovo.com/documents/lnvo-eula • https://lenovopress.lenovo.com/ (Product Guides/Datasheets/White papers) •...
  • Page 408 • (Service request) https://support.lenovo.com/servicerequest • (Stay up to date on firmware updates) https://datacentersupport.lenovo.com/solutions/ht509500 ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...

  • Page 409: Appendix C. Notices

    Lenovo representative for information on the products and services currently available in your area. Any reference to a Lenovo product, program, or service is not intended to state or imply that only that Lenovo product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any Lenovo intellectual property right may be used instead.

  • Page 410: Trademarks

    (TBW). A device that has exceeded this limit might fail to respond to system-generated commands or might be incapable of being written to. Lenovo is not responsible for replacement of a device that has exceeded its maximum guaranteed number of program/erase cycles, as documented in the Official Published Specifications for the device.

  • Page 411: Taiwan Region Bsmi Rohs Declaration

    Taiwan region BSMI RoHS declaration Taiwan Region import and export contact information Contacts are available for Taiwan Region import and export information. Appendix C. Notices...
  • Page 412 ThinkSystem SD650-N V3 Neptune DWC Tray User Guide...

This manual is also suitable for:

7d7n

Table of Contents