Page 2
52.227-14. TRADEMARKS AND ATTRIBUTIONS Altix, Silicon Graphics, SGI, and the SGI logo, are registered trademarks of Silicon Graphics, Inc., in the United States and/or other countries worldwide.
Page 3
Record of Revision Version Description April, 2008. Original printing. 007-5483-001...
Structure of this Guide xiii Related Publications . Conventions Product Support . Reader Comments . SGI Virtu VN200 Overview Product Description . Warnings and Cautions . Site Planning . Selecting a Delivery Route . Selecting a Final Location Unpacking the Rack .
Page 6
Industry Canada Notice (Canada Only) VCCI Class A Notice (Japan Only) . Shielded Cables System Features Server Racks. Virtu VN200 Graphics Node . Front Panel Components and Status LEDs . Rear Panel Components . Node Board . Node Board Features .
Page 7
System Procedures Connecting a Digital Monitor Connecting a Keyboard and Mouse . Configuring Firmware and Drivers . Power On/Off the Cluster Powering Off Manually . Verifying the Graphics Card Installation for Linux Starting the X Window System . Commands Documentation Start Graphics .
Page 8
Hardware Replacement Procedures . Before You Begin . Tools and Supplies Needed . Removing the Front Bezel Removing the Graphics Node from the Rack . Removing the Side Cover. Replacing the Power Supplies Replacing a Disk Drive Replacing the Chassis Fans . Replacing the Graphics Card .
This guide provides information about how to install, configure, administrate, and repair an SGI Virtu VN200. Note: Refer to “Related Publications” on page xiv, for a listing of relevant SGI Technical Publications that can provide more detailed information about SGI cluster head nodes, compute nodes, system software, and rack hardware.
This guide provides instructions administrating SGI Altix XE320 compute nodes in the SGI Virtu VN200. The SGI Virtu VN200 uses the same node board as the SGI Altix 320 server, so you can also use this guide for information about BIOS settings, power-on self test (POST) codes, and the baseboard management controller (BMC) intelligent platform management interface (IPMI) management utility.
Man page section identifiers appear in parentheses after man page man page(x) names. Product Support SGI provides a comprehensive product support and maintenance program for its products. SGI also offers services to implement and integrate Linux applications in your environment. • Refer to http://www.sgi.com/support/ •...
If you have comments about the technical accuracy, content, or organization of this document, contact SGI. Be sure to include the title and document number of the manual with your comments. (Online, the document number is located in the front matter of the manual.
Chapter 1 SGI Virtu VN200 Overview This chapter provides an introduction to the SGI Virtu VN200 graphics node (refer to Figure 1-1) and introduces information necessary to understand the system. SGI Altix XE250 Head Node GigE Switch I S R...
Message Passing Interface (MPI). An SGI Virtu VN200 system is a distributed memory server as opposed to a shared memory system like that used in the SGI Altix 450 or SGI Altix 4700 high-performance compute servers.
Page 19
Product Description The primary hardware components in an SGI Virtu VN200 system are: • Head node(s) (SGI Altix XE250 servers) • Compute nodes (SGI Altix XE320 servers) • Graphics nodes (SGI Virtu VN200 graphics nodes) • Network interconnect components (Gigabit Ethernet switches, InfiniBand switches, PCI cards, and cables) •...
SGI Virtu VN200 Overview Warnings and Cautions Read the following safety and warnings and cautions when operating your SGI Virtu VN200 cluster. Warning: System power on/off: The power button DOES NOT turn off the system AC power. To remove power from system, you must unplug the AC power cord from the wall outlet.
• SGI recommends that you use a pallet jack to position the rack as close as possible to its final destination. If the shipping crate cannot be transported to the...
Ensure that you have enough room to open the doors, remove boards, and accomplish other routine tasks. As a general rule, SGI systems require 48-in of clearance from the front and rear of the rack. Side access to the rack is not required for service.
Unpacking the Rack Unpacking the Rack Ensure that the temperature of the rack is acclimated to the installation environment and that the system crate is stable, and in an upright position. Refer to Figure 1-2 as you follow these steps; the numbered illustrations correspond to the numbered steps.
SGI Virtu VN200 Overview Warning: Serious personal injury or death could result if the rack becomes unbalanced or gains too much momentum when rolled. Always get the assistance of other qualified personnel when rolling a rack. 6. Get four qualified persons to carefully roll the rack down the ramp and onto the floor.
Unpacking the Graphics Nodes Unpacking the Graphics Nodes The SGI Virtu VN200 graphics nodes are packed as shown in Figure 1-3. Unpack each graphics node and place it on a stable ESD safe worksurface or position each graphics node on the rack shelf as shown in Figure 1-4.
SGI Virtu VN200 Overview Installing the Rack Shelf Install the rack shelf as shown in Figure 1-4. The graphics nodes require 4U of rack space. Five graphics nodes can be placed on the rack shelf. Position each graphics node on the...
System Configurations and Cabling Table 1-1 lists the system configurations for the SGI Virtu VN200. SGI Altix ICE clusters and SGI Altix IA64 configurations will be supported in a future release of the product. Table 1-1 SGI Virtu VN200 System Configurations...
System Configurations and Cabling Figure 1-6 shows an SGI Virtu VN200 graphics node configured with an SGI Altix ICE cluster. This system configuration will be supported in a future product release. SGI Altix ICE IRU IB cables Ethernet cables Customer Ethernet...
Electrical Requirements Electrical Requirements The SGI Virtu VN200 accepts single-phase input voltage only. It typically connects to a single-phase or 3-phase rack power distribution unit (PDU) that outputs single-phase 220 VAC. Ensure that the required voltage is available and is within a reasonable distance of the intended location.
SGI Virtu VN200 Overview Environmental Requirements Electromagnetic interference (EMI), electrostatic discharge (ESD), vibration, and humidity can cause problems for computer systems. Operating Temperature 10º to 40º C (50º to 104º F) Non-operating Temperature -40º to 70º C (-40º to 158º F)
Regulatory Specifications Electrostatic Discharge SGI designs and tests its products to ensure that they resist the effects of electrostatic discharge (ESD). However, it is still possible for ESD to cause problems that range from data errors and lockups to permanent component damage. To protect the systems from ESD, follow these precautions: •...
The “CE” symbol indicates compliance of the device to directives of the European Community. A “Declaration of Conformity” in accordance with the standards has been made and is available from SGI upon request. Electromagnetic Emissions This section provides the contents of electromagnetic emissions notices for various countries.
VCCI Class A Notice (Japan Only) Shielded Cables The SGI Virtu VN200 product is FCC compliant under test conditions that include the use of shielded cables between the server and its peripherals. Your server and any peripherals that you purchase from SGI have shielded cables. Shielded cables reduce the possibility of interference with radio, television, and other devices.
2-1) is an integrated compute environment that can scale to 288 (refer to Figure 2-2). The standard cooling method for the SGI Virtu VN200 is blower-driven air cooling. Optional supplemental water cooling will be available (in a 42U water-cooled rack) in a future release. Graphics node...
System Features Server Racks Each SGI Virtu VN200 rack contains the head nodes, compute nodes, an InfiniBand switch, a Gigabit Ethernet switch, a system console, power distribution units (PDUs) and graphics nodes (refer to Figure 2-2). The racks are typically air-cooled.
System Features Front Panel Components and Status LEDs The front panel of the SGI Virtu VN200 provides status LEDs, system disk access, and power supply access (refer to Figure 2-4). The front panel status LED definitions are listed below: •...
Virtu VN200 Graphics Node Rear Panel Components Figure 2-5 shows the location of the rear panel components on the SGI Virtu VN200 graphics node. The rear panel component descriptions are listed below: • Input AC - Connects to system input AC power cord •...
System Features Node Board At the heart of the SGI Virtu VN200 lies a dual-processor socket node board, which is based on the Intel 5400P chipset (refer to Figure 2-6 Figure 2-7). Memory DIMMs Processors Power connectors SATA ports Power...
Node Board Node Board Features Refer to the node board block diagram, Figure 2-8. • Dual Intel® 64-bit Xeon LGA 771 quad-core/dual-core processors at a front side bus speed of 1.600 GHz or 1.333 GHz Memory • Eight 240-pin DIMM sockets with support up to 32 GB DDR2 FBD (fully buffered) ECC 800 MT/s memory Chipset •...
Page 44
System Features ACPI Features • Slow blinking LED for suspend state indicator • Main switch override mechanism • ACPI Power Management Onboard I/O • Four SATA2 system disk ports (only two ports are used) • One SIMSO IPMI slot • One Intel Gigabit Ethernet 82563EB controller supports two Gigabit LAN ports •...
Node Board Node Board Connector Definitions (continued) Table 2-2 Connector Description JWOL Wake-on LAN header 0/1 back panel USB 0/1 for external use (JUSB1) 2/3 front panel USB 2/3 VGA Connector (JVGA1) SIMSO SIMSO IPMI slot SATA 0-3 Intel SATA 0-3 connectors Indicator description Onboard Power LED Indicator InfiniBand Link LED (green)
Branch 0, you must also configure 2GB of memory in Branch 1 (refer to Figure 2-9). • Unbalanced branch configurations can reduce overall memory performance up to 40%. Memory Part Numbers Table 2-4 lists the SGI Virtu VN200 memory sizes and their SGI part numbers. 007-5483-001...
BMC daughter card Figure 2-11 AOC-SIMSO (BMC) Daughter Card Note: Refer to the SGI Altix XE320 System User's Guide, publication number 007-5466-00x for more detailed information about the baseboard management controller (BMC) intelligent platform management interface (IPMI) management utility. Note: The add-on graphics card disables the KVM-over-LAN feature of the BMC.
Page 52
System Features IPMI controller enables you to communicate with the BMC via a Java-based web interface. The BMC has the following features: • Installs in the JIPMI slot of the serverboard • Enables you to: • Monitor the Server Event Log (SEL) •...
Node Board Power Supplies Each hot-pluggable power supply connects to the power distribution assembly in the center of the chassis. The power supplies are located in the chassis as shown in Figure 2-12. The power supply specifications are listed in Table 2-6.
System Features Cooling Fans Each power supply has built in cooling fans that pull cool air in from the front of the chassis. Two chassis cooling fans pull cool air in from the front of the chassis and blow it across the node board heatsinks and memory DIMMs.
The system disk drive are controlled by the SATA ports on the node board (refer to Figure 2-7 page 26. The following system disk drive options are supported by the SGI Virtu VN200 (refer to Figure 2-14): • 2.5-in, 80 GB, 7200 RPM SATA •...
System Features Graphics Cards The following graphics cards are supported on the SGI Virtu VN200. Other cards may be supported in a future release. • NVIDIA Quadro FX1700 • NVIDIA Quadro FX3700 • NVIDIA Quadro FX4600 • NVIDIA Quadro FX5600...
Accelerated graphics drivers from NVIDIA are preinstalled by SGI on the SGI Virtu VN200 graphics node. These drivers are not part of the standard SLES or RHEL distribution of Linux. A copy of these drivers is available on the SGI Propack CD and/or supportfolio.
Chapter 3 System Procedures Connecting a Digital Monitor The NVIDIA Quadro FX card supports two DVI-I outputs. The DVI-I output farthest away from the stereo connector is the primary channel for the system. By default this port outputs an analog signal when the system is not configured. If you attach digital monitor to the system, you can reset the DVI-I port by performing the following steps.
Connecting a Keyboard and Mouse A keyboard/mouse is not supplied as standard for the system but may be provided as an option from SGI. Connect a USB keyboard and a USB mouse to your system, as shown Figure 3-1. If your keyboard/mouse is further than 10 feet (3 meters) away from the USB ports on the system, then optional USB extension cables are required.
Configuring Firmware and Drivers Configuring Firmware and Drivers The SGI Virtu VN200 uses the same BIOS images as the SGI Altix XE320 server, with customized settings. The SGI Virtu VN200 also uses factory-installed, accelerated NVIDIA graphics drivers and X.org windowing system; SGI provides limited support for these SGI qualified NVIDIA graphics drivers.
System Procedures If your cluster uses the Scali Manage administrative software (release 5.6.x) or later, you can power on/off specific nodes or the entire system using the graphical user interface. Select Management Menu -> Power Mgt ->Power Off. The compute nodes can be halted from the Scali Manage GUI by selecting the nodes and choosing “halt system”...
Verifying the Graphics Card Installation for Linux Verifying the Graphics Card Installation for Linux To verify the graphics card is installed and seen correctly by Linux do the following steps. From a Linux root prompt enter: # /sbin/lspci | grep VGA Check the output for information similar to the following: 01:00.0 VGA compatible controller: nVidia Corporation G80 [Quadro FX xxx] (rev xx) 08:01.0 VGA compatible controller: ATI Technologies Inc ES1000 (rev 02)
The X Window System is the only window system provided for the SGI Virtu VN200 running Linux. The SGI Virtu VN200 uses an X server (Xorg) from the opensource xorg project (www.x.org).
Starting the X Window System Documentation NVIDIA driver documentation is located in /usr/share/doc/NVIDIA_GLX-1.0/README.txt. Refer to the man pages for the commands listed above and the X.Org website at: http://www.x.org. Start Graphics The init 5 command start the system in the graphical mode. When init 5 is used, the graphics console displays a login screen.Enter the following command to start graphics: # init 5...
System Procedures xorg.conf Configuration File Xorg.conf is the file containing configuration information for the default X server. It is best to think about the organization of the xorg.conf file as tree (refer to Figure 3-2). The file contains information about the following system properties: •...
Page 67
The xorg.conf file is located in the /usr/etc/X11 directory. Make a copy of the factory default xorg.conf file so that you can make your configuration changes. An example xorg.conf file for the SGI Virtu VN200 is listed below for reference. The X server can be modified by: •...
Starting the X Window System Monitor Section The monitor section contains information that defines the capabilities of your display You can make specific changes to the monitor section of the xorg.conf file by referring to the documentation that came with the monitor. An example monitor section is shown below: Section "Monitor"...
System Procedures Device Section The device section defines the configuration parameters for a specific graphics card. There is typically one "Device" section for each graphics card in the system. The device section can make one graphics card with two channels act like two graphics cards using Screen option.
Starting the X Window System You must also specify either: Option "SecondMonitorHorizSync" "<hsync range(s)>" Option "SecondMonitorVertRefresh" "<vrefresh range(s)>" Option "HorizSync" "<hsync range(s)>" Option "VertRefresh" "<vrefresh range(s)>" You may also use any of the following options, though they are not required: Option "TwinViewOrientation"...
System Procedures TwinView Edge Blending To configure TwinView displays that require edge-blending (refer to Figure 3-4) between the two display channels with a 10% overlap, use the following entries: Option "TwinView" "true" # turn on dual channel mode Option "MetaModes" "1920x1200, 1920x1200 +1728+0" The +1728 in the option line above commands the graphics card to start the second channel at 1728 offset from the lower left hand corner of the first channel (the origin).
Starting the X Window System Passive stereo uses the TwinView clone mode stereo and is not compatible with any other TwinView settings. Note: Stereo is not compatible with the “Composite extension being set”. Please refer to the NVIDIA README file for other stereo options and limitations. Setting up Multiple X Servers on a Single Graphics Card The NVIDIA graphics card can be configured such that each connected display device functions as a separate X server.
Page 74
System Procedures Then, create two Screen sections, each using one of the Device sections: Section "Screen" Identifier "Screen SG-0" Device "nvidia0" Monitor "Monitor0" DefaultDepth 24 Subsection "Display" Depth Modes "1600x1200" "1024x768" "800x600" "640x480" EndSubsection EndSection Section "Screen" Identifier "Screen SG-1" Device "nvidia1"...
Starting the X Window System ServerLayout Section The ServerLayout section defines the input/output keyboard/mouse is associated with a screen and where the multiple screens are located relative to one another. This example shows two screens: Section "ServerLayout" Identifier "Layout[all]" InputDevice "Keyboard0"...
Note: The NVIDIA installer may require you to install additional packages on the system. • Install the .rpm files supplied by SGI For SLES 10 with the NVIDIA 169.12 driver, install the following .rpm files: rpm -ivh x11-video-nvidiaG01-169.12-3.rpm nvidia-gfxG01-kmp-smp-169.12_2.6.16.46_0.12-3.x86_64.rpm For Redhat with the NVIDIA 169.12 driver, install the following .rpm files:...
Troubleshooting Upgrading Drivers SGI provides updated drivers through SupportFolio. If you upgrade drivers directly from the NVIDIA website, remove the .rpm files listed on page 60 first to ensure that there are no conflicts when installing the new drivers. Troubleshooting...
The following list describes basic troubleshooting tips that you can use to isolate BMC configuration problems. • By default, the SGI Virtu VN200 system enables serial-over-LAN (SOL) functionality via serial port B. • To configure SOL manually, enable serial-console redirection in BIOS, then...
DEFAULTP.EXE command from a DOS-bootable thumbdrive image Power Use the following tools and techniques to troubleshoot the power components on the SGI Virtu VN200: • Check the power supply status LEDs. Each power supply has an green LED that illuminates when DC power is present. When a supply fails, the PWR FAIL LED on the control panel illuminates and the power-supply status LED does not illuminate.
System Procedures Hardware Replacement Procedures Before You Begin Before working with your server product, pay close attention to “Warnings and Cautions” on page Tools and Supplies Needed • Phillips (cross head) screwdrivers (#1 bit and #2 bit) • Needle-nosed pliers •...
Hardware Replacement Procedures Removing the Graphics Node from the Rack Power off the system using ipmitool or BMC web interface. 2. Disconnect the AC input power cord from the rear panel of system. 3. Disconnect the video and data cables from the rear panel of the system. 4.
System Procedures Removing the Side Cover Remove the graphics node from the rack in accordance with the instructions on page 2. Remove the two screws (Figure 3-7) that secure the side cover to the chassis and remove the side cover. Side Cover Removal Figure 3-7 007-5483-001...
Hardware Replacement Procedures Replacing the Power Supplies Note: Power supplies can be hot-swapped while the system is in operation in a rack. Have the replacement power supply ready, so that it can be installed quickly without disrupting the airflow through the chassis. If necessary, remove the front bezel from the system in accordance with the instructions on page...
System Procedures Replacing a Disk Drive Note: Halt the operating system before replacing a disk drive. Disk drives can be hot-plugged while the system is in operation in a rack. Have the replacement disk drive ready, so that it can be installed quickly without disrupting the airflow through the chassis.
Hardware Replacement Procedures Replacing the Chassis Fans Remove the side cover from the chassis in accordance with the instructions on page 2. Remove two screws (refer to Figure 3-10) that secure the fan assembly to the chassis. 3. Disconnect the fan power cables from the node board. 4.
System Procedures Note: Step 3 is not required for the NVIDIA Quadro FX 1700 graphics cards. 3. If necessary, remove the three screws that secure the graphics card to the chassis air baffle (refer to Figure 3-11, B). 4. Remove two screws (refer to Figure 3-11, C) that secure the graphics card to the rear panel.
Hardware Replacement Procedures Replacing the AOC-SIMSO (BMC) Daughter Card The AOC-SIMSO daughter card (BMC) resides in a slot attached to the node board. The rear panel BMC LAN port is connected to the BMC using a short USB cable. Remove the graphics card in accordance with the instructions on page 2.
System Procedures Removing the PCI Riser Board The PCI riser board connects between the graphics card and the node board. Remove the graphics card in accordance with the instructions on page 2. Remove the screws that secure the riser board to the chassis (refer to Figure 3-13).
Hardware Replacement Procedures Removing the Air Baffles The air baffles deflect the airflow from the chassis fans across the node board and the graphics card. Remove the graphics card in accordance with the instructions on page 2. Remove two screws on each side of the chassis (refer to Figure 3-14).
System Procedures Replacing Node Board Memory DIMMs Remove the graphics card in accordance with the instructions on page 2. Remove the air baffle in accordance with the instructions on page 73, (refer to Figure 3-15, A and B). 3. Press down on the latches to release the DIMM (refer to Figure 3-15, C).
Hardware Replacement Procedures Replacing Node Board Processors Caution: When handling the processor, avoid placing direct pressure on the label area of the fan. Caution: Do not place the node board on a conductive surface. A conductive surface can damage the BIOS battery and prevent the system from booting. Always place the node board on an ESD-safe worksurface.
Page 92
System Procedures 5. Gently lift the load lever to open the load plate. 6. Use your thumb and your index finger to hold the CPU at opposite sides. 7. Carefully lift the CPU straight out of the socket. 007-5483-001...
Hardware Replacement Procedures Note: If you are removing the processor so that you can return the node board to SGI, install a PnP cap in the socket. Install a Processor Note: A black pick-and-place (PnP) cap is attached to the load plate to protect the CPU socket.
Page 94
System Procedures 2. Gently lift the load lever to open the load plate. Load plate released 3. Use your thumb and your index finger to hold the CPU at opposite sides. 007-5483-001...
Page 95
Hardware Replacement Procedures 4. Align pin 1 of the CPU (the corner marked with a triangle) with the notched corner of the CPU socket. Caution: In step 5, do not drop the CPU on the socket. Do not move the CPU horizontally or vertically, and do not rub the CPU against any surface or any of the contacts which may damage the CPU and/or contacts.
Page 96
System Procedures 8. Use your thumb to gently push the load lever down until it snaps into the retention clasp. Load lever 9. If the CPU is properly installed into the socket, the PnP cap is automatically released from the load plate when the lever locks. Caution: Retain the plastic PnP cap for shipping.
Hardware Replacement Procedures 10. Remove the PnP cap. PnP cap released from load plate 11. Repeat these steps to install a second CPU if desired. Install a Heatsink Ensure that you have replaced the thermal interface material (TIM) between the heatsink and the CPU die.
System Procedures 4. Install the two remaining screws (screw 3 and screw 4, shown in Figure 3-17); then, gently tighten all four screws. Screw (3) Screw (2) Screw (1) Screw (4) Figure 3-17 Install a CPU Heatsink 007-5483-001...
Hardware Replacement Procedures Remove the Heatsink SGI does not recommend that the CPU or the heatsink be removed from the node board. However, if you do need to remove the heatsink, use the instructions below to prevent damage to the CPU or the CPU socket.
System Procedures Removing TIM from a Processor You should always remove the TIM from a processor before you remove it from the node board. Warning: Always wear gloves when performing this procedure to avoid direct skin contact with the TIM, which is a potentially toxic material. Tools/Parts Required: •...
Page 101
Hardware Replacement Procedures Note: If you plan to re-use the heatsink with a replacement processor, you must clean the TIM off of the heatsink. (Refer to “Removing TIM from a Heatsink” on page 87.) 2. Gently push any loose TIM pieces to the center of the processor. 3.
Page 102
System Procedures 4. Use a dry, lint-free cloth to wipe any remaining TIM pieces off of the processor. 5. Use a different lint-free cloth moistened with isopropyl alcohol (IPA) to remove any remaining TIM from the processor. 007-5483-001...
Hardware Replacement Procedures Note: Be sure to remove TIM from the gaps between the processor and the load plate. Removing TIM from a Heatsink If you plan re-use the heatsink with a replacement processor, you need to remove the TIM from the heatsink.
Page 104
System Procedures Remove the heatsink from the socket and place the heatsink on a secure surface with the bottom facing upward. 2. Use a dry, lint-free cloth to wipe any loose TIM pieces off of the heatsink. 007-5483-001...
Page 105
Hardware Replacement Procedures 3. Use a different lint-free cloth moistened with isopropyl alcohol (IPA) to remove any remaining TIM from the heatsink. 007-5483-001...
System Procedures Replacing the Node Board Remove the graphics card in accordance with the instructions on Figure 3-11. 2. Remove the air baffles (refer to Figure 3-18). Air baffle Screws Figure 3-18 Air Baffles 3. Remove the BMC daughter card in accordance with the instructions on page 4.
Hardware Replacement Procedures Replacing the Control Panel The control panel is located at the front of the system (refer to Figure 3-19). Remove the disk drives or the front top cover to access the cable that connects to the control panel. 2.
Need help?
Do you have a question about the Virtu VN200 and is the answer not in the manual?
Questions and answers