Verifying The Infiniband Network - Oracle EXADATA X6-2 Installation And Configuration Manual

Verify the rack master serial number is set correctly using the following command:
4.
# dcli -l root -g group_file "ipmitool sunoem cli 'show
/SP system_identifier'" > /tmp/show-rack-csn.out
# more /tmp/show-rack-csn.out
If none of the serial numbers are set, then contact Oracle Support Services. If one
serial number is not set correctly, then run the following command to correct it:
ipmitool sunoem cli 'set /SP system_identifier= \
"Exadata Database Machine rack_type xxxxAKyyyy"'
In the preceding command, rack_type is
Ensure disk cache policy is set to
5.
root
dcli -g /opt/oracle.SupportTools/onecommand/all_group -l root \
/opt/MegaRAID/MegaCli/MegaCli64 -LdPdInfo -aALL | grep -i 'Disk Cache Policy'
The following is an example of the output from the command:
dm01db01: Disk Cache Policy
dm01db02: Disk Cache Policy
dm01db03: Disk Cache Policy
...
dm01cel01: Disk Cache Policy
dm01cel02: Disk Cache Policy
...
If any server shows different output, then identify the LUN and use the following
command to reset the policy to
MegaCli64 -LDSetProp -DisDskCache -LUNn -a0
In the preceding command, LUNn is the LUN number, such as L2.

5.7 Verifying the InfiniBand Network

The following procedure describes how to verify the InfiniBand network:
Visually check all the InfiniBand cable connections within the rack. The port lights
1.
should be on, and the LEDs should be on. Do not press each connector to verify
connectivity.
Log in as the
2.
Verify the InfiniBand topology using the following commands:
3.
# cd /opt/oracle.SupportTools/ibdiagtools
# ./verify-topology [-t rack_size]
The following example shows the output when the network components are
correct.
[DB Machine Infiniband Cabling Topology Verification Tool ]
Is every external switch connected to every internal switch......[SUCCESS ]
Are any external switches connected to each other................[SUCCESS ]
Are any hosts connected to spine switch..........................[SUCCESS ]
Check if all hosts have 2 CAs to different switches..............[SUCCESS ]
Leaf switch check:cardinality and even distribution..............[SUCCESS ]
user on the first database server in the cluster:
user on any component in the rack.
root
or , depending on the rack.
X2-2 X2-8
using the following command as the
Disabled
: Disabled
: Disabled
: Disabled
: Disabled
: Disabled
:
Disabled
Chapter 5
Verifying the InfiniBand Network
\
5-60