Ipoib Configuration (Layer-3) - Oracle Exalytics In-Memory Machine Owner's Manual

Link layer: IB
Port 2:
State: Active
Physical state: LinkUp
Rate: 40
Base lid: 73
LMC: 0
SM lid: 187
Capability mask: 0x02510868
Port GUID: 0x0021280001efe986
Link layer: IB
[root@exalytics ~]# sminfo
sminfo: sm lid 187 sm guid 0x10e00c16c0c0a0, activity count 124743184 priority
14 state 3 SMINFO_MASTER
Run the following command to make sure all the links are at 4X QDR:
4.
[root@exalytics ~]# ibdiagnet -ls 10 -lw 4x

8.5 IPoIB Configuration (Layer-3)

Make sure that a unique IP address is available in the IPoIB subnet used in the Oracle
Big Data Appliance rack. Never reuse an IP address that is already assigned, or
allocated for Big Data Appliance or other purposes in this subnet. To identify a unique
and available IP address with associated subnet mask, refer to the Oracle Big Data
Appliance configuration..
To identify IPoIB network information from an existing Oracle Big Data Appliance
compute node, run the following command:
[root@bda10 ~]# egrep '(IPADDR|NETMASK|NETWORK|BROADCAST)' /etc/sysconfig/network-
scripts/ifcfg-bondib0
IPADDR=192.168.41.10
NETMASK=255.255.248.0
NETWORK=192.168.40.0
BROADCAST=192.168.47.255
Using the information obtained, you can request for a free and unique IP address
belonging to Oracle Big Data Appliance's IPoIB subnet.
To find out the CIDR prefix, run the
[root@bda10 ~]# ipcalc -p 192.168.41.10 255.255.248.0
PREFIX=21
Note that in the example, the possible IP address host range is from 192.168.40.1 to
192.168.47.254. In the following step, IP 192.168.44.100/21 will be used as an IP
address for Exalytics' IPoIB bond0 interface.
Run the
configure the IPoIB (InfiniBand) network interface. It is a bonded interface named
bond0 using ib0 and ib1.
[root@exalytics root]# /opt/exalytics/bin/exalytics_configure_network.sh
Network Configuration Script Version exalytics-scripts-1.0.0.5ps3-52
Do you want to configure InfiniBand for Exadata connection? (y/n) [n]: y
Enter bond0 IP address:192.168.44.100
exalytics_configure_network.sh
command as follows:
ipcalc
script on Exalytics as root. When prompted,
Chapter 8
IPoIB Configuration (Layer-3)
8-5