Converged network switches for nx-os 4.0(1a)n1(1) (3 pages)
Summary of Contents for Cisco MDS 9216
Page 1
Cisco MDS 9000 Family SAN Volume Controller Configuration Guide Cisco MDS SAN-OS Release 1.3 September, 2004 Corporate Headquarters Cisco Systems, Inc. 170 West Tasman Drive San Jose, CA 95134-1706 http://www.cisco.com Tel: 408 526-4000 800 553-NETS (6387) Fax: 408 526-4100 Customer Order Number: DOC-7816119=...
Page 2
CCIP, CCSP, the Cisco Arrow logo, the Cisco Powered Network mark, Cisco Unity, Follow Me Browsing, FormShare, and StackWise are trademarks of Cisco Systems, Inc.; Changing the Way We Work, Live, Play, and Learn, and iQuick Study are service marks of Cisco Systems, Inc.; and Aironet, ASIST, BPX, Catalyst, CCDA, CCDP, CCIE,...
MDS switches Table 2 contains the history of the changes to the Cisco MDS 9000 Family SAN Volume Controller Configuration Guide, Release 1.3. When the document is updated for the next release, these changes are incorporated into the new revision and will no longer appear in this table.
Page 4
New and Changed Information Cisco MDS 9000 Family SAN Volume Controller Configuration Guide 78-16119-01, Cisco MDS SAN-OS Release 1.3...
Setting Up the Cisco MDS Switch Setting Up a New Cisco MDS Switch Updating an Existing Cisco MDS Switch SVC Role Authorization Understanding SVC Terminology Node I/O group Cluster MDisk Cisco MDS 9000 Family SAN Volume Controller Configuration Guide 78-16119-01, Cisco MDS SAN-OS Release 1.3...
Page 6
Configuring LUN Masking Identifying MDisks Configuring MDisk Groups Configuring Quorum Disks Managing Virtual Disks C H A P T E R Virtualization Policies Striped Sequential Cisco MDS 9000 Family SAN Volume Controller Configuration Guide 78-16119-01, Cisco MDS SAN-OS Release 1.3...
Page 7
C H A P T E R Upgrading Clusters Cluster Upgrade Prerequisites Recognizing Failure Situations Cluster Upgrade Guidelines Performing the Cluster Upgrade Sample Cluster Upgrade Cisco MDS 9000 Family SAN Volume Controller Configuration Guide 78-16119-01, Cisco MDS SAN-OS Release 1.3...
Page 8
Configuring a Dual Fabric SAN Environment 10-1 C H A P T E R Overview 10-2 Basic SVC Requirements 10-2 Dual Fabric Prerequisites 10-2 Sample Configuration 10-2 Cisco MDS 9000 Family SAN Volume Controller Configuration Guide viii 78-16119-01, Cisco MDS SAN-OS Release 1.3...
Configuration Guide. It also provides information on how to obtain related documentation. Audience This Cisco MDS 9000 Family SAN Volume Controller User Guide is for system administrators who configure and maintain network and storage systems using the Cisco CSM and the SAN-OS Command-Line Interface (CLI).
VDisks to hosts. Chapter Configuring Copy Services Discusses the copy services, including data migration, FlashCopy, and Remote Copy available in Cisco MDS 9216 Switches and in Cisco MDS 9500 Directors. Chapter Upgrading CSM Software Explains the procedures and processes for the different upgrade possibilities when a Caching Services Module (CSM) is used.
Cisco provides several ways to obtain documentation, technical assistance, and other technical resources. These sections explain how to obtain technical information from Cisco Systems. Cisco.com You can access the most current Cisco documentation on the World Wide Web at this URL: http://www.cisco.com/univercd/home/home.htm You can access the Cisco website at this URL: http://www.cisco.com...
Cisco Systems Corporate Headquarters (California, U.S.A.) at 408 526-7208 or, elsewhere in North America, by calling 800 553-NETS (6387). Documentation Feedback You can submit comments electronically on Cisco.com. On the Cisco Documentation home page, click Feedback at the top of the page. You can e-mail your comments to mdsfeedback-doc@cisco.com.
The Cisco TAC is available to all customers who need technical assistance with a Cisco product, technology, or solution. Two types of support are available: the Cisco TAC website and the Cisco TAC Escalation Center. The type of support that you choose depends on the priority of the problem and the conditions stated in service contracts, when applicable.
Cisco TAC website. Some services on the Cisco TAC website require a Cisco.com login ID and password. If you have a valid service contract but do not have a login ID or password, go to this URL to register: http://tools.cisco.com/RPF/register/register.do...
Page 15
You can access Packet magazine at this URL: http://www.cisco.com/go/packet • iQ Magazine is the Cisco bimonthly publication that delivers the latest information about Internet business strategies for executives. You can access iQ Magazine at this URL: http://www.cisco.com/go/iqmagazine •...
Page 16
Preface Obtaining Additional Publications and Information Cisco MDS 9000 Family SAN Volume Controller Configuration Guide 78-16119-01, Cisco MDS SAN-OS Release 1.3...
The combined SVC storage software along with the Cisco MDS 9000 Series CSM is delivered as a feature of the Cisco MDS 9000 Family. The SVC software runs on a clustered pair of CSMs within the switch. Based on virtualization technology, this solution is designed to support a virtualized pool of storage from the storage subsystems attached to a SAN.
Provides simple migration of storage. Provides a single set of copy services. • SVC Features This section explains the features provided by the combined SVC storage software and the Cisco MDS 9000 Series CSM solution. A central point for volume management control •...
Page 19
• High availability Like all other Cisco MDS 9000 Family modules, the CSM is hot-swappable and fully integrates into the Cisco MDS 9000 high availability architecture. The Cisco MDS 9000 SAN-OS software architecture offers an unparalleled level of availability including automatic restart of failed supervisor processes and fabric level availability via Cisco PortChannel capability which allows 16 physical links to be aggregated into one logical interface.
Page 20
Cisco MDS 9000 Family Command Line Interface (CLI), IBM’s SVC for Cisco MDS 9000 CLI, and IBM’s ICAT management GUI. For users who prefer a common interface for both SAN and Volume management, the Cisco SAN-OS CLI includes the full suite of capabilities necessary to manage your virtual storage environment from the SAN-OS command line.
The chapter explains the tasks required to set up and configure each building block of your system to run SVC for Switches. You will be performing these tasks using one or more Cisco MDS 9000 Family switches for setup and configuration. You may wish to have your Cisco and IBM documentation handy for reference to more detailed procedures.
Before setting up your hardware, ensure you have correctly installed the Cisco MDS chassis and its components as specified in “Chapter 2: Installing the Cisco MDS 9000 Family Switch” in the Cisco MDS [9216 Switch or the Cisco MDS 9500] Series Hardware Installation Guides.
The Cisco Fabric Manager provides an alternative to the CLI for most switch configuration commands. To use the Cisco MDS 9000 Fabric Manager, refer to the Cisco MDS 9000 Family Fabric Manager User Guide. To use the CLI, refer to the Cisco MDS 9000 Family Configuration Guide.
CLI or the Device Manager and Fabric Manager GUIs. To use the Cisco MDS 9000 Fabric Manager, refer to the Cisco MDS 9000 Family Fabric Manager User Guide.
A virtual representation of a LUN that is exposed by the cluster to the hosts in a SAN. Each VDisk is independently associated with a single I/O group. Cisco MDS 9000 Family SAN Volume Controller Configuration Guide 78-16119-01, Cisco MDS SAN-OS Release 1.3...
In order for SVC to virtualize the back-end storage to hosts, you must ensure that the hosts do not directly access the storage. You can separate hosts or disks using the concept of VSANs. The VSAN feature is specific to Cisco MDS 9000 Family switches. Alternatively, you can also use zones to separate hosts or disks.
Page 27
Initiator Initiator N-Port N-Port N-Port N-Port Disk VSAN VSAN 3 To configure a SVC interface and N-port VSANs in a Cisco MDS switch, follow these steps: Command Purpose Step 1 Enters configuration mode. switch# config t switch(config)# Step 2 Enters the configuration mode for SVC interface 2/1.
Page 30
Chapter 2 Getting Started Multiple Initiators and Targets Cisco MDS 9000 Family SAN Volume Controller Configuration Guide 2-10 78-16119-01, Cisco MDS SAN-OS Release 1.3...
Page 31
This section explains the steps required to create clusters. To configure other SVC features or to access the switch after initial configuration you can use one of the following CLI or Graphical User Interface (GUI) options: Cisco MDS 9000 Family CLI—to use the Cisco MDS CLI, follow the procedure specified in this • guide.
Physical Topology Figure 3-1, CSMs reside in slots 3 and 7 in a Cisco MDS 9500 Series switch. CSM 3 has two nodes identified as interface svc 3/1 and interface svc 3/2. CSM 7 has two nodes identified as interface svc 7/1 and interface svc 7/2.
Total number of entries = 4 Default zone configuration is permit for VSAN 2. Step 5 switch1(config)# zone default-zone permit vsan 2 switch1(config)# Cisco MDS 9000 Family SAN Volume Controller Configuration Guide 78-16119-01, Cisco MDS SAN-OS Release 1.3...
Unique id is 01:00:07:20:30:36:41:51 Node is in config mode Node is part of iogroup id 1 name io_grp0 The configured node is the only node in this cluster. Cisco MDS 9000 Family SAN Volume Controller Configuration Guide 78-16119-01, Cisco MDS SAN-OS Release 1.3...
Caution Do not add two nodes from the same CSM to the same I/O group of a cluster. Cisco MDS SVC does not allow this configuration as both nodes will be contained in one power domain. If both nodes are configured in the same I/O group of one cluster and a power failure occurs, both nodes will fail.
Enter the cluster configuration mode for the required cluster. Step 1 switch1# svc-config switch1(svc)# cluster config SampleCluster Issue the no node name command. Step 2 switch1(svc-cluster)# no node name node3 Cisco MDS 9000 Family SAN Volume Controller Configuration Guide 78-16119-01, Cisco MDS SAN-OS Release 1.3...
The MDS CLI does not use an explicit command to delete a cluster. The cluster is automatically deleted when the last node in the cluster is deleted. Cisco MDS 9000 Family SAN Volume Controller Configuration Guide 78-16119-01, Cisco MDS SAN-OS Release 1.3...
Page 40
Chapter 3 Creating and Managing Clusters Deleting a node from a Cluster Cisco MDS 9000 Family SAN Volume Controller Configuration Guide 3-10 78-16119-01, Cisco MDS SAN-OS Release 1.3...
MDisk are available for use in creating new VDisks data migration. Unmanaged mode • Mdisks in this mode do not belong to any Mdisk group Cisco MDS 9000 Family SAN Volume Controller Configuration Guide 78-16119-01, Cisco MDS SAN-OS Release 1.3...
3 switch(config-if)# no initiator vsan 1 switch(config-if)# exit Configure the default zone configuration as permit for VSAN3. Step 5 switch(config)# zone default-zone permit vsan 3 Cisco MDS 9000 Family SAN Volume Controller Configuration Guide 78-16119-01, Cisco MDS SAN-OS Release 1.3...
Target N-port WWN is 2e:a5:00:05:30:00:1a:e0, vsan is 1, FCID is 0xe80003 Initiator N-port WWN is 21:2e:00:05:30:00:00:21, vsan is 3, FCID is 0xea0004 Mgmt N-port WWN is 2f:af:00:05:30:00:1a:e0, vsan is 2, FCID is 0xe80000 Cisco MDS 9000 Family SAN Volume Controller Configuration Guide 78-16119-01, Cisco MDS SAN-OS Release 1.3...
Obtain a list of candidate MDisks using the show cluster cluster-name mdisk candidate command. Step 1 Select the group(s) to add the configured MDisk. switch(svc)# show cluster SampleCluster mdisk candidate ---------------------------------------------------- nwwn capacity ---------------------------------------------------- 50:05:07:63:00:c0:9c:f9 953 MB Cisco MDS 9000 Family SAN Volume Controller Configuration Guide 78-16119-01, Cisco MDS SAN-OS Release 1.3...
We recommend that you set quorum disks on multiple controllers to avoid the possibility of losing all of the quorum disks with a single failure. Cisco MDS 9000 Family SAN Volume Controller Configuration Guide 78-16119-01, Cisco MDS SAN-OS Release 1.3...
Page 48
If the quorum command fails due to the lack of sufficient extents in the new quorum disk, the old Caution quorum disk may no longer be operational. Cisco MDS 9000 Family SAN Volume Controller Configuration Guide 78-16119-01, Cisco MDS SAN-OS Release 1.3...
VDisk is independently associated with a single I/O group. This chapter includes the following sections: Virtualization Policies, page 5-2 • • Licensing Requirements, page 5-2 Configuring VDisks, page 5-3 • Cisco MDS 9000 Family SAN Volume Controller Configuration Guide 78-16119-01, Cisco MDS SAN-OS Release 1.3...
Virtualization is the process of creating a pool of storage that can be split into VDisks. VDisks are visible to the host systems that use them and provide a common way to manage SAN storage. VDisks in the Cisco MDS 9000 Family use one of three virtualization policies: striped, sequential, or image. Striped When a VDisk is created using a striped policy its extents are allocated from the specified ordered list of MDisks.
C H A P T E R Configuring Hosts To continue configuring the using the SVC application for a Cisco MDS 9216 switch or for any switch in the Cisco MDS 9500 Family, you must determine the number of hosts, isolate host traffic to VSAN 1, and map VDisks to hosts.
Multipath=no HostIPsforMP=<ip address of NIC1>,<ip address of NIC2> ConnFailTimeout=50 Refer to the Cisco MDS 9000 Family Configuration Guide for further details on iSCSI concepts and configuration options. Note Before configuring the iSCSI Hosts, be sure to configure the required level of iSCSI authentication.
Page 58
The following example is based on the persistent pWWN assigned to the initiator. You can obtain the pWWN from the output of the show iscsi initiator command. Cisco MDS 9000 Family SAN Volume Controller Configuration Guide 78-16119-01, Cisco MDS SAN-OS Release 1.3...
Page 59
Start the iSCSI clients on both hosts and verify that the sessions come up using the show iscsi session Step 10 command. Cisco MDS 9000 Family SAN Volume Controller Configuration Guide 78-16119-01, Cisco MDS SAN-OS Release 1.3...
Page 60
Chapter 6 Configuring Hosts Configuring iSCSI Hosts in SVC Cisco MDS 9000 Family SAN Volume Controller Configuration Guide 78-16119-01, Cisco MDS SAN-OS Release 1.3...
Page 61
C H A P T E R Configuring Copy Services The SVC copy services function available in all Cisco MDS 9216 switches and directors in the Cisco MDS 9500 Family enables you to copy virtual Disks (VDisks). These copy services include data migration, FlashCopy , and Remote Copy.
Each VDisk may be a member of only one FlashCopy mapping. VDisks participating in FlashCopy mapping cannot have their size increased or decreased while they remain participants of the FlashCopy mapping. Cisco MDS 9000 Family SAN Volume Controller Configuration Guide 78-16119-01, Cisco MDS SAN-OS Release 1.3...
The target VDisk, and the source VDisk in a FlashCopy mapping need to be of same size. Note Enter the cluster configuration submode Step 3 switch(svc)# cluster config SampleCluster switch(svc-cluster)# Step 4 Enable the FlashCopy feature. switch(svc-cluster)# feature enable flash-copy Cisco MDS 9000 Family SAN Volume Controller Configuration Guide 78-16119-01, Cisco MDS SAN-OS Release 1.3...
You can only start the FlashCopy process if you have configured a consistency group as specified in the “FlashCopy Consistency Groups” section on page 7-5. To start the FlashCopy process, follow these steps: Cisco MDS 9000 Family SAN Volume Controller Configuration Guide 78-16119-01, Cisco MDS SAN-OS Release 1.3...
A FlashCopy, once started, can be stopped by issuing the cluster name cluster-name flash-copy fcgrp stop command. Once stopped, use the cluster name cluster-name flash-copy fcgrp prepare command for the FlashCopy group before it is started. Cisco MDS 9000 Family SAN Volume Controller Configuration Guide 78-16119-01, Cisco MDS SAN-OS Release 1.3...
But a more significant use arises where the relationships contain VDisks that have Cisco MDS 9000 Family SAN Volume Controller Configuration Guide 78-16119-01, Cisco MDS SAN-OS Release 1.3...
Establish a remote copy partnership with the remote cluster at the local cluster. Step 7 local-switch(svc-cluster)# remote-copy cluster remote-cluster local-switch(svc-cluster)# Cisco MDS 9000 Family SAN Volume Controller Configuration Guide 78-16119-01, Cisco MDS SAN-OS Release 1.3...
VDisk are also mirrored in the corresponding auxiliary VDisk, in the remote copy relationship. To start remote copy relationships, follow these steps: Cisco MDS 9000 Family SAN Volume Controller Configuration Guide 7-10 78-16119-01, Cisco MDS SAN-OS Release 1.3...
VDisk, in a relationship, are not reflected at the corresponding auxiliary VDisk. local-switch(svc)# cluster name local-cluster remote-copy rcgrp stop local-switch(svc)# Cisco MDS 9000 Family SAN Volume Controller Configuration Guide 7-11 78-16119-01, Cisco MDS SAN-OS Release 1.3...
The Primary column indicate whether the auxiliary VDisks or source VDisks have the primary (or master) role. remote-cluster(svc)# show cluster remote-cluster remote-copy ---------------------------------------------------------- Name Remote Cluster Mappings Primary Status ---------------------------------------------------------- rcgrp remote consistent_synchronised remote-cluster(svc)# Cisco MDS 9000 Family SAN Volume Controller Configuration Guide 7-12 78-16119-01, Cisco MDS SAN-OS Release 1.3...
Page 73
C H A P T E R Upgrading CSM Software When CSMs are present in any switch in the Cisco MDS 9000 Family, several kinds of upgrade may be performed as required—a cluster software upgrade, an automatic upgrade when nodes are added, a service mode upgrade, or a switch software upgrade.
The force option can be used to override this restriction if you do not mind losing access to data during the upgrade. Cisco MDS 9000 Family SAN Volume Controller Configuration Guide 78-16119-01, Cisco MDS SAN-OS Release 1.3...
If any of nodes report a check failure, then an error message is issued with a table displaying all the check failures across all the nodes in the cluster. Cisco MDS 9000 Family SAN Volume Controller Configuration Guide 78-16119-01, Cisco MDS SAN-OS Release 1.3...
SampleCluster active active 1.3(1) svc8/2 SampleCluster active active 1.3(1) In this example, node svc4/1 has completed upgrade to the new version of the svc software. Cisco MDS 9000 Family SAN Volume Controller Configuration Guide 78-16119-01, Cisco MDS SAN-OS Release 1.3...
Issue the show node local command in SVC configuration mode to verify that the required node is Step 4 running with new node software. switch1(svc)# show nodes local ------------------------------------------------------------------------------- Node cluster config cluster node node status status version Cisco MDS 9000 Family SAN Volume Controller Configuration Guide 78-16119-01, Cisco MDS SAN-OS Release 1.3...
------------------------------------------------------------------------------- svc3/1 SampleCluster active active 1.3(1) svc3/2 SampleCluster active active 1.3(1) svc7/1 SampleCluster active active 1.3(1) svc7/2 SampleCluster active active 1.3(x) Cisco MDS 9000 Family SAN Volume Controller Configuration Guide 78-16119-01, Cisco MDS SAN-OS Release 1.3...
Exit the switch console and open a new terminal session to view the upgraded supervisor module using Step 4 the show module command. Sample Switch Upgrade switch# install all system scp://usery@171.69.16.22/auto/vwsvkd/usery/m9500-sf1ek9-mzg.1.3.x.bin For scp://usery@171.69.16.22, please enter password: Cisco MDS 9000 Family SAN Volume Controller Configuration Guide 78-16119-01, Cisco MDS SAN-OS Release 1.3...
RAID arrays or partitions to the cluster, you must modify the port groups that belong to the cluster because the nWWN or pWWN's of the node have changed. Cisco MDS 9000 Family SAN Volume Controller Configuration Guide 78-16119-01, Cisco MDS SAN-OS Release 1.3...
Replacing a CSM in the Same Slot If a CSM is replaced by another CSM in the same slot in any switch in the Cisco MDS 9500 Series or in a Cisco MDS 9216 Switch, the same nWWNs and pWWNs are automatically assigned for both interfaces on the new CSM.
The old nWWNs and pWWNs will be lost from the system and never reassigned for any purpose in that chassis. Replace the new CSM in the same slot as directed in the Cisco MDS 9216 Switch or the Cisco MDS 9500 Step 3 Series Hardware Installation Guide.
The Node ID of the replacement node is different from the node ID of the replaced node. Use the SDD management tool on the host systems to verify that all paths are online. Step 4 Cisco MDS 9000 Family SAN Volume Controller Configuration Guide 8-12 78-16119-01, Cisco MDS SAN-OS Release 1.3...
Caching Services Module (CSM) module. The SPAN feature is specific to switches in the Cisco MDS 9000 Family. It monitors network traffic though a interface. Traffic through any SVC interface can be replicated to a special port called the SPAN destination port (SD port).
Configure the destination interface for the SPAN session. Step 3 switch(config-span)# destination interface fc1/9 Configure the source interface svc1/1 for the initiator, target, and mgmt traffic-types in both directions. Step 4 Cisco MDS 9000 Family SAN Volume Controller Configuration Guide 78-16119-01, Cisco MDS SAN-OS Release 1.3...
Egress (tx) sources are svc1/1(inititator), svc1/1 (target), svc1/1 (mgmt) svc2/1(target) You have now configured SVC interfaces as SPAN sources. Refer to the Cisco MDS 9000 Family Configuration Guide for further details on the SPAN feature. Using VSAN Filters to Specify N-Port Traffic By default, traffic on all N-ports are spanned.
Page 88
Configures VSAN 4 and 5 as a session filter to additionally span initiator N-port traffic in these two Step 5 VSANS. switch(config-span)# source filter vsan 4-5 Cisco MDS 9000 Family SAN Volume Controller Configuration Guide 78-16119-01, Cisco MDS SAN-OS Release 1.3...
All SVC nodes must be able to use the same preferred target controller port for a given Mdisk. In a redundant, dual fabric configuration, the Cisco MDS Inter-VSAN Routing (IVR) feature must be used in order to satisfy these basic SVC requirements. IVR provides the same benefit of keeping the two fabrics separate while providing the connectivity required to implement SVC.
Page 91
2 switch1(config-if)# no target vsan 1 switch1(config-if)# target vsan 3 switch1(config-if)# no mgmt vsan 1 switch1(config-if)# mgmt vsan 4 switch1(config-if)# exit Cisco MDS 9000 Family SAN Volume Controller Configuration Guide 10-3 78-16119-01, Cisco MDS SAN-OS Release 1.3...
Page 92
Set up the IVR zone to allow SVC initiators in Switch 2 to talk to target port in Switch 1. switch1(config)# ivr zone name ivr_12_2 switch1(config-ivr-zone)# member pwwn 23:30:00:05:30:00:8d:e2 vsan 12 Cisco MDS 9000 Family SAN Volume Controller Configuration Guide 10-4 78-16119-01, Cisco MDS SAN-OS Release 1.3...
Page 93
14 switch2(config-vsan-db)# vsan 20 switch2(config-vsan-db)# exit Set the static domains in Switch 2. Step 3 switch2(config)# fcdomain domain 12 static vsan 12 Cisco MDS 9000 Family SAN Volume Controller Configuration Guide 10-5 78-16119-01, Cisco MDS SAN-OS Release 1.3...
Page 94
Set up the IVR zone to allow SVC initiators in Switch 1 to talk to the target port in Switch 2. Step 10 switch2(config)# ivr zone name ivr_2_12 Cisco MDS 9000 Family SAN Volume Controller Configuration Guide 10-6 78-16119-01, Cisco MDS SAN-OS Release 1.3...
Page 95
20 switch2(config-if)# exit You have now configured the second fabric for the dual fabric SAN environment displayed in Figure 10-1. Cisco MDS 9000 Family SAN Volume Controller Configuration Guide 10-7 78-16119-01, Cisco MDS SAN-OS Release 1.3...
Page 96
Chapter 10 Configuring a Dual Fabric SAN Environment Sample Configuration Cisco MDS 9000 Family SAN Volume Controller Configuration Guide 10-8 78-16119-01, Cisco MDS SAN-OS Release 1.3...