Page 3
Document History Title Publication number Summary of changes Date Fabric OS Encryption Administrator’s 53-1001114-01 New document. August 2008 Guide Fabric OS Encryption Administrator’s 53-1001114-02 Revised document to September 2008 Guide include additional best practices. Fabric OS Encryption Administrator’s 53-1001114-03 Revised document to September 2008 Guide include new performance...
Page 4
Fabric OS Encryption Administrator’s Guide 53-1001864-01...
Contents About This Document In this chapter ......... . . xiii How this document is organized .
Page 6
Chapter 2 Encryption configuration using the Management application In this chapter ......... . . 13 Encryption Center features.
Page 7
Adding Target Tape LUNs for encryption ..... . 59 Configuring encrypted tape storage in a multi-path environment ....... . 60 Re-balancing the encryption engine .
Page 8
Configuring cluster links ........98 Special consideration for blades ......98 IP Address change of a node within an encryption group.
Page 9
Tape pool configuration ........137 Tape pool labeling ........137 Creating a tape pool .
Page 10
Configuration upload and download considerations ...168 Configuration Upload at an encryption group leader node. . .168 Configuration upload at an encryption group member node .168 Information not included in an upload ....168 Steps before configuration download.
Page 11
Best practices for host clusters in an encryption environment . . .180 HA Cluster Deployment Considerations and Best Practices ..180 Chapter 6 Maintenance and Troubleshooting In this Chapter ......... .181 Encryption group and HA cluster maintenance.
Page 12
Appendix C NS-Based Transparent Frame Redirection Index Fabric OS Encryption Administrator’s Guide 53-1001864-01...
Supported hardware and software The following hardware platforms support data encryption as described in this manual. • Brocade DCX and DCX-4S with an FS8-18 encryption blade. • Brocade Encryption Switch. What’s new in this document Information about decommissioning a encrypted LUN, hosting disk and tape containers on the same encryption engine and support for replicated LUN environments is included in this document..
Page 15
Optional element. variable Variables are printed in italics. In the help pages, variables are underlined or enclosed in angled brackets < >. Repeat the previous element, for example “member[;member...]” value Fixed values following arguments are printed in plain font. For example, show WWN Boolean.
Notice to the reader This document may contain references to the trademarks of the following corporations. These trademarks are the properties of their respective companies and corporations. These references are made for informational purposes only. Corporation Referenced Trademarks and Products Microsoft Corporation Windows, Windows NT, Internet Explorer Net App...
• Best practice guides, white papers, data sheets, and other documentation is available through the Brocade Partner Web site. For additional resource information, visit the Technical Committee T11 Web site. This Web site provides interface standards for high-performance and mass storage applications for Fibre Channel, storage management, and other applications: http://www.t11.org For information about the Fibre Channel industry, visit the Fibre Channel Industry Association Web...
If you cannot use the licenseIdShow command because the switch is inoperable, you can get the WWN from the same place as the serial number, except for the Brocade DCX. For the Brocade DCX, access the numbers on the WWN cards by removing the Brocade logo plate at the top of the non-port side of the chassis.
Terminology Terminology The following are definitions of terms used extensively in this document. ciphertext Encrypted data. cleartext Unencrypted data. CryptoModule The secure part of an encryption engine that is protected to the FIPS 140-2 level 3 standard. The term CryptoModule is used primarily in the context of FIPS authentication.
Page 21
Terminology Recovery cards A set of smart cards that contain a backup master key. Each recovery card holds a portion of the master key. The cards must be gathered and read together from a card reader attached to a PC running the Brocade SAN Management Application to restore the master key.
The Brocade encryption switch The Brocade encryption switch The Brocade encryption switch (Figure 1) is a high performance 32 port auto-sensing 8 Gbps Fibre Channel switch with data cryptographic (encryption/decryption) and data compression capabilities. The switch is a network-based solution that secures data-at-rest for heterogeneous tape drives, disk array LUNs, and virtual tape libraries by encrypting the data, using Advanced Encryption Standard (AES) 256-bit algorithms.
The FS8-18 blade The FS8-18 blade The FS8-18 blade provides the same features and functionality as the encryption switch. The FS8-18 blade installs on the Brocade DCX and DCX-4S. Four FS8-18 blades may be installed in a single DCX or DCX-4S. Performance licensing Encryption processing power is scalable, and may be increased by purchasing and installing an encryption performance license.
Recommendation for connectivity Recommendation for connectivity In order to achieve high performance and throughput, the encryption engines perform what is referred to as “cut-through” encryption. In simple terms this is achieved by encrypting the data in data frames on a per frame basis. This enables the encryption engine to buffer only a frame, encrypt it and send the frame out to the target on write I/Os.
Brocade encryption solution overview Brocade encryption solution overview The loss of stored private data, trade secrets, intellectual properties, and other sensitive information through theft or accidental loss of disk or tape media can have widespread negative consequences for governments, businesses, and individuals. This threat is countered by an increasing demand from governments and businesses for solutions that create and enforce policies and procedures that protect stored data.
Brocade encryption solution overview Data flow from server to storage The Brocade encryption switch can be introduced into a SAN with minimum disruption, with no need for SAN reconfiguration, and with no need to reconfigure host applications. Frames sent from a host and a target LUN are redirected to a virtual target associated with the encryption switch.
Data encryption key life cycle management Data encryption key life cycle management Data encryption keys (DEKs) are generated by the encryption engine. Data is encrypted and decrypted using the same DEK, so a DEK must be preserved at least long enough to decrypt the ciphertext that it created.
Page 28
Data encryption key life cycle management FIGURE 5 DEK life cycle Fabric OS Encryption Administrator’s Guide 53-1001864-01...
Key management systems Key management systems Key management systems are available from several vendors. This release supports the following leading key management systems: • The NetApp LIfetime Key Manager (LKM) version 4.0 or later. • The RSA Key Manager (RKM) version 2.1.3 or later, available through EMC. •...
Support for Virtual Fabrics Support for Virtual Fabrics The Brocade encryption switch does not support the logical switch partitioning capability and can not be partitioned, but the switch can be connected to any Logical Switch partition or Logical Fabric using an E-Port. The FS8-18 encryption blades are supported in only in a default switch partition All FS8-18 blades must be placed in a default switch partition in DCX or DCX-4S.
Encryption Center features Encryption Center features The Encryption Center dialog box (Figure 6) is the single launching point for all encryption-related configuration in the Management application. It also provides a table that shows the general status of all encryption-related hardware and functions at a glance. FIGURE 6 Encryption Center dialog box Beginning with Fabric OS version 6.4, the Encryption Center is dynamically updated to reflect the...
Encryption user privileges Encryption user privileges In the Management application, resource groups are assigned privileges, roles, and fabrics. Privileges are not directly assigned to users; users get privileges because they belong to a role in a resource group. A user can only belong to one resource group at a time. The Management application provides three pre-configured roles: •...
Smart card usage Smart card usage Smart Cards are credit card-sized cards that contain a CPU and persistent memory. Smart cards can be used as security devices. You must have Storage Encryption Security user privileges to activate, register, and configure smart cards. Smart cards can be used to do the following: •...
Smart card usage 3. Select the Quorum Size. The quorum size is the minimum number of cards necessary to enable the card holders to perform the security sensitive operations listed above. The maximum quorum size is five cards. The actual number of authentication cards registered is always more than the quorum size, so if you set the quorum size to five, for example, you will need to register at least six cards in the subsequent steps.
Smart card usage De-registering an authentication card Authentication cards can be removed from the database and the switch by de-registering them. Use the following procedure to de-register an authentication card. 1. Select the authentication card on the Authentication Card table. 2.
Smart card usage Enabling or disabling the system card requirement If you want to use a system card to control activation of an encryption engine on a switch, you must enable the system card requirement. You can use the following procedure to enable or disable the system card requirement.
Smart card usage De-registering a system card System cards can be removed from the database by de-registering them. Use the following procedure to de-register a system card. 1. From the Register System Card dialog box, select the system card you want to de-register. 2.
Network connections Network connections Before you use the encryption setup wizard for the first time, you must have the following required network connections: • The management ports on all encryption switches and 384-port Backbone Chassis CPs that have encryption blades installed must have a LAN connection to the SAN management program, and must be available for discovery.
Encryption node initialization and certificate generation Encryption node initialization and certificate generation When an encryption node is initialized, the following security parameters and certificates are generated: • FIPS crypto officer • FIPS user • Node CP certificate • A self-signed Key authentication center KAC) certificate •...
Steps for connecting to an SKM appliance Steps for connecting to an SKM appliance The SKM management web console can be accessed from any web browser with Internet access to the SKM appliance. The URL for the appliance is as follows: https://<appliance hostname>:<appliance port number>...
Steps for connecting to an SKM appliance Configuring a Brocade group on SKM A Brocade group is configured on SKM for all keys created by Brocade encryption switches and blades. This needs to be done only once for each key vault. 1.
Steps for connecting to an SKM appliance Registering the SKM Brocade group user name and password The Brocade group user name and password you created when configuring a Brocade group on SKM must also be registered on each Brocade encryption node. 1.
Steps for connecting to an SKM appliance Setting up the local Certificate Authority (CA) on SKM To create and install a local CA, perform the following steps: 1. Login to the SKM management web console using the admin password. 2. Select the Security tab. 3.
Steps for connecting to an SKM appliance In the Trusted Certificate Authority List, click Edit. 8. From the list of Available CAs in the right panel, select the CA you just created. Repeat these steps any time another local CA is needed. Downloading the local CA certificate from SKM The local CA certificate you created using the procedure for “Setting up the local Certificate...
Steps for connecting to an SKM appliance 11. Enter the required data in the Sign Certificate Request section of the window. Select the CA name from the Sign with Certificate Authority drop down box. Select Server as the Certificate Purpose. Enter the number of days before the certificate must be renewed based on your site's security policies.
Steps for connecting to an SKM appliance Creating an SKM High Availability cluster The HP SKM key vault supports clustering of HP SKM appliances for high availability. If two SKM key vaults are configured, they must be clustered. If only a single SKM appliance is configured, it may be clustered for backup purposes, but the backup appliance will not be directly used by the switch.
Steps for connecting to an SKM appliance Adding SKM appliances to the cluster If you are adding an appliance to an existing cluster, select the Cluster Settings section of the window, click Download Cluster Key and save the key to a convenient location, such as your computer's desktop.
Steps for connecting to an SKM appliance Signing the Brocade encryption node KAC certificates The KAC certificate signing request generated when the encryption node is initialized must be exported for each encryption node and signed by the Brocade local CA on SKM. The signed certificate must then be imported back into the encryption node.
Gathering information Gathering information Before you use the encryption setup wizard for the first time, you should also have a detailed configuration plan in place and available for reference. The encryption setup wizard assumes the following: • You have a plan in place to organize encryption devices into encryption groups. •...
Creating a new encryption group Creating a new encryption group The following steps describe how to start and run the encryption setup wizard, and then create a new encryption group. NOTE When a new encryption group is created, any existing tape pools in the switch are removed. 1.
Page 53
Creating a new encryption group 4. Click Next. Create a new encryption Group is pre-selected. This is the correct selection for creating a new group. FIGURE 11 Designate Switch Membership dialog box 5. Enter an Encryption Group Name for the encryption group (the maximum length of the group name is 15 characters;...
Page 54
Creating a new encryption group FIGURE 12 Select Key Vault dialog box Select SKM as the Key Vault Type. Fabric OS Encryption Administrator’s Guide 53-1001864-01...
Page 55
Creating a new encryption group When you select SKM, the options are as shown in Figure a. Enter the IP address or host name for the primary key vault. b. Enter the name of the file that holds the primary key vault’s public key certificate or browse to the location by clicking the Browse button.
Page 56
Creating a new encryption group FIGURE 14 Specify Public Key Certificate filename dialog box 8. Specify the name of the file where you want to store the public key certificate that is used to authenticate connections to the key vault, and click Next. The certificate stored in this file is the switch’s public key certificate.
Page 57
Creating a new encryption group 10. Enter a file name, or browse to the desired location. 11. Enter the passphrase, which is required for restoring the master key. The passphrase can be between eight and 40 characters, and any character is allowed. 12.
Page 58
Creating a new encryption group FIGURE 17 Configuration Status dialog box The Management application sends API commands to verify the switch configuration. The CLI commands are detailed in the Fabric OS Encryption Administrator’s Guide, “Key vault configuration.” • Initialize the switch If the switch is not already in the initiated state, the Management application performs the command.
Adding a switch to an encryption group • Save the switch’s public key certificate to a file The Management application saves the KAC certificate into the specified file. • Back up the master key to a file The Management application saves the master key into the specified file. 15.
Page 60
Adding a switch to an encryption group FIGURE 18 Add switch to an encryption group - Designate Switch Membership dialog box a. Select Add this switch to an existing encryption group. b. Click Next. The Add Switch to Existing Encryption Group dialog box displays. FIGURE 19 Add Switch to Existing Encryption Group dialog box 5.
Page 61
Adding a switch to an encryption group FIGURE 20 Add switch to an encryption group - Specify Public Key Certificate filename dialog box 6. Specify the name of the file where you want to store the public key certificate that is used to authenticate connections to the key vault, and click Next.
Page 62
Adding a switch to an encryption group Click Next to confirm the displayed information. The Configuration Status displays. • A progress indicator shows that a configuration step is in progress. A green check mark indicates successful completion of all steps for that Configuration Item. A red stop sign indicates a failed step.
Replacing an encryption engine in an encryption group FIGURE 23 Add switch to an encryption group - Next Steps dialog box 9. Click Finish to exit the Configure Switch Encryption wizard. Replacing an encryption engine in an encryption group To replace an encryption engine in an encryption group with another encryption engine within the same DEK Cluster, complete the following steps.
Creating high availability (HA) clusters FIGURE 24 Engine Operations tab 2. Select the engine you want to replace in the Engine list. 3. Select the engine you want to use as the replacement in the Replacement list. 4. Click Replace. All containers hosted by the current engine (Engine list) are replaced by the new engine (Replacement list).
Creating high availability (HA) clusters 4. Select an available encryption engine, and a destination HA cluster under High-Availability Clusters. Select New HA Cluster if you are creating a new cluster. 5. Click the right arrow to add the encryption engine to the selected HA cluster. FIGURE 25 HA Clusters tab NOTE...
Creating high availability (HA) clusters Swapping engines in an HA cluster Swapping engines is useful when replacing hardware. Swapping engines is different from removing an engine and adding another because when you swap engines, the configured targets on the former HA cluster member are moved to the new HA cluster member. To swap engines, select one engine from the right tree (see Figure 25) and one unclustered engine...
Adding encryption targets Adding encryption targets Adding an encryption target maps storage devices and hosts to virtual targets and virtual initiators within the encryption switch. NOTE You must zone the physical host and physical target together to enable creation of a re-direction zone.
Page 68
Adding encryption targets 5. Click Next to begin. The Select Encryption Engine dialog box displays. The list of engines depends on the scope being viewed. • If the Targets dialog box is showing all targets in an encryption group, the list includes all engines in the group.
Page 69
Adding encryption targets 6. Select the encryption engine (blade or switch) you want to configure, and click Next. The Select Target panel displays. This panel lists all target ports and target nodes in the same fabric as the encryption engine. The Select Target list does not show targets that are already configured in an encryption group.
Page 70
Adding encryption targets Click Next. The Select Hosts panel displays. This panel lists all hosts in the same fabric as the encryption engine. There are two available methods for selecting hosts: select from a list of known hosts or manually enter the port and node world wide names. FIGURE 29 Select Hosts dialog box a.
Page 71
Adding encryption targets FIGURE 30 Name Container dialog box 10. Click Next. The Confirmation panel displays. FIGURE 31 Confirmation dialog box Fabric OS Encryption Administrator’s Guide 53-1001864-01...
Page 72
Adding encryption targets 11. Click Next to confirm the displayed information. The Configuration Status displays the target and host that are configured in the target container, as well as the virtual targets (VT) and virtual initiators (VI). NOTE If you can view the VI/VT Port WWNs and VI/VT Node WWNs, the container has been successfully added to the switch.
Page 73
Adding encryption targets 13. Click Next to confirm the configuration. The Important Instructions dialog box displays. FIGURE 33 Important Instructions dialog box 14. Review the instructions about post-configuration tasks you must complete after you close the wizard. 15. Click Finish to exit the Configure Storage Encryption wizard. Fabric OS Encryption Administrator’s Guide 53-1001864-01...
Configuring hosts for encryption targets Configuring hosts for encryption targets Use the Encryption Target Hosts dialog box to edit (add or remove) hosts for an encrypted target. NOTE Hosts are normally selected as part of the Configure Storage Encryption wizard but you can also edit hosts later using the Encryption Target Hosts dialog box.
Adding target disk LUNs for encryption Adding target disk LUNs for encryption You can add a new path to an existing disk LUN or add a new LUN and path by launching the Add New Path wizard. Take the following steps to launch the Add New Path wizard. 1.
Page 76
Adding target disk LUNs for encryption 5. Click Next. The Select Initiator Port dialog box displays. 6. Select the initiator port from the Initiator Port list. Click Next. LUN discovery is launched, and a progress bar displays. There are four possible outcomes: A message displays indicating No LUNs are discovered.
Adding Target Tape LUNs for encryption Adding Target Tape LUNs for encryption You configure a Crypto LUN by adding the LUN to the CryptoTarget container and enabling the encryption property on the Crypto LUN. You must add LUNs manually. After you add the LUNs, you must specify the encryption settings.
Re-balancing the encryption engine 8. Select the desired encryption mode. • If you change a LUN policy from Native Encryption or DF-Compatible Encryption to Clear Text, you disable encryption. • The LUNs of the target which are not enabled for encryption must still be added to the CryptoTarget container with the Clear Text encryption mode option.
Master keys 1. Select Configure > Encryption from the menu bar. The Encryption Center dialog box displays. 2. Select an encryption engine and select Engine > Re-Balance from the menu bar, or right click on the encryption engine, and select Re-Balance. A warning message displays, cautioning you about the potential disruption of disk and tape I/O, and telling you that the operation may take several minutes.
Master keys • Backup master key, which is enabled any time a master key exists. • Restore master key, which is enabled when no master key exists or the previous master key has been backed up. • Create new master key, which is enabled when no master key exists or the previous master key has been backed up.
Master keys 4. Select Backup Master Key as the Master Key Action. The Master Key Backup dialog box displays, but only if the master key has already been generated. FIGURE 38 Backup Destination (to file) dialog box 5. Select File as the Backup Destination. 6.
Page 82
Master keys Saving a master key to a key vault Use the following procedure to save the master key to a key vault. 1. Select Configure > Encryption from the menu bar. The Encryption Center dialog box displays. 2. Select an encryption group from the tree, and click Properties. 3.
Master keys Saving a master key to a smart card set A card reader must be attached to the SAN Management application PC to complete this procedure. Recovery cards can only be written once to back up a single master key. Each master key backup operation requires a new set of previously unused smart cards.
Master keys FIGURE 40 Backup Destination (to smart cards) dialog box 5. Select A Recovery Set of Smart Cards as the Backup Destination. 6. Enter the recovery card set size. Insert the first blank card and wait for the card serial number to appear. 8.
Page 85
Master keys Restoring a master key from a file Use the following procedure to restore the master key from a file. 1. Select Configure > Encryption from the menu bar. The Encryption Center dialog box displays. 2. Select an encryption group from the tree, and click Properties. 3.
Master keys Restoring a master key from a key vault Use the following procedure to restore the master key from a key vault. 1. Select Configure > Encryption from the menu bar. The Encryption Center dialog box displays. 2. Select an encryption group from the tree, and click Properties. 3.
Master keys Restoring a master key from a smart card set A card reader must be attached to the SAN Management application PC to complete this procedure. Use the following procedure to restore the master key from a set of smart cards. 1.
Master keys 10. Insert the next card, and repeat step 8 step 11. Continue until all the cards in the set have been read. 12. Click OK. Creating a new master key Though it is generally not necessary to create a new master key, you may be required to create one due to circumstances such as the following: •...
Zeroizing an encryption engine Zeroizing an encryption engine Zeroizing is the process of erasing all data encryption keys and other sensitive encryption information in an encryption engine. You can zeroize an encryption engine manually to protect encryption keys. No data is lost because the data encryption keys for the encryption targets are stored in the key vault.
Encryption Targets dialog box 4. Click YES to zeroize the encryption engine. Encryption Targets dialog box The Encryption Targets dialog box enables you to send outbound data that you want to store as ciphertext to an encryption device. The encryption target acts as a virtual target when receiving data from a host, and as a virtual initiator when writing the encrypted data to storage.
Encryption Targets dialog box FIGURE 45 Encryption Targets dialog box TABLE 1 Encryption Targets dialog box fields and components Feature Description Add button Launches the Storage Encryption Setup Wizard, which enables you to configure a new target for encryption. It is the first step in configuring encryption for a storage device.
Page 92
Encryption Targets dialog box TABLE 1 Encryption Targets dialog box fields and components (Continued) Feature Description Hosts button Launches the Encryption Target Hosts dialog box, where you can configure hosts to access the selected encryption target. LUNs button Launches the Encryption Target LUNs dialog box, where you can display existing LUNs and add new LUNs.
Disk device decommissioning Disk device decommissioning A disk device needs to be decommissioned when any of the following occur: • The storage lease expires for an array, and devices must be returned or exchanged. • Storage is reprovisioned for movement between departments. •...
Viewing and editing switch encryption properties Displaying and deleting decommissioned key IDs When disk LUNs are decommissioned, the process includes the disabling of the key record in the key vault and indicating that the key has been decommissioned. These decommissioned keys are still stored on the switch.
Page 95
Viewing and editing switch encryption properties FIGURE 46 Encryption Properties dialog box • Switch Properties table - the properties associated with the selected switch. • Name - the name of the selected switch. • Node WWN - the world wide name of the node. •...
Page 96
Viewing and editing switch encryption properties • Domain ID - the domain ID of the selected switch. • Firmware Version - the current encryption firmware on the switch. • Primary Key Vault Link Key Status - the possible statuses are as follows: Not Used –...
Viewing and editing switch encryption properties • HA Cluster Name - the name of the HA cluster (for example, Cluster1), if in an HA configuration. The name can have a maximum of 31 characters. Only letters, digits, and underscores are allowed.
Viewing and editing group properties Enabling the encryption engine state from Properties To enable the encryption engine, complete the following steps. 1. Find the Set State To entry under Encryption Engine Properties. 2. Click the field and select Enabled. 3. Click OK. Disabling the encryption engine state from Properties To disable the encryption engine, complete the following steps.
Viewing and editing group properties FIGURE 47 Encryption Group Properties dialog box General tab The properties displayed in the General tab are described below. • Encryption group name - the name of the encryption group. • Group status - the status of the encryption group, which can be OK-Converged or Degraded. Degraded means the group leader cannot contact all of the configured group members.
Page 100
Viewing and editing group properties Members tab The Group Members tab lists group switches, their role, and their connection status with the group leader. The tab displays the configured membership for the group (none of the table columns are editable). The list can be different from the members displayed in the Encryption Center dialog box if some configured members are unmanaged, missing, or in a different group.
Page 101
Viewing and editing group properties Consequences of removing an encryption switch Table 2 explains the impact of removing switches. TABLE 2 Switch removal impact Switch configuration Impact of removal The switch is the only switch in the encryption The encryption group is also removed. group.
Viewing and editing group properties Figure 49 shows the warning message that displays if you click Remove to remove an encryption group. FIGURE 49 Removal of switch in encryption group warning Fabric OS Encryption Administrator’s Guide 53-1001864-01...
Page 103
Viewing and editing group properties Security tab The Security tab (Figure 50) displays the status of the master key for the encryption group. NOTE You must enable encryption engines before you back up or restore master keys. Master key actions are as follows: •...
Page 104
Viewing and editing group properties HA Clusters tab HA clusters are groups of encryption engines that provide high availability features. If one of the engines in the group fails or becomes unreachable, the other cluster member takes over the encryption and decryption tasks of the failed encryption engine. An HA cluster consists of exactly two encryption engines.
Viewing and editing group properties Tape Pools tab Tape pools are managed from the Tape Pools tab. Figure 52 displays the tape pools tab. FIGURE 52 Encryption Group Properties - Tape Pools tab • If you want to remove a tape pool, select one or more tape pools in the list and click Remove. •...
Viewing and editing group properties Adding tape pools A tape pool can be identified by either a name or a number, but not both. Tape pool names and numbers must be unique within the encryption group. When a new encryption group is created, any existing tape pools in the switch are removed and must be added.
Page 107
Encryption-related acronyms in log messages Choices include Clear Text, DF-Compatible Encryption, and Native Encryption. DF-Compatible Encryption is valid only when LKM is the key vault. The Key Lifespan (days) field is editable only if the tape pool is encrypted. If Clear Text is selected as the encryption mode, the key lifespan is disabled.
Command validation checks Command validation checks Before a command is executed, it is validated against the following checks. 1. Active or Standby availability: on enterprise-class platforms, checks that the command is available on the Control Processor (CP). 2. Role Based Access Control (RBAC) availability: checks that the invoking user’s role is permitted to invoke the command.
Command RBAC permissions and AD types Command RBAC permissions and AD types There are two RBAC roles that are permitted to perform Encryption operations. 1. Admin and SecurityAdmin Users authenticated with the Admin and SecurityAdmin RBAC roles may perform cryptographic functions assigned to the FIPS Crypto Officer including the following: •...
Page 112
Command RBAC permissions and AD types TABLE 4 Encryption command RBAC availability and admin domain type (Continued) Command name User Admin Operator Switch Zone Fabric Basic Security Admin Domain Admin Admin Admin Switch Admin Admin createtapepool Disallowed decommission Disallowed deletecontainer Disallowed deleteencgroup Disallowed...
Page 113
Command RBAC permissions and AD types TABLE 4 Encryption command RBAC availability and admin domain type (Continued) Command name User Admin Operator Switch Zone Fabric Basic Security Admin Domain Admin Admin Admin Switch Admin Admin recovermasterkey Disallowed refreshDEK Disallowed regEE Disallowed reggroupleader Disallowed...
Cryptocfg Help command output Cryptocfg Help command output All encryption operations are done using the cryptocfg command. The cryptocfg command has an help output that lists all options. switch:admin> cryptocfg --help Usage: cryptocfg --help -nodecfg: Display the synopsis of node parameter configuration. --help -groupcfg: Display the synopsis of group parameter configuration.
Configuring cluster links Configuring cluster links Each encryption switch or FS8-18 blade has two GbE ports labeled Ge0 and Ge1. The Ge0 and Ge1 ports connect encryption switches and FS8-18 blades to other encryption switches and FS8-18 blades. These two ports are bonded together as a single virtual network interface. Only one IP address is used.
Configuring cluster links NOTE The IP address of the cluster link should be configured before enabling the encryption engine for encryption. If the IP address is configured after the encryption engine is enabled for encryption, or if the IP address of the cluster link ports is modified after encryption engine is enabled for encryption, the encryption switch needs to be rebooted, and the encryption blade needs to be powered off and powered on (slotpoweroff/slotpoweron) for the IP address configuration to take effect.
Steps for connecting to an SKM appliance Steps for connecting to an SKM appliance The following configuration steps are performed from the SKM management web console, which can be accessed from any web browser with Internet access to the SKM appliance. The URL for the appliance is as follows: https://<appliance hostname>:<appliance port number>...
Page 119
Steps for connecting to an SKM appliance 1. Login to the SKM management web console using the admin password. 2. Select the Security tab. 3. Under Certificates & CAs, click Local CAs. 4. Enter information required by the Create Local Certificate Authority section of the window to create your local CA.
Steps for connecting to an SKM appliance Repeat these steps any time another local CA is needed. Downloading the local CA certificate The local CA certificate you created using the procedure for “Setting up the local Certificate Authority (CA)” on page 100 must be saved to your local system. Later, this certificate must be imported onto the Brocade encryption group leader nodes.
Steps for connecting to an SKM appliance 11. Enter the required data in the Sign Certificate Request section of the window. Select the CA name from the Sign with Certificate Authority drop down box. Select Server as the Certificate Purpose. Enter the number of days before the certificate must be renewed based on your site's security policies.
Steps for connecting to an SKM appliance Creating an SKM High Availability cluster The HP SKM key vault supports clustering of HP SKM appliances for high availability. If two SKM key vaults are configured, they must be clustered. If only a single SKM appliance is configured, it may be clustered for backup purposes, but the backup appliance will not be directly used by the switch.
Steps for connecting to an SKM appliance Adding SKM appliances to the cluster If you are adding an appliance to an existing cluster, select the Cluster Settings section of the window, click Download Cluster Key and save the key to a convenient location, such as your computer's desktop.
Steps for connecting to an SKM appliance Initializing the Brocade encryption engines You must perform a series of encryption engine initialization steps on every Brocade encryption node (switch or blade) that is expected to perform encryption within the fabric. NOTE The initialization process overwrites any authentication data and certificates that reside on the node and the security processor.
Steps for connecting to an SKM appliance ARE YOU SURE (yes, y, no, n): y Operation succeeded. Register the encryption engine by entering the cryptocfg regEE command. Provide a slot number if the encryption engine is a blade. This step registers the encryption engine with the CP or chassis.
Steps for connecting to an SKM appliance Signing the Brocade encryption node KAC certificates The KAC certificate signing request generated when the encryption node is initialized must be exported for each encryption node and signed by the Brocade local CA on SKM. The signed certificate must then be imported back into the encryption node.
Steps for connecting to an SKM appliance Registering SKM on a Brocade encryption group leader An encryption group consists of one or more encryption engines. Encryption groups can provide failover/failback capabilities by organizing encryption engines into Data Encryption Key (DEK) clusters.
Page 128
Steps for connecting to an SKM appliance 6. Use the cryptocfg - - show groupcfg command to verify that the key vault state is Connected. Mace_127:admin> cryptocg --show groupcfg rbash: cryptocg: command not found Mace_127:admin> cryptocfg --show -groupcfg Encryption Group Name: mace127_mace129 Failback mode: Auto...
Generating and backing up the master key Generating and backing up the master key You must generate a master key on the group leader, and export it to a secure backup location so that it can be restored, if necessary. The master key is used to encrypt DEKs for transmission to and from SKM.
Page 130
Generating and backing up the master key State: DEF_NODE_STATE_DISCOVERED Role: MemberNode IP Address: 10.32.244.60 Certificate: enc1_cpcert.pem Current Master Key State: Configured Current Master KeyID: 8f:88:45:32:8e:bf:eb:44:c4:bc:aa:2a:c1:69:94:2 Alternate Master Key State: Not configured Alternate Master KeyID: 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00 EE Slot: SP state: Waiting for enableEE Current Master KeyID: 8f:88:45:32:8e:bf:eb:44:c4:bc:aa:2a:c1:69:94:2 Alternate Master KeyID:...
High Availability (HA) cluster configuration High Availability (HA) cluster configuration An HA cluster consists of two encryption engines configured to host the same CryptoTargets and to provide Active/Standby failover and failback capabilities in a single fabric. Failover is automatic (not configurable). Failback occurs automatically by default, but is configurable with a manual failback option.
High Availability (HA) cluster configuration 3. Enter cryptocfg commit to commit the transaction. Any transaction remains in the defined state until it is committed. The commit operation fails if the HA cluster has less than two members. 4. Display the HA cluster configuration by entering the cryptocfg show -hacluster -all command In the following example, the encryption group brocade has one committed HAC1 with two...
High Availability (HA) cluster configuration Failover/failback policy configuration Failover/failback policy parameters as outlined in Table 5 can be set for the entire encryption group on the group leader. Use the cryptocfg set command with the appropriate parameter to set the values for the policy. Policies are automatically propagated to all member nodes in the encryption group.
Enabling the encryption engine Enabling the encryption engine Enable the encryption engine by entering the cryptocfg enableEE command. Provide a slot number if the encryption engine is a blade. NOTE Every time a Brocade Encryption Switch or DCX or DCX-4S chassis containing one or more FS8-18 blade goes through power cycle event, or after issuing slotpoweroff <slot number>...
Zoning considerations Link IP Addr : 10.32.72.76 Link GW IP Addr : 10.32.64.1 Link Net Mask : 255.255.240.0 Link MAC Addr : 00:05:1e:53:89:03 Link MTU : 1500 Link State : UP Media Type : DISK Rebalance Recommended: NO System Card Label : System Card CID : Remote EE Reachability : Node WWN/Slot EE IP Addr EE State IO Link State...
Zoning considerations Frame redirection zoning Name Server-based frame redirection enables the Brocade encryption switch or blade to be deployed transparently to hosts and targets in the fabric. NS-based frame redirection is enabled as follows: • You first create a zone that includes host (H) and target (T). This may cause temporary traffic disruption to the host.
Page 137
Zoning considerations The nsshow command shows all devices on the switch, and the output can be lengthy. To retrieve only the initiator PWWN, do a pattern search of the output based on the initiator Port ID (a hex number). In the following example, The PID is 010600, where 01 indicates the domain and 06 the port number.
Page 138
Zoning considerations Create a zone that includes the initiator and a LUN target. Enter the zonecreate command followed by a zone name, the initiator PWWN and the target PWWN. FabricAdmin:switch>zonecreate itzone, "10:00:00:00:c9:2b:c9:3a; \ 20:0c:00:06:2b:0f:72:6d" 8. Create a zone configuration that includes the zone you created in step 4. Enter the cfgcreate command followed by a configuration name and the zone member name.
CryptoTarget container configuration CryptoTarget container configuration A CryptoTarget container is a configuration of virtual devices created for each target port hosted on a Brocade Encryption Switch or FS8-18 blade. The container holds the configuration information for a single target, including associated hosts and LUN settings. A CryptoTarget container interfaces between the encryption engine, the external storage devices (targets), and the initiators (hosts) that can access the storage devices through the target ports.
CryptoTarget container configuration LUN re-balancing when hosting both disk and tape If you are currently using encryption and running Fabric OS version 6.3.x or earlier, you are hosting tape and disk target containers on different encryption switches or blades. Beginning with Fabric OS version 6.4, disk and tape target containers can be hosted on the same switch or blade.
CryptoTarget container configuration Creating a CryptoTarget container Before you begin, have the following information ready: • The switch WWNs of all nodes in the encryption group. Use the cryptocfg show -groupmember -all command to gather this information. • The port WWNs of the targets whose LUNs are being enabled for data-at-rest encryption. •...
Page 142
CryptoTarget container configuration CAUTION When configuring a multi-path LUN, you must complete the CryptoTarget container configuration for ALL target ports in sequence and add the hosts that should gain access to these ports before committing the container configuration. Failure to do so results in data corruption. Refer to the section “Configuring a multi-path Crypto LUN”...
CryptoTarget container configuration Removing an initiator from a CryptoTarget container You may remove one or more initiators from a given CryptoTarget container. This operation removes the initiators’ access to the target port. If the initiator has access to multiple targets and you wish to remove access to all targets, follow the procedure described to remove the initiator from every CryptoTarget container that is configured with this initiator.
CryptoTarget container configuration 1. Log into the group leader as Admin or FabricAdmin. 2. Enter the cryptocfg delete -container command followed by the CryptoTarget container name. The following example removes the CryptoTarget container “my_disk_tgt”. FabricAdmin:switch>cryptocfg --delete -container my_disk_tgt Operation Succeeded 3.
Crypto LUN configuration Crypto LUN configuration A Crypto LUN is the LUN of a target disk or tape storage device that is enabled for and capable of data-at-rest encryption. Crypto LUN configuration is done on a per-LUN basis. You configure the LUN for encryption by explicitly adding the LUN to the CryptoTarget container and turning on the encryption property and policies on the LUN.
Crypto LUN configuration CAUTION When configuring a LUN with multiple paths, perform the LUN discovery on each of the Crypto Target containers for each of the paths accessing the LUN and verify that the serial number for these LUNs discovered from these Crypto Target containers are the same. This indicates and validates that these Crypto Target containers are indeed paths to the same LUN.
Crypto LUN configuration Log into the group leader as Admin or FabricAdmin. 3. Enter the cryptocfg add -LUN command followed by the CryptoTarget container Name, the LUN number or a range of LUN numbers, the PWWN and NWWN of the initiators that should be able to access the LUN.
Page 148
Crypto LUN configuration NOTE LUN policies are configured at the LUN-level but apply to the entire HA or DEK cluster. For multi-path LUNs exposed through multiple target ports and thus configured on multiple Crypto Target containers on different encryption engines in an HA cluster or DEK cluster, the same LUN policies must be configured.
Crypto LUN configuration TABLE 6 LUN parameters and policies (Continued) Policy name Command parameters Description Re-key policy -enable_rekey time_period Enables or disables the auto re-keying feature on a specified disk <days>| -disable_rekey LUN. This policy is not valid for tape LUNs. By Default, the Disk LUN: yes automatic re-key feature is disabled.
Crypto LUN configuration 10:00:00:00:c9:2b:c9:3a 20:00:00:00:c9:2b:c9:3a -encrypt Operation Succeeded NOTE When changing the tape LUN policy from encrypt to cleartext or from cleartext to encrypt, or the encryption format from Brocade native to DF-compatible while data is being written to or read from a tape backup device, the policy change is not enforced until the current process completes and the tape is unmounted, rewound, or overwritten.
Crypto LUN configuration Removing a LUN from a CryptoTarget container You can remove a LUN from a given CryptoTarget container if it is no longer needed. Stop all traffic I/O from the initiators accessing the LUN before removing the LUN to avoid I/O failure between the initiators and the LUN.
Crypto LUN configuration Modifying Crypto LUN parameters You can modify one or more policies of an existing Crypto LUN with the cryptocfg modify -LUN command. If the modification applies to multiple LUNs, you may specify a LUN number range. NOTE A maximum of 25 LUNs can be added or modified in a single commit operation.
Crypto LUN configuration Impact of tape LUN configuration changes LUN-level policies apply when no policies are configured at the tape pool level. The following restrictions apply when modifying tape LUN configuration parameters: • If you change a tape LUN policy from encrypt to cleartext or from cleartext to encrypt, or if you change the encryption format from Brocade native to DF-compatible while data is written to or read from a tape backup device, the policy change is not enforced until the current process completes and the tape is unmounted, rewound, or overwritten.
Page 154
Crypto LUN configuration When a device decommission operation fails on the encryption group leader for any reason, the crypto configuration remains uncommitted until a user-initiated commit or a subsequent device decommission operation issued on the encryption group leader completes successfully. Device decommission operations should always be issued from a committed configuration.
Tape pool configuration Tape pool configuration Tape pools are used by tape backup application programs to group all configured tape volumes into a single backup to facilitate their management within a centralized backup plan. A tape pool is identified by either a name or a number, depending on the backup application. Tape pools have the following properties: •...
Page 156
Tape pool configuration CommVault Galaxy labeling CommVault uses a storage policy for each backup. When configuring a tape pool to work with CommVault Galaxy, first create a storage policy on CommVault and then use the storage_policy_id (sp_id) as the label when creating the tape pool on the encryption switch or blade. 1.
Tape pool configuration Creating a tape pool Take the following steps to create a tape pool: 1. Log into the group leader as FabricAdmin. 2. Create a tape pool by entering the cryptocfg create -tapepool command. Provide a label or numeric ID for the tape pool and specify the encryption policies.
Tape pool configuration Deleting a tape pool This command does not issue a warning if the tape pool being deleted has tape media or volumes that are currently accessed by the host. Be sure the tape media is not currently in use. 1.
Configuring a multi-path Crypto LUN Configuring a multi-path Crypto LUN A single LUN may be accessed over multiple paths. A multi-path LUN is exposed and configured on multiple CryptoTarget Containers located on the same encryption switch or blade or on different encryption switches or blades.
Page 160
Configuring a multi-path Crypto LUN FIGURE 57 A LUN accessible through multiple paths The following steps may be used to configure multiple path access to the LUN in Figure 1. Create zoning between host port 1 and target port 1. Refer to the section “Creating an initiator - target zone”...
Page 161
Configuring a multi-path Crypto LUN Add host port 1 to the container CTC1. FabricAdmin:switch>cryptocfg --add -initiator <CTC1> <Host Port1 WWN> \ <Host NWWN> d. Add host port 2 to the container CTC2. FabricAdmin:switch>cryptocfg --add -initiator <CTC2> <Host Port2 WWN> <Host NWWN> e.
Page 162
Configuring a multi-path Crypto LUN 6. Validate the LUN policies for all containers. Display the LUN configuration for ALL CryptoTarget containers to confirm that the LUN policy settings are the same for all CryptoTarget containers. FabricAdmin:switch>cryptocfg --show -LUN CTC1 0 <Host Port1 WWN> -cfg FabricAdmin:switch>cryptocfg --show -LUN CTC2 0 <Host Port2 WWN>...
First time encryption First time encryption First time encryption, also referred to as encryption of existing data, is similar to the re-keying process described in the previous section, except that there is no expired key and the data present in the LUN is cleartext to begin with. In a first time encryption operation, cleartext data is read from a LUN, encrypted with the current key and written back to the same LUN at the same logical block address (LBA) location.
Data re-keying Data re-keying In a re-keying operation, encrypted data on a LUN is decrypted with the current key, re-encrypted with a new key and written back to the same LUN at the same logical block address (LBA) location. This process effectively re-encrypts the LUN and is referred to as “in-place re-keying.” It is recommended you limit the practice of re-keying to the following situations: •...
Data re-keying Configuring a LUN for automatic re-keying Re-keying options are configured at the LUN level either during LUN configuration with the cryptocfg add -LUN command, or at a later time with the cryptocfg modify -LUN command. For re-keying of a disk array LUN, the Crypto LUN is configured in the following way: •...
Data re-keying Initiating a manual re-key session If auto re-keying is disabled, you can initiate a re-keying session manually at your own convenience. All encryption engines in a given HA cluster, DEK cluster, or encryption group must be online for this operation to succeed.
Data re-keying Suspension and resumption of re-keying operations A re-key may be suspended or fail to start for several reasons: • The LUN goes offline or the encryption switch fails and reboots. Re-key operations are resumed automatically when the target comes back online or the switch comes back up. You cannot abort an in-progress re-key operation.
Page 168
Data re-keying Fabric OS Encryption Administrator’s Guide 53-1001864-01...
Chapter Deployment Scenarios In this chapter • Single encryption switch, two paths from host to target....152 • Single fabric deployment - HA cluster ......153 •...
Single encryption switch, two paths from host to target Single encryption switch, two paths from host to target Figure 58 shows a basic configuration with a single encryption switch providing encryption between one host and one storage device over two the following two paths: •...
Single fabric deployment - HA cluster Single fabric deployment - HA cluster Figure 59 shows an encryption deployment in a single fabric with dual core directors and several host and target edge switches in a highly redundant core-edge topology. Key Management Management Management Appliance...
Single fabric deployment - DEK cluster Figure 59, the two encryption switches provide a redundant encryption path to the target devices. The encryption switches are interconnected through a dedicated cluster LAN. The Ge1 and Ge0 gigabit Ethernet ports on each of these switches are attached to this LAN. This LAN connection provides the communication needed to distribute and synchronize configuration information, and enable the two switches to act as a high availability (HA) cluster, providing automatic failover if one of the switches fails, or is taken out of service.
Dual fabric deployment - HA and DEK cluster Figure 60, two encryption switches are required, one for each target path. The path from host port 1 to target port 1 is defined in a CryptoTarget container on one encryption switch, and the path from host port 2 to target port 2 is defined in a CryptoTarget container on the other encryption switch.
Multiple paths, one DEK cluster, and two HA clusters failover for the encryption path between the host and target in fabric 1. Encryption switches 2 and 4 act as a high availability cluster in fabric 2, providing automatic failover for the encryption path between the host and target in fabric 2.
Page 175
Multiple paths, one DEK cluster, and two HA clusters The configuration details shown in Figure 62 are as follows: • There are two fabrics. • There are four paths to the target device, two paths in each fabric. • There are two host ports, one in each fabric. •...
Multiple paths, DEK cluster, no HA cluster Multiple paths, DEK cluster, no HA cluster Figure 63 shows a configuration with a DEK cluster with multiple paths to the same target device. There is one encryption switch in each fabric. Management Network Management Link Management Link...
Deployment in Fibre Channel routed fabrics Deployment in Fibre Channel routed fabrics In this deployment, the encryption switch may be connected as part of the backbone fabric to another switch or blade that provides the EX_port connections (Figure 64), or it may form the backbone fabric and directly provide the EX_port connections (Figure 65).
Page 178
Deployment in Fibre Channel routed fabrics The following is a summary of steps for creating and enabling the frame redirection zoning features in the FCR configuration (backbone to edge). • The encryption device creates the frame redirection zone automatically consisting of host, target, virtual target, and virtual initiator in the backbone fabric when the target and host are configured on the encryption device.
Deployment as part of an edge fabric Deployment as part of an edge fabric In this deployment, the encryption switch is connected to either the host or target edge fabric. The backbone fabric may contain a 7500 extension switch or FR4-18i blade in a 48000 director, DCX, or DCX-4S, or an FCR-capable switch or blade.
Deployment with FCIP extension switches Deployment with FCIP extension switches Encryption switches may be deployed in configurations that use extension switches or extension blades within a DCX, DCX-4S or 48000 chassis to enable long distance connections. Figure 67 shows an encryption switch deployment in a Fibre Channel over IP (FCIP) configuration. Refer to the Fabric OS Administrator’s Guide for information about creating FCIP configurations.
VmWare ESX server deployments VmWare ESX server deployments VM ESX servers may host multiple guest operating systems. A guest operating system may have its own physical HBA port connection, or it may use a virtual port and share a physical HBA port with other guest operating systems.
Page 182
VmWare ESX server deployments Figure shows a VmWare ESX server with two guest operating systems where two guests access a fabric over a shared port. To enable this, both guests are assigned a virtual port. There are two paths to a target storage device: •...
Firmware download considerations Firmware download considerations The encryption engine and the control processor or blade processor are reset after a firmware upgrade. Disruption of encryption I/O can be avoided if an HA cluster is configured. If encryption engines are configured in an HA cluster, perform firmware upgrades one encryption engine at a time so that the partner switch in the HA cluster can take over I/O by failover during firmware upgrade.
Firmware download considerations • A firmware consistency check for Fabric OS version 6.4.0 is enforced in an encryption group if any of the Fabric OS version 6.4.0 features are enabled. If any Fabric OS version 6.4.0 feature is in an enabled state then any firmware download to Fabric OS version 6.3.x or earlier versions is blocked.
Configuration upload and download considerations 10. After all nodes in the Encryption Group have been upgraded, change back the failback mode to auto from manual, if required by issuing the following command. cryptocfg --set -failback auto Configuration upload and download considerations Important information is not included when you upload a configuration from an encryption switch or blade.
Configuration upload and download considerations Steps before configuration download The configuration download does not have any certificates, public or private keys, master key, or link keys included. Perform following steps prior to configuration download to generate and obtain the necessary certificates and keys: 1.
Configuration upload and download considerations Steps after configuration download For all key vaults except LKM, restore or generate and backup the master key. In cluster environments, the master key is propagated from group leader node. 1. Use the following command to enable the encryption engine. cryptocfg --enableEE [slot num] 2.
HP-UX considerations HP-UX considerations The HP-UX OS requires LUN 0 to be present. LUNs are scanned differently based on the type value returned for LUN 0 by the target device. • If the type is 0, then HP-UX only scans LUNs from 0 to 7. That is the maximum limit allowed by HP-UX for device type for type 0.
Tape data compression Tape data compression Data is compressed by the encryption switch or blade before encrypting only if the tape device supports compression, and compression is explicitly enabled by the host backup application. That means if the tape device supports compression, but is not enabled by the host backup application, then compression is not performed by the encryption switch or blade before encrypting the data.
Tape key expiry Tape key expiry When the tape key expires in the middle of a write operation on the tape, the key is used for the duration of any write operation to append the data on the tape media. On any given tape medium, the same key is used for all written blocks, regardless of the time in between append operations.
Configuring CryptoTarget containers and LUNs Configuring CryptoTarget containers and LUNs The following are best practices to follow when configuring CryptoTarget containers and crypto LUNs: • Host a target port on only one encryption switch, or one HA cluster. All LUNs visible through the target port are hosted on the same encryption switch, and are available for storing cipher text.
Redirection zones Redirection zones Redirection zones should not be deleted. If a redirection zone is accidentally deleted, I/O traffic cannot be redirected to encryption devices, and encryption is disrupted. To recover, re-enable the existing device configuration by invoking the cryptocfg commit command.
Tape library media changer considerations Tape library media changer considerations In tape libraries where the media changer unit is addressed by a target port that is separate from the actual tape SCSI I/O ports, create a CryptoTarget container for the media changer unit and CryptoTarget containers for the SCSI I/O ports.
Re-keying best practices and policies Re-keying best practices and policies Re-keying should be done only when necessary. In key management systems, DEKs are never exposed in an unwrapped or unencrypted state. You must re-key if the master key is compromised. The practice of re-keying should be limited to the following cases: •...
Changing IP addresses in encryption groups Do not change LUN configuration while re-keying Never change the configuration of any LUN that belongs to a Crypto Target Container/LUN configuration while the re-keying process for that LUN is active. If you change the LUN’s settings during manual or auto, re-keying or first time encryption, the system reports a warning message stating that the encryption engine is busy and a forced commit is required for the changes to take effect.
Recommendations for Initiator Fan-Ins Recommendations for Initiator Fan-Ins For optimal performance at reasonable scaling factors of initiators, targets, and LUNs accessed, Brocade Encryption Engines (EEs) are designed to support a fan-In ratio of between four and eight initiator ports to one target port, in terms of the number of distinct initiator ports to a Crypto Container (i.e., a virtual target port corresponding to the physical target port).
Best practices for host clusters in an encryption environment Best practices for host clusters in an encryption environment When host clusters are deployed in a encryption environment, please follow these recommendations: • If two encryption engines are part of an HA cluster, configure the host/target pair so they have different paths from both encryption engines.
Chapter Maintenance and Troubleshooting In this Chapter • Encryption group and HA cluster maintenance ..... . 181 • Troubleshooting examples using the CLI ......198 •...
Page 200
Encryption group and HA cluster maintenance SecurityAdmin:switch>cryptocfg --show -groupmember \ 10:00:00:05:1e:41:99:bc Node Name: 10:00:00:05:1e:41:99:bc (current node) State: DEF_NODE_STATE_DISCOVERED Role: MemberNode IP Address: 10.32.33.145 Certificate: 10.32.33.145_my_cp_cert.pem Current Master Key State: Saved Current Master KeyID: b8:2a:a2:4f:c8:fd:12:e2:a9:25:d9:5b:58:2c:96:7e Alternate Master Key State: Not configured Alternate Master KeyID: 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00 EE Slot:...
Encryption group and HA cluster maintenance FIGURE 71 Removing a node from an encryption group Deleting an encryption group You can delete an encryption group after removing all member nodes following the procedures described in the previous section. The encryption group is deleted on the group leader after you have removed all member nodes.
Encryption group and HA cluster maintenance Removing an HA cluster member Removing an encryption engine from an HA cluster “breaks” the HA cluster by removing the failover/failback capability for the removed encryption engines, However, the removal of an encryption engine does not affect the relationship between configured containers and the encryption engine that is removed from the HA cluster.
Encryption group and HA cluster maintenance Replacing an HA cluster member 1. Log into the group leader as Admin or SecurityAdmin. 2. Enter the cryptocfg replace -haclustermember command. Specify the HA cluster name, the node WWN of the encryption engine to be replaced, and the node WWN of the replacement encryption engine.
Page 204
Encryption group and HA cluster maintenance FIGURE 72 Replacing a failed encryption engine in an HA cluster Fabric OS Encryption Administrator’s Guide 53-1001864-01...
Page 205
Encryption group and HA cluster maintenance Case 2: Replacing a “live” encryption engine in an HA cluster 1. Invoke the cryptocfg replace -haclustermember command on the group leader to replace the live encryption engine EE2 with another encryption engine (EE3). This operation effectively removes EE2 from the HA cluster and adds the replacement encryption engine (EE3) to the HA cluster.
Encryption group and HA cluster maintenance Deleting an HA cluster member This command dissolves the HA cluster and removes failover capability from the participating encryption engines. 1. Log into the group leader as Admin or SecurityAdmin. 2. Enter the cryptocfg delete -hacluster command.
Encryption group and HA cluster maintenance • The failed EE2 has come back online, Failover is still active: SecurityAdmin:switch>cryptocfg --show -hacluster -all Encryption Group Name: brocade Number of HA Clusters: 1 HA cluster name: HAC3 - 2 EE entries Status: Committed Slot Number Status...
Page 208
Encryption group and HA cluster maintenance Recovery 1. Configure the IP address 0f the new node that is replacing the failed node, and the IP addresses of the I/O cluster sync ports (Ge0 and Ge1), and initialize the node with the cryptocfg initnode command.
Page 209
Encryption group and HA cluster maintenance A member node reboots and comes back up Assumptions N1, N2 and N3 form an encryption group and N2 is the group leader node. N3 and N1 are part of an HA cluster. Assume that N3 reboots and comes back up. Impact When N3 reboots, all devices hosted on the encryption engines of this node automatically fail over to the peer encryption engine N1, and N1 now performs all of the rebooted node’s encryption...
Page 210
Encryption group and HA cluster maintenance A member node lost connection to all other nodes in the encryption group Assumptions N1, N2 and N3 form an encryption group and N2 is the group leader node. N3 and N1 are part of an HA cluster.
Page 211
Encryption group and HA cluster maintenance • Each encryption group registers the missing members as “offline”. • The isolation of N3 from the original encryption group breaks the HA cluster and failover capability between N3 and N1. • You cannot configure any CryptoTargets, LUN policies, tape pools, or security parameters on any of the group leaders.
Encryption group and HA cluster maintenance Configuration impact of encryption group split or node isolation When a node is isolated from the encryption group or the encryption group is split to form separate encryption group islands, the defined or registered node list in the encryption group is not equal to the current active node list, and the encryption group is in a DEGRADED state rather than in a CONVERGED state.
General encryption troubleshooting I General encryption troubleshooting I Table 9 lists the commands you can use to check the health of your encryption setup. Table 10 provides additional information for failures you might encounter while configuring switches using the CLI. TABLE 9 General troubleshooting tips using the CLI Command...
Page 214
General encryption troubleshooting I TABLE 10 General errors and conditions Problem Resolution LUN state for some LUNS remains in "initialize" state on the This is expected behavior. The LUNs exposed through Passive paths of the passive path. target array will be in either Initialize or LUN Discovery Complete state so long as the paths remain n passive condition.
Page 215
General encryption troubleshooting I TABLE 10 General errors and conditions Problem Resolution Searching or viewing key IDs on an LKM server returns “Not In cases where there are many keys to search through (e.g., one MB or more) Responding” or “Unknown/Busy” for 20 or more minutes LKM database searches may get queued and may not get serviced fast while searching for decommissioned key records.
Troubleshooting examples using the CLI Troubleshooting examples using the CLI Encryption Enabled Crypto Target LUN The LUN state should be Encryption enabled for the host to see the Crypto LUN. switch:FabricAdmin> cryptocfg --show -LUN kmfvt 0 21:01:00:e0:8b:a9:ac:d2 -stat Container name: lkmfvt Type: disk...
Troubleshooting examples using the CLI Encryption Disabled Crypto Target LUN If the LUN state is Encryption Disabled the host will not be able to access the Crypto LUN. > switch:FabricAdmin> cryptocfg --show -LUN kmfvt 0 21:01:00:e0:8b:a9:ac:d2 -stat Container name: lkmfvt Type: disk EE node:...
Management application encryption wizard troubleshooting Management application encryption wizard troubleshooting • Errors related to adding a switch to an existing group ....200 • Errors related to adding a switch to a new group ....201 •...
Errors related to adding a switch to an existing group Errors related to adding a switch to a new group Table 12 lists configuration task errors you might encounter while adding a switch to a new group, and describes how to troubleshoot them. TABLE 12 Error recovery instructions for adding a switch to a new group Configuration task...
Page 220
Errors related to adding a switch to an existing group TABLE 12 Error recovery instructions for adding a switch to a new group (Continued) Configuration task Error description Instructions Create a new master key (if the key A failure occurred while attempting to Remove the switch from the group using the Group vault type is not NetApp) create a new master key.
Errors related to adding a switch to an existing group General errors related to the Configure Switch Encryption wizard Table 13 provides additional information for failures you might encounter while configuring switches using the Configure Switch Encryption wizard. TABLE 13 General errors related to the Configure Switch Encryption wizard Problem Resolution...
LUN policy troubleshooting LUN policy troubleshooting Table 14 may be used as an aid in troubleshooting problems related to LUN policies. TABLE 14 LUN policy troubleshooting Case Reasons for the LUN getting disabled by Action taken If you do not need to save the data: If you need to save the data: the encryption switch The LUN was modified from encrypt...
Loss of encryption group leader after power outage Loss of encryption group leader after power outage When all nodes in an encryption group, HA Cluster, or DEK Cluster are powered down due to catastrophic disaster or power outage to whole data center, and the group leader node either fails to come back up when the other nodes are powered on, or the group leader is kept powered down, the member nodes lose information and knowledge about the encryption group.
MPIO and internal LUN states MPIO and internal LUN states The Internal LUN State field displayed within the cryptocfg -show -LUN command output does not indicate the host-to-storage path status for the displayed LUN, but rather the internal LUN state as known by the given encryption engine.
Page 225
MPIO and internal LUN states FabricAdmin:switch> cryptocfg --show -rekey -all • Read all data off the LUN and write it to another LUN. In this case, you can cancel the re-key session by removing the LUN from its container and force committing the transaction. Refer to the section “Removing a LUN from a CryptoTarget container”...
Page 226
MPIO and internal LUN states Fabric OS Encryption Administrator’s Guide 53-1001864-01...
Appendix State and Status Information In this appendix • Encryption engine security processor (SP) states ....209 • Security processor KEK status ........210 •...
Security processor KEK status Security processor KEK status Table 16 lists security processor KEK status information. TABLE 16 Security processor KEK status KEK type KEK status Description Primary KEK (current MK or None Primary KEK is not configured. primary KV link key) Mismatch Primary KEK mismatch between the CP and the SP.
Page 229
Encrypted LUN states TABLE 17 Encrypted LUN states (Continued) LUN_1ST_TIME_REKEY_IN_PROG First time re-key is in progress. LUN_KEY_EXPR_REKEY_IN_PROG Key expired re-key is in progress. LUN_MANUAL_REKEY_IN_PROG Manual re-key is in progress. LUN_DECRYPT_IN_PROG Data decryption is in progress. LUN_WR_META_PENDING Write metadata is pending. LUN_1ST_TIME_REKEY_PENDING First time re-key is pending.
Page 230
Encrypted LUN states TABLE 17 Encrypted LUN states (Continued) LUN_DIS_WR_META_DONE_ERR Disabled (Write metadata done with failure). LUN_DIS_LUN_REMOVED Disabled (LUN re-discovery detects LUN is removed). LUN_DIS_LSN_MISMATCH Disabled (LUN re-discovery detects new device ID). LUN_DIS_DUP_LSN Disabled (Duplicate LUN SN found). LUN_DIS_DISCOVERY_FAIL Disabled (LUN discovery failure). LUN_DIS_NO_LICENSE Disabled (Third party license is required).
Page 231
Encrypted LUN states TABLE 18 Tape LUN states Internal Names Console String Explanation LUN_DIS_LUN_NOT_FOUND Disabled (LUN not found) No logical unit structure in tape module. This is an internal software error. If it occurs, contact Brocade support. LUN_TGT_OFFLINE Target Offline Target port is not currently in the fabric.
Page 232
Encrypted LUN states TABLE 18 Tape LUN states LUN_ENCRYPT Encryption enabled The tape medium is present, and is in ciphertext (encrypted). The encryption switch or blade has full read/write access, because its current tape policy for the medium is also encrypted.
Appendix LUN Policies In this appendix The following topics are covered in this appendix: • DF-compatibility support for disk LUNs ......215 •...
Page 234
DF-compatibility support for disk LUNs TABLE 20 Support matrix for disk LUNs for various configuration and modify options LUN state LUN policy Encrypt existing data Key ID Metadata Results encryption on LUN format Native Encrypted Encrypt NA when No error. If the LUN was previously (Brocade) LUN State = encrypt DF-encrypted, the LUN is set to Read Only until...
Page 235
DF-compatibility support for disk LUNs TABLE 20 Support matrix for disk LUNs for various configuration and modify options (Continued) LUN state LUN policy Encrypt existing data Key ID Metadata Results encryption on LUN format Native Cleartext Cleartext NA in case of cleartext The LUN is disabled for encryption.
Page 236
DF-compatibility support for disk LUNs TABLE 20 Support matrix for disk LUNs for various configuration and modify options (Continued) LUN state LUN policy Encrypt existing data Key ID Metadata Results encryption on LUN format Cleartext Encrypt The LUN is disabled for encryption. Metadata compatible is present on the LUN and the LUN is in encrypted state.
DF-compatibility support for tape LUNs DF-compatibility support for tape LUNs Table 21 Table 22 may be used as a reference for establishing tape LUN policies in support of DataFort firmware versions. NOTE On tapes written in DataFort format, the encryption switch or blade cannot read and decrypt files with a block size of one MB or greater.
Page 238
DF-compatibility support for tape LUNs TABLE 22 Compatibility support matrix for tape pools (Continued) Tape pool encryption format Tape pool policy Metadata present Results DF-compatible Encrypt No (new tape) No error. A new key is generated and both read and write are allowed in DF-compatible format.
Page 239
Appendix NS-Based Transparent Frame Redirection Table 23 provides the NS-based transparent frame redirection interoperability matrix. TABLE 23 NS-based transparent frame redirection interoperability matrix Frame FOS version Host and target edge switches/directors redirection support FOS only FOS and EOSc and EOSn FOS and EOSc and EOSn EOSc and EOSn only interop mode 2 “native...
Page 241
Index configuring target ports container adding a LUN to CryptoTarget using the CLI add commands creating a CryptoTarget --add -haclustermember deleting a CryptoTarget using the CLI --add -initiator discovering a Crypto LUN using the CLI --add -LUN moving a CryptoTarget using the CLI --add -membernode removing a LUN to CryptoTarget using the CLI removing an initiator using the CLI...
Page 242
cryptocfg command CryptoTarget container --add -haclustermember adding a LUN --add -initiator configuring --add -LUN creating --add -membernode deleting --commit discovering a LUN --create -container moving --create -encgroup removing a LUN --create -hacluster removing an initiator from --create -tapepool --delete -container --delete -encgroup --delete -hacluster --delete -tapepool...
Page 243
enable commands encryption group --enable -LUN adding a switch using the management application --enable -rekey advanced configuration --enable_rekey allowed configuration changes --enableEE basic configuration enableEE configuration impact of split or node isolation encrypted LUN states confirming configuration status creating using the CLI encryption creating using the encryption setup wizard adding a license...
Page 244
failback command, --failback -EE import commands, --import failover and failback, states of encryption engines during initialize commands --initEE initEE field replaceable unit --initnode See FRU initializing firmware download considerations encryption switch using the CLI frame redirection initiators, removing from CryptoTarget container creating and enabling in an FCR configuration (edge to initiator-target zone, creating edge)
Page 245
configuring for first-time encryption configuring for multi-path example NetBackup labeling configuring policies using the CLI NetWorker labeling force-enabling for encryption NS-based transparent frame redirection interop matrix impact of policy changes modifying parameters using the CLI multi-path configuration requirements policy for DF-compatibility disk LUNs policy for DF-compatibility tape LUNs policy for DF-compatibility tape pools policy parameters...
Page 246
resume commands --resume_rekey RKM key management system tape compression role based access control (RBAC) permissions for tape library media changer considerations cryptoCfg commands tape LUN, configuring tape metadata tape pool impact of policy changes tape pools security processor (SP) adding KEK status CommVault Galaxy labeling using the CLI states for encryption engines...
Page 247
validating commands verifying encryption engine status using the CLI virtual initiators, description of in an encryption configuration virtual targets, description of in an encryption configuration zeroize command --zeroize zeroizing effects of using on encryption engine zone creating an initiator-target using the CLI Fabric OS Encryption Administrator’s Guide 53-1001864-01...
Page 248
Fabric OS Encryption Administrator’s Guide 53-1001864-01...