Page 2
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE, BOTH OF WHICH ARE EXPRESSLY DISCLAIMED. In no event shall Spectra Logic be liable for any loss of profits, loss of business, loss of use or data, interruption of business, or for indirect, special, incidental or consequential damages of any kind, even if Spectra Logic has been advised of the possibility of such damages arising from any defect or error.
Contacting Spectra Logic Contacting Spectra Logic To obtain general information Spectra Logic web Site: www.spectralogic.com United States Headquarters European Office Spectra Logic Corporation Spectra Logic Europe Ltd. 1700 North 55th Street Magdalen Centre Boulder, CO 80301 Robert Robinson Avenue Oxford Science Park...
File Migrator software. For these documents and additional information about the Spectra nTier Intelligent Storage appliances and libraries, check the Spectra Logic web site at www.spectralogic.com/documentation. Spectra nTier Intelligent Secondary Storage Appliances The Spectra nTier500 Intelligent Secondary Storage Appliance User Guide provides detailed information about installing, configuring, and using the Spectra nTier500.
About This Guide Conventions Used in This Guide This manual uses the following conventions to highlight important information: Note: Provides additional information or suggestions about the current topic. Caution: Provides information you must know to avoid damage to the library or tape drives or avoid losing data. Warning: Provides information you must know to avoid personal injury.
Overview Introduction ® ™ The Spectra nTier File Migrator software manages data tapes and magnetic disk on a Windows file server (a Spectra nTier Intelligent Secondary Storage appliance) to create a highly scalable digital archive solution. The solution can be configured with a wide range of tape libraries, stand-alone tape drives and magnetic storage.
Page 12
Chapter 1 – Overview The nTier File Migrator software is hosted on an Windows server (for example, the Spectra nTier700) running Microsoft Storage Server 2003 R2. The Windows server is connected to a tape library or stand-alone tape drives and is always configured with a dedicated logical drive that is managed by the nTier File Migrator software.
Page 13
Introduction The online retention period is set by policy and can range from permanent retention on magnetic disk to flushing from disk as soon as the last tape replica has been successfully written. A typical policy is illustrated in Figure 1-2. Figure 1-2 A file storage policy example.
Chapter 1 – Overview The software presents the magnetic disk storage and tape library as a standard rewritable logical drive, even when unalterable WORM tape is being used within the library. The ability to delete and overwrite files is required by many standard applications.
Page 15
Software Feature Overview Multiple Tape Set Support—The software allows file groups to be allocated to specified groups of tapes. Benefit: The Administrator can group related files together on the same set of tapes. Cartridge Spanning—The system supports storage of large files across multiple tape cartridges.
Chapter 1 – Overview Benefit: These popular industry formats provide high capacities, high data transfer rates and estimated data lifetimes up to 30 years without need to rewrite data or rewind tapes. Highly Scalable—The nTier File Migrator software supports unlimited RAID capacity and tape libraries with capacities from less than 1 TB to over 1PB.
Page 17
Installations Using a Tape Library When configuring a system using a tape library, consider the following guidelines to ensure that the library includes the appropriate number of tape drives. With only one tape drive within the library, it is important to be careful in setting the file management policies to prevent the single drive from becoming a bottleneck and in turn degrading performance.
Chapter 1 – Overview Installations using Stand-alone Drives The nTier File Migrator software can be configured and licensed to manage one or more stand-alone tape drives connected to the Windows server. Figure 1-4 shows a Windows server (the nTier500) connected to two tape drives. Even though there are two tape drives in this system, all archived files appear on one logical drive on the Windows server, independent of whether those files are on a cartridge loaded in one of the tape drives or on a cartridge stored offline.
Windows Offline Attribute 1. The tape is moved to the beginning of the file (or a specified portion of the file). Note: When reading a file that is only available on an offline tape, the system generates a Windows event log entry that identifies the tape cartridge on which the file is store.
Chapter 1 – Overview The state of the offline file attribute for the three tiers of HSM is presented in the following table. HSM Tier Description Offline Attribute Online One instance of the file is on magnetic disk and, in addition, Not set there may be one or more instances on tape.
Page 21
Determining Disk Space Requirements Example calculation for an installation with only stand-alone tape drives: Up to a 250,000 files: 2 GB required for metadata Maximum file size of 1 GB: 20 GB allowed for caching Immediate file deletion and no replication: zero allowance is required for online retention A total dedicated logical drive capacity of 22 GB or higher is estimated for this installation.
Concepts Introduction Using the nTier File Migrator software, the system administrator defines policies that allow: Groups of files (termed File Groups) to be defined based on the file name and folder (that is, location in the directory structure) Groups of files to be stored on defined sets of tape cartridges (termed Volume Sets) Groups of files to be retained online on magnetic disk for predefined periods Automatic generation of tape replicas for off-site retention Fast partial restore of very large files and tape cartridge spanning...
Chapter 2 – Concepts File Group 3 contains all other files saved to the folder \project2\. These are retained online on magnetic disk for 24 hours after they were written or last read. They are also saved to tape to another Volume Set called “Tapes02.” A file cannot be written to the system if a File Group is not present that matches its file name and path.
Page 25
File Groups The system maintains a list of File Groups, each of which has a File name or path pattern. To establish which File Group to use for a particular file, the system starts at the top of the list of File Groups and tries to match the file name to the pattern for the File Group.
Chapter 2 – Concepts Examples of Allocating Files to File Groups The easiest way to illustrate how files are allocated to File Groups is by way of examples, shown below. In each case, files are allocated to three different File Groups. Example 1 One File Group contains all files with names ending in .tif;...
Page 27
File Groups Example 3 This is similar to example 2, but includes all sub-folders for the first two File Groups. One File Group contains all files written to a folder at the root called \project01\ and its sub-folders; another contains all files written to a folder at the root called \project02\ and its sub-folders;...
Chapter 2 – Concepts Example 6 This example illustrates the importance of File Group order. File Group File name or path pattern Exclude pattern File Group 1 \project01\* File Group 2 *.txt File Group 3 In this example, files ending in .txt in folder project01\ are allocated to the same File Group as the other files in this folder.
Tapes: Volumes, Volume Sets, and Automatic Replication Another unique Volume Set, termed the quarantined media set, contains all tapes that have been imported into the library but are unrecognized by the system. Typically, they are not recognized because a cartridge has previously been used by a different application (such as a backup application) or because an error occurred while the system was trying to identify the contents of the cartridge.
Chapter 2 – Concepts For automatic operation using a tape library, make sure sufficient tape cartridges of the correct type are maintained in the blank media set. When the Volumes within a Volume Set become full, the system automatically attempts to take the appropriate number of tapes from the blank media set to extend the Volume Set.
Partial File Restore and Cartridge Spanning Partial File Restore and Cartridge Spanning The ability to quickly restore a portion of a file from tape is often useful. For example, in professional video, administrators may want to retrieve a short video clip from a very large multi-gigabyte file.
Chapter 2 – Concepts Off-line File Management The nTier File Migrator software can be configured to provide three tiers of storage hierarchy: Online—One instance of the file on magnetic disk and typically one or more instances on tape. In this case the file is retrieved from disk when accessed. Nearline—No instances of the file are stored on disk.
File Security File Security The nTier File Migrator software integrates fully with the Microsoft Windows security model, based on Active Directory. Files and directories have user-definable security attributes just as they do with standard Microsoft file systems and access control checks are performed in the same way.
Installing nTier File Migrator Before You Begin Before installing and configuring the nTier File Migrator software: Complete the installation and initial configuration of your nTier Intelligent Secondary Storage appliance as described in the User Guide that accompanied it. Make sure you can access the internet—preferably with a direct connection to the nTier.
Page 36
Chapter 3 – Installing nTier File Migrator 2. Enter the IP address of your nTier in the browser address bar using the form https://nnn.nnn.nnn.nnn, where nnn.nnn.nnn.nnn is the IP address. Note: The BlueScale web interface uses a secure internet connection. You must prefix the IP address with https:// instead of just entering the IP address.
Prepare the nTier Windows Server Prepare the nTier Windows Server The Windows server in your nTier Intelligent Secondary Storage appliance (also referred to as the nTier) must be configured before you install and use the nTier File Migrator software. Configure the Archive Drive The nTier File Migrator software requires a dedicated logical drive on the Windows server for storing the file system archive metadata.
Chapter 3 – Installing nTier File Migrator Disable Microsoft Removable Storage Service Use the following steps to disable the Microsoft Removable Storage service. 1. From the nTier Windows Desktop, right-click on the My Computer icon under the Start menu, then select Manage. 2.
Page 39
Prepare the nTier Windows Server Removing NFS 1. From the nTier Windows Desktop, select Start > Settings > Control Panel > Add/ Remove Programs. 2. On the left side of the Add or Remove Programs window, select Add/Remove Windows Components. 3.
Chapter 3 – Installing nTier File Migrator Enable Internet Information Services (ISS) Internet Information Services (ISS) must be enabled before you install the nTier File Migrator software. 1. From the nTier Windows Desktop, select Start > Administrative Tools > Manage Your Server.
Install nTier File Migrator Install nTier File Migrator Use the following steps to install the nTier File Migrator software. 1. Ensure that the tape library or stand-alone drive is powered on and connected to the nTier. Refer to the User Guide that accompanied your nTier for information about connecting a library or stand-alone tape drive to the nTier.
Page 42
Chapter 3 – Installing nTier File Migrator Installing the Tape Drive Device Drivers Device drivers for all of the tape drives used in Spectra Logic libraries are included on the nTier Windows server. 1. From the nTier Windows Desktop, use Windows Explorer to locate the folder...
Install and Activate the Tape Drives 11. When the driver is installed, the Properties screen redisplays showing the new Spectra Logic T-Series Medium Changer (Figure 3-3). Figure 3-3 The Spectra Logic T-Series Medium Changer Properties screen. Enable Stand-Alone Drives If the nTier is connected to one or more stand-alone tape drives, use the Hardware Configuration Utility to enable them for use with nTier File Migrator.
Chapter 3 – Installing nTier File Migrator Activate the nTier File Migrator License Before you can begin configuring nTier File Migrator, you must activate the software. Activate the Archive Drive 1. From the nTier Windows Desktop, use Windows Explorer to locate the License Administration Utility located at: C:\Program Files\nTier File Migrator\Archive\Drives.msc 2.
Complete the Installation Activating Without Internet Access If Internet access is not available for the nTier but is available on another computer, use the following steps to activate the license. 1. Enter the Activation code you received with the nTier File Migrator software in the appropriate box.
Chapter 3 – Installing nTier File Migrator 3. After the nTier restart completes, import tapes into the tape library using the tape library controls, or for configurations with only a stand-alone tape drive, manually insert one of the tapes. The nTier File Migrator software automatically tries to identify any tape cartridge that is imported into a library or tape drive under its control.
Configuring nTier File Migrator Before You Begin Review the following check list of key points to be considered when configuring the nTier File Migrator for your environment. Read Chapter 2 – Concepts to learn about Volume Sets and File Groups and their relationship to how nTier File Migrator handles files.
Chapter 4 – Configuring nTier File Migrator When configuring the File Group and Volume Settings, make sure you consider the capabilities of the hardware in your configuration, including the capacity of the logical drive used for the nTier File Migrator archive on the nTier, the cartridge storage capacity of the library, the number of available tape drives, and the data capacity of the cartridges.
Configure Volume Sets 4. Click on the + sign next to the drive letter to display the available options, then expand the option you want to configure. When first configuring the system, you must use these options in the following order: Configure Volume Sets on page 49 Allocate Files to File Groups on page 53.
Chapter 4 – Configuring nTier File Migrator 4. If desired, change the default name assigned to the Volume Set to something more meaningful. a. Right-click on the name of the newly created Volume Set and select Rename. b. Type the new name. c.
Configure Volume Sets 8. Enter a value to specify the percent full point at which nTier File Migrator automatically allocates additional tape cartridges (Volumes) from the Blank Media Set to this Volume Set. 9. When you are satisfied with the configuration options, select .
Chapter 4 – Configuring nTier File Migrator 4. To enable replication for the Volume Set, check Enable replication and enter the number of replicas you want created into the Number of additional copies of each cartridge field. 5. Configure the replication timing. An essential part of defining the replication strategy for a Volume Set is defining the timing of replica updates.
Allocate Files to File Groups If insufficient fresh media is available, then the system reports that it is out of disk space when any attempt is made to write data to files that have been allocated to that Volume Set. Use the following steps to allocate cartridges to a Volume Set.
Chapter 4 – Configuring nTier File Migrator Create One or More New File Groups By default, nTier File Migrator software has a single Default File Group defined into which all files it manages are placed. If you want more specific groupings, use the following steps to define one or more File Groups into which files can be allocated.
Allocate Files to File Groups Configure the File Group Listing Order The order in which File Groups are listed in the Archive Management Console is important because an individual file can be allocated to only one File Group. The rules governing the allocation of files to File Groups are applied in the order that the File Groups appear in the left pane of the console.
Chapter 4 – Configuring nTier File Migrator 2. Click on the name of the File Group to which you want to allocate files. The Configuration of file group panel displays in the right pane of the Archive Management Console window. Figure 4-8 Enter the file name extensions or path pattern and exclude pattern for the File Group.
Page 57
Allocate Files to File Groups 4. Select Enable file fragmentation if you want to be able to perform partial file restores. File fragmentation is typically required only for very large files (see Partial File Restore and Cartridge Spanning on page 31.) 5.
Page 58
Chapter 4 – Configuring nTier File Migrator Flush Written Files From Disk When a file is first written to the drive letter under nTier File Migrator control, it is always first written to the system disk drive. The Flush written files from disk options lets you configure the system so that, after a file has been securely written to tape, the instance stored on disk is flushed (deleted) to release the disk space that was occupied by the file.
Page 59
Allocate Files to File Groups Option Description Files are flushed a Select Flush written files from disk and After and put an appropriate preset length of time number in the box. With these options selected, files will be flushed a after writing defined length of time after they were written, or when the last replica has been completely written to tape, whichever comes later.
Page 60
Chapter 4 – Configuring nTier File Migrator Option Description Files are flushed Select Flush written files from disk and When tape replica has been immediately after written. Also, select Flush read files from disk and As soon as the file writing and after is read.
Page 61
Allocate Files to File Groups Option Description File are flushed from Select both the Flush written files from disk and Flush read files from disk a preset time disk boxes. Select the required timing of the flush operation following after writing unless writing.
Page 62
Chapter 4 – Configuring nTier File Migrator 2. Select the advanced options you want to use. Keep all versions on disk. The nTier File Migrator software normally maintains a complete version history of every file under its control. However, only the most recent version of a file is available to standard applications through the file system interface;...
Page 63
Allocate Files to File Groups However, in the event of a shortage of blank tapes or a failure of the tape library hardware, no data can be written to the system. For high-integrity applications that require data to be written immediately to WORM media there is no alternative but to stop writing data until the error condition has been resolved.
Page 64
Chapter 4 – Configuring nTier File Migrator To overcome this, File Migrator implements a data pipeline that allows the application to continue writing data to the next file before the last file is completely written to tape. Selecting the Force applications to wait... option turns off the data pipeline, forcing applications to wait until the data has been written to the tape drive before they can continue.
Using nTier File Migrator This chapter describes using the nTier File Migrator software to perform day-to-day operations. Accessing Files The magnetic disk storage and tapes managed by the nTier File Migrator software appear as a single standard Windows logical drive to computers on the same network as the nTier system.
Page 66
Chapter 5 – Using nTier File Migrator Start History Explorer Use the following steps to start History Explorer. 1. Using a computer connected to the same network as your nTier, access the Windows Desktop on your nTier (see Access the nTier Windows Desktop on page 35).
Page 67
Accessing Files To view the files in a particular folder, click on the folder name in the left pane. Click the + sign next to the folder icon to expand a folder to show a list of subfolders. Clicking again on the folder icon collapses the folder list. The upper right pane lists files (and subfolders) contained within the folder selected on the left pane.
Tape cartridges are imported into a tape library using the library controls, which vary from one model to another. In Spectra Logic libraries, the BlueScale user interface is used to import (and export) cartridges. If the nTier system is connected to a stand- alone tape drive, a tape cartridge is manually inserted into the drive.
Page 69
Working with Tape Cartridges If you import a tape cartridge that was managed by nTier File Migrator software on a different Windows server (nTier) and the Volume Set does not exist on this server, the nTier File Migrator software automatically creates the required Volume Set. However, files from the imported tape are not be available until you import the tape (or its catalog) into the nTier File Migrator archive on this server (Importing an External Tape).
Chapter 5 – Using nTier File Migrator Reformatting Tapes Reformatting a tape erases all the data that is stored on the tape and moves the tape into the Blank Media Set. When a tape is reformatted, the files that were stored only on that tape (for example, files that have been flushed from magnetic disk in accordance with the File Group policies) become inaccessible.
Page 71
Working with Tape Cartridges 4. Select the tape you want to reformat to display the Medium Properties in the right pane. Figure 5-4 The Medium Properties pane for a tape cartridge. 5. Click Reformat. 6. Choose a reformat option, then click Ok to begin reformatting the tape. Mark as Blank will instruct the system to treat the tape as blank, and move it to the blank media set.
Chapter 5 – Using nTier File Migrator Long Erase uses the SCSI long erase command which overwrites all the data on the tape. This can be a time consuming operation but it provides a good degree of certainty that the data on the tape cannot be recovered. Overwrite Entire Medium uses write commands to overwrite all the data recorded on the tape with a different data pattern than that used by long erase.
Working with Tape Cartridges Verifying the Data on a Tape The verify function allows you to confirm that the data on a tape can still be read, without transferring it to magnetic disk (that is, it performs media verification). You can use this function to check the integrity of your archived data.
Chapter 5 – Using nTier File Migrator Finalizing a Tape Finalizing a tape cartridge results in special tape marks being written to the tape that prevent additional data files or file fragments being written to that tape. In addition, when a tape is finalized, a Tape Contents Catalog is automatically written to the tape.The presence of a Tape Contents Catalog on a tape greatly reduces the time required to perform an Import Catalog operation.
Working with Tape Cartridges Write-Protecting Tapes To stop the system from writing data to a particular tape (or Volume) before the tape is full, you can write-protect the cartridge through the Archive Management Console. You may want to write protect a volume or a volume set, for example, if the archive strategy requires that the system start writing to a new Volume at the beginning of each month.
Chapter 5 – Using nTier File Migrator Managing Volume Sets and File Groups When you configured File Migrator, you created one or more Volume Sets (Configure Volume Sets on page 49) and File Groups (Create One or More New File Groups on page 54).
Retensioning Tapes Retensioning Tapes Under normal circumstances it should not be necessary to retension tapes. In exceptional circumstances, the administrator can retension a tape cartridge. Some drive types do not support the retension operation. In these cases, the option is not available in the Archive Management Console.
Page 78
Chapter 5 – Using nTier File Migrator Notes...
System Recovery System Rebuild from Tape About File System Rebuild From Tape The system provides a facility to rebuild the file system from tape, for example in the event of a hardware failure. If an imported cartridge belongs to a Volume Set that has not previously been defined on this particular server, the new Volume Set is created and the tape cartridge is allocated to it automatically.
Chapter 6 – System Recovery 4. Right-click on the tape and select Import catalog or Import data. Import Catalog loads file and folder information (metadata) into the system making the entire directory tree visible to users, but it does not restore the actual file data to disk cache.
Disabling and Re-Enabling a Tape Drive or Library Disabling a Tape Drive or Library 1. Open the Archive Management Console. 2. Navigate to the Diagnostics section. 3. Right-click on the component and select Disable. Figure 6-2 The Disable option of the Diagnostics section. Re-enabling a Tape Drive or Library 1.
Chapter 6 – System Recovery Options In Case of Tape Library or Stand-Alone Drive Failure In normal operation, the nTier File Migrator software writes files to tape as soon as they have been written to magnetic disk. By doing this, the system provides the improved data integrity and security necessary for an archival storage system by ensuring that a second instance of all data written to disk is generated almost instantly.
Diagnostics & Maintenance Event Log Whenever the nTier File Migrator software encounters an unexpected error condition, it puts a message in the nTier File Migrator Event Log and generates a Trace Log file. Certain other conditions, such as running out of space because all available tapes are full, writes warnings to the event log.
Chapter 7 – Diagnostics & Maintenance Viewing More Detail about an Error or Warning Double-click on the error or warning of interest to view more information. An Event Properties dialog opens for the event. Figure 7-2 The Event Properties for a selected event. Diagnostic Trace Log It is sometimes useful to be able to see what is happening internally within the system.
Diagnostic Trace Log 3. Click on the System icon to open the trace log in the right pane of the window. Figure 7-3 The trace log. Changing the Level of Detail in the Trace Log The trace log can generate a large amount of information, which can scroll past very quickly.
Chapter 7 – Diagnostics & Maintenance Automatic Generation of Trace Files Whenever the nTier File Migrator software encounters an unexpected error condition, it puts a message in the nTier File Migrator Event Log and generates a trace file. The trace file contains a record of what the system was doing in the time preceding the ®...
Tape Drive Cleaning Tape Drive Cleaning Most types of tape drive require periodic cleaning to maintain peak performance. Normally, a drive automatically issues a request for cleaning at the appropriate time. When the nTier File Migrator software receives this request from drives within tape libraries, the File Migrator sends a command that inserts a cleaning cartridge in the drive.
Chapter 7 – Diagnostics & Maintenance Tape Cartridge Error States Certain error conditions, such as media errors, leave the affected tape (or, in the case of replicated tapes, the affected Volume) in an Error state. The system does this to protect data by ensuring that it will not attempt to write to media that have problems associated with them.
Archive Management Console replication configuring volume volume sets sets contacting Spectra Logic setting policies for file e-mail addresses, Spectra Logic groups enabling stand-alone tape starting drives using to allocate files to file event log, using to diagnose groups...
Page 90
See HSM mailing address, Spectra Logic pattern matching with maintenance, preparing hardware History Explorer wildcards description renaming media sales, Spectra Logic opening a file required configuration...
Page 91
Knowledge Base web phone numbers, Spectra Logic site offices tiered storage, See HSM troubleshooting using hardware tape catalog, description diagnostics tape contents catalog,...
Page 92
Spectra Logic manuals Spectra Logic home wildcards, using to create file groups Windows desktop, See Windows Remote Desktop Windows Explorer, using to access files...