1. Manuals
  2. Brands
  3. napp-it Manuals
  4. Network Storage Server
  5. ZFS Storage
  6. User manual

napp-it ZFS Storage User Manual

Zfs storage server
Hide thumbs
Table of Contents

Advertisement

Quick Links

Download this manual
napp- it
ZFS Storage Server
User's Guide
Setup on OmniOS
First steps
published: 2016-Oct-12 (c) napp-it.org
Licence:
CC-BY-SA see http://creativecommons.org/licenses/by-sa/2.0/

Advertisement

Table of Contents
loading

Summary of Contents for napp-it ZFS Storage

  • Page 1 ZFS Storage Server User‘s Guide Setup on OmniOS First steps published: 2016-Oct-12 (c) napp-it.org Licence: CC-BY-SA see http://creativecommons.org/licenses/by-sa/2.0/...

  • Page 2: Table Of Contents

    Rollback and Clones Operational settings Appliance Security Appliance Tuning Appliance Maps Disaster and general data security napp-it Free vs Pro & Extensions Appliance Z-RAID / ZPOOL-RAID with SSF (Storage/Service Failover) HA/ RSF-1 Clustering (3rd party extension) Addendum: About Storage Problems and Solutions...
  • Page 3 1. The Solaris Family - OPENSOLARIS based operating systems Developed by Sun with its initial release in 2008, based on Unix System 5, Release 4 with revolutionary features like ZFS (http://en.wikipedia.org/wiki/ZFS), Comstar (enterprise ready iSCSI and FC technology), Dtrace, Crossbow virtual networking, virtualization with Zones (http://en.wikipedia.org/wiki/Solaris_Zones), ZFS inte- grated kernel based CIFS/SMB and NFS server among other features like ZFS boot mirrors, disk unique WWN enumeration or fault management service with auto hotspare support.

  • Page 4: Zfs Distributions

    2. ZFS Distributions Unlike storage appliances that are based on their own distribution of BSD, Illumos or Linux, napp-it is a „Nasifier“ for some general-use enterprise operating systems that you can keep up to date like Based on Solarish - Oracle Solaris 11.3+ (commercial OS)
  • Page 5 You can run napp-it on OpenIndiana Hipster 2016.04 (OpenSource) or Oracle Solaris 11.3. Both support SMB 2.1 and come with a GUI. You manage storage via the napp-it Web-UI but the local GUI helps to transfer and organize data locally or to setup things like ip v6. If you want the GUI in production systems, prefer Solaris.
  • Page 6 2.3. Storage-Management via napp-it on OmniOS, OpenIndiana or Oracle Solaris ZFS Filesystems and disk details...
  • Page 7 You should mainly care about the network and the disk controller. Rules for a trouble free minimal system - use Sata/ AHCI with a 30GB Bootsdisk (60GB when using the napp-it ToGo image) - prefer Intel nics - use at least 4 GB ECC RAM (2 GB is the absolute minimum, more gives performance as readcache)
  • Page 8 3.1 ZFS Configurations...

  • Page 9: Napp-It Togo Setup (Preconfigured)

    This is the easiest and suggested setup method of installing, backup and recover a napp-it barebone appliance based on a preconfigured image for a ready to use ZFS storage server. It works best for OEM use with known hardware. It may not work with any hardware or disk. On success, please send a mail to community@napp-it.org with your mainboard, interface, disk type and remark.
  • Page 10 4.1.2 Screenshots of a base setup (asuming you have a Clonezilla USBStick) Copy the napp-it imagefolder to you USB stick (root directory) and boot your server from the USB stick 1. Start Clonezilla 2. select device-image mode 3. select local_dev (USB drive) 4.
  • Page 11 Optionally update OmniOS On problems with the restored image: 1. If napp-it boots, but network not working: do a manual network setup, see 5.2 2. Image is not compatible with your SSD, use the ones that were used to create the image 3.
  • Page 12 IO::Socket::SSL notest install Net::SMTP::TLS exit; 9. disable services SSH root access and sendmail service in napp-it (menu Service) set netbios_enable=true (Windows network browsing in OmniOS 151016 and up, menu Service>SMB>prop.) 10. set basic tunings in napp-it System > Tuning, 11.
  • Page 13 All services that require configuration like databases, webservers etc should be virtualized on stable ZFS storage with online backup and snap support. The VM requires a DHCP server. You must only import the VM (OVA Template). For disaster recovery, you can export your individual template from ESXi.
  • Page 14 5.1 Manual setup of a napp-it storage appliance with DHCP Download the OmniOS or Solaris installer (ISO dvd or USB installer), boot the installer and install the OS. Select UTC timezone, with bloody versions keep all defaults incl keyboard After OS setup, your need to configure your network either with DHCP (5.1) or manually (5.2).
  • Page 15 Napp-it installer creates a bootenvironment with the state prior the installation so you can always go back. As napp-it is a pure copy and run installation, it copies everything to /var/web-gui If you delete this folder, the init file /etc/init.d/napp-it, the user nappit and an entry in /etc/sudoers and user_attr you have wiped it beside tools that are installed during setup like smartmontools, midnight commander, iperf or netcat.
  • Page 16 6. ZFS manuals and infos You should now download and optionally printout some basic manuals. 6.1 napp-it manuals http://www.napp-it.org/doc/downloads/napp-it.pdf http://www.napp-it.org/doc/downloads/napp-in-one.pdf http://www.napp-it.org/doc/downloads/advanced_user.pdf http://www.napp-it.org/doc/manuals/flash_x9srh-7tf_it.pdf 6.2 manuals for Oracle Solaris 11 Download and print out needed manuals from https://docs.oracle.com/en/operating-systems/ 6.3 manuals for Oracle Solaris 11 Express and OmniOS Download and print out needed manuals.

  • Page 17: Remote Management

    7. Remote management A Server can be managed remotely, use these tools 7.1 IPMI IPMI ist a must have for a server: https://en.wikipedia.org/wiki/Intelligent_Platform_Management_Interface IPMI is a remote management microcontroller on serverclass hardware like Supermicro mainboards that ends with a „-F„ . You can connect the microcontroller remotely with a webbrowser even when the server is in a power-off state.
  • Page 18 So you must either create a regular user than can login. After this you can gain admin permissions wit a su command. Other option is to enable remote root access in the napp-it Web-GUI in menu „Services >>...
  • Page 19 To use WinSCP you must enable SSH on OmniOS. This is the case per default but per default only allows that regular users can login, not root. An option is to enable remote root access in the napp-it Web-GUI in menu „Services >>...
  • Page 20 If you are unsure about your ip, enter the following console command ifconfig -a If you start napp-it the first time, you are asked to setup napp-it passwords and email. Setup options (Menu About >> Settings): User accounts: Admin and oprtator...
  • Page 21 Napp-it free is not crippleware or a product that is limited in essential features. It is sufficient for many cases. It is a stable state of napp-it that is updated from the dev release from time to time. If you want to support napp-it or use Pro features or require immediate access to bugfix releases commercially or as a homeoffer, check http://napp-it.org/extensions/quotation_en.html...
  • Page 22 9. Create a ZFS datapool From other systems like Windows, you know disks where you can create one or more partitions with a fixed size. You can combine single disks or partitions to a Raid that is treated like a single disk. It is possible to increase a partition up to disk or Raid-array size.
  • Page 23 10. Create a ZFS filesystem From other systems like Windows, you know partitions that you can format in FAT32 or NTFS. This is similar to OmniOS and ZFS with the difference, that you always format to ZFS and that the size of a filesystem can grow dynamically up to poolsize.

  • Page 24: Zfs Filesystems

    Folder-ACL Reset ACL is free. Other features are part of an extension. You can set ACL via Windows in napp-it free. list all ZFS filesystem properties...

  • Page 25: Solarish Smb Server

    - permissions are based on Unix UID/GID/ Posix ACL, this is a plus if you work mainly in a Unix world https://en.wikipedia.org/wiki/Access_control_list some Advantages of Solarish CIFS over SAMBA (used by napp-it) - fully integrated in ZFS as a filesystem property, easy handling via zfs set command there is no configuration file, enable it and set permissions as file/share attribut.
  • Page 26 The SMB service is started automatically when you enable a share. Some modifications (like share level ACL) require a service restart. This is done automatically by napp-it. On problems with the SMB server or if you are in a AD Domain that was temporarely unavailable, it may be needed to restart the service manually in menu „Services >>...
  • Page 27 If you have enabled the napp-it backup function in menu Jobs, you can restore napp-it and all user settings in menu Users > Restore settings (the ACL extension on a napp-it Pro or Dev edition is required) Howto: Reinstall OS, setup napp-it, insert a license key, update napp-it to a Pro edition and import datapool.
  • Page 28 You can modify ACL permissions with the /usr/bin/chmod command, per Windows or per napp-it ACL extension. Modifying ACL via CLI command is really stupid. Especially with napp-it free, you can use Windows (beside Home editions) to modify permissions. To do this, you can login to the SMB share from Windows as user root.
  • Page 29 SMB share share tank/userdata, you find the share control file as /tank/userdata/.zfs/shares/userdata You can set share level ACL via napp-it/ ACL extension, or remotely via Windows server management (You must connect a share/ server management as a user that is a member of the SMB admin group.)
  • Page 30 ACL other than generating the ne- cessary ACL entries to represent the new mode of the file or directory. The default mode for the aclmode is pass-through (napp-it only). “ aclmode = restricted is added in Illumos/OmniOS to avoid permission...

  • Page 31: User And Groups/ Active Directory

    If you do not need to restrict access to a SMB share, you can enable guestaccess and you are ready. If you want to restrict access, you can create users with napp-it menu „User“ When you create a user, you only need to enter a username and a password. This user is valid for SMB access and is a valid Unix user.
  • Page 32 ACL settings for multi-user SMB access some basic examples for File and Folder ACL settings Goal: - everyone can access and read files from a share like data (data is a filesystem below a pool named tank) - everyone can modify files in data/common and below - user paul is the only one to access data/paul and below needed ACL settings: folder /tank/data:...

  • Page 33: Active Directory

    If OmniOS lost connectivity to your AD server, you can disable/enable the SMB service or rejoin the domain. Care about Your napp-it Server can be either a member of a workgroup (use local user) or a domain (can use either local or domain user).
  • Page 34 Enable iSCSI via menu ZFS Filesystems For smaller installations, Comstar is quite complex. Napp-it offers a way where you can enable iSCSI on a per filesystem way with a on/off switch in menu ZFS filesystems in the row of a filesystem under iSCSI.

  • Page 35: Data Scrubbing

    If you want to check the whole pool you can start a scrub manually or as a planned napp-it Job. With desktop disks I would do this once a month ex on a low io day like saturday. Unlike a traditional fschk that require an offline filesystem for quite a long time without a real repair option, a scrub is a online process that runs in the background with low priority and verifies/repair all data.

  • Page 36: Data Snapshots/ Versioning/ Backup

    Even ten-thousands of snaps can be hold without any problem (Okay as the former state blocks capacity, sometimes the pool is full). As this is done on ZFS storage, checksum verification, scrubbing and autorepair is working - does not matter how old a snap is - ideal for long term storage and archives with regular scrubs.
  • Page 37 If you need realtim sync with the exact same datastate at any time you can use a mirror between appliances. You need two or more storage nodes (independent ZFS storage servers) that offer a ZFS file- system over a fast network connection as a FC or iSCSI target. A storagehead can the built a ZFS pool over these iSCSI targets as a mirror or raid-Z over nodes.
  • Page 38 For most rollback cases, using Windows and Previous version is the suggested method for a roll back as you can restrict the rollback to files and folders. You can use napp-it menu „ZFS-Filesystem >> Rollback to initiate a rollback 18.3 Create a Clone (writeable filesystem from a snap).

  • Page 39: Operational Settings

    - more see www.pushalot.com (Windows8 and -Phone, free) and www.pushover.net (ios, Android) - Set a backup job (Jobs >backup >> create backup job) tp backup basic OS and napp-it settings to a pool Restore all user, SMB groups, idmappings and other napp-it settings then via User > Restore (ACL extension) - Set autoscrub jobs (see 15.)
  • Page 40 Firewall settings/ Security panel (available on a valid Pro or Dev edition) You can use the napp-it Pro security panel to restrict access based on a set of ip adresses or local networks or based on a network adapter. With napp-it free, set the according rules manually.

  • Page 41: Appliance Tuning

    1000MB/s or more. This is why we need faster networks like 10Gb/s Ethernet (about 1000 MB/s) and for this we need some tuning settings to achieve full performance on your hardware or for a workload.. Napp-it Appliance Tuning Panel (napp-it Pro complete only)
  • Page 42 - napp-in-one tuning If you use napp-in one to connect ESXi over the internal vswitch with the ZFS storage VM, all transfers are in software so most netwok and Ethernet centric tunings aspects are not needed, You should use vmxnet3 as vnic as it is much faster than the e1000 vnic with the base vmxnet3 tuning.
  • Page 43 X520 or 640 adapter. If you enable the bridging functionality, napp-it acts like a regular 1/10/40G switch. Every computer can con- nect the NAS, each other and the internet that is connected for example on the 1G onboard nic of the NAS.

  • Page 44: Appliance Maps

    Appliance Maps works with Sata or LSI HBAs where it uses the sasircu tool (like Disks > DISK Location). It allows to display maps of your enclosures (up from napp-it 16.8). You can create up to 9 maps and assign disks to the map.
  • Page 45 Example: Map Chenbro 50 x 3,5“ Bay...

  • Page 46: Disaster And General Data Security

    23. Disaster and General Data Security ZFS is the best available free technology to protect your data. It adresses nearly all problems of conventional filesystems and raid technologies with many disks or large capacity. Unlike checksummed backup technologies like Snapraid it works in realtime and protects your data from the moment when you click save. It can detect every problem due data and metadata checksums in the chain disk-controller-driver >...
  • Page 47 OS that writes the data. The key is that you must ensure that after a commit from the storage unit, the data must to be on stable storage and not lost in the disk writeback or ZFS storage cache on a crash. One option would be, disable all write caching but this is a bad idea.
  • Page 48 Sync Write (Log every commited write to a ZIL device) ZFS always collects small and slow randow writes in a rambased write cache for a few seconds and write them together as a single large sequential write. A commit to a writing application means, yes data is in cache. It does not mean data is on disk.
  • Page 49 Level 2: SoHo use Add another small ZFS storage server and place it on another physical location/ room. This can be a cheap HP Microserver that gives you 4 disk bays where you can insert disks for a backup pool. Replicate data from your primary storage on a hourly or daily base.
  • Page 50 The Free edition includes all functions to run a NAS/SAN with many enterprise features. There are no restrictions regarding OS features and it comes with all basic features to run and manage a ZFS Storage System in a SoHo, lab, office or school environment. Napp-it is not an OpenSource application but all sources are open (Perl) and you are allowed to edit or extend the functionality of napp-it.
  • Page 51 24.2 Complete extension (not working on Linux) If you want to unlock all Pro features from all extensions you can use the Napp-it Pro complete edition. Complete adds Appliance menus like Appliance Maps, Appliance Security and Appliance Tuning. Complete additionally adds email support for setup problems but only for regular commercial licenses.
  • Page 52 24.5 Async highspeed/ network replication (Solarish and Linux) - Async Replication between appliances (near realtime) with remote appliance management and monitoring - Based on ZFS send/ receive and snapshots - After an initial full transfer, only modified datablocks are transferred - High speed transport via (buffered on Solaris) netcat - (unencrypted transfer, intended for secure LANs) - Replication is always pull data.
  • Page 53 The replication itself is done via a netcat datastream on a port > 50000. These ports must be open. Check if napp-it is working on both machines. If you have a firewall, check that port 81 and the replication port (see menu Jobs) are open.
  • Page 54 If you reinstall the OS you must either restore group and job settings. The easiest way is when you have used a napp-it backup job (Menu Jobs) that creates a backup of group and job settings on your first datapool under backup_napp-it.
  • Page 55 25. Appliance Z-RAID with SSF (Storage and Service Failover) based on Comstar iSCSI targets/ initiators (under development, not ready) Z-RAID vs RAID-Z ZFS and Raid-Z is a perfect solution when you want data protection against bitrot with end to end data checksums and a crash resistent CopyOnWrite filesystem with snaps and versioning.
  • Page 56 25.2 Setup Appliance Z-RAID SSF (ZPOOL-RAID with Storage and Service Failover) The basic option is using two Storage Appliances where each offer a local Pool as a LUN via iSCSI. An Initiator on one of them creates a Z-RAID Pool over both LUNs and offers services like NFS or SMB. On problems, the Initiator on the second server imports the Z-RAID and switch NFS/SMB services over a common virtual HA ip.
  • Page 57 This menu shows the state of the network Z-RAID and allows to change the state of heads (Master/ Slave ) and nodes. You can do a manual failover between heads or a role switch from this menu. more, see Z-RAID SSF manual http://www.napp-it.org/doc/downloads/z-raid.pdf...
  • Page 58 The RSF-1 Cluster Plugin can be obtained with support directly from high-availability.com or from distributors specialised to HA/ Clustering solutions. Napp-it only offers a menu to make handling easier. The RSF-1 Plugin or support for clustering is a 3rd party offer and not available from napp-it.org.

  • Page 59: Addendum: About Storage Problems And Solutions

    27. Addendum: About Storage Problems and Solutions RAID, Backup and ECC exist in order to mitigate the probability of data loss. While the technical basics are clear, one now has to estimate the data loss endangerment and relevance of all components involved. Included in this is an assessment of which problems can occur and how often they occur in addition to an evaluation of how one can minimize each risk on its own as well as relative to the file system.
  • Page 60 Involved System Components RAID RAID mainly exists in order to make failure of one or several disks possible without data loss. It may include Self-Healing Features, meaning that corrupted or changed files can be repaired or restored on access. Contrary to RAID-similar backup solutions (like Snapraid and Unraid), RAID protects the data in real time because it distributes, stripes or mirrors data blocks to different drives on every write..
  • Page 61 ECC is something like RAID for RAM. It can detect bit errors and repair them from redundancy. Those data errors must not be due damaged RAM in all cases, but might just be a statistical problem which occurs more often if more RAM is in use.
  • Page 62 tl;dr: Solution in order to minimize the probability of a data loss Next to good hardware and backups, use advanced file systems, like ZFS. They were developed in order to work with large arrays (multiple terabyte up to petabytes) safely. Unlike arrays with ext4 or NTFS which do not implement CopyOnWrite and checksums, they are crash resistent, do not require a long running offline fschk and offer checksums to repair bitrot on access or scrubs without the write hole problem of hardware raid.

  • Page 63: Other Manuals

    28. Other manuals and more tuning infos .. All-In-One (virtualised SAN)_ http://www.napp-it.org/doc/downloads/napp-in-one.pdf Build Examples http://www.napp-it.org/doc/downloads/napp-it_build_examples.pdf SMB 10G Tunings on OSX/ Windows: http://napp-it.org/doc/downloads/performance_smb2.pdf Advanced user: http://www.napp-it.org/doc/downloads/advanced_user.pdf Tuning: http://napp-it.org/manuals/tuning_en.html more: http://napp-it.org/manuals/index_en.html From OmniOS 151019: OmniOS suppports LX branded zones for lightweight virtualisation of Linux VMs more http://www.napp-it.org/doc/downloads/zones.pdf...

Table of Contents