ZFS Server Greatness

June 20, 2011 Leave a comment

As I mentioned previously, I like to run a PC as a server in my home to centralize all the files instead of having documents, pictures, videos, etc., spread out on various PCs.  But as time goes on, and the amount of files being collected grows, you really start to panic about losing files.  Especially when you have years of digital photos.  So I went in search of a solution for safe storage.  For a while, I used a first generation Drobo.  Great product.  You can swap in and out new drive to easily grow the amount of space available.  But as I was running Linux and trying to use it in a server setup for a year or so, it just wasn’t working for me. Then I found the breakthrough of a lifetime, which is ZFS file system.  It make traditional hardware Raid look like a joke for ease of setup, especially for what I needed.

If you haven’t heard of ZFS, watch this great video I found that explained it all good and got me up and going in no time at http://blogs.oracle.com/video/entry/becoming_a_zfs_ninja.  But when it comes to ZFS, you’re limited by the options for operating systems. ZFS, a technology in then Sun Microsystems Solaris, now Oracle Solaris.  When I first started using it, I started with OpenSolaris as the operating system.  A great operating system with a great community.  Since then, I have migrated to Oracle Solaris 11 Express.

Setup of a ZFS file system was really easy.  What I needed was to have a pair of drives act as a mirror so I have a complete copy of the files on each drive.  As an example, here is how easy it was.

1. First, you’ll need to figure out what Solaris calls your drive. You can find this out by using the format command.

root@nas:~# format
Searching for disks...

AVAILABLE DISK SELECTIONS:
       0. c11d0 < cyl 19454 alt 2 hd 255 sec 63>
          /pci@0,0/pci-ide@11/ide@0/cmdk@0,0
       1. c11d1 <ST320082-         3ND2FLA-0001-186.31GB>
          /pci@0,0/pci-ide@11/ide@0/cmdk@1,0
       2. c12d1 <WDC WD20-  WD-WMAZA339211-0001-1.82TB>
          /pci@0,0/pci-ide@11/ide@1/cmdk@1,0
       3. c13d0 <ST320082-         5ND0KM1-0001-186.31GB>
          /pci@0,0/pci-ide@14,1/ide@0/cmdk@0,0
       4. c13d1 <WDC WD20-  WD-WMAZA320422-0001-1.82TB>
          /pci@0,0/pci-ide@14,1/ide@0/cmdk@1,0

2. Look at the output and you will see at the start of each entry, an ID like c13d1. This is the name to remember.  So to create my mirror with disks c13d1 and c12d1, I did the following.

zpool create files mirror c13d1 c12d1

That’s it.  A new ZFS pool was created called ‘files’ using the two disks in a mirrored setup.  It also automatically mount the pool as /files in the file system and is “formated” as a ZFS file system.  How easy was that?

Now that other part of ZFS that I like, but is hard to explain is a ZFS dataset.  It’s kind of like a partition, but not bound to a particular size, but easiest to think of it like a partition.  The reason I wanted this was to keep track of the amount of space certain files were taking up. So within my ‘files’ zpool, I created several datasets such as one for photos, another for videos, another for general files, etc.  Each dataset within a zpool can take up as much disk space as it needs but needs to share the space with the other datesets in the zpool.

So to create one of these datasets, as an example, I will create a dataset for my videos under the files zpool.

zfs create files/video

Now I have a new dataset within ‘files’ for videos.  What’s really interesting is that you can nest these datasets. So as I mentioned, I created several datasets for my different data.

root@nas:~# zfs list
NAME               USED  AVAIL  REFER  MOUNTPOINT
files             1.09T   715G    34K  /files
files/Audio       3.66G   715G  3.66G  /files/Audio
files/Pictures    23.1G   715G  23.1G  /files/Pictures
files/files       34.8G   715G  34.8G  /files/files
files/gallery     1.92G   715G  1.92G  /files/gallery
files/home        28.1G   715G  28.1G  /files/home
files/mail        3.26G   715G  3.26G  /files/mail
files/video       865G    715G   865G  /files/video

As you can see, you will notice that the AVAIL space is identical for each dataset.  This is because they are all sharing the space within the zpool ‘files’.  So they look like partitions but aren’t fixed to a size.  Now I can see that my photos are taking up 23.1 GB of space.

Well, this is the first post of my adventures with ZFS.  If you haven’t had a chance, watch the video that I linked to above about how fun and easy it is to use.

Advertisement

Filed under Solaris Tagged with ,

Overview

June 20, 2011 Leave a comment

I guess I’ll first start with an overview.  My network setup at home is not your usual setup.  I discovered a long time ago, before Windows Home Server was around, of the advantages of having a PC act as a server to centralize files and information in your computer life. So currently I have two PCs that I use as my server backend. One as just the file storage and the other as the main server frontend that runs multiple virtual machines to do different functions.

Networking wise, I believe that if you can run a wire to a location, that’s the best option instead of wireless.  So if I can get a Cat5 ethernet cable to a room, then that is what to do. Wireless is great, but it can have issues with interference and signal strength. I prefer to run a nice fast Gigabit Ethernet to as many rooms as possible.

PC wise, as it takes a while for me to through out an old PC, I have many in the house, functional or not, usually not.  Basically, I have the family PC that I’m not really allowed to tinker with, the wife’s PC, my Mac in the basement and my server(s).

But what amazes me is the amount of IP enabled devices in the home.  At one time, my DHCP server seemed to have listed between 20 to 30 addresses allocated.  That’s hard to believe, but if you sit down one day and think about it, it adds up quickly these days with Tivo’s, media extenders, iPods, the work laptop, home PCs, game consoles, some TVs, etc. The other day I looked behind my TV in the living room and realized that there was five things with an ethernet cable plugged into it.

Well, that is a pretty good summary with each topic above having its own crazyness.

 

Filed under Misc

Welcome

June 20, 2011 Leave a comment

So I decided to start writing about my activities that I do at home and maybe related to work as it all relates to living as a sys admin for my home.  Since I am in the hitech business, it’s hard not to bring some of that knowledge home and for anyone else that is in my position, try to not get carried away.  So you’ll hear about my successes and failures of being the home sys admin, as well as how it has interconnected itself into my hobby of model railroading.

Enjoy,

Greg

Filed under Misc