In Defiance of Titles

Just another weblog

Posts Tagged ‘home nas

My Home NAS, Part 10: Mac OS X Automounting

leave a comment »

So, uh, some time ago when I wrote the last post in my home NAS tutorial (for reference, here’s all previous posts in the series), I made a rather bold omission:

Once you’ve got the tunnel running (ideally you’d set it up to run automatically), all that’s left is to mount the NFS share(s) to appropriate locations in your filesystem. This process varies by operating system (even across UNIXes), so for now I’ll leave that up to you.

Unfortunately, yesterday I found myself deeply regretting the decision not to document this step; somehow, when I upgraded my Mac to Snow Leopard, my NFS share configurations were wiped out, leaving me with nothing but scp to get files off of the NAS. So, to avoid future headaches, I’ve decided to finish my documentation.

First, a word of warning for you Linux folks out there: Mac OS X handles NFS mounting differently than a lot of other UNIXes do; you’ll be tempted to start out in /etc/fstab, but don’t do it…it’s deprecated. OS X offers a more flexible solution allowing you to automatically mount various volumes from various kinds of directory services…but that’s kind of beyond what we’re looking for, so we’re just going to make a few simple changes to the automount configs instead.

The end goal here is to take the NFS shares we previously exported (critical, standard, pictures, and music) and mount them all in one directory under /Volumes. To do this, we’ll first want to create an automount map file called “/etc/auto_your-server-name”. The contents should look something like this, assuming you used the LOST-inspired port forwarding from my previous post:

critical -tcp,port=48151,mountport=62342,resvport,locallocks localhost:/export/critical
standard -tcp,port=48151,mountport=62342,resvport,locallocks localhost:/export/standard
pictures -tcp,port=48151,mountport=62342,resvport,locallocks localhost:/export/pictures
music    -tcp,port=48151,mountport=62342,resvport,locallocks localhost:/export/music

So far so good. Next, we’ll need to tell the automounter to use this new file to populate a particular directory under /Volumes; to do so, add the following line to /etc/auto_master:

/Volumes/your-server-name auto_your-server-name

Finally, tell the automounter to re-read the configs by executing the following command:

sudo automount -vc

If all goes well, you should see your four new mount points show up as shared directories under /Volumes/your-server-name; assuming, of course, that you’ve got your SSH tunnel running as per the previous tutorial. Hope this helps someone!


Written by jazzslider

September 10, 2009 at 6:38 pm

My Home NAS, Part 8: Hardware Manifest

with 2 comments

I know I said my next post was going to be about NFS setup, but I thought it might be useful instead to take a momentary break for listing off the final hardware manifest. My previous posts have been a little unclear on this subject, so to avoid confusion, here’s a list of everything I bought that’s currently part of the machine, along with links to NewEgg product pages:

Component Purpose Price (as of today)
MSI Wind PC Barebone Motherboard, processor and case, all pre-assembled in a nice, neat, power-saving package $139.99
Kingston Elite Pro Compact Flash Card Operating System Storage $22.99
512 DDR2 533 Memory (don’t remember which brand I used…) $7-8
Western Digital Caviar SE16 WD5000AAKS 500GB Hard Drive, /dev/sdb in the main RAID array $64.99
Seagate Barracuda ST3750640AS 750GB Hard Drive, /dev/sda in the main RAID array (no longer available) $80
VIGOR iSURF II HCC-S2BL Aluminum HDD Cooler for the 750GB drive $22.99

All of this is variable to suit your tastes, of course…this is just what I used. If you’ve got a bit more money lying around, it might be worth investing in larger hard drives. Make sure they’re SATA (ideally SATA 3Gb/s) drives rather than IDE, as the Wind PC doesn’t have any IDE ports. 7200RPM is also a good idea for quick access times.

Also, I’ve found that 8GB for the operating system is a bit on the high side; on my current setup, Debian is only taking up a measly 1.7GB. You could probably get away with a 4GB card instead, but the prices are low enough that it doesn’t save you much.

Oh, and as far as memory is concerned…512MB has been plenty for my purposes (my swap file is pretty much never used), but if you can afford 1 or 2 gigabytes instead, why not? Just make sure it’s DDR2 533 laptop memory (yes, this is basically a laptop without all the mobility-related bits), and only one stick of it (the Wind PC only has one memory slot).

Meanwhile, the entire software stack is free and open source, so there’s nothing else to buy beyond what I’ve listed above (unless you want something else, of course).  Based on the components/prices above (and assuming you don’t need to buy any of the extra stuff I’m about to list off), you’re looking at a total project cost of about $340, not including any shipping costs you might incur.  Not a bad deal considering the degree of customization you get along with it.

There are a couple of other things you’ll need, but only temporarily; for these, it’s best to just use something you’ve got lying around if you can:

  • 1GB SD card or USB flash drive, to use as the installer media
  • Monitor
  • USB keyboard (PS/2 won’t work; the Wind PC doesn’t have a PS/2 port)
  • Another computer which (among other things) will be used to set up the installer media; to really get this right, you’ll need to install something like VirtualBox and create a dummy Debian installation that can access your installer media…which I suppose I should document 🙂

Anyway, next time we’ll get back after the step-by-step setup; there’s still a fair amount of software setup to do, but we’re almost there.

Written by jazzslider

January 13, 2009 at 7:14 am

Posted in Uncategorized

Tagged with , ,

My Home NAS, Part 7: Breaking Things Down with LVM

leave a comment »

As I mentioned at the end of my RAID setup post, I want the storage space on my home NAS divided up into several fixed-size filesystems, each associated with a different purpose. Now, one approach here would have been to divide the physical disks up into several partitions and create several separate RAID arrays on top of those…but that seems a bit like overkill, and certainly isn’t very flexible if I later increase the size of the array. So, after a little research, I discovered a better solution: Logical Volume Management, or LVM.

Linux LVM allows you to create flexible logical volumes on top of an existing set of devices, either to join them together into one giant filesystem, or to separate them out by logical purpose. In my case, I wanted the final setup to look something like this:

Purpose Allocated Space Mount Point
Non-Critical Documents 45GB /export/standard
Critical Documents 25GB /export/critical
Operating System Backup 8GB /root/os-backup
Pictures/Home Videos 182GB /export/pictures
Music/Purchased Videos 182GB /export/music
MySQL Storage 8GB /var/lib/mysql
Cacti RRD Files 8GB /var/lib/cacti
Extra Swap Space 2GB /root/swap

I know, the math doesn’t quite work out right…but ignore that, you’ll enjoy this more.

Anyway, there are a couple of unusual details here that I’d like to explain. First…I broke my “Documents” storage up into two separate volumes, one labeled “Critical” and the other “Non-Critical.” The “Non-Critical” documents are things I’ve already backed up to DVDs, but might want immediate access to. The “Critical” documents are things I’m working on right now that I haven’t quite gotten backed up yet; everything on that volume is backed up nightly (using the duplicity command-line utility) to Amazon S3, so I needed to make sure it couldn’t get too large. I am not, after all, made of money. Meanwhile, the “Operating System Backup” volume is, as its name implies, a place to store a copy of everything on the CompactFlash card in case it fails and I need to put in a new one. Can’t be too careful.

Anyway, it’s reasonably simple to set up a logical volume structure like this. Make sure you’ve got the lvm package installed (apt-get install lvm2); then, create an LVM “physical volume” out of the device (or devices) that you want LVM to manage. In our case, we’ll use the RAID array we created last time (/dev/md0):

pvcreate /dev/md0

The next step is to create a “volume group;” volume groups are used to collect various physical volumes so that they can be treated as a single unit. Since we’ve only got a single physical volume to worry about, this is easy:

vgcreate -s 16M export /dev/md0

The -s parameter there is the size of the “physical extents” that make up the volume. Because this device is so large, it was important to choose a reasonably large physical extent size; unfortunately I can’t remember exactly why. “export”, meanwhile, is the name I used for the volume group; we’ll be using that again in a moment.

To figure out the total number of physical extents in the new volume group, you can run vgdisplay export; in my case, it came to 29808. However, you don’t necessarily have to know this to get things working properly, since LVM allows you to create your logical volumes using actual byte-size values instead. It’s just useful to know it’s there.

Anyway, creating each logical volume is pretty simple; for each volume you want, run something like this:

lvcreate -L 25G export -n critical

The above command would give you a 25GB logical volume in the “export” group named “critical”. For each logical volume you set up, you’ll also need to create a new filesystem as follows:

mkfs.ext3 /dev/export/critical

See how the device filename works? The volume group is the second path component, and the logical volume name is the third.

Finally, once you’ve got your filesystems created, you just need to pick appropriate mount points and add entries to your /etc/fstab to get them assigned. I put pretty much everything in /export, since I’ll be “exporting” these filesystems via NFS later. One thing to note: it’s easiest if you don’t mount any of these inside each other, since NFS will get a little confused by that. Keep things simple. So, for instance, to add a mount point for our new “critical” volume, we do the following:

mkdir -p /export/critical
echo "/dev/export/critical /export/critical ext3 defaults,errors=remount-ro 0 1" >> /etc/fstab
mount /dev/export/critical

Easy enough. Next time I’ll show you how to get these things shared to other UNIX-based computers using NFS.

Written by jazzslider

January 11, 2009 at 10:43 am

Posted in Uncategorized

Tagged with , , ,

My Home NAS, Part 6: RAID Setup

leave a comment »

Now that my home NAS has its hard drives installed, it’s time to set up the RAID-1 array. As it turns out, this is pretty simple work. First, create a single Linux RAID Autodetect partition on each disk, taking up its entire usable space. You can do this by running fdisk /dev/sda; fdisk is pretty powerful, so just in case you’ve never done this before, I’ll walk you through the steps. The listing below shows you fdisk’s prompts, with the appropriate response in bold; press the {enter} key after each command, of course.

Command (m for help): o
{several lines of output from fdisk}

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
Partition number (1-4): 1
First cylinder (1-x, default 1): {just press enter to keep the default}
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-x, default x): {again, just press enter}
Using default value x

Command (m for help): t
Selected partition 1
Hex code (type L to list codes): fd
Changed system type of partition 1 to fd (Linux raid autodetect)

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

Repeat the process for your second disk (e.g., /dev/sdb). When you’ve finished with this process for both disks, you’ll have two new device special files in /dev, corresponding to the new partitions you created. These should be called /dev/sda1 and /dev/sdb1 (i.e., partition 1 on /dev/sdb). You’re now ready to create your RAID-1 array out of these partitions.

Make sure you’ve got the mdadm package installed (apt-get install mdadm); then, create your new RAID array as follows:

mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sd[ab]1

Like I said, dang simple. The arguments work as follows:

  • --create /dev/md0 tells mdadm that the device special filename for the new RAID device should be /dev/md0. This is conventional; I think you could name it something else if you wanted, but it’s best not to get too creative with this kind of thing.
  • --level=1 tells mdadm that this is a RAID-1 array. In a RAID-1 array, each disk is basically a copy of every other disk, getting you full redundancy in case one disk fails.
  • --raid-devices=2 tells mdadm that this array will have two devices in it, something which it could probably figure out from the fact that we’re about to tell it which two devices are in the array. But it never hurts to specify.
  • Finally, the flagless argument /dev/sd[ab]1 is actually expanded by the shell into two arguments, /dev/sda1 and /dev/sdb1 (that’s what the square brackets do; fun trick); these are the devices that will make up the array.

It’ll take awhile to finish setting up the array, but you can actually start using /dev/md0 right away (if, for instance, you want to move on to my next post about LVM setup). You can always monitor the new array at any point by running one of two commands; either cat /proc/mdstat or mdadm --detail /dev/md0 will give you useful information about how things are running. Once it’s had time to create the array, both of these commands should show you that the state of the array is clean. But that may take awhile, so be patient.

Now, things aren’t quite usable yet in this state. For reasons related to my backup policy (which I will explain later hopefully), I wanted to separate the array out into several distinct filesystems, each with a fixed size. Unfortunately, I’m out of time for now, so I’ll have to show you that process in another post.

Written by jazzslider

January 11, 2009 at 10:13 am

Posted in Uncategorized

Tagged with , ,