FreeNAS Users Rejoice!
Unetbootin[1] now supports FreeNAS! Take a look at these awesome little snippets of code:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 | //distrolst.cpp if (nameDistro == "FreeNAS") { if (isarch64) { cpuarch = "amd64"; } else { cpuarch = "i386"; } instIndvfl("memdisk", QString("%1ubnkern").arg(targetPath)); if (islivecd) { downloadfile(QString("http://downloads.sourceforge.net/sourceforge/lubi/FreeNAS-%1-LiveCD-%2.img.gz").arg(cpuarch, relname), QString("%1ubninit").arg(targetPath)); } else { downloadfile(QString("http://sourceforge.net/projects/freenas/files/stable/0.7/FreeNAS-%1-embedded-%2.img/download").arg(cpuarch, relname), QString("%1ubninit").arg(targetPath)); } } |
1 2 3 4 5 6 | //distrover.cpp distroselect->addItem("FreeNAS", (QStringList() << "0.7.4919" << unetbootin::tr("<b>Homepage:</b> <a href=\"http://freenas.org/\">http://www.freenas.org</a><br/>" "<b>Description:</b> FreeNAS is an embedded open source NAS (Network-Attached Storage) distribution based on FreeBSD.<br/>" "<b>Install Notes:</b> The LiveCD version creates a RAM drive for FreeNAS, and uses a FAT formatted floppy disk or USB key for saving the configuration file. The embedded version allows installation to hard disk.") << "0.7.4919" << "0.7.4919_x64" << "0.7.1.5024_Live" << "0.7.1.4997_Live_x64")); |
Segue:
I'm actually considering forking the unetbootin project to add support for a master distro list which can be updated remotely eliminating the requirement for users to download a new copy of the program if they wish to get the latest version of the list of pre-configured distros.
This has a little bit to do with the fact that I'll be required to take a few C++ courses at the University of Wyoming since Java is the standard language taught at the University of Arizona while I was there and I've never used C++ before. Can't be that hard right?
FreeNAS on Acer Altos easyStore H340
I've been reading more about the Acer easyStore and thinking about the possibility of installing FreeNAS on it instead of the preinstalled Windows Home Server. There's a multitude of problems associated with this but it still might be possible to kludge something together to make it work.
The first of these problems is the lack of a display port, this complicates things a little bit, but not terribly. First of all there would be no troubleshooting available if there were install troubles. However given that FreeNAS is based off of m0n0wall which is in turn based off of FreeBSD and it's designed with hardware compatibility in mind it would be easy enough to simply DD the embedded image to a CF card or some small flash storage and plug it in inside the server and boot it up without having to worry too much about making sure it installs all the right drivers.
This presents yet two more problems, both of which are closely related. Where are we going to plug in the flash that will have the image? It probably won't have a PATA//IDE port but we'd never know until we looked inside. And how can we make sure it boots from the right drive? There's no BIOS to jump into and configure since there's no display port and even no serial port for doing this remotely. The only solution I could come up with is this: Hope that there's a spare SATA port, which there likely isn't since it's designed for 4 drives and SATA ports usually come in pairs and since 4 % 2 == 0 we're sort of back to square one. Now if for some odd reason the motherboard rolled for this appliance just *happened* to have an unused SATA port our problem is pretty much solved. We could simply move cable of the bottom bay (which has the preinstalled drive with WHS) to another port and plug in a small SATA flash storage device which the system would already have been configured to boot from.
Once that's all said and done we could simply just go to the web interface of the FreeNAS install and configure it how we like. Though you could theoretically modify the configuration before hand in a virtual machine. Mount the storage device to a virtual machine, boot and make changes you deem necessary then simply shutdown and plug the storage device into the easyStore. All the new hardware would automatically be detected and all would be well with the setup.
One other thing I've been considering is the possibility of using one of the new Display Link USB video adapters. Although I'm very skeptical of the idea that this would work without drivers installed in a minimally functioning mode such as in the BIOS or on a simple terminal like what FreeNAS presents the user after booting. If it did work though, it would be easy enough to get one of these and use it for troubleshooting or modifying the BIOS settings if that's even possible in what looks to be a very locked down device.
Now lets just say that the DisplayLink device would work in a very simple mode and we could indeed access a BIOS to change boot order, lets assume that the board has a mini PCI-E expansion slot in it for kicks since quite a few Atom boards do (and Acer might have been too lazy to just exclude it's installation on what i assume is a custom board). We could put in a PCI-E SSD like this one and configure the BIOS to boot from that. But I have a feeling that's just wishful thinking and we'd be better off hoping that there would be an extra SATA port. You could also theoretically just use a thumbdrive assuming the BIOS would allow you to boot from a removable USB device. Again probably just wishful thinking.
Acer easyStore vs. My homebrew
If you've read back far enough here you'll remember I was in the process of building my own NAS for media storage and backup. I was recently reading through the RSS feed from engadget and came across the Acer easyStore. While this is a lot more polished in the end than my home-brew NAS box it ended up being a little more expensive as well. The parts I used are as follows:
- Jetway VIA CN700 Mini-ITX $99.99
- Corsair 1GB DDR2 533 $19.99
- Syba IDE-CF adapter $12.99
- A-DATA 2GB CF Card $8.49
- Athena Power 3x3.5" SATA Backplane $61.99
- Apex Mini-ITX Case + 250W PSU $55.99
- Promise SATA II Controller $59.99
- Western Digital Caviar SE16 640GB (QTY:3) $227.96 ($69.99 ea)
Ignoring extraneous things like shipping and sales tax the grand total is $547.39. If you take out the hard drives it was only $319. While the Acer Altos easyStore will be $400 (with one preinstalled 1TB drive). Granted Acer's looks much prettier than mine and depending on the internals it might even have hardware raid where mine doesn't. Though considering it's got Windows Home Server installed i'm doubting very much that it has hardware raid.
When I first started building my NAS box atom processors had only just started to come out let alone be available in mini-itx packages like the one I bought. I could just as easily upgrade the motherboard in mine for somewhere between $80 and $120 which might bring new life to my system, but for what it does now and it's general duties the Via C7 it's got is good enough for me.
I think if I ever decided to get one of the Acer Altos easyStore NAS's I'd just stick an IDE-CF adapter inside (assuming it's got PATA) and load freeNAS onto that and use all 4 bays for storage instead of OS + Storage on the 4th drive.
Podcast Downloading on FreeNAS (Followup)
I've been following the referrals to my blog lately and I noticed the FreeNAS Podcast Downloader post was getting a lot of traffic but that the post wasn't descriptive enough in it's actual use. So I've got some instructions how to actually USE the project.
First off this will involve installing a package that isn't default for FreeNAS, if this worries you then skip down a step or two to the link to download all the project files manually. All this involves is installing subversion for checking out the code and all necessary files for the project to run. SSH into your FreeNAS server and execute this command:
1 | pkg_add -r subversion |
This will take a while to run so don't worry if it looks like it froze. You should reboot your server after this is finished for all the new settings to take effect. This is just so you can download the latest copy of the project.
Next step is to check out the project which can be done using the package you just installed. Choose a folder on your data partition for this to go because in an embedded install any other location would be overwritten on reboot. I made a directory in /mnt/Main/Content/.db/ (This is where I've chosen for all the databases for UPnP and DAAP to be stored along with the scripts for my server that I've written):
1 | svn checkout http://svn.xp-dev.com/svn/bemasher-FreeNASPodder/ FreeNASPodder |
This will check out the latest copy of the code to a new folder FreeNASPodder in /mnt/Main/Content/.db/
If you're uncomfortable with installing new packages to your server and just want a copy of the project you can simply browse to http://svn.xp-dev.com/svn/bemasher-FreeNASPodder and download each individual file manually. The benefit of using subversion is that getting the latest version of the file requires only browsing to the folder it resides in and executing:
1 | svn update |
Once this is all done you've checked out a working copy of the code. There are a few minor things you'll need to change in the script and in the configuration file. First in bashpodder.sh you'll need to change podcast_dir to the directory that you'd like your podcasts to be downloaded to, do the same in select_podcasts.sh. Make sure this folder exists as the script won't create it for you, it will only create folders for individual podcast feeds inside this folder. Be sure to escape spaces with a backslash.
The next change you'll need to make is in feeds.list. This will have a list of the url to each podcast feed you'd like to download episodes from. Be sure to keep one blank line at the very end of the file it will skip the last feed if you don't.
If you don't want to download all the current episodes all at once run select_podcasts.sh from the shell using:
1 | sh select_podcasts.sh |
This will create the basic folder structure that all of your podcasts will go into and compile episode.txt files for each podcast feed in their respective folders wherever you specified podcast_dir to be. To download a particular episode just remove the url for the episode you'd like to download from the episode.txt of the cooresponding podcast. Then simply run:
1 | sh bashpodder.sh |
If you'd like to execute this using cron in the web interface go to: System -> Advanced -> Cron. Click the add button and use the following command to check for and download new scripts however often you'd like:
1 | sh /path/to/FreeNASPodder/bashpodder.sh |
Followup:
Good news! I've moved all of my subversion repositories over to GitHub, the latest code for FreeNASPodder can be found at http://github.com/bemasher/FreeNASPodder. And even better news: you no longer have to install svn or download each final manually, GitHub has the option to download an archive in tgz or zip format of the latest source files.
Socks FTW!
Well, not the fabric-y variety of SOCKS, the proxy variety. My friend Pete mentioned at some point a few weeks ago that I was "doing it the wrong" when I was doing some SSH tunneling to my FreeNAS server.
The original setup involved tunneling port 80 and a few other random ports for things like the web interfaces of the DAAP and UPnP servers. Which was a bit of a pain to add each time I wanted to tunnel the ports on a new computer whether it was my laptop, desktop or workstation. I could save the profile and just use that though it was a pain to be thorough enough to include all the ports I'd need. At the time Pete mentioned something about a SOCKS proxy which I didn't really understand and originally thought it would involve installing new software somewhere which I didn't really want to do at the time.
After a little bit of research later I found that one of the most common SSH servers OpenSSH has a built in SOCKS proxy. This is great because it meant installing no new software or even any new configurations for that matter. In Putty I discovered it was as simple as adding a dynamic port of my choice (1080 is probably best here since it's the default SOCKS proxy port). Once I did this I installed FoxyProxy plugin for firefox and was on my way to a quick and simple proxy to any service I desired on my FreeNAS server.
To do this from a command line is just as simple:
1 | ssh -D1080 user@example.com |
A quick way to test this is to go to a "what is my IP" site of some variety then connect to the server with the SOCKS proxy enabled and setup your browser with that proxy and reload the page, if your external IP changes to the external IP of the server you just ssh'd into then you've done it.
Podcast Downloading on FreeNAS.
I was thinking earlier this week about how files move too and from my file-server. I discovered that it's more convenient to have the server "pull" files to itself. This actually made me think a little bit more about podcast shows I watch on a pretty regular basis. I thought to myself "Wouldn't it be great if my server would get new episodes for me?". Once all that's done I can easily pull up the Podcasts directory on my friend's PS3 (which this will broadcast to using UPnP DLNA) and watch any new episodes that happen to be there.
I then set out almost immediately to figure out the simplest way to do this. I stumbled upon a script called BashPodder written by Linc. So I figured I'd just give it a try in it's original state. That turned out badly since I discovered that this was meant specifically for linux-based systems. My file-server runs FreeNAS a FreeBSD based OS. FreeBSD has an equivalent but different set of basic system tools. Fetch instead of wget, xml instead of xsltproc and so on.
After about 8 hours of learning my way around the basic set of FreeBSD tools and just generally refreshing myself on shell scripting I had heavily modified the BashPodder script to work on FreeNAS.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 | #!/bin/bash cd "`dirname "$0"`" script_dir=`pwd` podcast_dir=/mnt/Main/Content/TV\ Shows/Podcasts/ while read podcast do cd "$script_dir" file=$(fetch -q -o - $podcast | xml tr parse_enclosure.xsl) cd "$podcast_dir" data_dir=$(fetch -q -o - $podcast | xml sel -T -t -v "//channel/title" | sed "s/ (.*)//") echo \"$data_dir\" mkdir -p "$data_dir" chown ftp:wheel "$data_dir" cd "$data_dir" for url in $file do filename=`echo $url | sed "s/^\(.*\)*\///g"` echo $filename if [ -z "`grep $url "episodes.txt"`" ]; then fetch -m -q $url echo $url >> "episodes.txt" if [ -n "`echo $filename | grep -v ".m4v"`" ]; then mv $filename `basename $filename m4v`mp4 chown ftp:wheel `basename $filename m4v`mp4 else chown ftp:wheel $filename fi fi done done < feeds.list /etc/rc.d/fuppes updatedb |
I think the only thing I didn't end up changing at all was the xml stylesheet Linc had written since it works perfectly. The basic flow of this program is thus:
- Move into the directory that the script exists in.
- For each line in feeds.list do the following.
- Move into the scripts directory (there are files we need to parse the feed here).
- Get the rss feed.
- Parse out the title of the feed.
- Parse the urls for each episode in the feed.
- Move into the podcast directory.
- Make a directory of the same name as the title if necessary.
- Move into the directory we just made (or already exists).
- For each url we parsed do the following.
- Determine the filename from the url.
- If the episode is not in episodes.txt do the following.
- Download the episode.
- Add the episode to episodes.txt
- Change the owner to ftp. (We're running this as root).
- Update the Fuppes database.
After all that was done and over with I just added a new file extension to the Fuppes (UPnP DLNA server) configuration for telling the PS3 that m4v files are really just mp4 files.
Also I wrote a slightly modified version of the above script for just getting lists of episodes. This is useful because if I don't want to download every single episode in each feed (for when I add new feeds to the feeds list) all I have to do is remove the urls from the episodes.txt file that is generated and run the main script. The modification is done in the for loop that handles episodes of each feed:
1 2 3 | filename=`echo $url | sed "s/^\(.*\)*\///g"` echo $filename grep -q $url "episodes.txt" || echo $url >> "episodes.txt" |
So instead of downloading each file it just makes sure it's in the episodes.txt file. If you've got any suggestions for optimization of the script let me know!
If you'd like to check out a read-only copy of the script and necessary configurations:
1 | svn checkout http://svn.xp-dev.com/svn/bemasher-FreeNASPodder/ FreeNASPodder |
Keep in mind that the subversion is the most up-to-date copy I'll have available and it may often contain broken code or errors though I'll try to keep that down to a minimum.
Mini ITX Shuffle
Recently, even with my current lack of money I've been ogling an Intel mini ITX board at newegg.com. The board has an Atom 330 embedded as well as a 100/1000 NIC.
The reason that I bring this up is that I watched an episode of Hak5 which talked about pfSense a freeBSD based firewall//router OS (a fork of M0n0wall). I've used pfSense before and found it to be awesome I've just never had the hardware to use it as a permanent solution. However now I think I have the perfect set of hardware.
For a while I've been planning to upgrade the motherboard//proc on my file server which is running FreeNAS another freeBSD based OS (also based off of M0n0wall) meant for network attached storage systems. The board that the fileserver is running on now is a JetWay VIA C7 mini ITX board which does it's duty decently well. The JetWay board has dual-gigabit NIC's which is exactly what I'd like to use for a router box. So the plan is that once I've got the money for doing all this I'll be buying the Intel Atom 330 mini ITX board and replacing the JetWay board in my fileserver with the intel board while using the JetWay board for the router.
I figure it will work well enough because for the router I'll need at least 2 NIC's (which the JetWay has) and for the fileserver I'll only really need the one NIC. Either way it's a win-win situation, my fileserver gets a little beefier while I get guts for a Router!
QR Code
Blogroll
Comics
Info
Archives
www.bemasher.net by BeMasher is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.