A Little Off

Summer is usually a bad thing for my hardware and projects. Once I’ve gone through all the trouble of getting them setup and working the way I like I get bored and wonder what else I could do to//with¹ them.

My next project for my server² was to install pfSense³. I wasted most of my time on this project trying to do it the ways that were either not recommended or not documented.

Things like using unetbootin⁴ to run the LiveCD⁵ image since I have this strange hatred for optical media, it seems too wasteful to me, not to mention I rarely have the specific kind of media I need for the right project. That failed miserably of course since FreeBSD based LiveCD’s never seem to like the extraction and customization process unetbootin does to linux based iso’s.

After trying and failing to run the LiveCD from a thumb drive I did what I usually do to install OS’s that require optical media for installation: I use my IDE-USB adapter⁶ to chain it to the USB KVM⁷ I use so I don’t have to take my optical drive out to do installs. Lo and behold pfSense just happens to not support USB optical drives, it boots to the point where it would normally mount the iso9660⁸ formatted volume it expects to be at /dev/acd0 which isn’t because FreeBSD 7.0 doesn’t seem to have support for USB optical drives.

Anyway next in line was to use the embedded image. If you didn’t already know the main storage for my server is a SYBA SD-CF-2IDE-U adapter⁹ or simpler: a Compact Flash to IDE adapter. I figured this would be simple enough as I used to do the same thing for running FreeNAS on my server. I’d just pop the CF card into the multi-reader on my desktop and use dd¹⁰ to dump the embedded image that pfSense provides to it. My first instinct after getting the image downloaded was to decompress it since it was packaged as a GNU Zip file. Booting from this produced only a pipe character with a blinking cursor immediately beneath it. Reading through instructions further I discovered that the embedded version doesn’t have keyboard or video support, only serial which I don’t have on any of my systems anymore never mind that for once I have the proper cable for that. Also apparently I’m supposed to dump the compressed image to the card which produces no pipe character at all let alone a bootable card.

Clearly I was off to a good start¹¹. I finally gave up on the whole idea of doing it any other way than what was tried and true. This lead to me dismantling my desktop to use the only working optical drive I’ve got left and plug it into my server to install from the LiveCD. Also took me a good long time to find the manual for the IDE-CF adapter to figure out which jumper¹² needed to be changed so it would act as a slave. Once that was all said and done and the system booted to the LiveCD I ran the install to harddisk option. After formatting the disk, partitioning it and setting up appropriate slices it started the install. I wasn’t done yet with my troubles since it decided it was going to hang at 43%. Upon further investigation I noticed the light on the optical drive I was using wasn’t on or indicating any accessing at all. Found out that power had somehow disappeared from the drive, either through mechanical error¹³ or something else entirely. Rebooted and restarted the install once more, ended up disconnecting power and reconnecting it whilst installing. That seemed to do the trick as the install finished without further hitch excluding the fact that it wouldn’t install GRUB citing an error return code of 1 so ditched that idea and just used their default setting.

You’d think I would have been done with installing and general mucking about at the low-level end of this whole thing and you’d be wrong. After putting the optical drive back in my desktop and rebooting the server I notice it did the same thing it did when I tried to boot it from the optical drive plugged in through USB, it couldn’t find//mount the volume it thought the system was on. This is due to the CF adapter being a slave during the install and a master during boot after removing the optical drive. Instead of mucking around with the KVM switch and switching monitor display ports anymore¹⁴ I popped the CF card out of my server and into the CF port on the multi-reader in my desktop. Instead of using the USB filtering that VirtualBox has since it rarely ever works//tends to break anything I touch with it, I fired up a command prompt and created a vmdk that points to the physical disk. Mind you the command prompt must be run as administrator if you’re doing this in Windows 7 like I am. It will fail with a VERR_ACCESS_DENIED exception if you don’t.

After creating CF.vmdk I made a new virtual machine for it to go into along with mounting the pfSense LiveCD. Starting a command prompt with option 8 I mounted /dev/ad0s1a to /mnt and edited /mnt/etc/fstab to change ad1s1a and ad0s1b to ad0s1a and ad0s1b respectively.

Reboot and it is finished! After all that is simple web-interface based configuration that didn’t take very much time at all. Actually as I write this I am posting it through my new pfSense router.

1. Read: dismantle and start a new project [↩]
2. http://www.newegg.com/Product/Product.aspx?Item=N82E16813153062 JetWay J7F4K1G5D-PB, main component anyway. Any compatibility issues would be with this as there are no other peripherals at the moment. [↩]
3. http://www.pfsense.com/ pfSense is a free, open source customized distribution of FreeBSD tailored for use as a firewall and router. [↩]
4. http://unetbootin.sourceforge.net/ UNetbootin allows you to create bootable Live USB drives for a variety of Linux distributions from Windows or Linux, without requiring you to burn a CD. [↩]
5. LiveCD^W A LiveCD or LiveDVD is a CD or DVD containing a bootable computer operating system. [↩]
6. http://www.newegg.com/Product/Product.aspx?Item=N82E16812156101 Not sure if this is the exact one I have but looks to be about the same. [↩]
7. KVM Switch^W A KVM switch is a hardware device that allows a user to control multiple computers from a single keyboard, video monitor and mouse. [↩]
8. ISO 9660^W ISO 9660, also referred to as CDFS (CD File System) is a file system standard for optical disc media. [↩]
9. http://www.newegg.com/Product/Product.aspx?Item=N82E16822998002 Closest thing to what I’m using, they don’t seem to be making the revision I’m using anymore. [↩]
10. http://www.chrysocome.net/dd dd for Windows. [↩]
11. Read: Sarcasm [↩]
12. Which are descriptively named: jmp1, jmp2, jmp3 [↩]
13. I may or may not have been using a 12V molex to SATA power adapter in between. [↩]
14. Like I had been doing the entire time until this point. [↩]

Every couple of days I stop and take a look through the stats on my blog to see how things are progressing, and I notice plenty of times where people have used certain search terms to come to my blog. Sometimes the terms are very specific, sometimes they are very vague, but I always wonder the same thing: did they find what they were looking for?

I often times wish I could some how contact the person that made a certain query to ask them if they found what they wanted to find. I wonder if they took the time to read through the post i wrote that relates to their query and found their answer. I know how I read blogs and it’s often not very thorough, I figure that a majority of the time I don’t find what i’m looking for specifically because I just skim over it. I do admit though that I’m probably not making it very easy for the readers to find very specific information, there’s often a lot of fluff and cruft surrounding the important bits of information in my posts.

Google can you help me? I want to be able to answer the questions posed by your users’ queries. Yes I already know you’re going to tell me that that’s what comments are for but sadly not enough people use comments. I also already know you’ve implemented what you call a SearchWiki¹ and that seems to have failed miserably, but I like the idea, just wish I could contribute my own results and findings to other’s SearchWiki’s.

1. http://www.google.com/support/websearch/bin/answer.py?hl=en&answer=115764 SearchWiki lets you customize your Google Web Search results by ranking, removing, and adding notes to them. [↩]

One of the only things that kept me from installing Windows 7 permanently during the school year was that the few times I tried, I had never gotten MikTeX¹ to work. This of course was a major problem since nearly all of my assignments are done with MikTeX now. When installing MikTeX I always ran into a BSOD that I ignored because I figured that it was only because Windows 7 was only an RC².

I’ve had Windows 7 on my desktop now for about 2 weeks and up until this point I’ve been making due with the MikTeX Portable Edition which is pretty buggy to say the least. More than half of the time it would hang on compiling a document to pdf at something due to it not thinking initexmf.exe was an operable program. And of course upon googling this problem, nothing of use could be found.

Well tonight I decided I’d give the install another go to see if either Windows 7 had been patched to fix this issue, or if MikTeX had fixed the problem. On my first try, it did exactly as it had always done before BSOD’d. Since I had never set windows to not automatically restart upon catastrophic system failure³ it would just instantly restart without giving me enough time to read the type of error. I fixed this and ran the installer once more. The BSOD was a PAGE_FAULT_IN_NON_PAGED_AREA error, which was pretty vague as usual but I figured it had to do with system paging, so I disabled the Virtual Memory restarted and ran the installer once more. This time it worked exactly as it should.

On another interesting note, I discovered that pdflatex is significantly faster than texify. I found this out when I was trying different methods of compiling my TeX documents into pdf’s using the MikTeX Portable Edition which was giving me fits with my old method.

I used to use the following in NotePad++’s NppExec plugin to compile a pdf and view in Adobe Reader:

Now I use pdflatex:

1. MiKTeX (pronounced mick-tech) is an up-to-date implementation of TeX and related programs for Windows (all current variants). [↩]
2. Release Candidate [↩]
3. Read: Blue Screen Of Death [↩]

I’m kind of pathetic, I still have access to my workstation even when I’m on vacation and I needed to do a little bit of work while I was here. So I fired up Logmein only to discover it didn’t like any of the credentials I had tried to give it.

A quick google later and I found that Windows 7 RC requires the computer’s name be prefixed as the domain of the login credentials like so:

Lets suppose your username is bob and your computer’s name is bobs-computer. In order to login you would have to use bobs-computer\bob in for the username field then simply bob’s password for that username.

So I did manage to remote into my workstation to do some work and all is well.

If you’ve read back far enough here you’ll remember I was in the process of building my own NAS for media storage and backup. I was recently reading through the RSS feed from engadget and came across the Acer easyStore. While this is a lot more polished in the end than my home-brew NAS box it ended up being a little more expensive as well. The parts I used are as follows:

• Jetway VIA CN700 Mini-ITX $99.99
• Corsair 1GB DDR2 533 $19.99
• Syba IDE-CF adapter $12.99
• A-DATA 2GB CF Card $8.49
• Athena Power 3×3.5″ SATA Backplane $61.99
• Apex Mini-ITX Case + 250W PSU $55.99
• Promise SATA II Controller $59.99
• Western Digital Caviar SE16 640GB (QTY:3) $227.96 ($69.99 ea)

Ignoring extraneous things like shipping and sales tax the grand total is $547.39. If you take out the hard drives it was only $319. While the Acer Altos easyStore will be $400 (with one preinstalled 1TB drive). Granted Acer’s looks much prettier than mine and depending on the internals it might even have hardware raid where mine doesn’t. Though considering it’s got Windows Home Server installed i’m doubting very much that it has hardware raid.

When I first started building my NAS box atom processors had only just started to come out let alone be available in mini-itx packages like the one I bought. I could just as easily upgrade the motherboard in mine for somewhere between $80 and $120 which might bring new life to my system, but for what it does now and it’s general duties the Via C7 it’s got is good enough for me.

I think if I ever decided to get one of the Acer Altos easyStore NAS’s I’d just stick an IDE-CF adapter inside (assuming it’s got PATA) and load freeNAS onto that and use all 4 bays for storage instead of OS + Storage on the 4th drive.

In lieu of my Git vs. Mercurial post I’ve pretty much decided to stick with GitHub due to it’s extreme ease of migrating subversion repositories to it. While I’m sure it’s going to take me a little while to get used to the GUI for MSysGit (or find a better client) I’ve decided to move all of my current version control to GitHub.

You can find all my repositories here: http://github.com/bemasher/. Feel free to follow me on there, I’m always looking to meet other tech-friendly people.

A co-worker and I were talking about version control software and he made an interesting comparison: Git is like MacGuyver and Mercurial is like James Bond. He then proceeded to point me in the direction of GitHub and Bitbucket.

From first impression both seem to have the same basic set of tools and features. Both have built in simple wiki’s and issue trackers. Both allow unlimited public repositories limited only by disk space, 150MB for Mercurial and 300MB for GitHub. Mercurial allows one private repository while GitHub allows no private repositories without a paid plan.

Both of them have fatal flaws for Windows users. First off, GitHub is the most enticing because they offer an integrated subversion repository importing, just give it the URL to a SVN repository and you can import all the authors and history of it. Bitbucket however doesn’t seem to have this feature, I’ll continue looking for a simple way to do this but right now that’s a major flaw in Bitbucket because all of my repositories until this point have been on xp-dev a free SVN hosting service I talked about in my Free Subversion Hosting post.

The major fatal flaw in GitHub for Windows users is that there’s not really any solid Git clients to use with it at the moment. There’s MSysGit which provides a very basic GUI and command line tools which is great but not very integrated or simple to pick up and use. There’s a project for porting TortoiseSVN to TortoiseGit, though at the moment it heavily relies on MSysGit except it uses it’s own ssh client plink instead of the OpenSSH library that MSysGit uses which means running pageant to manage your keys and importing and creaking a ppk of your private keys for use with GitHub.

Mercurial seems to have a major fix for the main problem with Git’s poor selection of Windows clients is a major issue for usefulness in Windows. Mercurial wins this particular aspect of the competition. Mercurial has TortoiseHg which is pretty stable so far.

When it’s all said and done Bitbucket and GitHub are more or less equivalent services built on top of two different version control projects.

If you’re interested in the cross-platform-y-ness of python, you’ll probably find this interesting. Initially i’ve written a few programs for work in python. They needed to be more or less cross-platform and simple to use since it’s going to be distributed to users who have pretty much no idea how to do anything in the command line or anything of that nature.

My first experience with packaging python such that it could run on Windows desktops without installing the python interpreter before hand involved using a program called py2exe. Which does just what it says, it compiles all the dependencies and libraries of a particular python program into a single zip file along with a few basic things and generates an executable file for Windows based systems. The only drawback to this is that it only seems to work with Python 2.5 right now, it also has several extra files that must be distributed with the program and are really good at confusing users.

As for running python on Linux and OS X based systems it was a breeze, I had written a few very basic interfaces using Tkinter which is installed by default with python. The scripts by themselves run great on Linux and OS X systems since both come with them by default (RedHat Desktop is the Linux distro in question).

I still wanted an easier // more fool-proof way to distribute the scripts between computers with different flavors of OS’s without having to have a different version for each OS. A co-worker suggested I look at Jython which to my surprise happens to be awesome. Jython is a full implementation of python on the Java VM. This is great for me because ALL of the systems I’m writing this stuff for have Java. You’re probably wondering why I’m not just writing this stuff in Java to begin with and my answer to that is, these are really simple tasks that take way too much code//time to implement in Java when they took a few hours and about a 4th of the amount of code to perform the same task in python.

I discovered much to my surprise that it ended up being as simple as switching from Tkinter to Swing for the GUI’s and then fixing a few random things like OS detection for configuring default paths of certain system files before the code all ran flawlessly in Jython. Now onto the really fun part. I discovered that I could compile the code including all dependencies and libraries for the Jython interpreter into a single Jar file usually about 1MB total. All the user has to do is double-click on the jar and everything fires up and does it’s thing.

The first problem I ran into though was that OS X distributes their own version of the Java VM, 1.5.0_16 instead of the latest 1.6.0_13. I eventually discovered that it was as simple as adding the -target option to the java compiler and telling it to compile for the target version. Once all that was said and done I had a single jar file that contained all the necessary files to run the program by itself without any external dependencies.

If you’re interested in knowing what command I use to compile to jar it is as follows:

Mind you that I run this in Notepad++ on the current open file. $(NAME_PART) is the name portion of the filename excluding the extension. $(FULL_CURRENT_PATH) is you guessed it: the full current path including the filename and extension.

I was looking through my starred posts in Google Reader and found a post from a while ago. Turns out i can make a “Wordle” just by giving the rss feed of my blog to the tool it will process it. Another thing that makes this interesting is that I can change various settings to my liking.

I’ve been following the referrals to my blog lately and I noticed the FreeNAS Podcast Downloader post was getting a lot of traffic but that the post wasn’t descriptive enough in it’s actual use. So I’ve got some instructions how to actually USE the project.

First off this will involve installing a package that isn’t default for FreeNAS, if this worries you then skip down a step or two to the link to download all the project files manually. All this involves is installing subversion for checking out the code and all necessary files for the project to run. SSH into your FreeNAS server and execute this command:

This will take a while to run so don’t worry if it looks like it froze. You should reboot your server after this is finished for all the new settings to take effect. This is just so you can download the latest copy of the project.

Next step is to check out the project which can be done using the package you just installed. Choose a folder on your data partition for this to go because in an embedded install any other location would be overwritten on reboot. I made a directory in /mnt/Main/Content/.db/ (This is where I’ve chosen for all the databases for UPnP and DAAP to be stored along with the scripts for my server that I’ve written):

This will check out the latest copy of the code to a new folder FreeNASPodder in /mnt/Main/Content/.db/

If you’re uncomfortable with installing new packages to your server and just want a copy of the project you can simply browse to http://svn.xp-dev.com/svn/bemasher-FreeNASPodder and download each individual file manually. The benefit of using subversion is that getting the latest version of the file requires only browsing to the folder it resides in and executing:

Once this is all done you’ve checked out a working copy of the code. There are a few minor things you’ll need to change in the script and in the configuration file. First in bashpodder.sh you’ll need to change podcast_dir to the directory that you’d like your podcasts to be downloaded to, do the same in select_podcasts.sh. Make sure this folder exists as the script won’t create it for you, it will only create folders for individual podcast feeds inside this folder. Be sure to escape spaces with a backslash.

The next change you’ll need to make is in feeds.list. This will have a list of the url to each podcast feed you’d like to download episodes from. Be sure to keep one blank line at the very end of the file it will skip the last feed if you don’t.

If you don’t want to download all the current episodes all at once run select_podcasts.sh from the shell using:

This will create the basic folder structure that all of your podcasts will go into and compile episode.txt files for each podcast feed in their respective folders wherever you specified podcast_dir to be. To download a particular episode just remove the url for the episode you’d like to download from the episode.txt of the cooresponding podcast. Then simply run:

If you’d like to execute this using cron in the web interface go to: System -> Advanced -> Cron. Click the add button and use the following command to check for and download new scripts however often you’d like:

Followup:
Good news! I’ve moved all of my subversion repositories over to GitHub, the latest code for FreeNASPodder can be found at http://github.com/bemasher/FreeNASPodder/tree. And even better news: you no longer have to install svn or download each final manually, GitHub has the option to download an archive in tgz or zip format of the latest source files.