A Little Off Code, Computers, Photography and Guns

3Jul/090

Installing pfSense on Jetway J7F4K1G5D-PB

Summer is usually a bad thing for my hardware and projects. Once I've gone through all the trouble of getting them setup and working the way I like I get bored and wonder what else I could do to//with[1] them.

My next project for my server[2] was to install pfSense[3]. I wasted most of my time on this project trying to do it the ways that were either not recommended or not documented.

Things like using unetbootin[4] to run the LiveCD[5] image since I have this strange hatred for optical media, it seems too wasteful to me, not to mention I rarely have the specific kind of media I need for the right project. That failed miserably of course since FreeBSD based LiveCD's never seem to like the extraction and customization process unetbootin does to linux based iso's.

After trying and failing to run the LiveCD from a thumb drive I did what I usually do to install OS's that require optical media for installation: I use my IDE-USB adapter[6] to chain it to the USB KVM[7] I use so I don't have to take my optical drive out to do installs. Lo and behold pfSense just happens to not support USB optical drives, it boots to the point where it would normally mount the iso9660[8] formatted volume it expects to be at /dev/acd0 which isn't because FreeBSD 7.0 doesn't seem to have support for USB optical drives.

Anyway next in line was to use the embedded image. If you didn't already know the main storage for my server is a SYBA SD-CF-2IDE-U adapter[9] or simpler: a Compact Flash to IDE adapter. I figured this would be simple enough as I used to do the same thing for running FreeNAS on my server. I'd just pop the CF card into the multi-reader on my desktop and use dd[10] to dump the embedded image that pfSense provides to it. My first instinct after getting the image downloaded was to decompress it since it was packaged as a GNU Zip file. Booting from this produced only a pipe character with a blinking cursor immediately beneath it. Reading through instructions further I discovered that the embedded version doesn't have keyboard or video support, only serial which I don't have on any of my systems anymore never mind that for once I have the proper cable for that. Also apparently I'm supposed to dump the compressed image to the card which produces no pipe character at all let alone a bootable card.

Clearly I was off to a good start[11]. I finally gave up on the whole idea of doing it any other way than what was tried and true. This lead to me dismantling my desktop to use the only working optical drive I've got left and plug it into my server to install from the LiveCD. Also took me a good long time to find the manual for the IDE-CF adapter to figure out which jumper[12] needed to be changed so it would act as a slave. Once that was all said and done and the system booted to the LiveCD I ran the install to harddisk option. After formatting the disk, partitioning it and setting up appropriate slices it started the install. I wasn't done yet with my troubles since it decided it was going to hang at 43%. Upon further investigation I noticed the light on the optical drive I was using wasn't on or indicating any accessing at all. Found out that power had somehow disappeared from the drive, either through mechanical error[13] or something else entirely. Rebooted and restarted the install once more, ended up disconnecting power and reconnecting it whilst installing. That seemed to do the trick as the install finished without further hitch excluding the fact that it wouldn't install GRUB citing an error return code of 1 so ditched that idea and just used their default setting.

You'd think I would have been done with installing and general mucking about at the low-level end of this whole thing and you'd be wrong. After putting the optical drive back in my desktop and rebooting the server I notice it did the same thing it did when I tried to boot it from the optical drive plugged in through USB, it couldn't find//mount the volume it thought the system was on. This is due to the CF adapter being a slave during the install and a master during boot after removing the optical drive. Instead of mucking around with the KVM switch and switching monitor display ports anymore[14] I popped the CF card out of my server and into the CF port on the multi-reader in my desktop. Instead of using the USB filtering that VirtualBox has since it rarely ever works//tends to break anything I touch with it, I fired up a command prompt and created a vmdk that points to the physical disk. Mind you the command prompt must be run as administrator if you're doing this in Windows 7 like I am. It will fail with a VERR_ACCESS_DENIED exception if you don't.

1
VBoxManage.exe internalcommands createrawvmdk -filename CF.vmdk -rawdisk \\.\PhysicalDrive3

After creating CF.vmdk I made a new virtual machine for it to go into along with mounting the pfSense LiveCD. Starting a command prompt with option 8 I mounted /dev/ad0s1a to /mnt and edited /mnt/etc/fstab to change ad1s1a and ad0s1b to ad0s1a and ad0s1b respectively.

1
2
mount ufs:/dev/ad0s1a /mnt
vI /mnt/etc/fstab

Reboot and it is finished! After all that is simple web-interface based configuration that didn't take very much time at all. Actually as I write this I am posting it through my new pfSense router.

  1. Read: dismantle and start a new project [↩]
  2. http://www.newegg.com/Product/Product.aspx?Item=N82E16813153062 JetWay J7F4K1G5D-PB, main component anyway. Any compatibility issues would be with this as there are no other peripherals at the moment. [↩]
  3. http://www.pfsense.com/ pfSense is a free, open source customized distribution of FreeBSD tailored for use as a firewall and router. [↩]
  4. http://unetbootin.sourceforge.net/ UNetbootin allows you to create bootable Live USB drives for a variety of Linux distributions from Windows or Linux, without requiring you to burn a CD. [↩]
  5. LiveCD: A LiveCD or LiveDVD is a CD or DVD containing a bootable computer operating system. [↩]
  6. http://www.newegg.com/Product/Product.aspx?Item=N82E16812156101 Not sure if this is the exact one I have but looks to be about the same. [↩]
  7. KVM Switch: A KVM switch is a hardware device that allows a user to control multiple computers from a single keyboard, video monitor and mouse. [↩]
  8. ISO 9660: also referred to as CDFS (CD File System) is a file system standard for optical disc media. [↩]
  9. http://www.newegg.com/Product/Product.aspx?Item=N82E16822998002 Closest thing to what I'm using, they don't seem to be making the revision I'm using anymore. [↩]
  10. http://www.chrysocome.net/dd dd for Windows. [↩]
  11. Read: Sarcasm [↩]
  12. Which are descriptively named: jmp1, jmp2, jmp3 [↩]
  13. I may or may not have been using a 12V molex to SATA power adapter in between. [↩]
  14. Like I had been doing the entire time until this point. [↩]
Tagged as: , , , , , , , , No Comments
5Mar/093

Podcast Downloading on FreeNAS.

I was thinking earlier this week about how files move too and from my file-server. I discovered that it's more convenient to have the server "pull" files to itself. This actually made me think a little bit more about podcast shows I watch on a pretty regular basis. I thought to myself "Wouldn't it be great if my server would get new episodes for me?". Once all that's done I can easily pull up the Podcasts directory on my friend's PS3 (which this will broadcast to using UPnP DLNA) and watch any new episodes that happen to be there.

I then set out almost immediately to figure out the simplest way to do this. I stumbled upon a script called BashPodder written by Linc. So I figured I'd just give it a try in it's original state. That turned out badly since I discovered that this was meant specifically for linux-based systems. My file-server runs FreeNAS a FreeBSD based OS. FreeBSD has an equivalent but different set of basic system tools. Fetch instead of wget, xml instead of xsltproc and so on.

After about 8 hours of learning my way around the basic set of FreeBSD tools and just generally refreshing myself on shell scripting I had heavily modified the BashPodder script to work on FreeNAS.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
#!/bin/bash

cd "`dirname "$0"`"
script_dir=`pwd`

podcast_dir=/mnt/Main/Content/TV\ Shows/Podcasts/

while read podcast
do
    cd "$script_dir"
    file=$(fetch -q -o - $podcast | xml tr parse_enclosure.xsl)
    cd "$podcast_dir"
    data_dir=$(fetch -q -o - $podcast | xml sel -T -t -v "//channel/title" | sed "s/ (.*)//")
    echo \"$data_dir\"
    mkdir -p "$data_dir"
    chown ftp:wheel "$data_dir"
    cd "$data_dir"
    for url in $file
    do
        filename=`echo $url | sed "s/^\(.*\)*\///g"`
        echo $filename
        if [ -z "`grep $url "episodes.txt"`" ]; then
            fetch -m -q $url
            echo $url >> "episodes.txt"
            if [ -n "`echo $filename | grep -v ".m4v"`" ]; then
                mv $filename `basename $filename m4v`mp4
                chown ftp:wheel `basename $filename m4v`mp4
            else
                chown ftp:wheel $filename
            fi
        fi
    done
done < feeds.list

/etc/rc.d/fuppes updatedb

I think the only thing I didn't end up changing at all was the xml stylesheet Linc had written since it works perfectly. The basic flow of this program is thus:

  1. Move into the directory that the script exists in.
  2. For each line in feeds.list do the following.
    1. Move into the scripts directory (there are files we need to parse the feed here).
    2. Get the rss feed.
    3. Parse out the title of the feed.
    4. Parse the urls for each episode in the feed.
    5. Move into the podcast directory.
    6. Make a directory of the same name as the title if necessary.
    7. Move into the directory we just made (or already exists).
    8. For each url we parsed do the following.
      1. Determine the filename from the url.
      2. If the episode is not in episodes.txt do the following.
        1. Download the episode.
        2. Add the episode to episodes.txt
        3. Change the owner to ftp. (We're running this as root).
  3. Update the Fuppes database.

After all that was done and over with I just added a new file extension to the Fuppes (UPnP DLNA server) configuration for telling the PS3 that m4v files are really just mp4 files.

Also I wrote a slightly modified version of the above script for just getting lists of episodes. This is useful because if I don't want to download every single episode in each feed (for when I add new feeds to the feeds list) all I have to do is remove the urls from the episodes.txt file that is generated and run the main script. The modification is done in the for loop that handles episodes of each feed:

1
2
3
filename=`echo $url | sed "s/^\(.*\)*\///g"`
echo $filename
grep -q $url "episodes.txt" || echo $url >> "episodes.txt"

So instead of downloading each file it just makes sure it's in the episodes.txt file. If you've got any suggestions for optimization of the script let me know!

If you'd like to check out a read-only copy of the script and necessary configurations:

1
svn checkout http://svn.xp-dev.com/svn/bemasher-FreeNASPodder/ FreeNASPodder

Keep in mind that the subversion is the most up-to-date copy I'll have available and it may often contain broken code or errors though I'll try to keep that down to a minimum.

Tagged as: , , , , , , , , , , , , , 3 Comments
   

QR Code

Blogroll

Comics

Info

Archives

RSS Recommended Reading