A Little Off Code, Computers, Photography and Guns

30Mar/093

Podcast Downloading on FreeNAS (Followup)

I've been following the referrals to my blog lately and I noticed the FreeNAS Podcast Downloader post was getting a lot of traffic but that the post wasn't descriptive enough in it's actual use. So I've got some instructions how to actually USE the project.

First off this will involve installing a package that isn't default for FreeNAS, if this worries you then skip down a step or two to the link to download all the project files manually. All this involves is installing subversion for checking out the code and all necessary files for the project to run. SSH into your FreeNAS server and execute this command:

1
pkg_add -r subversion

This will take a while to run so don't worry if it looks like it froze. You should reboot your server after this is finished for all the new settings to take effect. This is just so you can download the latest copy of the project.

Next step is to check out the project which can be done using the package you just installed. Choose a folder on your data partition for this to go because in an embedded install any other location would be overwritten on reboot. I made a directory in /mnt/Main/Content/.db/ (This is where I've chosen for all the databases for UPnP and DAAP to be stored along with the scripts for my server that I've written):

1
svn checkout http://svn.xp-dev.com/svn/bemasher-FreeNASPodder/ FreeNASPodder

This will check out the latest copy of the code to a new folder FreeNASPodder in /mnt/Main/Content/.db/

If you're uncomfortable with installing new packages to your server and just want a copy of the project you can simply browse to http://svn.xp-dev.com/svn/bemasher-FreeNASPodder and download each individual file manually. The benefit of using subversion is that getting the latest version of the file requires only browsing to the folder it resides in and executing:

1
svn update

Once this is all done you've checked out a working copy of the code. There are a few minor things you'll need to change in the script and in the configuration file. First in bashpodder.sh you'll need to change podcast_dir to the directory that you'd like your podcasts to be downloaded to, do the same in select_podcasts.sh. Make sure this folder exists as the script won't create it for you, it will only create folders for individual podcast feeds inside this folder. Be sure to escape spaces with a backslash.

The next change you'll need to make is in feeds.list. This will have a list of the url to each podcast feed you'd like to download episodes from. Be sure to keep one blank line at the very end of the file it will skip the last feed if you don't.

If you don't want to download all the current episodes all at once run select_podcasts.sh from the shell using:

1
sh select_podcasts.sh

This will create the basic folder structure that all of your podcasts will go into and compile episode.txt files for each podcast feed in their respective folders wherever you specified podcast_dir to be. To download a particular episode just remove the url for the episode you'd like to download from the episode.txt of the cooresponding podcast. Then simply run:

1
sh bashpodder.sh

If you'd like to execute this using cron in the web interface go to: System -> Advanced -> Cron. Click the add button and use the following command to check for and download new scripts however often you'd like:

1
sh /path/to/FreeNASPodder/bashpodder.sh

Followup:
Good news! I've moved all of my subversion repositories over to GitHub, the latest code for FreeNASPodder can be found at http://github.com/bemasher/FreeNASPodder. And even better news: you no longer have to install svn or download each final manually, GitHub has the option to download an archive in tgz or zip format of the latest source files.

Tagged as: , , , , , 3 Comments
5Mar/093

Podcast Downloading on FreeNAS.

I was thinking earlier this week about how files move too and from my file-server. I discovered that it's more convenient to have the server "pull" files to itself. This actually made me think a little bit more about podcast shows I watch on a pretty regular basis. I thought to myself "Wouldn't it be great if my server would get new episodes for me?". Once all that's done I can easily pull up the Podcasts directory on my friend's PS3 (which this will broadcast to using UPnP DLNA) and watch any new episodes that happen to be there.

I then set out almost immediately to figure out the simplest way to do this. I stumbled upon a script called BashPodder written by Linc. So I figured I'd just give it a try in it's original state. That turned out badly since I discovered that this was meant specifically for linux-based systems. My file-server runs FreeNAS a FreeBSD based OS. FreeBSD has an equivalent but different set of basic system tools. Fetch instead of wget, xml instead of xsltproc and so on.

After about 8 hours of learning my way around the basic set of FreeBSD tools and just generally refreshing myself on shell scripting I had heavily modified the BashPodder script to work on FreeNAS.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
#!/bin/bash

cd "`dirname "$0"`"
script_dir=`pwd`

podcast_dir=/mnt/Main/Content/TV\ Shows/Podcasts/

while read podcast
do
    cd "$script_dir"
    file=$(fetch -q -o - $podcast | xml tr parse_enclosure.xsl)
    cd "$podcast_dir"
    data_dir=$(fetch -q -o - $podcast | xml sel -T -t -v "//channel/title" | sed "s/ (.*)//")
    echo \"$data_dir\"
    mkdir -p "$data_dir"
    chown ftp:wheel "$data_dir"
    cd "$data_dir"
    for url in $file
    do
        filename=`echo $url | sed "s/^\(.*\)*\///g"`
        echo $filename
        if [ -z "`grep $url "episodes.txt"`" ]; then
            fetch -m -q $url
            echo $url >> "episodes.txt"
            if [ -n "`echo $filename | grep -v ".m4v"`" ]; then
                mv $filename `basename $filename m4v`mp4
                chown ftp:wheel `basename $filename m4v`mp4
            else
                chown ftp:wheel $filename
            fi
        fi
    done
done < feeds.list

/etc/rc.d/fuppes updatedb

I think the only thing I didn't end up changing at all was the xml stylesheet Linc had written since it works perfectly. The basic flow of this program is thus:

  1. Move into the directory that the script exists in.
  2. For each line in feeds.list do the following.
    1. Move into the scripts directory (there are files we need to parse the feed here).
    2. Get the rss feed.
    3. Parse out the title of the feed.
    4. Parse the urls for each episode in the feed.
    5. Move into the podcast directory.
    6. Make a directory of the same name as the title if necessary.
    7. Move into the directory we just made (or already exists).
    8. For each url we parsed do the following.
      1. Determine the filename from the url.
      2. If the episode is not in episodes.txt do the following.
        1. Download the episode.
        2. Add the episode to episodes.txt
        3. Change the owner to ftp. (We're running this as root).
  3. Update the Fuppes database.

After all that was done and over with I just added a new file extension to the Fuppes (UPnP DLNA server) configuration for telling the PS3 that m4v files are really just mp4 files.

Also I wrote a slightly modified version of the above script for just getting lists of episodes. This is useful because if I don't want to download every single episode in each feed (for when I add new feeds to the feeds list) all I have to do is remove the urls from the episodes.txt file that is generated and run the main script. The modification is done in the for loop that handles episodes of each feed:

1
2
3
filename=`echo $url | sed "s/^\(.*\)*\///g"`
echo $filename
grep -q $url "episodes.txt" || echo $url >> "episodes.txt"

So instead of downloading each file it just makes sure it's in the episodes.txt file. If you've got any suggestions for optimization of the script let me know!

If you'd like to check out a read-only copy of the script and necessary configurations:

1
svn checkout http://svn.xp-dev.com/svn/bemasher-FreeNASPodder/ FreeNASPodder

Keep in mind that the subversion is the most up-to-date copy I'll have available and it may often contain broken code or errors though I'll try to keep that down to a minimum.

Tagged as: , , , , , , , , , , , , , 3 Comments
   

QR Code

Blogroll

Comics

Info

Archives

Creative Commons License
www.bemasher.net by BeMasher is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.