Installing Ubuntu via Network
At some point in the last 6 months or so I may or may not have accidentally left my 1GB Sandisk Cruzer in a pair of jeans when they went through the washer AND the dryer. As such it's not exactly in peak physical condition[1] and for whatever reason I've had issues with using it for installing certain things[2] lately[3].
Anyway it has become time again to get my file server back up and running and I needed to reinstall Ubuntu on it. Given my extreme laziness when it comes to doing this sort of stuff I was in no mood to move everything to the top of my desktop[4] so I opted to try pxe booting[5] again.
I've messed with pxe booting in the past, particularly with GeeXboX[6] for my media center and that was a nightmare at the time and essentially required you to have a linux system in order to do it. Since then a wonderful application has made its way into the internet: tftpd32[7]. Tftpd32 greatly simplifies the whole process by not requiring you to install anything or make any major system changes.
Before you continue take note, these instructions assume a few things:
- You're serving the netboot images from a windows system.
- You have a tomato based router, although these instructions can be easily modified to work with any router firmware that uses DNSMasq or allows you to change advanced settings for the DHCP server.
Things you'll need:
- Ubuntu Alternative ISO: This will be used for setting up the local http repository.
- Ubuntu NetBoot Image: Grab netboot.tar.gz
- tftpd32: This will be used for serving files during PXE booting.
- HFS ~ HTTP File Server: This will be used for setting up a local http repository for installing from our local network instead of having ubuntu download everything from a mirror.
Router Settings:
- Advanced -> DHCP / DNS -> Dnsmasq Custom Configuration
- dhcp-boot=pxelinux.0,,[tftpd32 server ip address]
- Save.
For ease of readability from this point forward files will be bolded and directories will be italicized.
- Untar netboot.tar.gz into a folder, which I'll refer to as netboot from now on.
- Delete pxelinux.0 and pxelinux.cfg from netboot/ as these are symlinks which will not work in windows.
- Create the directory netboot/pxelinux.cfg/
- Copy pxelinux.0 from netboot/ubuntu-installer/i386/ to netboot/
- Copy sysconfig.cfg from netboot/unbuntu-installer/i386/boot-screens/ to netboot/pxelinux.cfg/
- Rename netboot/pxelinux.cfg/sysconfig.cfg to netboot/pxelinux.cfg/default
Preparing tftpd32:
- Run tftpd32
- Browse to the netboot folder we just finished setting up.
- Tftpd32 should be serving the files in that directory at this point.
Preparing the local HTTP Ubuntu Repository:
- Run HFS.exe
- Extract all of the files from ubuntu-10.10-alternate-i386.iso to a folder which I'll refer to as ubuntu-alt from this point on.
- In the Virtual File System pane right click -> Add Folder from disk...
- Browse to and select ubuntu-alt
- When HFS prompts you to ask what kind of folder it should be added as, select Real Folder
- Note the link in the address bar next to Open in browser, you'll use this link when installing ubuntu.
Installing Ubuntu:
- Boot the system you're attempting to install Ubuntu on from your network device.
- If you have tftpd32 up on another monitor at this point you should see a deluge of requests in the tftp server tab.
- Ubuntu should show a boot menu select install.
- Now I'm not going to go into full detail on how to install Ubuntu but when you get to mirror selection at the very top of the list there should be the option to enter a mirror manually this is where you should enter the address from the address bar in HFS, be sure to also include the port value.
- If all goes well it should start installing and you should see another deluge of requests in HFS.
- In fact it's pretty far from peak physical condition. [↩]
- Like ubuntu for example. [↩]
- I'm not entirely sure if this is due to washing it or just from it being nearly 5 years old. [↩]
- So the cable for the USB adapter I've got my DVD drive connected to in my desktop can reach my mini-itx board. [↩]
- Preboot Execution Environment [↩]
- GeeXboX [↩]
- tftpd32: An open-source tftp//dhcp//syslog server for Windows. [↩]
Coping with a puny outbound bandwidth limit.
If you were unaware I recently moved to Laramie, WY. Which is a significantly smaller town//city//something-or-other than Tucson, AZ is by quite a lot. Anyway if you didn't know Wyoming existed there are a few basic facts about Wyoming I should explain so you'll know what I'm dealing with here.
Wyoming is very very spacious and very sparse. Everything here is incredibly spread out. As such laying cable for connecting cities is generally expensive and still in the early stages of deployment. It is still possible to get broadband cable in the cities but if you don't live in the city and want broadband you're stuck with WiMax[1] like wireless broadband which is pretty wide-spread here and effective since the terrain in most areas of the state permit line-of-sight wireless connections.
I went the broadband cable route since all of the experience I've had with Wifi stuff has been unreliable and I'm just a big fan of having a cable from one point to the next, there is a lot less downtime and possibility for interference.
Unfortunately having chosen broadband cable left me with two providers, Qwest and Bresnan. Qwest limits me to 3Mbps down and what i'm guessing is no more than 384Kbps up. Bresnan on the other hand provides 8Mbps/500Kbps which is better.
I'm consistently getting more and more frustrated with the emphasis broadband cable providers put on download speeds, upload speeds are getting to be just as important if not more important than inbound bandwidth. People are starting to learn that it's much much easier to share media and personal data online instead of the alternatives. By comparison I moved from Tucson where we had 20Mbps/3Mbps.
Now you can see where my problem is. 50KB/s up is pretty miserable compared to what I used to have. And you can imagine where my congestion comes in. If I saturate my outbound bandwidth Bresnan essentially ruins my inbound bandwidth. I recently discovered that Tomato the firmware I've installed on my router comes with TCP Vegas[2] integrated.
Initially I couldn't find any proper documentation on what alpha, beta and gamma meant and how to tune them properly to help control my congestion problem. After several hours of searching and research I found only two configurations that seemed to provide people with some measure of congestion prevention: 3,3,2 and 2,6,2[3]
So I switched TCP Vegas on while downloading the first Pioneer One episode[4] and reset all the connections since TCP Vegas only affects connections created since enabled. Immediately I noticed the saw-tooth that TCP Reno[5] causes disappeared, TCP Vegas immediately smoothed out the connection and allowed for a sustained outbound rate with an oscillating inbound rate around the maximum throughput achievable while maintaining sustained outbound traffic. I've read that tuning TCP Vegas properly can eliminate the oscillation of inbound traffic but I haven't had much luck with actually getting this done.
Once TCP Vegas began to smooth things out I figured out that in order to optimize inbound throughput I could simply limit the outbound rate to the magical 70% of outbound capacity recommended for P2P apps now that congestion was no longer an issue. This increased the inbound rate significantly and everything has been running smoothly since then. I wish I knew more about the algorithm used and how to tune it but there doesn't seem to be much in the way of documentation anywhere.
- WiMax: Worldwide Interoperability for Microwave Access [↩]
- TCP Vegas: a TCP congestion control, or network congestion avoidance, algorithm that emphasizes packet delay, rather than packet loss, as a signal to help determine the rate at which to send packets. It was developed at the University of Arizona by Lawrence Brakmo and Larry L. Peterson. [↩]
- Alpha, Beta and Gamma respectively. [↩]
- A free TV Series designed specifically for P2P distribution. [↩]
- TCP Reno: a TCP Congestion Avoidance Algorithm [↩]
Open-Source Print Server
One of the things I had always wanted to play with until I moved was a wireless router that was "deluxe" enough to have a USB port installed.
Most of the time these sorts of routers expect that you're just going to plug an external hard drive into it and pretend like it's a full-fledged NAS becuase typically they manage to squeeze CIFS into the firmware that governs the router and for the most part this is good enough for the most basic of NAS needs. Except they tend to forget that throughput with such a simple processor is usually miserable. I've never actually tested any of this myself but all of the forums I've read on the subject seem to report that there is a throughput ceiling of around 5-6MB/s which is pretty bad.
The other purpose for the USB port that I bet people would get more use out of is the ability to use your router as a simple print server. The first reason this is a lot more practical is that nearly all consumer printers these days use USB for their interface. This means that essentially any printer that can be made to work for your current computer with USB can also be put behind a print server without too much trouble. The other way this is useful is for the growing number of laptops you'll find in any given household. Except for minority of people that do enough computer work that owning a desktop is a necessity most every house will mostly be comprised of laptops and all of them will likely communicate with Wifi.
Every time you need to print something from your laptop you'll have to take it back to wherever you've parked your printer and plug it in. But you'll also need to make sure that you plug it back into the same USB port you installed it with in the first place or your computer will go into full-retard mode and "Duhhh... new printer! I'll look for drivers for your awesome new printer."
Now you see where the real hassle is. But this is the point at which a lot of people go the wrong direction. The first thought a lot of people have[1] is that they'll just solve this problem by throwing money at it which usually means shopping for and buying a printer with network capabilities built in.
The main problem with this is that, network attached printers are expensive, quite a lot more expensive than their non-network-aware brethren. The secondary problem is that a grand majority of home networks are made possible by DHCP[2] which divvies out IP addresses as devices connect to the network. This presents a problem when the main method for connecting to network attached printers involves knowing the printer's IP address, which can be problematic when your router arbitrarily hands out IP addresses on a regular basis. Every time the lease is up for your printer's IP address there is the possibility of that printer to get a new IP address which causes issues. So unless you are tech-savvy enough to setup static DHCP leases this will cause problems.
The next option for a lot of people is to shoehorn round peg into a square hole by buying a print server to make their current printer network friendly. See the above problem for why this isn't an optimal solution.
In comes the router with a USB port. I recently purchased a Netgear WNR3500L[3] from newegg. If you're interested in my adventurous experience with installing Tomato on it see my previous[4] post about that.
Now you're probably wondering how on earth a router with a USB port is any better than a print server or a network attached printer. The reason it is better is that in typical household networks the router will have the same IP address no matter how you've got DHCP configured. For the most part your router lives at 192.168.1.1 or 192.168.0.1 and this will for most situations never change.
So given a router with a USB port and proper firmware to allow for print serving you can host your printer on your router which will almost always have the same IP address and you can do this all with one device instead of having to buy a separate device.
In my situation I got lucky. I bought the router originally only intending to install dd-wrt[5] on it. Later I found Tomato[6] which looked like it would suit my needs a lot better than dd-wrt would. Except there was some initial stupidity on my part and eventually I got that all sorted out by installing a fork[7] of the Tomato project which added USB support to Tomato.
For now the TomatoUSB fork only supports broadcom based routers like the original firmware but adds support for a few others which have USB ports. This is where I got lucky, I had never checked before I bought my router to see if it would be supported since I originally only intended on installing dd-wrt on it and it just happened to be supported.
Eventually I got around to unpacking my printer and decided to give it a try. While watching the USB section of the web interface of my router which at this point was running Tomato[8]. I plugged in my printer's USB cable into my router and about 3 seconds later[9] the printer showed up and it began serving the printer using raw data on port 9100 and with LPR[10] queue lp0.
So that was easy... a little too easy. Lo' and behold, it was just that easy. All that was left to do was add the printer using a TCP/IP port with 192.168.1.1 as the address and use the driver that I had previously installed to use the printer via USB. That was pretty much all I had to do, it worked exactly as it was meant to the first time I tried it.
- If they're tech-savvy enough to realize that this is possible. [↩]
- Dynamic Host Configuration Protocol [↩]
- Netgear WNR3500L [↩]
- 3rd Party Router Firmware [↩]
- http://www.dd-wrt.com/ [↩]
- Tomato Firmware [↩]
- http://tomatousb.org/ [↩]
- Which has in my opinion a much more beautiful interface that dd-wrt does. [↩]
- The default refresh time of pages that have dynamic content on Tomato. [↩]
- Line Printer Remote [↩]
3rd Party Router Firmware
Until Monday I'm without any kind of proper internet connection which means until this point I've just been using my phone to browse the internet and chat with friends, which is frustrating to say the least. This prompted me to do a little more research on 3rd party firmware for wireless routers.
The first project that comes to mind is obviously DD-WRT[1] the most popular and probably the most powerful of them all right out of the box. Before I moved here I researched and purchased a new router to use at my new place. I stumbled upon a Netgear router which seemed to match all the features I needed at a reasonable price. I've never really been a huge fan of Netgear routers but I checked that it was compatible at least with DD-WRT before I bought it. The router I'm talking about is a Netgear WNR3500L[2] which includes:
- 802.11n WiFi
- 4x 10/100/1000 Ethernet ports
- 1x USB 2.0 port
I think grand total it was ~$90[3]. Anyway first thing I did was install DD-WRT which is standard practice for me. Ran exactly as intended except when I tried to set it up to act as a wifi client which failed miserably, I never did figure out how to make it do what I wanted. Everything else worked as intended. But I recently discovered a new firmware I wanted to try, which was Tomato[4] another open-source project like DD-WRT.
Tomato is essentially a watered down version of DD-WRT. I think the only useful feature it's missing that DD-WRT has is virtual wifi interfaces, but that's not such a big deal. On the other hand though Tomato has the most polished bandwidth monitoring features of any other project I've ever seen or used. DD-WRT has essentially the same feature but it's much weaker and not nearly as thought out and well designed as Tomato's is.
My main gripe with Tomato is it's lack of community support. DD-WRT is so popular that just about any router you can get your hands on has a forum post somewhere about someone's woes with installing something on it and getting it working the way they needed it. Tomato isn't quite the same way. Also it appears that Tomato mostly only supports broadcom chipsets which is what my new router has.
Well anyway I just decided to download the latest version and try it out. Bad idea. I hadn't really considered that putting firmware on it that doesn't support USB would brick it. Figured USB just wouldn't work, WRONG. Got the firmware uploaded and reset it nothing. Nothing at all. So first thing I did was look up instructions for uploading a new firmware (one that I knew worked) using tftp. No luck, the router responds to pings for about 2 seconds immediately after booting but then ceases to respond. This was a good sign at least, means some basic features were still working properly. Also discovered that because Windows 7 implements CTCP instead of a simpler TCP protocol this breaks most ability to upload new firmware via tftp. So I downloaded an atftp.deb for my linux box and that didn't work either.
Eventually I stumbled upon an article about using a USB-TTL cable to unbrick the router. This article was mostly useless because all they were doing was using a USB serial adapter and dissecting the cable to work with the serial connection on the routers board, which I could just as easily have done with my arduino. But hidden deep in the comments was a far simpler trick than that. Only thing I needed to buy was a torx screw driver set to get the thing open. I looked up the chip used for storing settings and found that there were two pins on it that could be shorted to erase nvram (the structure responsible for storing all the router's settings).
So I busted open the router and proceeded to power on the router while shorting the two pins. No effect. I tried powering it on and then shorting the pins. No effect. Finally I tried shorting the pins exactly when the router responded to pings during that 1-2 second window. Success!
Then I fired up tftp and uploaded a modified version of Tomato to support USB and vaula! Tomato works properly on my router now. As well as the wifi client mode and wifi bridge mode. Anyway that occupied several hours of tinkering where I would have otherwise been bored out of my mind.
Multi-tracker Problems
I can't even begin to describe how annoying it is to download a torrent and to discover that the person that compiled it added a bunch of trackers to it but didn't bother to take into consideration that not all torrent clients (at least one) handle multi-trackers properly if you don't put blank lines in between each tracker.
Like so:
1 2 3 4 5 | http://tracker.openbittorrent.com/announce udp://tracker.openbittorrent.com:80/announce http://tracker.publicbt.com:80/announce udp://tracker.publicbt.com:80/announce |
Now I know for a fact that μtorrent requires this but I'm not sure about any other torrent clients. But the above list of trackers won't work properly. In fact only the http based trackers will register because of the blank line. For all of the trackers to be used each tracker must have a blank line following it.
Like so:
1 2 3 4 5 6 7 | http://tracker.openbittorrent.com/announce udp://tracker.openbittorrent.com:80/announce http://tracker.publicbt.com:80/announce udp://tracker.publicbt.com:80/announce |
So please if you're going to post create a multi-tracker torrent at least list all of the trackers properly because it will only do any good if every peer in the swarm has all of the trackers listed in the torrent. Unfortunately most people treat bittorrent like a "set and forget" sort of file sharing protocol but if you setup one small part like that wrong everyone suffers and most people simply won't notice.
Comcast’s Data Usage Meter
It looks like Comcast is starting to roll out a data usage meter to customers in the Portland, OR area so they can gauge how far along they are in their 250GB per year limit. According to Gizmodo, Comcast says their median data usage is 2-4GB per month. I thought this was hilarious so I decided to do a little calculating of my own.
I've got a Linksys WRT-something-or-other router which I've installed DD-WRT on. Recent versions of the firmware have a section that keeps track of overall traffic through WAN that your router handles. It also makes it pretty easy to do a little calculation of your own with it since you can download the data in text format. It logs in terms of total data in and out per day of each month. November was my first full month of data excluding the the first of the month (something broke that day I guess), so I downloaded the log and looked at November's data.
On average we downloaded 1917MB per day and uploaded 562MB per day. This is the total traffic between 3 people. Grand total we downloaded 54GB and uploaded 16GB. If we take a look at the ratio between the two I can approximate what our actual bandwidth is. We're supposed to have a 20Mb down connection and the ratio suggests that our up bandwidth is ~5.86Mb which means our maximum upload rate is 750KB/s which we've never achieved before. When I use bittorrent to download Linux ISO's I assume that in order to not choke our router with ACK's I need to throttle the upload rate to about 70% of the maximum which hovers around about 120-130KB/s which is ~1Mb/s even and that's only 70% of the max.
Basically I wouldn't survive if I had Comcast and a 250GB limit per year.
QR Code
Blogroll
Comics
Info
Archives
www.bemasher.net by BeMasher is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.