A Little Off Code, Computers, Photography and Guns

Coping with a puny outbound bandwidth limit.

If you were unaware I recently moved to Laramie, WY. Which is a significantly smaller town//city//something-or-other than Tucson, AZ is by quite a lot. Anyway if you didn't know Wyoming existed there are a few basic facts about Wyoming I should explain so you'll know what I'm dealing with here.

Wyoming is very very spacious and very sparse. Everything here is incredibly spread out. As such laying cable for connecting cities is generally expensive and still in the early stages of deployment. It is still possible to get broadband cable in the cities but if you don't live in the city and want broadband you're stuck with WiMax^[1] like wireless broadband which is pretty wide-spread here and effective since the terrain in most areas of the state permit line-of-sight wireless connections.

I went the broadband cable route since all of the experience I've had with Wifi stuff has been unreliable and I'm just a big fan of having a cable from one point to the next, there is a lot less downtime and possibility for interference.

Unfortunately having chosen broadband cable left me with two providers, Qwest and Bresnan. Qwest limits me to 3Mbps down and what i'm guessing is no more than 384Kbps up. Bresnan on the other hand provides 8Mbps/500Kbps which is better.

I'm consistently getting more and more frustrated with the emphasis broadband cable providers put on download speeds, upload speeds are getting to be just as important if not more important than inbound bandwidth. People are starting to learn that it's much much easier to share media and personal data online instead of the alternatives. By comparison I moved from Tucson where we had 20Mbps/3Mbps.

Now you can see where my problem is. 50KB/s up is pretty miserable compared to what I used to have. And you can imagine where my congestion comes in. If I saturate my outbound bandwidth Bresnan essentially ruins my inbound bandwidth. I recently discovered that Tomato the firmware I've installed on my router comes with TCP Vegas^[2] integrated.

Initially I couldn't find any proper documentation on what alpha, beta and gamma meant and how to tune them properly to help control my congestion problem. After several hours of searching and research I found only two configurations that seemed to provide people with some measure of congestion prevention: 3,3,2 and 2,6,2^[3]

So I switched TCP Vegas on while downloading the first Pioneer One episode^[4] and reset all the connections since TCP Vegas only affects connections created since enabled. Immediately I noticed the saw-tooth that TCP Reno^[5] causes disappeared, TCP Vegas immediately smoothed out the connection and allowed for a sustained outbound rate with an oscillating inbound rate around the maximum throughput achievable while maintaining sustained outbound traffic. I've read that tuning TCP Vegas properly can eliminate the oscillation of inbound traffic but I haven't had much luck with actually getting this done.

Once TCP Vegas began to smooth things out I figured out that in order to optimize inbound throughput I could simply limit the outbound rate to the magical 70% of outbound capacity recommended for P2P apps now that congestion was no longer an issue. This increased the inbound rate significantly and everything has been running smoothly since then. I wish I knew more about the algorithm used and how to tune it but there doesn't seem to be much in the way of documentation anywhere.

1. WiMax: Worldwide Interoperability for Microwave Access [↩]
2. TCP Vegas: a TCP congestion control, or network congestion avoidance, algorithm that emphasizes packet delay, rather than packet loss, as a signal to help determine the rate at which to send packets. It was developed at the University of Arizona by Lawrence Brakmo and Larry L. Peterson. [↩]
3. Alpha, Beta and Gamma respectively. [↩]
4. A free TV Series designed specifically for P2P distribution. [↩]
5. TCP Reno: a TCP Congestion Avoidance Algorithm [↩]