A Little Off Code, Computers, Photography and Guns

11Aug/100

Choosing an SSD

Before I started my new job I had an inordinate amount of free time and for a majority of that time, nothing to spend it doing[1]. I was still thinking about my desktop wishlist[2] and about choosing a better SSD than the one I had previously selected[3].

A long time ago when I was following the HDD market since I was looking to buy some bulk storage I wrote a php script which loaded newegg's product list based on some search parameters you provided newegg's productlist.xml[4]. The script would then parse the list and produce a list sorted based on price per gigabyte. Which is useful when you're in the market for capacity[5].

I decided to do more or less the same thing with SSD's except this time I did it in python since I'm rusty on PHP and I didn't want to mess with setting up a web server to test on. So I got started by doing a power search on newegg for the specific flavor of SSD I was looking for.

The search parameters are as follows:

  • 2.5" Form Factor
  • SATA II/III
  • 120GB or Greater
  • Less than $300
  • Retail or OEM
  • Support TRIM Command

As of this writing those particular search parameters narrows the result to 17 SSD's. Now comes the code. Before I started coding I needed some way to sort them according to what I thought was important. The metric is as follows:

\frac{\text{Read} \times \text{Write} \times \text{Capacity}}{|\text{Read} - \text{Write}| \times \text{Price}}

After looking closer at the scores this produces I noticed that it heavily penalizes drives with huge differences between read and write speeds which effectively weeds out drives that still have acceptable read//write speeds. So I removed that section of the metric producing:

\frac{\text{Read} \times \text{Write} \times \text{Capacity}}{\text{Price}}

The basic idea behind this scoring measure is that sequential read and write speeds are important, as well as capacity. Price and difference between sequential read//write are considered bad[6]. In the equation read and write refer to sequential read and write speeds. The ratio of these will produce a score of the SSD's overall performance for capacity, read//write speeds and price.

The code is relatively simple in purpose. Load the data and parse it into a dictionary then sort based on the metric above.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
import urllib2, re

# url = "
# http://www.newegg.com/Product/ProductList.aspx?Submit=Property&Subcatego
# ry=636&Description=&Type=&N=100008120&IsNodeId=1&srchInDesc=&MinPrice=&M
# axPrice=&OEMMark=1&OEMMark=0&PropertyCodeValue=4213:30854&PropertyCodeVa
# lue=4214:30848&PropertyCodeValue=4214:39416&PropertyCodeValue=4214:30849
# &PropertyCodeValue=4214:39415&PropertyCodeValue=4215:55552&PropertyCodeV
# alue=4215:41071&PropertyCodeValue=4215:46319"

# data = open("temp.html", "w")
# data.write(urllib2.urlopen(url).read())
# data.close()
raw = open("temp.html").read()

item_re = re.compile(r'<div class="itemCell".*?>(.*?)<br class="clear".*?</div>')
feature_re = re.compile(r"<li>&nbsp;(.*?)</li>")
feature_list_re = re.compile(r'<b>(.*?)\s?\#?\s?:\s?</b>\s?(.*?)</li>')
speed_re = re.compile(r"(up to )?(\d+).*?MB/s")
capacity_re = re.compile(r"(\d+)GB")
price_re = re.compile(r"</span>\$<strong>(\d+)</strong><sup>.(\d+)</sup>")

item_list = []
valid = ['Read', 'Item', 'Interface', 'Capacity', 'Model', 'Write', 'Size']

for item in item_re.findall(raw):
    current = {}
    no_label = []
    features = feature_re.findall(item)
    current["Size"] = features[0]
    current["Capacity"] = features[1]
    current["Interface"] = features[2]
   
    for feature in feature_list_re.findall(item):
        if feature[1].find("\r") != -1:
            current[feature[0]] = feature[1].split("\r")[0]
        else:
            current[feature[0]] = feature[1]
    current["Read"] = int(speed_re.findall(current["Sequential Access - Read"])[0][1])
    current["Write"] = int(speed_re.findall(current["Sequential Access - Write"])[0][1])
    current["Capacity"] = int(capacity_re.findall(current["Capacity"])[0])
    for feature in current.keys():
        if feature not in valid:
            del current[feature]
    current["Price"] = float('.'.join(price_re.findall(item)[0]))
    current["Item"] = "http://www.newegg.com/Product/Product.aspx?Item=%s" % (current["Item"])
    item_list.append(current)
   
sorted = {}
for item in item_list:
    ratio = (item["Read"] * item["Write"] * item["Capacity"]) / (item["Price"])
    sorted[ratio] = item
   
sort_order = sorted.keys()
sort_order.sort()
sort_order.reverse()
for key in sort_order:
    #print '\t'.join(map(lambda x: str(x), sorted[key].keys()))
    print '\t'.join(map(lambda x: str(x), sorted[key].values()))

Now given that there is quite a lot of data to present and analyze all at once I've decided it would be easiest to just provide you with a pretty graph[7]:


If you look closely at the scores of all the disks in the query, you'll notice that this is a noticeable gap between the top 3 and the rest. They are as follows:

Manufacturer: A-DATA Patriot G.Skill
Series: S599 Inferno Phoenix Series
Capacity: 128GB 120GB 120GB
Read: 280MB/s 285MB/s 285MB/s
Write: 270MB/s 275MB/s 275MB/s
Item: N82E16820211471[8] N82E16820220510[9] N82E16820231372[10]
Price: $295.99 $289.99 $299.00


I noticed that if you ignore capacity in the metric then the Patriot Inferno is the clear winner here. So as it turns out the Western Digital SiliconEdge I had selected when I first wrote the wishlist wasn't the best drive for my needs. But then I've always had a soft-spot for Western Digital. But now I'm convinced that the Patriot Inferno is the SSD I'll be getting unless by the time I get around to buying one there are better options[11].

  1. Nothing worth-while anyway [↩]
  2. See previous post: Wishlist. [↩]
  3. Western Digital SiliconEdge 128GB SSD [↩]
  4. Which no longer exists in it's original form. [↩]
  5. Which I was. [↩]
  6. Although we're excluding read//write speed difference. [↩]
  7. Scores have been normalized to 100%. [↩]
  8. A-Data S599 [↩]
  9. Patriot Inferno [↩]
  10. G.Skill Phoenix Series [↩]
  11. Which there probably will be. [↩]
Tagged as: , , , , , , , , , , No Comments
28Jul/100

Matplotlib and Live Data: A Tale of Two Technologies

Being unemployed over the summer is never usually a good thing for me. I get bored very easily if I don't have something to occupy myself with. This last bout of boredom led me to unpack some of my electronics. Dusted off my multimeter, Arduino and a digital thermometer I bought a little while ago. Figured I could use these to solve one of my current problems.

Living in Laramie usually subjects people to harsh winters which leaves most housing developments without central air conditioning installed since, well it's never really needed except maybe one or two days over the summer where it gets above 85 oF. This summer has apparently been hotter than previous summers and It's left my condo in an "uncomfortable state". Mind you I'm used to living in hot weather so this isn't such a terrible thing to me, I'm used to it.

What I'm not used to is not having AC and it cooling off enough at night that it's worthwhile to open a few windows and stick a fan in one of them. Which leaves me with this problem: When is the optimal time to open the windows and turn on the fan to get my condo cooled off earliest//fastest?

In comes my Arduino + digital thermometer[1]. Once I rigged up the proper power//data connections on a breadboard for my Arduino I set out to find code for the thermometer. I've setup the thermometer with a sketch on my Arduino before I just didn't feel like wasting a few hours trying to do it from scratch again. Soon enough I found some code[2] that worked perfectly. So I trimmed out some code I didn't need for the project and set it up to just write the temperature as fast as possible[3] to the serial port it's connected to.

After that I wrote a logging program on my desktop in Python to record temperatures sent via serial to my desktop. The program is incredibly simple and uses the pySerial library[4] to read temperatures from the serial port of my desktop and append them to a temperature log. I used a simple windows command to do this since it wouldn't lock the file so I could read data from it simultaneously. There are still occasionally collisions with the processing program locking the file and the logger not being able to write the data to the file but these are rare enough that it's negligible in my situation.

1
2
3
4
5
import serial, os

ser = serial.Serial(2)
while True:
    os.system("echo %s>>out.txt" % (ser.readline().strip()))

The next step in this project was visualizing the data. I've used matplotlib[5] before and I was thinking this time I would like to see if I could write the program to update data live as it recieves it. My first foray into this goal was a miserable disaster. Most of the solutions I could find involved just setting up an infinite loop with a short time delay in it. Which works great except that it sleeps the thread running the plot which makes it impossible to resize the plot or do anything at all with the GUI for that matter. So obviosly this wouldn't work at all.

After poking around for different solutions to this and crashing my computer once from spawning an infinite number of instances of the plot I gave up for a bit, only to discover that there was an example in the documentation which wasn't obviously named. I quickly discovered the best way to do this. I even added some pretty annotations and such.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
import gobject
import matplotlib
matplotlib.use('GTKAgg')

import matplotlib.pyplot as plt

current_pos = 0
temps = []
pad = 5.0

f = plt.figure()

def update(vars):
    # Unpack variables that need to be persistent between
    # executions of this method.
    temps = vars[0]
    current_pos = vars[1]
    pad = vars[2]
   
    # Open the data file and get any new data points since
    # the last time we read from this file
    data = open("out.txt", "r")
    data.seek(current_pos)
    new_temps = map(lambda x:
        float(x) * (1 + 4.0/5.0) + 32.0,
        data.read().split("\n")[:-1])
    current_pos = data.tell()
    data.close()
   
    # If we got new data then append it to the list of
    # temperatures and trim to 750 points
    if len(new_temps) > 0:
        temps.extend(new_temps)
        temps = temps[-750:]
   
    f.clear()
    f.suptitle("Live Temperature")
    a = f.add_subplot(111)
    a.grid(True)
    l, = a.plot(temps)
    plt.xlabel("Time (Seconds)")
    plt.ylabel(r'Temperature $^{\circ}$F')
   
    # Get the minimum and maximum temperatures these are
    # used for annotations and scaling the plot of data
    min_t = min(temps)
    max_t = max(temps)
   
    # Add annotations for minimum and maximum temperatures
    a.annotate(r'Min: %0.2f$^{\circ}$F' % (min_t),
        xy=(temps.index(min_t), min_t),
        xycoords='data', xytext=(20, -20),
        textcoords='offset points',
        bbox=dict(boxstyle="round", fc="0.8"),
        arrowprops=dict(arrowstyle="->",
        shrinkA=0, shrinkB=1,
        connectionstyle="angle,angleA=0,angleB=90,rad=10"))

    a.annotate(r'Max: %0.2f$^{\circ}$F' % (max_t),
        xy=(temps.index(max_t), max_t),
        xycoords='data', xytext=(20, 20),
        textcoords='offset points',
        bbox=dict(boxstyle="round", fc="0.8"),
        arrowprops=dict(arrowstyle="->",
        shrinkA=0, shrinkB=1,
        connectionstyle="angle,angleA=0,angleB=90,rad=10"))
   
    # Set the axis limits to make the data more readable
    a.axis([0,len(temps), min_t - pad,max_t + pad])
   
    f.canvas.draw_idle()
   
    # Repack variables that need to be persistent between
    # executions of this method
    vars = {0: temps, 1: current_pos, 2: pad}
   
    return True

vars = {0: temps, 1: current_pos, 2: pad}

# Execute update method every 500ms
gobject.timeout_add(500, update, vars)

# Display the plot
plt.show()

This code generates a plot which updates every 500ms. This is based on an example in the matplotlib examples[6]. An example of the program's output can be seen below.

I imagine that I could have made this simpler by not using the GTK libraries which are a pain to install since there are 3 or 4 modules you have to install in order to make all this work including the GTK+ runtime. I may come back later and post a version written using TK since it can be used without installing extra modules and stuff.

  1. DS18S20 Digital Thermometer Datasheet [↩]
  2. Temperature Measurement using the Dallas DS18B20 by Peter H. Anderson [↩]
  3. Somewhere in the range of 750ms between readings since it is in parasite mode, may change this later to run in non-parasite mode. [↩]
  4. pySerial Python Library [↩]
  5. matplotlib Python Library [↩]
  6. Animation example code: simple_anim_gtk.py [↩]
Tagged as: , , , , , , , , , , , No Comments
22Jul/100

Wishlist

I've noticed recently that I tend to spend a lot of time shopping for things I can't afford when I don't have any excess income. I can't really tell if it's just because I'm bored a lot more often over the summer. Especially this one since I've been unemployed for the maojority of it so far[1].

As it stands there is a rather long list of things I intend on buying//upgrading//replacing in the future. First and foremost on this list is a new laptop since my current ASUS Eee-PC 1000H is driving me nuts. It's useful for... writing, and not even that sometimes. For the last year I've used it almost strictly for taking notes in class, which it does well enough. But using it for anything else is essentially impossible. I've found this to be even more true in the last few weeks since I've been spending every other weekend with my parents on their ranch or in Cheyenne. I've just been using it when I went since it's pretty impractical to take my desktop with me everytime. Especially considering a lot of the stuff I work on needs a decent amount of bandwidth and my parents' internet connection is satellite based on their ranch at least so it would be pointless to try and get any real work done.

I've essentially decided that my next laptop will be a 13" Macbook Pro. The main reason is that for the amount of money I intend on spending on a new laptop the Macbook Pro is far superior in both build quality and components to the equivelant Dell which is the manufacturer I've used for all my mobile computing needs until my netbook. Easy decision don't you think?

The next item on my list was building a new desktop. I only really need to replace the core components of my desktop since everything else is more or less in good working order. But that's boring so I've made an entire list of components including core and secondary components to build a new desktop, excluding optical drive and hard drives[2]

The first part I always start with when building a wishlist[3] is the CPU and for this particular one it was a pretty simple choice. Intel's Core i7 series is pretty much the way to go when building a workstation. In this case there were only really two requirements I had for selecting the particular Core i7 I need for this build.

  • $0 < Price < $500
  • Supports Triple Channel DDR3

These requirements narrow down the selection to two processors. The Core i7-920 and the Core i7-930. There are only two differences. The 930 is 2.8Ghz and the 920 is 2.66Ghz and the 930 is $10 cheaper than the 920, so it's pretty obvious which one is the one to go with.

Intel Core i7-930 Bloomfield 2.8Ghz LGA 1366 130W Quad-Core
Model #: BX80601930
Item #: N82E16819115225
Price: $289.99

The second component I select after the CPU is the motherboard. Now this is where it gets tricky because the restrictions I use for selecting a motherboard have a lot less to do with technical capabilities than they do with reliability and proper functionality. This is where newegg becomes the right place to shop. Their product review system is by far the best in the online tech shopping world. I tend to score motherboards based on the number of reviews they receive and the score of the review. This is of course after I've removed motherboards incompatible with the other components I intend on using in the system.

  • LGA 1366
  • Intel X58
  • Intel ICH10R
  • ATX form factor

The motherboard that comes out on top after these restrictions is an EVGA board.

EVGA E758-TR Intel X58

http://www.newegg.com/Product/Product.aspx?Item=N82E16813188046

Model #: 132-BL-E758-TR
Item #: N82E16813188046
Price: $269.99 IMIR -$40.00: $229.99

Next up is system memory. This almost always follows from motherboard since some motherboards support odd RAM speeds and I tend to stick with standard speeds since they are a lot less prone to compatibility issues and just work. In this case the motherboard calls for DDR3 1333[4]. I usually filter out for the specific kind of RAM I want which leaves me with a few dozen sets. Then I score based on CAS latency and price. I've used G.SKILL before and was pleased with it and in this case a G.SKILL set won on both price and CAS latency.

G.SKILL 3GB (3 x 1GB) DDR3 1333 (PC10666) Triple Channel
Model #: F3-10666CL7T-3GBPK
Item #: N82E16820231229
Price: $84.99

Now you're probably thinking "Why does he only want 3GB that's puny!". There are some very good reasons for it. First of all I'm not really all that into 64-bit yet, I still have a few devices without 64-bit compatible drivers[5]. For the most part 2GB of RAM has suited me just fine for nearly anything I've ever needed//wanted to do on my desktop until this point, why should I pile in twice or even three times that amount? Besides if I so desire I could just purchase a second set in the future. The only reason I might consider doing that is if I suddenly became obsessed with running a dozen virtual machines simultaneously[6].

Next in line isn't exactly a component I need to buy, but I've been wanting to upgrade for a long time now and I figure a wishlist is the best place to do it. Ever since I saw an article on Gizmodo[7] about the new NVIDIA GeForce GTX 460 I've pretty much been set on that specific chipset. It was pretty easy to select a brand since they're all exactly the same price at this point and I only want the 768MB model. I've been wanting to do some CUDA development so here's my chance.

EVGA NVIDIA GeForce GTX 460 (Fermi) 768MB
Model #: 768-P3-1360-TR
Item #: N82E16814130562
Price: $199.99

Another component I consider to be core but isn't necessarily a core component is the storage device used for the OS. In the past I've strictly used HDD's for my desktop. But since I installed a Patriot 32GB SSD in my laptop I've fallen in love with SSD's for the OS//Programs drive. You might hear people moan and complain about SSD's being disproportionatly priced based on their capacity. Well I've got news for you, you don't buy SSD's for capacity, you buy them for speed. Anyone reasonably knowledgeable about computer components and their functionality would know that. I'm not interested in price per GB as quite a lot of people might be, at least not for SSD's[8]. I'm a lot more interested in price per MB/s sequential read-write. The particular disk that won in this case is one of the new Western Digital SSD's.

Western Digital SiliconEdge Blue 128GB SSD MLC 220/170 SR/SW
Model #: SSC-D0128SC-2100
Item #: N82E16820250002
Price: $199.99

In case the title of that product doesn't make sense the drive is 220MB/s Sequential Read and 170MB/s Sequential Write.

Everything from this point on I consider to be secondary components as they don't directly do any computation or data transfer//storage.

For this build I've decided that even though I don't need to get a new case, I've added one to the list anyway since my current case is due for an upgrade, especially in regards to aesthetics. I've been oggling this particular case for quite a while now, since it replaced it's predecessor at least. This case won by a long shot in aesthetics and functionality.

Antec P183 Black Aluminum/Steel ATX-Mid
Model #: P183
Item #: N82E16811129061
Price: $179.99 IMIR -$25.00: $154.99

The next item due for an upgrade was actually necessary considering the major increase in power needs for the core components. I've always hated shopping for power supplies because there are far more factors to consider when it comes to selecting one that matches your needs and is of reasonable build quality. If you don't have a decent power supply you may as well just give up. In this case I stuck as close as possible to the power supply I have now. I was only really interested in making sure that there were enough PCI-e power connectors since my current PSU has none. I let the reviews do the majority of selecting for me in this case.

Corsair 650W (ATX|EPS)12V
Model #: CMPSU-650TX
Item #: N82E16817139005
Price: $119.99 IMIR -$30.00: $89.99

This power supply matched most closely to the one I had now, it's simple, doesn't have too many "certifications" and marketing nonsense tacked onto the name and the cables are sheathed in black mesh[9].

Now I don't normally bother with purchasing a 3rd party heatsink//cooling system for my CPU but in this case I had heard mention of a self-contained water cooling system with radiator, pump and CPU waterblock from Corsair. So I checked it out and I am impressed. Since it is self-contained it removes a lot of the frustration with resevoirs and replacing the coolant on a regular basis.

Corsair H50 CPU Cooler
Model #: CWCH50-1
Item #: N82E16835181010
Price: $74.99

The last item in the list is more for interior neatness and organization. I've always hated just leaving components without a proper fastening inside the case. In this situation the SSD I select[10] is 2.5" form factor, suitable for notebooks and less suitable for desktops. So I looked around for a set of 2.5" to 3.5" brackets to secure the drive in one of the HDD bays.

iStarUSA 2.5" to 3.5" HDD Bracket
Model #: DIY-RP-HDD2.5
Item #: N82E16816215157
Price: $5.99

The subtotal for the build excluding shipping and including all instant mail in rebates comes out to $1330.91. Pretty good wouldn't you say? For a decently beefy workstation that would likely last me another 5-6 years before upgrading again. I'm currently on the 5th year since a major overhaul of my current system an Intel Core 2 Duo based rig.

  1. I do have an interview coming up so wish me luck. [↩]
  2. Again, excluding SSD from this list of parts I don't intend to buy. [↩]
  3. Always on Newegg.com, they're the de facto standard in online computer components. [↩]
  4. DDR3 SDRAM PC10666 [↩]
  5. And will probably never be compatible for that matter. [↩]
  6. Which I won't, so I won't. [↩]
  7. I think. [↩]
  8. The only component I consider price per GB on is standard HDD's. [↩]
  9. Which lends nicely to aesthetics should I ever decide to show someone my desktop's inards. [↩]
  10. Like all SSD's. [↩]
Tagged as: , , , , , , , , , , , , , , , No Comments
27Jun/100

WriteMonkey and Markdown

Recently Download Squad had a post[1] about a practical way to get features and support for open-source programs, specifically through donations. The post was about a program called WriteMonkey which is a minimalistic writing program that the author had originally written about previously[2]. Think of the best code editing program you know of, mine is Notepad++[3]. Now take that program and refactor it specifically for writing articles or blog posts, you've just created WriteMonkey[4].

Something that interested me about WriteMonkey was the Download Squad author's post specifically mentioned writing posts using Markdown[5] syntax. Markdown is a simple plain-text syntax which is parsed into html removing the need to tediously enter html[6] as you write. At first glance it didn't really seem like it would really help all that much when it came to writing blog posts. But I was completely wrong and am better off for it. Now the especially useful part is that WriteMonkey supports this completely as well as having a very useful shortcut for parsing and copying html straight from Markdown source. This is incredibly useful since I can then just go to my website and paste the resulting html into a blog post and hit save and be done with it.

As I looked through the program I realized, this is much much more than just a Markdown IDE. It includes all sorts of useful features like a "progress bar" which tells you how far along you are in a certain quota you specify in the preferences. This led me to write a little bit of SQL[7] to calculate the average word-count of posts in my blog. Excluding the outliers it came out to ~350 words per post. So I just set the quota to 350 words and it displays a bar at the top or bottom of the screen depending on what you choose showing your current progress on the quota.

It also does several other useful things like displaying current battery life as a percentage in the progress bar, showing the file you're writing in. There's also this feature called repository//main. This allows you to store text clippings in repository and then write the blog post in main. When exported as html the repository is ignored and only main is copied. Makes it useful to write notes and such in the middle of authoring a post to keep with everything you write and it's easy enough to switch between the two to make it useful. For this post I just made a list of points I wanted to cover.

After using WriteMonkey for an hour or so I think I've found the new environment I'll be writing all my posts in for the foreseeable future.

  1. Download Squad: Amazing software tip: Pay free software developers to get stuff fixed! [↩]
  2. Download Squad: WriteMonkey is an unbelievable full-screen text editor [↩]
  3. Notepad++ [↩]
  4. WriteMonkey [↩]
  5. Markdown [↩]
  6. HTML: HyperText Markup Language, is the predominant markup language for web pages. [↩]
  7. SQL: Structured Query Language [↩]
Tagged as: , , No Comments
23Jun/100

Coping with a puny outbound bandwidth limit.

If you were unaware I recently moved to Laramie, WY. Which is a significantly smaller town//city//something-or-other than Tucson, AZ is by quite a lot. Anyway if you didn't know Wyoming existed there are a few basic facts about Wyoming I should explain so you'll know what I'm dealing with here.

Wyoming is very very spacious and very sparse. Everything here is incredibly spread out. As such laying cable for connecting cities is generally expensive and still in the early stages of deployment. It is still possible to get broadband cable in the cities but if you don't live in the city and want broadband you're stuck with WiMax[1] like wireless broadband which is pretty wide-spread here and effective since the terrain in most areas of the state permit line-of-sight wireless connections.

I went the broadband cable route since all of the experience I've had with Wifi stuff has been unreliable and I'm just a big fan of having a cable from one point to the next, there is a lot less downtime and possibility for interference.

Unfortunately having chosen broadband cable left me with two providers, Qwest and Bresnan. Qwest limits me to 3Mbps down and what i'm guessing is no more than 384Kbps up. Bresnan on the other hand provides 8Mbps/500Kbps which is better.

I'm consistently getting more and more frustrated with the emphasis broadband cable providers put on download speeds, upload speeds are getting to be just as important if not more important than inbound bandwidth. People are starting to learn that it's much much easier to share media and personal data online instead of the alternatives. By comparison I moved from Tucson where we had 20Mbps/3Mbps.

Now you can see where my problem is. 50KB/s up is pretty miserable compared to what I used to have. And you can imagine where my congestion comes in. If I saturate my outbound bandwidth Bresnan essentially ruins my inbound bandwidth. I recently discovered that Tomato the firmware I've installed on my router comes with TCP Vegas[2] integrated.

Initially I couldn't find any proper documentation on what alpha, beta and gamma meant and how to tune them properly to help control my congestion problem. After several hours of searching and research I found only two configurations that seemed to provide people with some measure of congestion prevention: 3,3,2 and 2,6,2[3]

So I switched TCP Vegas on while downloading the first Pioneer One episode[4] and reset all the connections since TCP Vegas only affects connections created since enabled. And immediately i noticed the saw-tooth that TCP Reno[5] causes a saw-tooth wave in throughput of outbound traffic which is very bad for maintaining throughput. TCP Vegas immediately smoothed out the connection and allowed for a sustained outbound rate with an oscillating inbound rate around the maximum throughput achievable while maintaining sustained outbound traffic. I've read that tuning TCP Vegas properly can eliminate the oscillation of inbound traffic but I haven't had much luck with actually getting this done.

Once TCP Vegas began to smooth things out I figured out that in order to optimize inbound throughput I could simply limit the outbound rate now that the congestion was eliminated. This increased the inbound rate significantly and everything has been running smoothly since then. I wish I knew more about the algorithm used and how to tune it but there doesn't seem to be much in the way of documentation anywhere.

  1. WiMax: Worldwide Interoperability for Microwave Access [↩]
  2. TCP Vegas: a TCP congestion control, or network congestion avoidance, algorithm that emphasizes packet delay, rather than packet loss, as a signal to help determine the rate at which to send packets. It was developed at the University of Arizona by Lawrence Brakmo and Larry L. Peterson. [↩]
  3. Alpha, Beta and Gamma respectively. [↩]
  4. A free TV Series designed specifically for P2P distribution. [↩]
  5. TCP Reno: a TCP Congestion Avoidance Algorithm [↩]
Tagged as: , , , , , , , , No Comments
15Jun/100

Open-Source Print Server

One of the things I had always wanted to play with until I moved was a wireless router that was "deluxe" enough to have a USB port installed.

Most of the time these sorts of routers expect that you're just going to plug an external hard drive into it and pretend like it's a full-fledged NAS becuase typically they manage to squeeze CIFS into the firmware that governs the router and for the most part this is good enough for the most basic of NAS needs. Except they tend to forget that throughput with such a simple processor is usually miserable. I've never actually tested any of this myself but all of the forums I've read on the subject seem to report that there is a throughput ceiling of around 5-6MB/s which is pretty bad.

The other purpose for the USB port that I bet people would get more use out of is the ability to use your router as a simple print server. The first reason this is a lot more practical is that nearly all consumer printers these days use USB for their interface. This means that essentially any printer that can be made to work for your current computer with USB can also be put behind a print server without too much trouble. The other way this is useful is for the growing number of laptops you'll find in any given household. Except for minority of people that do enough computer work that owning a desktop is a necessity most every house will mostly be comprised of laptops and all of them will likely communicate with Wifi.

Every time you need to print something from your laptop you'll have to take it back to wherever you've parked your printer and plug it in. But you'll also need to make sure that you plug it back into the same USB port you installed it with in the first place or your computer will go into full-retard mode and "Duhhh... new printer! I'll look for drivers for your awesome new printer."

Now you see where the real hassle is. But this is the point at which a lot of people go the wrong direction. The first thought a lot of people have[1] is that they'll just solve this problem by throwing money at it which usually means shopping for and buying a printer with network capabilities built in.

The main problem with this is that, network attached printers are expensive, quite a lot more expensive than their non-network-aware brethren. The secondary problem is that a grand majority of home networks are made possible by DHCP[2] which divvies out IP addresses as devices connect to the network. This presents a problem when the main method for connecting to network attached printers involves knowing the printer's IP address, which can be problematic when your router arbitrarily hands out IP addresses on a regular basis. Every time the lease is up for your printer's IP address there is the possibility of that printer to get a new IP address which causes issues. So unless you are tech-savvy enough to setup static DHCP leases this will cause problems.

The next option for a lot of people is to shoehorn round peg into a square hole by buying a print server to make their current printer network friendly. See the above problem for why this isn't an optimal solution.

In comes the router with a USB port. I recently purchased a Netgear WNR3500L[3] from newegg. If you're interested in my adventurous experience with installing Tomato on it see my previous[4] post about that.

Now you're probably wondering how on earth a router with a USB port is any better than a print server or a network attached printer. The reason it is better is that in typical household networks the router will have the same IP address no matter how you've got DHCP configured. For the most part your router lives at 192.168.1.1 or 192.168.0.1 and this will for most situations never change.

So given a router with a USB port and proper firmware to allow for print serving you can host your printer on your router which will almost always have the same IP address and you can do this all with one device instead of having to buy a separate device.

In my situation I got lucky. I bought the router originally only intending to install dd-wrt[5] on it. Later I found Tomato[6] which looked like it would suit my needs a lot better than dd-wrt would. Except there was some initial stupidity on my part and eventually I got that all sorted out by installing a fork[7] of the Tomato project which added USB support to Tomato.

For now the TomatoUSB fork only supports broadcom based routers like the original firmware but adds support for a few others which have USB ports. This is where I got lucky, I had never checked before I bought my router to see if it would be supported since I originally only intended on installing dd-wrt on it and it just happened to be supported.

Eventually I got around to unpacking my printer and decided to give it a try. While watching the USB section of the web interface of my router which at this point was running Tomato[8]. I plugged in my printer's USB cable into my router and about 3 seconds later[9] the printer showed up and it began serving the printer using raw data on port 9100 and with LPR[10] queue lp0.

So that was easy... a little too easy. Lo' and behold, it was just that easy. All that was left to do was add the printer using a TCP/IP port with 192.168.1.1 as the address and use the driver that I had previously installed to use the printer via USB. That was pretty much all I had to do, it worked exactly as it was meant to the first time I tried it.

  1. If they're tech-savvy enough to realize that this is possible. [↩]
  2. Dynamic Host Configuration Protocol [↩]
  3. Netgear WNR3500L [↩]
  4. 3rd Party Router Firmware [↩]
  5. http://www.dd-wrt.com/ [↩]
  6. Tomato Firmware [↩]
  7. http://tomatousb.org/ [↩]
  8. Which has in my opinion a much more beautiful interface that dd-wrt does. [↩]
  9. The default refresh time of pages that have dynamic content on Tomato. [↩]
  10. Line Printer Remote [↩]
Tagged as: , , , , , , , , , , , No Comments
8Jun/100

3rd Party Router Firmware

Until Monday I'm without any kind of proper internet connection which means until this point I've just been using my phone to browse the internet and chat with friends, which is frustrating to say the least. This prompted me to do a little more research on 3rd party firmware for wireless routers.

The first project that comes to mind is obviously DD-WRT[1] the most popular and probably the most powerful of them all right out of the box. Before I moved here I researched and purchased a new router to use at my new place. I stumbled upon a Netgear router which seemed to match all the features I needed at a reasonable price. I've never really been a huge fan of Netgear routers but I checked that it was compatible at least with DD-WRT before I bought it. The router I'm talking about is a Netgear WNR3500L[2] which includes:

  • 802.11n WiFi
  • 4x 10/100/1000 Ethernet ports
  • 1x USB 2.0 port

I think grand total it was ~$90[3]. Anyway first thing I did was install DD-WRT which is standard practice for me. Ran exactly as intended except when I tried to set it up to act as a wifi client which failed miserably, I never did figure out how to make it do what I wanted. Everything else worked as intended. But I recently discovered a new firmware I wanted to try, which was Tomato[4] another open-source project like DD-WRT.

Tomato is essentially a watered down version of DD-WRT. I think the only useful feature it's missing that DD-WRT has is virtual wifi interfaces, but that's not such a big deal. On the other hand though Tomato has the most polished bandwidth monitoring features of any other project I've ever seen or used. DD-WRT has essentially the same feature but it's much weaker and not nearly as thought out and well designed as Tomato's is.

My main gripe with Tomato is it's lack of community support. DD-WRT is so popular that just about any router you can get your hands on has a forum post somewhere about someone's woes with installing something on it and getting it working the way they needed it. Tomato isn't quite the same way. Also it appears that Tomato mostly only supports broadcom chipsets which is what my new router has.

Well anyway I just decided to download the latest version and try it out. Bad idea. I hadn't really considered that putting firmware on it that doesn't support USB would brick it. Figured USB just wouldn't work, WRONG. Got the firmware uploaded and reset it nothing. Nothing at all. So first thing I did was look up instructions for uploading a new firmware (one that I knew worked) using tftp. No luck, the router responds to pings for about 2 seconds immediately after booting but then ceases to respond. This was a good sign at least, means some basic features were still working properly. Also discovered that because Windows 7 implements CTCP instead of a simpler TCP protocol this breaks most ability to upload new firmware via tftp. So I downloaded an atftp.deb for my linux box and that didn't work either.

Eventually I stumbled upon an article about using a USB-TTL cable to unbrick the router. This article was mostly useless because all they were doing was using a USB serial adapter and dissecting the cable to work with the serial connection on the routers board, which I could just as easily have done with my arduino. But hidden deep in the comments was a far simpler trick than that. Only thing I needed to buy was a torx screw driver set to get the thing open. I looked up the chip used for storing settings and found that there were two pins on it that could be shorted to erase nvram (the structure responsible for storing all the router's settings).

So I busted open the router and proceeded to power on the router while shorting the two pins. No effect. I tried powering it on and then shorting the pins. No effect. Finally I tried shorting the pins exactly when the router responded to pings during that 1-2 second window. Success!

Then I fired up tftp and uploaded a modified version of Tomato to support USB and vaula! Tomato works properly on my router now. As well as the wifi client mode and wifi bridge mode. Anyway that occupied several hours of tinkering where I would have otherwise been bored out of my mind.

  1. DD-WRT Firmware [↩]
  2. Netgear WNR3500L [↩]
  3. Newegg.com: Netgear WNR3500L [↩]
  4. Tomato Firmware [↩]
Tagged as: , , , , , , No Comments
3May/100

FreeNAS Users Rejoice!

Unetbootin[1] now supports FreeNAS! Take a look at these awesome little snippets of code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
//distrolst.cpp
if (nameDistro == "FreeNAS") {
    if (isarch64) {
        cpuarch = "amd64";
    } else {
        cpuarch = "i386";
    }
    instIndvfl("memdisk", QString("%1ubnkern").arg(targetPath));
    if (islivecd) {
        downloadfile(QString("http://downloads.sourceforge.net/sourceforge/lubi/FreeNAS-%1-LiveCD-%2.img.gz").arg(cpuarch, relname), QString("%1ubninit").arg(targetPath));
    } else {
        downloadfile(QString("http://sourceforge.net/projects/freenas/files/stable/0.7/FreeNAS-%1-embedded-%2.img/download").arg(cpuarch, relname), QString("%1ubninit").arg(targetPath));
    }
}
1
2
3
4
5
6
//distrover.cpp
distroselect->addItem("FreeNAS", (QStringList() << "0.7.4919" <<
unetbootin::tr("<b>Homepage:</b> <a href=\"http://freenas.org/\">http://www.freenas.org</a><br/>"
    "<b>Description:</b> FreeNAS is an embedded open source NAS (Network-Attached Storage) distribution based on FreeBSD.<br/>"
    "<b>Install Notes:</b> The LiveCD version creates a RAM drive for FreeNAS, and uses a FAT formatted floppy disk or USB key for saving the configuration file. The embedded version allows installation to hard disk.") <<
"0.7.4919" << "0.7.4919_x64" << "0.7.1.5024_Live" << "0.7.1.4997_Live_x64"));

Segue:
I'm actually considering forking the unetbootin project to add support for a master distro list which can be updated remotely eliminating the requirement for users to download a new copy of the program if they wish to get the latest version of the list of pre-configured distros.

This has a little bit to do with the fact that I'll be required to take a few C++ courses at the University of Wyoming since Java is the standard language taught at the University of Arizona while I was there and I've never used C++ before. Can't be that hard right?

  1. http://unetbootin.sourceforge.net/ [↩]
Tagged as: , , , , , , , , No Comments
3May/100

Analog Monitor Calibration

Many of you probably already are aware that LCD Monitors typically will come with an analog VGA port[1] but the typical way that LCD Monitors work is that they reference individual pixels to draw something where CRT's use a scanning electron beam to illuminate pixels. Because of the way this works LCD Monitors usually come with an "Auto" button or "Auto-calibrate" which will align the analog image to the displayable area of the monitor. But doing this with just any image won't always give you the sharpest alignment and calibration.

The method I've used for quite some time is a program that generates a black and white cross-hatch that basically makes a checkerboard of every single pixel on the screen. This makes it very easy for the auto-calibration feature of the monitor to almost exactly align the image to exact pixels. The program i use to do this is called lcdtest[2]. There are even windows binaries[3] that can be found by digging through the page, I've provided a link in the footnotes to the page I usually get them from.

The program starts and immediately draws the test pattern on the entire screen. You'll want to press w to change the color to white on black. Then you'll want to press x to change the pattern to crosshatch. Then using the - key to zoom out until it looks nearly grey. This is a good point at which you can see how well your monitor is calibrated already. If it looks very grey this is a sign that the scanning frequencies may not be exact, you might also see waves where it's clear and areas where it is blurry, these will likely go away once we're done. At this point you should use the auto-calibration feature of your monitor to calibrate the display using the crosshatch being displayed.

For those of you running more than one monitor at a time, this program will only display the pattern on the primary monitor. But never fear, there is an easy fix. Using paint or paint.net or your favorite image editing program you can create an image of the test pattern and set it as the wallpaper for the other monitors and calibrate them as well. Using Alt+PrtScn to capture only the current application's area of the screen (the test pattern) and pasting into a new image you can then save the image as a png[4] preferably as it will compress it losslessly.

Once you've saved the image simply set it as the wallpaper of your system and use tiling in the case that your other monitor is not the same dimensions as your primary monitor and then run the auto-calibration process on the remaining monitors. You'll find that this will significantly improve the quality of the image displayed on your monitors if they were improperly calibrated to begin with. But of course you could completely avoid this by switching to DVI[5]

  1. [W: VGA_connector] [↩]
  2. http://www.brouhaha.com/~eric/software/lcdtest/ [↩]
  3. http://code.google.com/p/lcdtest-win32/ [↩]
  4. [W: Portable_Network_Graphics] [↩]
  5. [W: Digital_Visual_Interface] [↩]
Tagged as: , , , , , , , , No Comments
9Apr/104

DVD Ripping Made Easy

Reading through my normal list of RSS feeds I stumbled upon a post claiming to have found some software that greatly simplifies the process of decrypting and ripping DVD's. And surprisingly for the most part they were right.

The software in question is called MakeMKV[1]. The software seems to do a decent job of both decrypting and ripping DVD's. Mind you this software is not meant for transcoding DVD video into a different format.

The software functions much like most DVD decrypting software does. DVDFab[2] and DVD Decryptor[3] provide the same basic functions as MakeMKV with one major exception. Where both DVDFab Decryptor and DVD Decryptor will provide you with the ability to decrypt a DVD and dump it's contents to a directory, MakeMKV instead muxes all of the video, audio and subtitle streams into a single container instead of having several VOB files from the entire DVD. Each title on the disk is muxed into a single container, which really simplifies the process when backing up TV Seasons from DVD to your computer since each episode is typically it's own title.

All that is left to do once you've ripped to a Matroska[4] container is either leave it by itself since it's a perfectly fine container and format (albiet nearly the same size as the original content) or transcode it into your favorite format//container. Typically when ripping DVD's I use Handbrake and encode the DVD using the High Profile preset which performs decombing and detelecine. The high profile preset is also uses constant quality encoding which seems to be the preferred method for encoding these days since it provides the best perceptible quality vs. compression ratio.

Now the first thing that bothered me about MakeMKV is the fact that the site specifically states that it is free for the beta. But when you read closer it does get better:

Functionality to open DVD discs is free and will always stay free.

So that's promising. At least if you're only interested in ripping DVD's and not Blu-Ray or HDDVD then you'll be golden with this software.

Overall I've decided to just stick with MakeMKV for all my decrypting//backup needs from now on since it seems to do as good of a job or better than any of the other DVD rippers on the market at the moment.

  1. http://makemkv.com/ [↩]
  2. http://www.dvdfab.com/ [↩]
  3. http://www.dvddecrypter.org.uk/ [↩]
  4. Matroska [↩]
Tagged as: , , , , , , , , , , , , 4 Comments
   Older Entries »

QR Code

Blogroll

Comics

Info

Archives

RSS Recommended Reading