TweetMachine started 2 or 3 years ago when I wanted to reuse an old laptop. For one reason or another (lack of tools, lack of parts, forgetting about it) it sat on my "open projects" self through 2 moves. Last week, though, it occured to me that I had acquired the last piece of hardware I needed and hadn't realized it. So, it was time to finish the project and make room on the projects shelf.

The device is basically a slightly deconstructed laptop in a shadow box frame running a custom Twitter client. What follows are the details so you can build your very own TweetMachine!

Hardware

First thing, the screen needs to be separated from the bottom half of the laptop enough so that it can be easily manipulated. In my experience there is only one main cable connecting the base to the screen and it connects through one of the hinges. By taking apart the screen first you should be able to see this.

In the Dell Inspiron I was dealing with there were two metal hinges that ran along the screen from top to bottom to keep the screen stiff and restrict movement. Those needed to be removed so that the screen could go back as far as I needed it to. After the hinges were removed I put the screen back together.

To suspend the laptop in a way that would work within the shadow box I decided to mount it to pegboard. This is the piece that took the most time to figure out. What I settled on was using plumber's strap to make brackets. Each individual bracket isn't very strong, but by distributing the weight of the laptop between 6 or 8 of them things were stable.

The laptop fit snugly in the frame, but I was concerned about overheating so I cut a vent with a Dremel and a hole for the power cable. The back board for the frame also worried me, so I cut out most of the center and left only enough of a frame to secure the laptop. 

Originally I had planned to use a nice photo matte to cover up the rough edges and only reveal the screen. But after putting it together with and without that set up, it felt less interesting with it. Cara convinced me it looked cooler without the matte, so I left it just like that.

Software

I installed the latest long term release of Ubuntu and configured it to start without requiring you to log in. (This is normally a bad idea, by the way.) I then needed to unlock the keyring automatically so that it could connect to the wifi without you needing to type in your password. (Again, a bad idea normally.) Next I installed unclutter, which automatically hides the mouse cursor. I disabled the screensaver among other power management tweaks. Finally, in power managment, I configured the power button to shut the computer down (without asking) whenever it was pressed.

The end result is a laptop that will start up, log in, and connect to the wifi without intervention. Even better, when you press the power button again it'll shut itself down.

I wanted a twitter client with no branding and a simple but attractive interface. Twitter's API is pretty easy to work with so I decided to write one myself. With Adobe Air, I was able to easily make a full screen app with no window controls.

The app supports following one user's timeline and a pre-specified search. By left clicking anywhere in the app it'll toggle between the two modes. (The left mouse button is easily accessible from the back of the device.)

Quick styling note: I was using pretty light colors as the background for the app. But, once I finished putting it together and put it in my living room I noticed how the terrible viewing angle of the laptop screen washed out the colors. My solution was to bump up the saturation of the blue and purple quite a bit. Looking at it straight on you see really bright colors, but anywhere else in the room you see the colors as I was originally picturing them.

Finally, the last piece was to configure my app to start whenever I logged in. 

You can grab the code here: https://github.com/doowttam/TweetMachine It won't win any code design awards, but it's simple and runs smoothly.

Quick note about OAuth: Twitter moved to OAuth recently for user authentication. This is great for users but can be annoying for developers. Luckly though, Twitter allows you to shortcut the OAuth process if you're the developer of the app. For simplicity I went this route and hard-coded the access keys. This does mean, though, that the app can only work with one user's timeline.

Future Improvements

I'd love to have more control over the app remotely. So, I'm planning on running a light webserver and webapp that'll allow anyone connecting to it within my apartment to tweak the setttings and change the search term. Right now the search is hard-coded and although it's easy to change, it requires SSHing into the machine and restarting the app.

I'd also like to have the app expand links and show videos (muted) and images that people link to. I don't think it'd be too hard and would really increase the interesting-ness of the appliance.

Permalink | Leave a comment  »

Benchmarking MySQL can be a tricky task. There are many variables that interact with each other and it can be difficult to run good isolated experiments and get data you can use. I ran into trouble benchmarking with sysbench recently that stumped me at first.

I was upgrading a database server to MySQL 5.1 from MySQL 5.0 in the hopes of getting better concurrency performance from InnoDB. The first step in that upgrade process was to run some benchmarks on 5.0 and 5.1 to make sure we were getting the benefits we were aiming for and not taking any performance hits. I performed an end-to-end application benchmark, a concurrency benchmark, and two standard benchmarks. The standard benchmarks were sysbench (read only and read/write) and the Database Test Suite’s OLTP test.

The Mystery

All of my benchmarks looked great except for the sysbench test. I was getting an order of magnitude better performance from 5.0 than from 5.1. This kind of performance regression was pretty disconcerting and needed to be investigated before we could continue with the upgrade.

The first step was isolating the variables. I got two cloud servers with a base Debian Lenny install and installed MySQL on both. (One server had 5.0, and the other 5.1.) I wanted to be sure that none of the other config our servers normally have could be causing the performance regression. I tweaked the config identically on both servers to give some reasonable values for things like the InnoDB buffer pool size. I then had sysbench create a test table with 1000000 rows. Finally I dumped the table and imported it into the other MySQL server.

The next step was to confirm that the problem still existed in my new isolated setups. The benchmark I used was the following:

sysbench --test=oltp --oltp-table-size=1000000 --mysql-db=sysbench --mysql-user=root --max-time=60 --max-requests=0 --num-threads=4 --oltp-read-only=on run

Running that on the new setups gave these results:

Transactions Per Second

• MySQL 5.0: 1309.28
• MySQL 5.1: 425.23

This confirms that after removing all of the variables associated with our setup and our applications the performance regression was still present. MySQL 5.0 was performing more than three times better than 5.1 on this read only benchmark.

The Investigation

The first thing to investigate were the MySQL status variables. I noticed that MySQL 5.1 had a really high query cache lowmem prune rate, especially when compared to the insert rate. I checked the same query cache variables in 5.0 and found that the query cache wasn’t being used at all. This was a surprise since the query cache was turned on in both installs and configured in the exact same way.

MySQL 5.1

mysql> show status like '%Qcache%';
+-------------------------+---------+
| Variable_name           | Value   |
+-------------------------+---------+
| Qcache_free_blocks      | 2549    |
| Qcache_free_memory      | 9036656 |
| Qcache_hits             | 74182   |
| Qcache_inserts          | 298916  |
| Qcache_lowmem_prunes    | 291970  |
| Qcache_not_cached       | 92      |
| Qcache_queries_in_cache | 6946    |
| Qcache_total_blocks     | 16442   |
+-------------------------+---------+
8 rows in set (0.00 sec)

MySQL 5.0

mysql> show status like '%Qcache%';
+-------------------------+----------+
| Variable_name           | Value    |
+-------------------------+----------+
| Qcache_free_blocks      | 1        | 
| Qcache_free_memory      | 16759744 | 
| Qcache_hits             | 0        | 
| Qcache_inserts          | 0        | 
| Qcache_lowmem_prunes    | 0        | 
| Qcache_not_cached       | 1105842  | 
| Qcache_queries_in_cache | 0        | 
| Qcache_total_blocks     | 1        | 
+-------------------------+----------+
8 rows in set (0.00 sec)

You can see that MySQL was pruning almost as many queries from the query cache as it was inserting in 5.1.

As an experiment I turned off the query cache in both installs and ran the benchmarks again. This time I got these results:

Transactions Per Second

• MySQL 5.0: 1305.35
• MySQL 5.1: 1313.37

It was now pretty clear that I had found the performance problem. It had something to do with a change in the way the query cache operated between 5.0 and 5.1. A quick googling for changes in the query cache between versions didn’t turn anything up, so it was time to dig a bit deeper.

I realized that I didn’t know much about the queries sysbench was running. This can be found out pretty easily by turning on the general log in MySQL. I ran the benchmarks again and checked the logs. I found that sysbench uses 10 different types of queries and that it executes them using prepared statements. Now this time google was able to give me some answers. As of 5.1.17, the query cache can cache prepared statements.

From skimming the general log after running sysbench it was clear that the read only benchmark was a worst case scenario for the query cache. It was a lot of simple, generally non-repeating, selects. This means that there is going to be a high query cache insert rate, but low hit rate (because they’re unlikely to repeat), and because so many are being inserted the cache is going to be constantly pruned.

Determining the value of the query cache is a whole separate problem. Some would argue that you’re better off without it. But, I wasn’t trying to evaluate the query cache, and since our apps don’t rely on prepared statements the results I was getting didn’t give a fair representation of how our app would perform.

The Takeaway

It was important to figure out what caused the performance regression. It couldn’t just be ignored, even though the other three benchmarks all agreed with each other. Once I figured out what caused the strange results (prepared statements are cached using the query cache) I could make a decision as to whether or not that was important. If you use a lot of prepared statements in your apps, then this information is very valuable and should be considered in your decision to upgrade.

By setting db-ps-mode to disabled you can tell sysbench not to use prepared statements. Once I determined that this better reflected the actual kinds of selects we used in our apps I could run the benchmarks again without the prepared statements:

sysbench --test=oltp --oltp-table-size=1000000 --mysql-db=sysbench --mysql-user=root --max-time=60 --max-requests=0 --num-threads=4 --oltp-read-only=on --db-ps-mode=disable run

Permalink | Leave a comment  »

I may be a little late to the Dependency Injection (DI) party, but it's something I've been thinking about a lot recently. We're experimenting with writing more isolated unit tests at work and one of the tools we're planning on using is DI.

Here's the basic idea as I see it. When you design software in an object oriented way you end up with an object graph. I define an object graph as the image (mental or otherwise) that defines the relationships between your objects. With DI you take those links between objects and make them more flexible.

For example, let's imagine that object A needs objects B and C to do some task. Object A could just instantiate objects B and C and then use them to perform work, but that means the links between object A and objects B and C are very rigid.

Now, refactor the code so that object A requires objects B and C to be passed in. The links on the object graph are now a little less rigid because as long as an object follows the interface offered by object B it could be passed in instead of B. Object A doesn't care as long as it works the same.

The real value to me is that when testing object A I can be less concerned about objects B and C. For my test I'll assume they work correctly and pass in mocked versions of them that return expected values for their methods. Then I focus on making sure object A works correctly. In a different test I'll test object B. With those two pieces tested I've tested each piece in isolation which is easier to process when writing a test, but I've also tested everything as long as B's interface stays the same.

That's the biggest downside to unit testing in isolation I've seen. You'll also need a functional test that ensures object A and object B interact properly and agree on B's interface.

One of the main criticisms I've seen is that DI is the enemy of encapsulation. Encapsulation is the idea that every object only exposes the things it must to other objects. This criticism feels very valid to me. At the moment I'm giving up that encapsulation for more isolation in testing.

I also like DI for passing in other dependencies. For example, passing in a logging function. Maybe in production you want to write to the syslog. In your tests you may just want to make sure the correct values get sent to the log but you don't want to have your tests check the syslog. One way around this is to have your object take in a logging function and in your tests have that function write to STDOUT or some variable you can check.

While looking around the web on the subject I was surprised to find that it appears to be controversial. Some complain that it's not new idea at all and others that it's actually a bad idea. I was sold on the theory the first time I read about it, but haven't used it a lot in practice. I'm still learning and it's something I'm trying to think about as I write code. It'll be interesting to come back to this post and topic in a few months and see how I'm doing.

Here are a few links to read about the disagreement on DI's usefulness or correctness. The Wikipedia talk page is especially interesting.

• Wikipedia talk page on Dependency Injection
• Does Dependency Injection Pay Off?
• Defending Dependency Injection

Permalink | Leave a comment  »

Here’s my Tron Legacy inspired Tron-O-Lantern. The design is based on the costume design seen in promotions for the upcoming movie. The glowing comes from 4 LEDs (3 blue, 1 green) and is powered by an Arduino. The Arduino is over-kill in this case, but it’s what I had.

I’d do a couple things differently with the design if I did another pass. There were issues that I didn’t notice until it was carved. For instance, the center rings should be the focal point, but they’re not because they’re too small and dim.

Here’s a clearer picture of the pumpkin:

Here’s the design I put together in Inkscape:

Here’s a shot of the Arduino set up:

I just put the LEDs and resistors directly on the Protoshield, powered it with a 9 volt battery and put the whole set up in a gallon size plastic bag.

Permalink | Leave a comment  »