Archive for the 'Python' Category

Geek Beacon
Greg Abbas

One of the things that I learned during my time at the MIT Media Lab is that computers are more interesting if you make them interact with us in our world, instead of demanding that humans do things that are convenient for computers, like using mice and keyboards. That's what I like about the idea of so-called "eXtreme Feedback" (XF)... if your team is developing software and there's a metric that the team should pay attention to (like whether the build is broken, whether unit tests are passing, etc) then make a device that reports that metric in some way that's more "real" than a web page on a computer monitor. Instead of just sending out an automated email that says "the build is broken!", put a device in the real world that shows whether the build is currently in good shape or not. Such a device is called an "Extreme Feedback Device" (or "XFD" for short) and people have thought of some creative ones, including lava lamps, bears, and orbs.

200907292301.jpg

If the software your team is building is compiling correctly, then the green lava lamp turns on. If there's an error, then the red one turns on instead. I wanted to give an XFD to my team, but like so many engineers, I had the strong desire to build it myself instead of buying one off-the-shelf. I decided to try a more "steam punk" aesthetic, to see if I could give a device a less modern look. The result is the Geek Beacon.

IMG_0480.jpg

The base is made from an antique wooden coffee grinder, and the shade is a vintage ceiling light from a 1939 Sears & Roebuck catalog. But despite the old-fashioned appearance, it has a microcontroller with an embedded TCP/IP server that can make the lamp shine any color of the rainbow, including animations. If you look carefully at the photo, you'll see two cables coming out the back: one is for power, and the other one is ethernet. Here's what it looks like when it's turned on (showing blue):


IMG_0481.jpg

I'm not as good with hardware as I am with software, so a few years ago this would've been an extremely challenging project for me. But around the time I learned about XFDs, I also discovered the Arduino Project. Arduino is an "open-source electronics prototyping platform", which basically means they've already done the hard work developing a well-designed microcontroller that has excellent input/output support. You can combine several circuit boards together by stacking them on top of each other, so that's what I did and it was great fun. I started with an Arduino Diecimila that I got for my birthday (thanks Mom & Dad!), added an "Ethernet Shield" to handle the network interface, and finally a "Proto Shield" to hold the LED driver circuits that I wanted to build. Here's what the boards look like (ethernet on the left, LED driver in the middle, and Diecimila on the right):

IMG_0492.JPG

They fit together using the headers on the sides of each board. When stacked up, they fit inside the drawer in the coffee grinder.

IMG_0489.jpg

For output, on top I hooked up four "Super Flux" RGB LEDs:

IMG_0496.jpg

Getting all sixteen LED leads soldered just right was probably the most challenging part of the whole project, so if I had to do it over again I think I would probably use something that required a little less assembly, like maybe the BlinkM RGB Blaster.

For the enclosure (all the non-electronic stuff) I scoured several local antique shops trying to find stuff that struck the right chord. But I didn't find anything I loved, so I ended turning to the internet. Got the coffee grinder on Craigslist in Louisville Kentucky, and the lamp shade on Ebay. And then I needed a few bits of obscure lamp hardware that I bought from a company in Connecticut (through a series of tubes).

I can sketch the circuit if anyone's really interested, but it's pretty simple... just a transistor per LED to give the PWM output lines a little more kick.

Right now this gizmo is still in my home office, but stay tuned for part 2, "Geek Beacon goes to Apple!"


A Lego Teapot
Greg Abbas

SIGGRAPH's conference this year in Boston is going to include a Teapot Exhibit, celebrating the Utah Teapot. That got me thinking, "I wonder if I could build a teapot out of Legos?" Round objects aren't exactly Lego's forte of course, but I figured a computer program that translated the model into a voxel lattice would go a long way towards getting me started. I wrote that in C++, and then wrote another program in Python for turning the lattice into an arrangement of Lego bricks.

In the process of pursuing this project, I'm learning just how serious some people take their Legos. There are several CAD programs to choose from, and I ended up using Bricksmith because it has a nice Mac OpenGL implementation. (Thanks Allen!)

Then there's the question of how to buy Lego bricks to actually build the teapot with in real life. BrickLink provides a huge online market for Lego pieces. Wow! You can search for exactly what part you need, and you get a list of people across the country who will sell it to you. So I bought a pile of all kinds of white bricks, plates, and slopes, and started trying to put my masterpiece together. It's harder than I thought... I'm definitely not a "master builder" as they call it. But I'm getting there. I may have to go back to Python for some more help from the computer. :-)


MetaWeblog and security
Greg Abbas

I've been using MetaWeblog to allow me to write articles for this blog in Ecto, and it's still working very well. There's a security issue that's worth mentioning, though, which is that MetaWeblog sends authentication information (i.e., your password) over the network without encrypting it. It'd be really nice if it used an MD5 digest scheme or something instead of cleartext, but it doesn't. As I mentioned previously, unfortunately MetaWeblog has several deficiencies and this is one of them. As Jacob Kaplan-Moss points out, this means that unless you encrypt your connection with HTTPS, your password is vulnerable to snooping, so hooking up HTTPS is probably a good idea.

Now that I think about it, I suppose the same argument applies anywhere you might POST a password in Django, including the Admin app. So if you don't want the other folks in your internet cafe getting unauthorized access to your website, then I guess you should encrypt the Admin app too. Just because you're paranoid doesn't mean they're not out to get you. :-)


Avatars
Greg Abbas

Pictures are more interesting than words... it's cool to represent a person or a web site using an icon instead of just a name. I'm obviously not the first person to realize this; avatars are cropping up on the web in more and more places. I like how Lisa McMillan added favatars to her "hall of fame", for instance. But her comment regarding the fact that technically she's stealing their bandwidth made me think that it'd be courteous to set up a Django app that cached those images, rather than linking straight to them. The extensibility to add features like that is exactly why I'm using Django! Here's a description of how it works.

My goal was to provide avatars for users who write comments on my posts. Two styles of avatars are supported: favatars (the favicon from the user's web site) and gravatars (images associated with the users email address).

Usage. To insert an avatar, use the "avatar" template tag that takes two parameters: the name of an object, and a size (in pixels). The object is queried for "email" and "url" attributes, which django.contrib.comments doesn't support so I copied the comments app and added them myself. The size attribute works best if it's a multiple of 16 (because that's how big a favicon is) and less than or equal to 80 (because that's how big a gravatar is), so I use 48. Here's what my template looks like:

  {% for comment in comment_list %}
  <li class="item" id="comment-{{ comment.id }}">
    {% avatar comment 48 %}
    ...etc...
  </li>
  <% endfor %>

The object doesn't have to define both email and url attributes... if you just want favatars, then all you need is a url. If an object does have both kinds of avatars, the gravatar gets priority (because they're typically higher res). Also, there are some variables to define in your settings.py to control how the tag operates. Two are required, the rest are optional:

AVATAR_STORAGE_DIR. Required. The name of a directory where the downloaded avatar images may be stored.
STATIC_DIRS. Required. A tuple of pairs, each one containing the name of a directory and the URL with which to access it. For instance, (('/var/htdocs/media', '/site_media'),) would let the avatar tag know that if the AVATAR_STORAGE_DIR is /var/htdocs/media/my_avatars, that it can generate urls that start with "/site_media/my_avatars/".
AVATAR_REFETCH. Default: 1 day. The minimum amount of time between requests to the original web server to see if an image has changed.
AVATAR_LIMIT. Default: 0. Like AVATAR_REFETCH, but this is the minimum amount of time between requests if the user hits the "reload" button (as indicated by HTTP_PRAGMA=no-cache).
AVATAR_DEFAULT_IMAGE. Default: no image. The image to show if no real avatar is available.

And here's the source code: avatar.py

Next, I want to use this to support favicons for a blogroll, of course. But one thing at a time. :-) And there are other features that could be added, like pre-fetching images asynchronously or deleting old files from the cache, but I don't get enough traffic to justify things like those.


XMLRPC in Django
Greg Abbas

In setting up a blog server, one of my requirements was that I wanted to use a desktop client to write articles, instead of doing it in an HTML form in a web browser. I accomplished that by purchasing Ecto (which I'm using to type this article) and adding a MetaWeblog interface to the server. Here's the first of two articles on how I did it. Today's article talks about the XMLRPC layer, and the second in this series will conclude with the MetaWeblog implementation itself.

MetaWeblog is a network API for posting blog articles, and it's one of the ways that Ecto can talk to a Blog server. There are some other common blog APIs, including Typepad, Moveable Type, Blogger, and Atom, and I could have chosen any of them. As I mentioned in a previous post, I strongly considered Atom, because it seems to be the newest contender and claims to address some of MetaWeblog's shortcomings. And it's RESTful, which is elegant. But when I tested Ecto with the Atom API, it didn't seem to support images. I don't know whether that's Ecto's fault or Atom's fault, but it would be a problem for me in any case. So I decided to be pragmatic and go with MetaWeblog for now.

MetaWeblog is based on XMLRPC, which (like the name suggests) is a protocol for making remote procedure calls over HTTP. So the first order of business was to add XMLRPC support to Django. I'm not the first person to do that; Amit Upadhyay and Georg "hugo" Bauer have done some good work in this area, and have some excellent ideas about how it ought to fit into Django as you can see from their comments on Django ticket #115. Amit's implementation looked similar to what I had in mind, so I started with his code. Here's what I came up with: xmlrpc.py.

The basic idea of XMLRPC is that a client (like Ecto) makes a procedure call (like "publish new post") by POSTing an XML document over HTTP. The web server (running Django, in this case) parses the XML into a method name and an unmarshalled set of arguments. It then locates the method that the client requested, executes it, and marshalls the returned value which is packaged inside an XML document and returned to the client.

How does that fit into the Django world-view? It seems to correspond to one entry (per API) in urls.py, because XMLRPC requests aren't sent to different URLs depending on which method you're calling or what object you're manipulating. But it makes sense for methods in an API (such as metaWeblog.newPost or metaWeblog.getCategories) to map to methods in a Python module (like views do) so we need an "XMLRPC dispatch view" to take care of this mapping. You specify the module containing your API methods by passing the name of the module as an argument to the view. For instance, here's a line from my urls.py:

(r'^metaweblog/', 'ayp.xmlrpc.view', {'module':'ayp.blog.metaweblog'}),

"ayp.xmlrpc.view" is the name of the dispatcher, and when it receives a request, it looks in the "ayp.blog.metaweblog" module for a corresponding method. This makes it slightly different from Amit's version, in a few ways:

  • The API methods are implemented as functions in a module, not methods in a class.
  • If a client requests a method whose name has a period in it (as many XMLRPC method names seem to) then it'll try translating the period into an underscore. This is a convenience feature that lets you call your methods things like "metaWeblog_newPost".
  • I made ayp.xmlrpc provide a callable view instead of a class, so that you don't have to explicitly instantiate a class or explicitly register an instance in your API module.

I did keep his "public" attribute which lets you declare which functions should be web-accessible, and I added a decorator (Python 2.4 style) for setting it. This is a safety feature so that you don't accidentally expose private methods.

Here's an example. Let's say we have an XMLRPC API that contains one method, "arithmetic.add". To implement it, we'd write an arithmetic.py that looks something like

from ayp.xmlrpc import public
@public
def arithmetic_add(a, b):
    return a+b

and then expose it in our urls.py with something like

urlpatterns = patterns('',
    (r'^arithmetic/', 'ayp.xmlrpc.view', {'module':'arithmetic'}),
)

If you put that into a running Django server, you can test it from a Python shell:

>>> from xmlrpclib import ServerProxy
>>> server = ServerProxy("http://localhost:8000/")
>>> print server.add(3,4)
7

Good times! Read "MetaWeblog & Django" to see how we apply this to blogging.