WhatHeaders.com - HTTP client inspection tool

Here's a small web application you may find useful:

What HTTP Response Headers - WhatHeaders.com

I made it this past Saturday, when the rain outside wouldn't stop. Basically, when any HTTP client makes a request to this URL, the request headers are reported back. It's useful when you are working with any kind of "black box" HTTP client you want to understand better. I made it to easily get this information for mobile phones and handheld devices, whose behavior is often mysterious and aggressively undocumented.

As a convenience, you can have the header info emailed to you (again, quite handy (har!) on mobile devices). Let me know what you think.

A little helper for us command-liners

Here's a small Unix/Linux script I coded up spontaneously this afternoon.

1. #!/bin/bash
2. #
3. # dl - fetch remote file for convenient local reading
4. #
5. # (That's lowercase "DL".)
6. #
7. # dl will fetch a document at a given URI, save to a temporary file,
8. # and then print its local path to standard output.  This behavior
9. # makes dl handy on the Unix/Linux command line for treating remote
10. # URIs like local files.  For example:
11. #
12. #  less `dl http://www.ietf.org/rfc/rfc3454.txt`
13. #  xpdf $(dl http://www.cs.harvard.edu/~malan/resources/cheatsheet.pdf)
14. #  emacs $(dl ftp://rtfm.mit.edu/pub/net/internet.text)
15. #
16. # dl supports the HTTP, HTTPS and FTP protocols.  It depends on wget
17. # for downloading; thus you must have wget installed.
18. #
19. # Error handling: In the event the document cannot be fetched, dl will
20. # emit a warning to stderr, print nothing to stdout, and yield a
21. # nonzero exit code.  You might want to consider whether the invoked
22. # primary command (e.g., "less", "xpdf" or "emacs" in the examples
23. # above) will fail gracefully enough if this happens.
24. #
25. # dl is in the public domain.  If it breaks, you get to keep both pieces.
26. #
27. # Enjoy.  -Aaron (amax@redsymbol.net)
28.  
29. url=$1
30. if [ -z "$url" ]; then
31.     echo "usage: dl URL"
32.     exit 1
33. fi
34. bucket=$(mktemp -d -t)
35. dest="$bucket/object"
36. wget -q -O "$dest" "$url"
37. wget_ec=$?
38. if [ $wget_ec -ne 0 ]; then
39.     echo "ERROR: dl: url '$url' does not return a document (wget exit code $wget_ec)" 1>&2
40.     exit 2
41. fi
42. echo "$dest"
43.  

Dealing with multiple Django versions

More than a few Django websites I built in the past two years are still live and active. And because they were made at different times, they were built against different Django versions.

If a (virtual) web host accidentally imports a different version of django, the site will completely break at best... or exhibit subtle, hard-to-discover runtime errors at worst. Guard against this by placing a version check in settings.py:

from django import VERSION as DJANGO_VERSION
assert (1,1) == DJANGO_VERSION[:2], DJANGO_VERSION

(If, for example, this web site is built for version 1.1. Change the tuple in the assert statement to match whatever version is required.)

This will cause the host start-up to immediately fail, loudly and visibly, if the incorrect Django version has been imported.

Django: AttributeError: 'str' object has no attribute 'resolve'

Here is today's obscure error message and its solution.

Say you are working on a Django project, using its development web server, and you get this exception when you try to load a page in the browser:

AttributeError: 'str' object has no attribute 'resolve'

It's because you forgot to type the word "patterns".

Specifically, in some url.py, you typed something like this:

urlpatterns = ('',
  (r'^$', direct_to_template, {'template' : 'a.html'}),
  # ...

when you should have typed this:

urlpatterns = patterns('',
  (r'^$', direct_to_template, {'template' : 'a.html'}),
  # ...

See the difference? In the first one, I'm incorrectly assigning urlpatterns to be a tuple. In the second, I'm correctly using the django.conf.urls.defaults.patterns function.

File this under "issues that cost me an annoying amount of time to solve, and which I'm documenting so it doesn't have to happen to anyone else."

How To Build an Automated Testing System

What is the value of automating your software testing as much as possible?

At a talk I gave once to a large room of software engineers, I asked for a show of hands: how many write unit tests all the time? And, how many use version control for nearly all their code? Almost every hand went up on both counts. I was proud of them for this, and told them so!

A little later in the same talk, I asked how many had use a code coverage tool on their code in the past week. Only fifteen percent! A tiny fraction. And these were undeniably excellent engineers. What would the ratio have been for a more mediocre group?

Automation is a form of leverage. If each attendee's organization had an automated QA system that simply did a source checkout each night, and executed all tests within a code coverage tool while everyone slept, then everyone would have been able to raise their hands... even if they did not know how to invoke the tool!

This situation demonstrates why an automated testing system is important. By silently working for you and your team, all measurable metrics are just there and available. You won't use them all the time. That's fine. They are there when you need them, without distracting you from more important matters.

A great automated quality assurance system will have a number of good ingredients. Let's look at them.

(And by the way, this list is unapologetically opinionated. It's not necessary to agree with each item. But everything is there for a reason. I urge you to at least understand the rationale behind it. Then you are well equipped to decide whether to do it this way or another way. Send questions and/or criticisms to amax@redsymbol.net.)

Read the full version of Building an Automated Testing/Quality Assurance System

Does it take money to make money?

Let's talk about a phrase you have heard before:

"It takes money to make money."

Like many aphorisms, it must be handled with great care. It can be amazing that the words you listen to can affect what you believe, and thus how you act.

"It takes money to make money." One could use this idea to keep themselves poor, reasoning that they can never afford to become wealthier than they are now.

I would prefer that this not happen to you. So let me suggest a different wording that may be more profitable.

"It takes SOME money to make MORE money."

And very importantly,

"You can create SOME money out of nothing."

Is this dual aphorism true? Just as true as the original, perhaps more.

If you have money, can you utilize it to create even more? Yes, absolutely. Can you create money out of thin air? Yes, there are many ways. One way is to sell your time and labor to those who will pay for it. There are other ways.

Some recommended reading:

• How To Make a Million Dollars
• Making Money Twice
• Is effort a myth?

Django and Lighttpd: one niggling fastCGI detail

If you are setting up a Django website using the lighttpd web server, one of the easiest ways to configure it is via fastcgi, which lighttpd has built-in support for. As I write this, the official Django FastCGI docs do a very good job of explaining everything, provided you read the whole document. (Don't just skip to the lighttpd section - relevant info is contained earlier, perhaps even in the Apache sections.)

The docs skimp over one detail that happened to affect me, however, which I'd like to document here in hopes that it will save you some time: the FORCE_SCRIPT_NAME setting. At first, when I called up the admin site and clicked the submit button to log in, I got a 404 error, for a url like "/mysite.fcgi/admin/". mysite.fcgi (not its real name :) is the url prefix I configured for the fastcgi rewrite rule in the lighttpd configuration.

After much research and fruitless tweaking of the config files, I thought to look at the HTML source of the admin login page (which served just fine). Turns out the action attribute of the login form element was set to "/mysite.fcgi/admin/", not "/admin/" or ".", like it should be. At the tail end of the above docs, I got a clue as to the cause. Long story short, defining FORCE_SCRIPT_NAME to the empty string in settings.py solved the problem.

There... hope this saves someone an hour or two!