BitWorking

Christopher, my 13 year old, has been playing with Flash for quite a while and has produced his first game: Gravity Mouse. It's a really neat concept and has already gotten 28 reviews and received a score of 2.89/5.0 on Newgrounds. I'm really proud of him.

Of course, this is what he was doing when he should have been working on his C++ homework, but I think I'll cut him some slack in this case.

I'm not following HTML5, so this was pleasantly surprising news:

The last few weeks there has substantive discussion on the HTML WG mailing list (public-html) and the Math WG public dicussion mailing list (www-math) regarding embedding non-HTML languages in the text/html serialization of HTML focusing mostly on MathML and SVG.

It will be nice when the majority of the web has access to both SVG and MathML.

Brian Redfern

Just look at the Css Zen Garden website to see a great demo of the power of xhtml at work.

Excuse me? CSS Zen Garden was about - you know - CSS.

The latest draft of the URI Template spec is available:

http://www.ietf.org/internet-drafts/draft-gregorio-uritemplate-03.txt

As usual you can find the HTML, diffs from the previous version, and an updated URI Template explainer web service here:

http://bitworking.org/projects/URI-Templates/

I folded in much of the feedback on the 02 draft. Here are the notes from the revision history:

Added more examples. Introduced error conditions and defined their handling. Changed listjoin to list. Changed -append to -suffix, and allowed -prefix and -suffix to accept list variables. Clarified the handling of unicode.

As usual feedback is welcome on the W3C URI mailing list:

http://lists.w3.org/Archives/Public/uri/

Official Google Data APIs Blog:

We've always encouraged other developers to adopt Atom, the Atom Publishing Protocol, and the extensions that Google has created on top of those standards, but we realized the issue of patents may have held back some adopters. Well, those concerns end today as we are giving a no-charge, royalty-free license to any patents we have that you would need to implement Atom, AtomPub, or any of those extensions.

Cliff Click, chief JVM architect at Azul Systems as quoted in InfoWorld:

As your program grows in size, the lack of strong typing basically kills your ability to handle a very large program and so you don't find the million-line Perl program

That line in particular has elicited some reactions, including this from chromatic:

Second, the reason that there aren’t many million-line Perl programs is that the people who are capable of writing and managing million-line Perl programs have better ways to organize their projects than glomming a million lines of Java into a single shared-everything instance. That’s setting aside the qualities of encapsulation and abstraction that Java-the-language doesn’t have, preferring instead to push that problem to tool vendors and AbstractFactoryFactoryInjectors which consume vast swaths of XML to get around Java’s static code fetish. I can only imagine how much larger the Java code would be without all of those XML files.

I was always baffled with Java folks love affair with XML until I realized it was just a crutch to make up for a lack of map and array literals in the language.

Another quote from the InfoWorld article:

Java is not the slowpoke of old days and performance now matches or exceeds applications developed in C

I found that interesting that only 13 years after being released Java is finally on par with C performance, particularly given that some people believe that a garbage collected language should be faster than a non-garbage collected language.

One of things that's been most puzzling about Bush is his reaction to failure, that maddening wacky grin and jaunty manner as the entire country, if not the entire planet, careens towards the abyss - Iraq, climate change, torture, Katrina, warrantless surveillance, and two recessions - just to name a few issues. This article, "Bush and the Psychology of Incompetent Decisions", does a great job of explaining the psychology behind his behavior.

As his decisions go awry, he exudes a troubling, uncanny aura of certitude (though some find it reassuring). He seems to expect to feel despised and alone (and probably has always felt that), as he has always secretly expected to fail.

We'll spend the next fifty years mopping up after him and his daddy issues.

So let me get this straight, you can use hydrogen sulfide (the toxic part of sewer gas) to induce suspended animation in mice, and someone's actually regrown a fingertip after applying extracellular matrix made from pig bladders. That's just so Harry Potter.

DeWitt Clinton:

...a number of interesting relationships were still hidden inside my private Google Reader subscription list. This short guide demonstrates how I extract that information and publish it publicly via plain HTML decorated with XFN markup.

Sweet, I didn't know you could do that. The secret sauce is the new options that appear once you've tagged a bunch of feeds and then made that group of tagged items 'public'. I've now used the same technique to add yet another column to my me page. [code]

AppClientTest has been updated again. Every warning or error generated now contains a link to the section of the specification that is being violated. I also have incorporated Mike Samuel's Javascript Code Prettifier so that all the XML requests and responses are easier to read. Finally, the request method and URI are also present in the report for every request. Here is an example report if you want to see these enhancements in action.

Of all the responses to Joel Spolsky's tortured pantomime of impartiality, Henri's is the best, and Mark's is the funniest.

AP:

President Bush is poised to veto legislation that would bar the CIA from using waterboarding — a technique that simulates drowning — and other harsh interrogation methods on terror suspects.

Now we know what the W stands for.

Brad Fitzpatrick:

I'm enjoying this rise. The sooner it gets to $5 or $10, the sooner people might care?

Agreed. A doubling or tripling of the relative price of gas is inevitable, and is probably the best long-term thing that could happen to the country, fully realizing that in the short term it's going to be painful.

IEBlog:

Under the hood, an Activity uses a simple XML format called OpenService. Just implement an XML file that describes how your service works with URL templates.

URL Template? Is that like a URI Template?

OpenService Format Specification for Activities

All variables are treated as required unless it [sic] the modifier "?"

Rats, those aren't URI Templates. But thanks for picking a different name.

Just for a little context, when discussing URI Templates the use of the trailing '?' was considered for marking a template variable optional, but in the end that was considered metadata and could appear in the surrounding document, just like it could appear the OpenService Format.

Question: URIs have a very limited character set. What happens if I have a URL Template that contains {selection} and I happen to select "☄"?

I just checked in the last of some changes I've made to appclienttest to support testing. Yes, that's right, tests for appclienttest itself. This actually required a bunch of small changes through a lot of different code, but in the end I think it's a general solution that will be useful.

The big problem with testing appclienttest is that it runs through a lot of different steps as it puts an AtomPub service through its paces. Instead of it being like a series of function calls it's more like a little play, a drama in HTTP, where each scene relies on the last scene completing successfully.

Now I already had MockHttp in atompubbase which reads files out of a directory to emulate the behavior of httplib2.Http, but creating a directory structure populated with all the files needed for a run of appclienttest was going to be tedious, so I added mockhttp.MockRecorder. This class is another mock for httplib2.Http, (probably not really a mock, but more a wrapper) that wraps httplib2.Http and records every response it receives into a directory structure. It's the mirror image functionality of MockHttp.

With those in place I was now able to add the 'record' and 'playback' command-line options to appclienttest:

$ python validator/appclienttest.py --help
usage: appclienttest.py [options]

options:
  -h, --help          show this help message and exit
  --credentials=FILE  FILE that contains a name and password on separate lines
                      with an optional third line with the authentication type
                      of 'ClientLogin <service>'.
  --output=FILE       FILE to store test results
  --verbose           Print extra information while running.
  --quiet             Do not print anything while running.
  --debug             Print low level HTTP information while running.
  --html              Output is formatted in HTML
  --record=DIR        Record all the responses to be used later in playback
                      mode.
  --playback=DIR      Playback responses stored from a previous run.

So now I can run appclienttest against my site and record the traffic:

$ python validator/appclienttest.py --record=./validator/rawtestdata/complete/ --html --output=test.html

And play it back any time I want:

$ python validator/appclienttest.py --playback=./validator/rawtestdata/complete/ --html --output=test.html

The one caveat with this system is that appclienttest generates random slugs when it adds new entries, and obviously the recorded responses won't reflect those changing slugs on playback so all playback will have Slug warnings.

Here's what the directory structure looks like when I record a run of appclienttest run against my AppTestSite:

$ tree validator/rawtestdata/complete/
validator/rawtestdata/complete/
|-- DELETE
|   `-- projects
|       `-- apptestsite
|           `-- app.cgi
|               `-- service
|                   |-- entry
|                   |   |-- 1.file
|                   |   |-- 2.file
|                   |   `-- 3.file
|                   `-- media
|                       `-- 1.file
|-- GET
|   `-- projects
|       `-- apptestsite
|           `-- app.cgi
|               |-- service
|               |   |-- entry
|               |   |   |-- 1.file
|               |   |   |-- 1.file.2
|               |   |   |-- 2.file
|               |   |   |-- 2.file.2
|               |   |   |-- 2.file.3
|               |   |   |-- 3.file
|               |   |   `-- 3.file.2
|               |   |-- entry.file
|               |   |-- entry.file.2
|               |   |-- entry.file.3
|               |   |-- media
|               |   |   `-- 1.file
|               |   |-- media.file
|               |   `-- media.file.2
|               `-- service.file
|-- POST
|   `-- projects
|       `-- apptestsite
|           `-- app.cgi
|               `-- service
|                   |-- entry.file
|                   |-- entry.file.2
|                   |-- entry.file.3
|                   `-- media.file
`-- PUT
    `-- projects
        `-- apptestsite
            `-- app.cgi
                `-- service
                    |-- entry
                    |   `-- 2.file
                    `-- media
                        `-- 1.file

26 directories, 30 files

Which gives me a nice directory of files I can later playback, or more importantly, can modify and playback to test different scenarios.

Now we're all set for some testing. Just record a good session, tweak the responses in the recording directory to trigger specific error conditions and then check for those errors on the output of a playback run against that directory, which is exactly what runtests.py in the validator subdirectory does.

One last change you may notice is that the error and warning messages have been enhanced, giving the specification and section number being violated. The next step will be to turn those into real links in the HTML output.

Sam Ruby:

Spend some time up front specifying the behaviors that you want to address.  In the case of Atom, adding an entry, deleting an entry, adding a category to an entry, fixing a typo in the content are examples of common scenarios.  Feel free to use the Atom wiki for this purpose.

To move the discussion of PATCH forward I've posted some examples on the Atom Wiki. They're good examples because the highlight the problem with coming up with a PATCH format for Atom. The largest of those problems is ordering in Atom. That is:

   <entry xmlns="http://www.w3.org/2005/Atom">
       <title>Atom-Powered Robots Run Amok</title>
       <link href="http://example.org/2003/12/13/atom03"/>
       <id>urn:uuid:1225c695-cfb8-4ebb-aaaa-80da344efa6a</id>
       <updated>2003-12-13T18:30:02Z</updated>
       <summary>Some text.</summary>
    </entry>

and

   <entry xmlns="http://www.w3.org/2005/Atom">
       <updated>2003-12-13T18:30:02Z</updated>
       <title>Atom-Powered Robots Run Amok</title>
       <link href="http://example.org/2003/12/13/atom03"/>
       <id>urn:uuid:1225c695-cfb8-4ebb-aaaa-80da344efa6a</id>
       <summary>Some text.</summary>
    </entry>

are the same even though the <updated> element has moved. You can create a patch format that required the server to always serialize elements the same order. For example, in the first example we could order our elements:

   <entry xmlns="http://www.w3.org/2005/Atom">                #1
       <title>Atom-Powered Robots Run Amok</title>            #2
       <link href="http://example.org/2003/12/13/atom03"/>    #3 
       <id>urn:uuid:1225c695-cfb8-4ebb-aaaa-80da344efa6a</id> #4
       <updated>2003-12-13T18:30:02Z</updated>                #5
       <summary>Some text.</summary>                          #6
    </entry>

Then our patch format becomes a matter of sending over the element number, and information on either the updated element value or updated attribute values. For example, to change the summary to "foo" we could send:

{
  "n": 6,
  "value": "foo"
}

But given that order is not significant for Atom we may not want to use that approach.

Once you step outside of ordering you need some other way of addressing the elements and attributes that are changed. Note that relying on an already existing XML technology like XPath doesn't solve the problem. For example, here is a severely elided example from Google Calendar:

<?xml version='1.0' encoding='utf-8'?>
<entry xmlns='http://www.w3.org/2005/Atom'
    xmlns:batch='http://schemas.google.com/gdata/batch'
    xmlns:gCal='http://schemas.google.com/gCal/2005'
    xmlns:gd='http://schemas.google.com/g/2005'>
  <id>http://www.google.com/calenda...ervobk3ng</id>
  ...
  <gd:who rel='http://schemas.google.com/g/2005#event.attendee'
      valueString='Fred Flintstone' 
      email='fred@example.com'>
    <gd:attendeeStatus value='http://schemas.google.com/g/2005#event.invited' />
  </gd:who>
  <gd:who rel='http://schemas.google.com/g/2005#event.organizer'
      valueString='Joe Gregorio' 
      email='joe@bitworking.org'>
    <gd:attendeeStatus value='http://schemas.google.com/g/2005#event.accepted' />
  </gd:who>
  <gd:where />
</entry>

If I wanted to send back a patch to update Fred's gd:attendeeStatus, what XPath would I construct to isolate that element and attribute?

atom:entry/gd:who/gd:attendeeStatus/@value

No, since that will capture both attendees.

atom:entry/gd:who[position()=0]/gd:attendeeStatus/@value

No, since that relies on the ordering of the elements.

atom:entry/gd:who[valueString="Fred Flintstone"]/gd:attendeeStatus/@value

Maybe, but is there any guarantee that valueStrings are unique?

atom:entry/gd:who[email="fred@example.com"]/gd:attendeeStatus/@value

This is right, but how would a generic client know how to construct such an XPath?

I am bringing this up to highlight the questions that need to be answered.

1. Does the patch format work only for base Atom elements?
2. Will it work for any and all extensions?
3. Do we assume that all extensions are order independent?
4. Do we presume that the server can always reconstruct the element ordering?
5. Do we require out-of-band information to use or construct the patch representation?

The gd:attendeeStatus example points out that we either need out-of-band information, i.e. that @email uniquely identifies a gd:who element, or that order is preserved. The only other option is to include "in-band" information that makes constructing patches possible, for example, adding id attributes to each element.

I've just checked in an updated version of the AppClientTest, a program for testing your AtomPub service. It has been updated to use the atompubbase module and also sports HTML output as an option. To make it easier to develop I've transcluded via svn:externals all the libraries you will need as long as you are running Python 2.5. You will need to install elementtree if you are on something older than 2.5. The transcluding means it is this easy to get up and running from svn:

$ svn co http://feedvalidator.googlecode.com/svn/trunk/apptestsuite/client/validator/ validator
$ python validator/appclienttest.py --output=results.html "http://bitworking.org/projects/apptestsite/app.cgi/service/;service_document" 
$ firefox results.html

Here is a sample report (looks better with JavaScript enabled) after running it against my APP Test Site. The icons are courtesy of Tango.

Here are the command line options:

$ python validator/appclienttest.py --help
usage: appclienttest.py [options]

options:
  -h, --help          show this help message and exit
  --credentials=FILE  FILE that contains a name and password on separate lines
                      with an optional third line with the authentication type
                      of 'ClientLogin <service>'.
  --output=FILE       FILE to store test results
  --verbose           Print extra information while running.
  --quiet             Do not print anything while running.
  --debug             Print low level HTTP information while running.
  --html              Output is formatted in HTML


The code is far from complete. It does most of the tests of Tim's APE (with the exception of categories), but there are many more tests to add. The next step for me is to add unit tests for appclienttest.py, which should be fairly simple given the structure of atompubbase, and the use of MockHttp.

The Guardian:

Appetites whetted, Stiglitz and Bilmes dug deeper, and what they have discovered, after months of chasing often deliberately obscured accounts, is that in fact Bush's Iraqi adventure will cost America - just America - a conservatively estimated $3 trillion. The rest of the world, including Britain, will probably account for about the same amount again.

Yes, you too can create you own failed-state-terrorist-haven with nothing more than a lie and $3,000,000,000,000.

Byrne Reese:

Anyways, today on my favorite Gawker blog I saw the new BSG Season 4 cast photos.

I guess it's safe to come out and admit that io9 is my favorite new blog.

The kids have been spending hours playing with Phun:

The more I play with it, and watch them interact with it, the more I wonder why more user interfaces don't include physics.