New AtomAPI Implementation Release

We came, we saw, we did, we got spanked, we did it right.

Last week Mark Pilgrim and I released an implementation of the AtomAPI, both a client and a server. That implementation included a new authorization scheme that we came up with. Now we would have liked to used HTTP Digest authentication, and the AtomAPI should support Digest authentication, but for many users setting up Digest just isn't possible. Many users, like myself, are using a server that does not have Digest authentication turned on. Similarly not everyone has the ability to use .htaccess files, which you need to be able to modify to setup Digest autentication, that or modify your servers httpd.conf which is even rarer still. However we heard from CMS vendors that they want at least the level of password security that HTTP Digest offers. So we needed to come up with a scheme that:

1. Is a challenge-response Digest authentication scheme.
2. Is able to be handled by a CGI program with no .htaccess, httpd.conf tricks or requirements to be running as an Apache module, ala mod_perl or mod_python.
3. Gracefully extends current authentication schemes.
4. Is the simplest thing that could possibly work.

So what we did was a simple transposition of Digest authentication into custom HTTP headers:

1. Used sha1 instead of md5.
2. Triggered authentication by rejecting a request with an HTTP status code of 447 instead of 401.
3. The server response when rejecting a request with a 447 also included an Atom-Authenticate: header that included the 'nonce'. This parallels the role of the WWW-Authenticate: header in HTTP Digest.
4. The client sends an Atom-Authorization: header with all the Digest authentication information in it.
5. The server sends back an Atom-Authentication-Info: header with the 'nextnonce'.

Now this was a good first pass but we did several things wrong as Ken Coar pointed out on his site and on the atom-syntax mailing list. Basically it boils down to the fact that we ignored the built-in extensiblity of the HTTP authenciation mechanism and went around it starting with returning an HTTP status code of 447.

What we should have done:

1. Triggered an auth by rejecting a request with an HTTP status code of 401.
2. The server response includes an Authenticate: header that includes Atom as an authentication scheme.
3. The client then sends an Authorization: header with the scheme of Atom with all the Digest authentication information going into X-Atom-Authorization: header.
4. With every request the server sends back an X-Atom-Authentication-Info: header with the 'nextnonce'.

Note that this now uses the extensibility of the HTTP authentication scheme. When the server responds with a WWW-Authenticate: header, that header lists, in order of preference, the authentication schemes that the server supports. The client then selects the most preferred scheme that it also knows. Note that the whole authentication mechanism is designed to be robust to the addition of new/unknown schemes. We are leveraging this extensibilty by passing in an authentication scheme to Apache that it doesn't know about. The Apache server reacts appropriately by passing the request onto the CGI program to give it a chance to handle the authentication.

That is exactly what we did, and now have released new versions of the client and server that use the new authentication scheme. Just to reiterate that this scheme is another HTTP authentication scheme that works alongside Basic and Digest authentication. Servers can support one or more of Atom, Basic and Digest authentication. Note that the server side has changed so the old client will no longer be able to create or update entries. Sorry about that, such is life on the bleeding edge.

Here the details of how the Atom authentication scheme works:

1. The client tries to do something that requires authentication, for instance, POSTing a new entry to http://diveintomark.org/cgi-bin/atom.cgi/blog_id=14. The server sends back and HTTP error code of "401 Unauthorized", and a WWW-Authenticate header like this:

   WWW-Authenticate: Atom realm="some server realm name", qop="atom-auth", algorithm="SHA", nonce="some unique server-specific value"

2. The client takes the username, the realm given by the server, and the password, and concatenates them to create an intermediate value which we will call A1:

   A1 = username + ":" + realm + ":" + password

3. The client takes the HTTP verb it wants to use (in this case "POST") and the path part of the URL it wants to post to (in this case "/cgi-bin/atom.cgi/blog_id=14"), and concatenates them into an intermediate value which we will call A2:

   A2 = verb + ":" + uri

4. The client creates a unique client-specific value, which we will call "cnonce". How this happens is completely client-specific, but it should change on every request, and future values should not be guessable.

5. The client takes A1, A2, the qop given by the server, the nonce given by the server, and the cnonce created by the client, and creates a digest, which we will call "response":

   response = sha(sha(A1) + ":" + nonce + ":" + "00000001" + ":" + cnonce + ":" + qop + ":" + sha(A2))

6. The client resends its original request, with the addition of two headers. First the X-Atom-Authentication header with all of the following values filled in:

   X-Atom-Authentication: Atom username="...", realm="...", nonce="...", uri="...", qop="atom-auth", nc="00000001", cnonce="...", response="..."

   N.B.Fixed the above line, which should use Atom instead of Digest.

   And then the Authentication header signalling that we are using the Atom authentication scheme:

   Authentication: Atom key="value"

   Note that the key-value pair after the 'Atom' have no significance for us as Apache will not pass down the Authentication: header to the CGI program even when it is an authentication scheme it doesn't know, which is why all the real useful information was moved into X-Atom-Authentication: which will get passed to a CGI program.

7. If the username/password is not valid, the server will respond with an HTTP error code 403, and a new WWW-Authenticate header, and the client starts all over.

   If the client screwed something up (forgot a value, sent a malformed authentication request), the server will respond with an HTTP error code 400, and a new WWW-Authenticate header.

   If the client successfully authenticated, the server will do what the client asked (in this case, post a new entry). Every subsequent response from the server may contain an X-Atom-Authentication-Info header that includes a "nextnonce" value. If present, the client must discard the previous nonce value and use the new nonce value to recalculate the digest response on the next request. (This protects against replay attacks.) The client should cache and reuse the other values given by the server (realm, qop, algorithm). Either way, only one extra round trip is required per session (before the first action, to get the initial authentication challenge). The client does not need to do additional round trips once they have successfully authenticated, as long as they stay current with their nonce values.

   If the server does not return a new nonce value, the client should continue using the old one, and increment the value of nc as a hexadecimal number, and recalculate the digest response. So on the second request, the client would recalculate the response like this:

   response = sha(sha(A1) + ":" + nonce + ":" + "00000002" + ":" + cnonce + ":" + qop + ":" + sha(A2))

   Many many thanks to Ken Coar for working patiently with Mark and I on fixing our intial version into the better scheme we have just released. Any errors and omissions lie solely with Mark and I. Thanks again Ken!

   Update - 26-Aug-2003: Fixed the header X-Atom-Authentication: which should start with Atom instead of Digest.

   Update 2 - 27-Aug-2003: Fixed the response calculation in step 7 which was missing the outer sha() around it.