App Engine and httplib2

The 1.1.9 release of the App Engine SDK includes support for httplib, urllib and urllib2. This is great since httplib2 depends on httlib. So far I've only found one problem, which is that httplib2 presumes that httplib returns all headers in lower case, and have already committed a fix to httplib2 for that. The change is currently in trunk and will appear in the next release.

Setting up is easy, just pull the httplib2 directory into your project.

The nice part is that you can hook up httplib2 to the memcache support in App Engine very easily:

import wsgiref.handlers
from google.appengine.ext import webapp

import httplib2
from google.appengine.api.memcache import Client

mem = Client()
http = httplib2.Http(mem)  

class MainHandler(webapp.RequestHandler):

  def get(self):
    
    headers, body = http.request("http://example.org/")

    self.response.headers['content-type'] = "text/plain"
    self.response.out.write("Reported status = %d\n" % headers.status)
    self.response.out.write(headers)
    self.response.out.write("\n\n")
    self.response.out.write(body)

The actual utility of httplib2 goes down on App Engine since the system that URLFetch runs through is an HTTP 1.1 compliant proxy and takes care of redirects and caching, but httplib2 still provides benefits such as authentication and lost update support. In addition you could supply a cache handler to httplib2 that persisted the cached pages to the datastore instead of memcache.