Test stubbing httplib2

I'm writing some HTTP client code that uses httplib2, and the following code has allowed me to create unit tests without creating a bunch of resources on the web to test against. It's a stub of httlib2.Http that reads its responses off the disk. I monkey-patch it in place for unit tests, and restore the original after I'm done, so as not to interfere with other unit tests that might run and require the original class.

As an aside I want to point out that this isn't particularly clever or unique, I'm writing this up just to have a nice place drop the code, and document its usage, to save time for anyone else building with httplib2.

Here is the stub class and a unittest.TestCase that uses it.

import unittest
import urlparse
import httplib2
import os
from email import message_from_string, message_from_file
HTTP_SRC_DIR = "./tests/model/"
class MyHttpReplacement:
    """Build a stand-in for httplib2.Http that takes its
    response headers and bodies from files on disk"""
def __init__(self, cache=None, timeout=None):
        self.hit_counter = {}

    def request(self, uri, method="GET", body=None, headers=None, redirections=5):
        path = urlparse.urlparse(uri)[2]
        fname = os.path.join(HTTP_SRC_DIR, path[1:])
        if not os.path.exists(fname):
            index = self.hit_counter.get(fname, 1)
            if os.path.exists(fname + "." + str(index)):
                self.hit_counter[fname] = index + 1
                fname = fname + "." + str(index)
        if os.path.exists(fname):
            f = file(fname, "r")
            response = message_from_file(f)
            f.close()
            body = response.get_payload()
            headers = httplib2.Response(response)
            return (headers, body)
        else:
            return (httplib2.Response({"status": "404"}), "")

    def add_credentials(self, name, password):
        pass
class Test(unittest.TestCase):

    def setUp(self):
        self.old_httplib2 = httplib2.Http
        httplib2.Http = MyHttpReplacement

    def tearDown(self):
        httplib2.Http = self.old_httplib2

The MyHttpReplacement class takes incoming requests and uses the request path to look up files under the directory HTTP_SRC_DIR. If a file is found it is presumed to be a single message in mbox format, where all the headers will be used as the response headers and the body of the message will be used as the response body. If no file is found a 404 is returned.

For example, if you place the file 'fred' under the HTTP_SRC_DIR directory, with the following contents:

status: 200
content-type: text/plain

Hello World!

Then a request to http://example.org/fred will return a response that is plain text with a body of "Hello World!". Only the path is used, so requests to http://bitworking.org/fred or http://example.com/fred?someparameter=somevalue will all access the same file and produce the same results.

The only other feature MyHttpReplacement has is for changing the response over a series of requests. If you create the files fred.1, fred.2, etc. then requests for http://example.org/fred will return each one of those files in order, and then start returning 404's once the last file in order has been server.

The setUp() and tearDown() members of the TestCase class swap my stub class in for the original httplib2.Http class, and then restore the original after the test is complete. That allows other unit tests to run that may want to use the original class.