Intro to the “Abstractions as Leverage” series

At PyCon 2009, I had the privilege of seeing Alex Martelli’s “Abstractions as Leverage” talk. I’m not going to go into the details (you can watch it here), but I walked away from it thinking about the wonderful abstracted place where I get to work (Python on Google App Engine).

Thing is, I know that I’m often thinking about what’s going on under the hood implicitly, but I don’t ever really stop to explicitly consider the low-level operations that execute my Python & App Engine code. As a result, I’m going to write up a few articles about how things work under the hood (because I think it’s great mental exercise).

Intro to the Dev_AppServer

I’ll be brief about the dev_appserver; there’s lots of other resources to learn about it. The gist of it is that the dev_appserver provides a local development environment that tries to emulate the APIs on Google App Engine.

If you’ve used it, though, you’ve probably noticed that it’s a lot slower than running code on the real App Engine. This is because it’s single-threaded and can only serve results back sequentially. This is the question I want to answer: “Why is the dev_appserver single-threaded? Why can it only handle one request at a time?

There’s a few important pieces that we need to be familiar with before answering those questions.

The WSGI Protocol

http://www.python.org/dev/peps/pep-0333/

The WSGI (Web Server Gateway Protocol) is standardized way for a web server to talk to a web application or framework. Practically, it’s a few extensions on top of the CGI protocol.

The CGI Protocol

http://hoohoo.ncsa.uiuc.edu/cgi/interface.html

http://www.w3.org/Daemon/User/CGI/Overview.html

For reference, the second link has a copyright from 1995. This stuff has been used for a long time. An important part to note is that he CGI protocol passes information into the target process through it’s standard input and environment variables.

Reading through the WSGI standard, it looks like they’ve identified that standard input and environment variables are a poor way to pass information into a request handler — these are both process-global state; that is, shared across all threads. The WSGI interface provides a dict-object, environ, containing all of the CGI variables, that isn’t tied to the os.environ (global) state.  WSGI, on its own then, can be multithreaded (see CherryPy, for example).

google.appengine.ext.webapp.util.run_wsgi_app

This is the App Engine entry point for calling a WSGI-compatible application. Here’s the first line of that function:

env = dict(os.environ)

First things first, it makes a copy of os.environ. If the dev_appserver were multi-threaded, we would absolutely want to ensure that run_wsgi_app (and whatever is calling it) had intelligent locking semantics, otherwise there would be a race between different requests (if Thread A is handling a request, and Thread B overwrites os.environ to handle a new request, the responses could get mixed up).

If you have a look at http://code.google.com/appengine/docs/python/runtime.html#The_Environment, it’s pretty clear that there’s an expectation that os.environ can be accessed anywhere within your request handler; since this is a process-global state, there’s no way to handle multiple requests in separate threads.

There are other instances of os.environ within the App Engine code. It looks like the Users API is implemented through USER_EMAIL in os.environ.

The os.environ global state is just starting to scratch the surface, too. Consider all of the stubbed-out functionality included with the dev_appserver, like the Datastore and Memcache. These would require significant work to make them thread-safe as well (at least, based on my understanding of their implementations). And… all of this work would only be relevant on the dev_appserver anyway, because they work over RPC calls on App Engine.