A while ago, Rob Arnold wrote a simple python caching proxy server for use with our Talos tests — the idea was that you’d run your test once against the proxy server in “record” mode, and then after that you can use the server for consistent local playback.

I was giving some WebGL demos recently, and needed a way to have all the content from the web-hosted demos locally.  As anyone who’s tried to create a local cache of any “Web 2.0″ app knows, it’s painful, given all the server requests, XMLHttpRequests, etc. that go on.  However, with the proxy server, this was actually ridiculously easy.

You can grab the proxy server here — it still lives in Mozilla CVS — proxyserver.py.  It works fine on Win32, OS X, and Linux.  On Win32, the python that’s part of mozilla-build works well.  Run it like this:

python proxyserver.py

and then in Firefox’s proxy settings (or the system proxy settings), set your HTTP proxy to localhost:8000.  You can change the port via a command line option.  Then, visit all the pages/sites that you want cached (don’t forget to shift-reload or clear Firefox’s cache beforehand to ensure that Firefox actually goes out to the network!).  After you’ve got everything going, restart the proxy server in local-only mode:

python proxyserver.py -l

… and make sure that your demos work.  You can also run without -l live, especially if you will have a network connection (even a slow one) to give you the option of going out to the network if necessary.  Also, if you want to copy the proxy cache to another machine, just copy proxy_cache.db that gets created in the same directory as proxyserver.py.

The proxy server currently supports HEAD and GET requests.  It doesn’t support POST, so if you have something that depends on POST, you’re out of luck.  It wouldn’t be too hard to add though; patches accepted if someone wants to tackle that.

With the proxy server, I was able to give a bunch of demos that made heavy use of XHR, including some that loaded video, without having to rely on a network or spend time downloading and fixing up URLs.  It really made demo prep much easier.