The SoCal Piggies had their fourteenth meeting at USC (Salvatori Computer Science Center) on June 1st 2006 at 7:00 PM. Nine Piggies attended: Daniel Arbuckle, Grig Gheorghiu, Mark Kohler, Diane Trout, Titus Brown, Howard Golden, Steve Wedig (his first participation -- welcome to the group!), George Bullis and Greg Abbas.
Mark was the first presenter of the night. He first showed us how Python, used in a novel way -- as a scripting language (it's a joke, laugh) -- makes his life much easier when it comes to debugging the Linux kernel. Yes, debugging the Linux kernel, you heard it right. Not many people have to do this in their day jobs, but Mark is one of those few. If it's a blessing or a curse, he hasn't really specified.
In any case, Mark launched into a [http://oss.sgi.com/projects/kdb/ kdb] session, using a custom roll-your-own non-Linus-approved kernel that included kdb capabilities. It was a bit scary to see how hard it is to make sense of anything at the kdb prompt. Certain incantations resulted in more humanly readable output, but not by much. So Mark proceeded to talk about [http://www.gnu.org/software/binutils/manual/html_chapter/binutils_4.html objdump], a nifty utility that allows you to inspect object files and make sense of things like how much memory space does a certain C structure take. These things, correlated with kdb commands, would then allow you to pinpoint more exactly where the problem is. Mark showed us a Python script he wrote that helped in extracting information out of the objdump output. Python to the rescue once again. Mark also mentioned a nifty utility called [http://ctags.sourceforge.net/ Exuberant ctags], which helps in indexing a body of source code, and then allows indexed items (such as function and variable definitions) to be quickly located by the editor of your choice (if you choice happens to be Vim or Emacs of course).
Mark then talked about the [http://www.python.org/doc/lib/module-subprocess.html subprocess] module. He showed us how easy it is to replace old-style os.system or popen2.* commands with subprocess functionality. Mark showed many examples that I won't repeat here. Suffice to say that subprocess is a lifesaver, especially on win32 platforms. Mark also underlined how sanely program exit codes are interpreted by subprocess -- as opposed to how os.system interprets them. I bet many of you weren't quite familiar with this astounding fact: when os.system returns the exit status, the low-order byte of the exit status is the signal number that killed the process, and the high-order byte is the exit status (if the signal number is 0). BTW, this only works under Unix; on Windows the exit status is always 0. Great stuff all around.
The next presenter was Titus, who first talked about [http://issola.caltech.edu/~t/scotch/doc/ scotch], his collection of modules for WSGI applications (get it? -- WSGI, whiskey, scotch). Scotch offers two self-contained tools: scotch.recorder and scotch.proxy.
scotch.recorder offers the capability of recording all Web traffic coming into and out of your application, with the extra feature of playing back the same exact traffic at a later time -- this of course is the ultimate regression test for any Web application. You can instrument your own WSGI-capable application using scotch.recorder:
[ Web server ] <--> [ scotch.recorder ] <--> [ Web app ]
||
\/
disk(image copyright Titus Brown)
You can also instrument any application by WSGI-wrapping it and using scotch.proxy:
[ browser ] <- net -> [ scotch.recorder ] <--> [ scotch.proxy ] <- net -> the WWW
||
\/
disk(image copyright Titus Brown)
The WSGI wrapping and proxying is as easy as saying cheese[cake]. The following example is out of the [http://issola.caltech.edu/~t/scotch/doc/recipes.html scotch recipes] page:
To set up a Web proxy,
import scotch.proxy app = scotch.proxy.ProxyApp()
Then run 'app' in any WSGI server, e.g. wsgiref:
from wsgiref.simple_server import WSGIServer, WSGIRequestHandler
server_address = ('', 8000)
httpd = WSGIServer(server_address, WSGIRequestHandler)
httpd.set_app(app)
while 1:
httpd.handle_request()This is now a fully functional Web proxy running on port 8000.
Titus demo-ed nose.recorder and used it to intercept HTTP traffic to Google, including Ajax-generated traffic via the Google Suggest functionality. Cool stuff, folks, and smooth as scotch.
Next, Titus talked about his recently written [http://ivory.idyll.org/articles/nose-intro.html introduction to nose]. [http://somethingaboutorange.com/mrl/projects/nose/ Nose] is a unit test discovery and execution framework, with many bells and whistles that make life for Python developers who unit test their code a whole lot easier. If you don't unit test your code, you should. If you do unit test your code and write Python code, you should look into nose. There's no excuse for complaining about lack of documentation now, as Titus took care of that. Titus also showed his nose extensions, aptly named [http://darcs.idyll.org/~t/projects/pinocchio/doc/ pinocchio]. They take advantage of the nice plugin mechanism offered by nose, and they offer capabilities such as running subsets of unit tests based on various features/qualities/attributes such as speed of execution, failing vs. passing tests, and in fact any [http://darcs.idyll.org/~t/projects/pinocchio/doc/#decorator-adding-attributes-to-tests attributes] you choose to choose.
The conclusion? Titus has a nose for unit testing.
We spent other enjoyable moments talking about various things such as the IBM Winchester hard drive ([http://en.wikipedia.org/wiki/Hard_drive according to Wikipedia], project head designer/lead designer Kenneth Haughton named it after the Winchester 30-30 rifle after the developers called it the "30-30" because of its two 30 MB spindles), and we concluded we're a gathering of geeks when Mark asked if anybody has a USB flash drive, and 80% of the people stood up and reached in their pockets.
Many thanks to Mark and Titus for presenting, and to Daniel for providing the room at USC.
Proposed agenda for the next meeting:
- Daniel Arbuckle: "Erlang for Python programmers"
- Greg Abbas: "Introduction to Django"