Fourth Floor Studio

August 24, 2007

Newbie python

Filed under: programming, python, R, science — chris @ 12:09 pm

I’ve finally hit a problem requiring multidimensional data structures, mathematical grunt, and speed, coupled with good DB bindings and text/file handling. Normally, I would use a perl script to fetch data from the db and process it with R (as I’ve been having trouble with the R DBI library). However, processing 31K chunks of data for 13K variables each just won’t work in R.

So I’m delving into python as a one-stop shop for all my woes. I’ve been procrastinating about learning the language, because, let’s face it, why write bad code in a new language when you can write ugly code in your long-term favourites.

Some resources I’ve found are: a python tutorial, the NumPy library for data arrays, SciPy and the python DB-API module. I’ll also have a look at StatPy for statistical computing in python.



  1. Why not have the best of both worlds:

    Comment by Greg Tyrelle — August 24, 2007 @ 5:47 pm

  2. Ah yes – my next port of call. I’m not yet sure I understand how everything fits together, though.

    Besides, I have to get the &*#% data out of the db first 😦

    Comment by chris — August 24, 2007 @ 6:01 pm

  3. I found myself editing a MODELLER script (in Python) recently, to accept files as command-line arguments rather than as hard-coded variables. Perhaps we all turn to Python eventually 🙂

    Like you, I’ve also taken the “learn it as and when you need it” approach. You won’t catch me reading programming manuals for fun.

    Comment by Neil — August 26, 2007 @ 12:49 am

  4. I’ve been meaning to make the switch for a while, but being the lazy sod I am have been waiting for a critical mass of stuff hanging over my head. What really draws me to python is the natural handling of 2+ dimensional data, with an eye on generating subsets, summaries on those, etc etc.

    Oh, and the R bindings, too 🙂

    Comment by chris — August 30, 2007 @ 2:21 pm

RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

Create a free website or blog at

%d bloggers like this: