[python-hpc] recipes on Python job submission to the Grid

Guy K. Kloss guy.kloss at aut.ac.nz
Wed Feb 23 00:57:59 CET 2011


here in New Zealand we've just finished our first Summer of eResearch, a 
summer student scholarship project slightly aligned with the idea of running 
the Google Summer of Code. We've had a bunch of students getting familiarised 
with the Grid (particularly the NZ BeSTGRID infrastructure). Some students 
have developed code already in Python using the Grid, others would've liked to 
use the Grid, but the learning curve was just not easy. We've had a group 
making some sample codes/recipes for Java on wiring in with the Grid, which 
was well received.

So my idea is to have some similar recipes available from Python. Simple code 
bites, with some documentation, that will make it easier to do some of the 

 * Write Python code to submit a job to a remote system
   (Grid or other infrastructure).

 * Write Python code to *run* on a remote system. This is much more versatile,
   as systems are very different. So it would most likely also benefit from
   a simple recipe on how to set up a virtualenv with local dependencies to
   host the application's needs.

 * Access information to manage remotely running jobs.

 * Access and store data on Grid resources (GridFTP, ...)

Any ideas on this? This was just an initial spark, so I guess many more things 
can be found, and some things may actually not be so simple/clear cut. If 
they're not, then they may need to be disected and disambiguated, so that one 
knows in which case what can be used.


Guy K. Kloss
School of Computing + Mathematical Sciences
Auckland University of Technology
Private Bag 92006, Auckland 1142
phone: +64 9 921 9999 ext. 5032
eMail: Guy.Kloss at aut.ac.nz
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.python.org/pipermail/python-hpc/attachments/20110223/1b1a5f1e/attachment.pgp>

More information about the Python-HPC mailing list