[python-hpc] recipes on Python job submission to the Grid
Guy K. Kloss
guy.kloss at aut.ac.nz
Wed Feb 23 00:57:59 CET 2011
here in New Zealand we've just finished our first Summer of eResearch, a
summer student scholarship project slightly aligned with the idea of running
the Google Summer of Code. We've had a bunch of students getting familiarised
with the Grid (particularly the NZ BeSTGRID infrastructure). Some students
have developed code already in Python using the Grid, others would've liked to
use the Grid, but the learning curve was just not easy. We've had a group
making some sample codes/recipes for Java on wiring in with the Grid, which
was well received.
So my idea is to have some similar recipes available from Python. Simple code
bites, with some documentation, that will make it easier to do some of the
* Write Python code to submit a job to a remote system
(Grid or other infrastructure).
* Write Python code to *run* on a remote system. This is much more versatile,
as systems are very different. So it would most likely also benefit from
a simple recipe on how to set up a virtualenv with local dependencies to
host the application's needs.
* Access information to manage remotely running jobs.
* Access and store data on Grid resources (GridFTP, ...)
Any ideas on this? This was just an initial spark, so I guess many more things
can be found, and some things may actually not be so simple/clear cut. If
they're not, then they may need to be disected and disambiguated, so that one
knows in which case what can be used.
Guy K. Kloss
School of Computing + Mathematical Sciences
Auckland University of Technology
Private Bag 92006, Auckland 1142
phone: +64 9 921 9999 ext. 5032
eMail: Guy.Kloss at aut.ac.nz
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 198 bytes
Desc: This is a digitally signed message part.
More information about the Python-HPC