Discussion: Python and OpenMP

[Paul McGuire]

"Carl J. Van Arsdall" <cvanarsdall at mvista.com> wrote in message
news:mailman.5638.1147456348.27775.python-list at python.org...
> Hey everyone,
>
> I know I've posted several questions regarding python and python's
> parallel capabilities so bear with me as I've never attempted to incite
> discussion.  However, today I'm interested in sparking discussion over
> having an OpenMP style of interface in python.
>
> For those of you familiar with OpenMP, its a pragmatic api for
> parallelizing software.  For more information I invite anyone to do some
> google searches, there's a plethora of information available. Just to
> give a simple example of what i'm talking about, in OpenMP you would
> insert a pragma above a section of code you want to parallelize.
>
> In C it might look something like:
>
> int main(int argc, char* argv[])
> {
>   #pragma omp parallel
>   printf("Hello, world.\n");
>   return 0;
> }
>
> In which case threads are spawn and handled.  OpenMP of course has more
> than this available for developers, but I'm just trying to paint a
> picture before I start asking questions.
>
> Anyhow, onto the meat of the discussion.  Would the python community be
> interested in parallel programming this way?  Although I understand
> python already supports threading I thought that this was a real
> interesting (and easy) way of writing parallel code.
>
> For example, new keywords could be added to the python interpreter, such
> as parallel:
>
> Ex:
> #######
> #!usr/bin/python
>
> parallel:
>   print "I am a thread"
>
> #######
>
>
> Taking this a step further, OpenMP or an OpenMP style implementation
> could be added to python.  In addition easy to use/read, one possible
> benefit I could see of writing parallel python code this way would be
> providing a layer of abstraction between parallel code and threading
> constructs.  For example, as a developer or community standards change
> threading in python code would not have to be re-written.  Developers
> would create an interface between python's OpenMP style code and
> whatever their new threading libraries may be (this is one of many ways
> it could happen).  Ultimately providing more code portability between
> people using different threading standards (should they ever become
> available).  I see other use cases as well, but I just wanted to throw a
> couple ideas to see if this was worth thinking about further.
>
> Thanks for reading this one, I know it was long but I'd really
> appreciate your comments!
>
>
>
> -Carl
>
>
> -- 
>
> Carl J. Van Arsdall
> cvanarsdall at mvista.com
> Build and Release
> MontaVista Software
>

My first reaction to this post was that an @parallel decorator might be a
reasonable approach, since this would not require any hacking into the
Python interpreter.  Googling "python parallel decorator" led me to this
recipe in the Python Cookbook:
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/474127.  While this
uses yield to create generators to simulate threads (and if you can simulate
threads without incurring the overhead, is that so wrong?), it may give you
some additional ideas on some implementation alternatives.

-- Paul