[pypy-dev] Talk in the Supercomputing Day, Madrid

[holger krekel]

Hi Guillem, 

On Tue, Jan 13, 2009 at 16:17 +0100, Guillem Borrell i Nogueras wrote:
> Hi!
> 
> El Tuesday 13 January 2009 13:31:07 holger krekel escribió:
> 
> >
> > Could you give examples on the concrete challenges you face?
> >
> 
> Multicore architectures are killing supercomputing because two kinds of 
> parallelism are needed at the same time. A couple of years ago the computer 
> architecture departments started suggesting multiple OpenMP threads in every MPI 
> processes. This required a huge amount of work and never worked as expected 
> (maybe compilers are to blame?).
> 
> Now the trend seems to be using MPI for processes and a superscalar compiler 
> that uses all the cores and SPU for just a few kind of operations. This is not 
> more useful than using vectorized versions of Blas, levels 1 and 2.
> 
> Some years ago I started experimenting with python and MPI. I coded a very 
> limited MPI bindings collection for Python and launched python interpreters as 
> MPI processes. Instantly thought that a JIT-accelerated, multicore-aware 
> interpreter and a fast MPI-like message passing library would lower the 
> complexity of using parallel computers. That would not give the highest 
> performance but one has to be realistic. There's no need to distribute a VM, 
> message passing is not that hard.

For that scenario PyPy is currently missing the JIT, multicore-awareness and bindings 
to numeric libraries.  
 
> > > In addition, most of postprocessing tools are written in matlab, an
> > > interpreted language.  Running not-so-high performance tasks in a
> > > workstation efficiently is sometimes as important as running a simulation
> > > in a 12000 node supercomputer. It yould be nice if someone would remind
> > > the audience that matlab is not the only suitable (or best) tool for that
> > > job.
> >
> > Right, I guess that something like Numpy combined with PyPy's JIT
> > would could make a killer combination.  That's work, though.
> 
> I think that you'll have to start explaining what a JIT is. See the note on the 
> audience below. I've been using Python+Numpy+Scipy+... as a complete replacement 
> for matlab for the last two years. I suggested all of my colleagues to do the 
> same but they all believe that python is a toy language (!). I think that you 
> could give a more authorized opinion on this.

If that is your main idea - and considering the audience - 
i'd rather suggest to get someone authoritative from the
Numpy/Scipy community - they did two dedicated conferences
last year, the November issue of the Python Magazine was about
Python in scientific environments.   So it shouldn't be hard
to get someone who authentically confirms that Python is
a seriously used language for scientific computing.  

Of course, we'd be happy to discuss and incorporate a "PyPy"
perspective in such a talk.  

cheers,
holger

> > > I hope that I managed to explain the topics I'm interested in.
> >
> > sure, let us know what you think makes most sense for
> > the audience - who will be the audience btw?
> 
> The talks will be in the school of aeronautics. Most of people in the audience 
> will be engineers, mathematicians and physicists users of supercomputing without 
> any academic background on Computer Science. 
> 
> Thany you so much!
> -- 
> guillem
> 
> Guillem Borrell i Nogueras
> Laboratorio de Mecánica de Fluidos Computacional
> Escuela Técnica Superior de Ingenieros Aeronáuticos
> guillem at torroja.dmt.upm.es
> Web: http://torroja.dmt.upm.es/guillem/blog
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
> 

-- 
collaborative expert contracting: http://merlinux.eu 
PyPy  Python/Compiler tool chain: http://codespeak.net/pypy 
pylib py.test/greenlets/svn APIs: http://pylib.org