[pypy-dev] Talk in the Supercomputing Day, Madrid

holger krekel holger at merlinux.eu
Tue Jan 13 13:31:07 CET 2009


Hi Guillem! 

On Mon, Jan 12, 2009 at 12:10 +0100, Guillem Borrell i Nogueras wrote:
> Hi again
> 
> Let's discuss the details.
> 
> I'll try to explain why I've thought about pypy when planning the conference 
> sessions.
> 
> My work as Computational Fluid Dynamics researcher is intimately related to 
> supercomputing for obvious reasons. Most of applications we work on are fine 
> tuned codes with pieces that are more than twenty years old.  They are rock 
> solid, fortan implemented, and run for hours in clusters of thousand of 
> computing nodes.
> 
> Since the last couple of years computer architectures are becoming more and more 
> complex. I'm playing with the cell processor lately and that little bastard is 
> causing me real pain. While programming is easier every day, supercomputing is 
> harder and harder.  Think about an arhitecture like the roadrunner, AMD Opteron 
> PPU with PowerPC SPU... Two assemblers in one chip!

Could you give examples on the concrete challenges you face?

> Talking with Stanley Ahalt (Ohio Supercomputing Center) about a year ago he 
> called that the "software gap". In computing, as times goes, low performance is 
> easier but high performance is harder. And that gap gets wider. Platform SDK are 
> helpful but they are not a huge leap.
> 
> I've always thought that virtual machines could help
> supercomputing like they have helped grid and cloud
> computing.  This is the point where I need someone to proof
> if I am wright or wrong.  Pypy is the most versatile, albeit
> complex, dynamic language implementation. I've been
> following the project during the last year and a half or so
> and I am impressed. I've thought that you could have a
> vision on how interpreted languages and virtual machines can
> help managing complexity.

The PyPy project is mostly concerned with reducing the complexity 
of writing efficient dynamic language interpreters. 
PyPy's JIT and its flexible backend implementation 
should help with bridging the software gap IIUC. 
It probably makes sense to stress and elaborate
on this point and to discuss examples in the talk. 

Managing the complexity of software deployment and integration
is not PyPy's primary concern but it provides some interesting
building blocks.  For example, i am interested in using using
PyPy's sandboxing and virtualization capabilities to build an
open "cloud at home" network: people would run a bare-bones
executable, configure & control CPU, RAM, Network, Disk
resources and access rights. On this network e.g. open source
developers or scientists could launch programs and
computations or start os-level virtual machines where
non-python code runs.  

> In addition, most of postprocessing tools are written in matlab, an interpreted 
> language.  Running not-so-high performance tasks in a workstation efficiently is 
> sometimes as important as running a simulation in a 12000 node supercomputer. It 
> yould be nice if someone would remind the audience that matlab is not the only 
> suitable (or best) tool for that job.

Right, I guess that something like Numpy combined with PyPy's JIT 
would could make a killer combination.  That's work, though. 

> I hope that I managed to explain the topics I'm interested in.

sure, let us know what you think makes most sense for
the audience - who will be the audience btw? 

thanks & best,

holger


 
> Those are the speakers that accepted the invitation at the moment:
> 
> * J.Mª. Cela, Computer Architecture Department, Universitat Politècnica de 
> Catalunya and Barcelona Supercomputing Center. Future supercomputers, CFD 
> applications and the Cell processor.
> 
> * Sun Microsystems. Distributed Filesystems and Lustre.
> 
> * S. Hoyas. School of Aeronautics. Universitat Politècnica de València. Running 
> a supercompuingr simulation in more than 2000 processors.
> 
> I'm very interested in your comments.

> cheers.
> 
> guillem
> 
> El Friday 09 January 2009 19:34:21 holger krekel escribió:
> > Hi Guillemi!
> >
> > thanks for mailing - this definitely sounds interesting to me.
> > I discussed earlier today with Maciej Fijalkowski who is also
> > interested to come.  So let's stay in contact and discuss details.
> >
> > cheers,
> > holger
> >
> > On Fri, Jan 09, 2009 at 10:54 +0100, Guillem Borrell i Nogueras wrote:
> > > Hello everyone
> > >
> > > My name is Guillem Borrell and I am a researcher in the Computational
> > > Fluid Dynamics Laboratory in the School of Aeronautics, Universidad
> > > Politécnica de Madrid.
> > >
> > > This year I'm in charge of organising the Supercomputing Day. Its aim is
> > > to present in form of talks the recent and future developments in high
> > > performance computing in science and engineering. It holds five talks,
> > > open to anyone, on one day and we expect that it will take place on Agust
> > > the 1st of 2009.
> > >
> > > Most of supercomputing software use static languages such as Fortran or C
> > > but postprocessing is usually done in dynamic  languages, mostly matlab.
> > > While it uses dynamic languages on a daily basis the scientific community
> > > has almost never taken a look at what the virtual machines can offer in
> > > terms of versatility, ease of use and performance.
> > >
> > > Pypy is now at the bleeding edge of dynamic language implementation and I
> > > think that some of you can give some insight on what dynamic languages
> > > (and pypy, of course) can offer in the field of supercomputing.  I know
> > > that pypy is not only focused in performance but our intention is to get
> > > an overview, not only to talk about a single project.
> > >
> > > My question is ¿Is some of the pypy developers interested in give such a
> > > talk? If the answer is yes  ther's still plenty of time to think about
> > > the details.
> > >
> > > I personall think that pypy is the most mind challenging software project
> > > and this can be a little effort to present some of its achievements to a
> > > broader audience.
> > >
> > > We have two confirmed talks at the moment.  The first one will be on
> > > future microprocessor architecture development and the Cell architecture
> > > in particular and the second one will be about distributed file systems
> > > and Lustre.
> > >
> > > Sincerely yours.
> > > --
> > > guillem
> > >
> > > Guillem Borrell i Nogueras
> > > Laboratorio de Mecánica de Fluidos Computacional
> > > Escuela Técnica Superior de Ingenieros Aeronáuticos
> > > guillem at torroja.dmt.upm.es
> > > Web: http://torroja.dmt.upm.es/guillem/blog
> > > _______________________________________________
> > > pypy-dev at codespeak.net
> > > http://codespeak.net/mailman/listinfo/pypy-dev
> 
> -- 
> guillem
> 
> Guillem Borrell i Nogueras
> Laboratorio de Mecánica de Fluidos Computacional
> Escuela Técnica Superior de Ingenieros Aeronáuticos
> guillem at torroja.dmt.upm.es
> Web: http://torroja.dmt.upm.es/guillem/blog
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
> 

-- 
collaborative expert contracting: http://merlinux.eu 
PyPy  Python/Compiler tool chain: http://codespeak.net/pypy 
pylib py.test/greenlets/svn APIs: http://pylib.org 



More information about the Pypy-dev mailing list