From Perl to Python: restructuring a HPC workflow

Joel Goldstick joel.goldstick at gmail.com
Wed Mar 27 13:23:55 CET 2013


On Wed, Mar 27, 2013 at 7:51 AM, rusi <rustompmody at gmail.com> wrote:

> On Mar 27, 4:29 pm, neurino <lelli.l... at googlemail.com> wrote:
> > In the need for restructuring our daily workflow, i think it might be a
> > good idea to ask the Python community and hopefully initiate a thread
> > about pros and cons.
> >
> > We are a small group of people (approx. 10), working separetely on
> > their own projects (each employee manages approx. 2-3 projects). We
> > deal with high loads of data everyday.
> >
> > While the processing is accomplished with fortran and C programs mainly
> > on three systems (one cluster, two standalone IBM HPCs,  8852 and p770,
> > all managed by a grid-Engine), networking, pre/postprocessing, jobs
> > queue administration and numerical analysis have been accomplished with
> > Perl.
> >
> > This workflow has been flawless now for at least 15 years. New
> > generations of employees have been given Perl scripts and they
> > developed the tools further.
>

So for 15 years the workflow is flawless, yet you want to throw it all
out?  I don't understand that

> >
> > If i think at the actual situation of Perl, i can't see a shiny time
> > ahead. Perl 6 is far to be a reliable solution, the CPAN archive is
> > slowing down. My idea is to persuade my colleagues to move toward
> > Python-based solutions.
>

So you want to change things but your colleagues don't?


> But our concerns are that, in 3-4 years from
> > now, the tools we are going to develop must be still scalable,
> > mantainable, portable and of high-performance.
>



> >
> > We don't have any solid in-house know-how on Python. We just have to
> > start everything from scracth. Where do you see advantages and
> > drawbacks in switching from Perl to Python, given the work picture
> > above?
> >
> > Thanks in advance for any opinions you might have.
>
> Python is a lot easier to read than perl.


> Switching is always a con; see
> http://www.joelonsoftware.com/articles/fog0000000069.html
> Assuming you have that under your belt
> - if python is the way to go, asking on the scipy/numpy and ipython
> lists may give you more specific answers.
> - And if the 'rewrite-bug' has really got you, remember that if perl
> is old, C/Fortran are older.
> There are options today for rewriting the whole system, such as
> haskell and julia http://julialang.org/
>
> WARNING: If the Spolsky warning above for perl->python is X units,
> take it 2X for Haskell and 4X for Julia!
> --
> http://mail.python.org/mailman/listinfo/python-list
>



-- 
Joel Goldstick
http://joelgoldstick.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20130327/7092688d/attachment.html>


More information about the Python-list mailing list