[Numpy-discussion] "import numpy" is slow

Hanni Ali hanni.ali at gmail.com
Thu Jul 31 08:31:49 EDT 2008


Hi All,

I've been reading this discussion with interest.

I would just to highlight an alternate use of numpy to interactive use. We
have a cluster of machines which process tasks on an individual basis where
a master tasks may spawn 600 slave tasks to be processed. These tasks are
spread across the cluster and processed as scripts in a individual python
thread. Although reducing the process time by 300 seconds for the master
task is only about a 1.5% speedup (total time can be i excess of 24000s). We
process large number of these tasks in any given year and every little
helps!

Hanni



2008/7/31 Stéfan van der Walt <stefan at sun.ac.za>

> 2008/7/31 Andrew Dalke <dalke at dalkescientific.com>:
> > The user base for numpy might be .. 10,000 people?  100,000 people?
> > Let's go with the latter, and assume that with command-line scripts,
> > CGI scripts, and the other programs that people write in order to
> > help do research means that numpy is started on average 10 times a day.
> >
> > 100,000 people * 10 times / day * 0.1 seconds per startup
> >    = almost 28 people-hours spent each day waiting for numpy to start.
>
> I don't buy that argument.  No single person is agile enough to do
> anything useful in the half a second or so it takes to start up NumPy.
>  No one is *waiting* for NumPy to start.  Just by answering this
> e-mail I could have (and maybe should have) started NumPy three
> hundred and sixty times.
>
> I don't want to argue about this, though.  Write the patches, file a
> ticket, and hopefully someone will deem them important enough to apply
> them.
>
> Stéfan
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20080731/1a547ca0/attachment.html>


More information about the NumPy-Discussion mailing list