[AstroPy] Using quantities is slow

Paul Kuin npkuin at gmail.com
Mon Aug 24 20:58:19 EDT 2015


I'm converting my code to use astropy.units, and decided to use it mainly
at the interface level, and use my own set of fixed units inside the
software units.

That said, it seems to me that it would be useful if you could set some
flag to tell the code that the units are not going to change until you
unset the flag. Then the "y = xx.value" steps are not needed. Or perhaps
this is a really stupid idea because I don't know the ins and outs. .. ;-)

Paul

On Mon, Aug 24, 2015 at 11:41 PM, Erik Bray <embray at stsci.edu> wrote:

> On 08/24/2015 06:32 PM, Derek Homeier wrote:
> > On 24 Aug 2015, at 3:57 pm, Kevin Gullikson <kevin.gullikson at gmail.com>
> wrote:
> >>
> >> I am writing an MCMC fitting code, and my current implementation uses
> the (very useful) astropy quantities framework to take care of unit
> conversions and such. However, the code is surprisingly slow so I profiled
> it with %prun and it looks like the quantity utilities that wrap numpy
> functions are taking a huge chunk of time. Is that just the price of
> convenience?
> >>
> > I am not an expert on the quantities/units implementation, but I would
> imagine that they could indeed
> > introduce a significant overhead, as the class adds a number of
> validations and checks for a function
> > calling a quantity object. Especially with rather numerous operations on
> comparatively small arrays
> > this could become problematic.
> > Did you experiment with using decorators as described in
> >
> http://astropy.readthedocs.org/en/stable/units/quantity.html#functions-accepting-quantities
> >
> > - iiuc this would allow bypassing some of the validations?
>
> I don't think this is really relevant to the OP's question (though always
> good
> to know about).  This decorator just makes it easy to bypass writing a lot
> of
> boilerplate code for checking the units of its inputs--it doesn't remove
> any
> overhead (in fact it adds it--but this was used for cases where that
> overhead
> already existed--the code was just highly repetitive).
>
> > Another option might be to only pass plain numpy arrays to the innermost
> loops of your code with .value,
> > of course at the expense of some of the convenience gained in the first
> place...
>
> Right, that's the big problem.  The thing Quantity supports (which most
> similar
> libraries in Python don't) is that it *is* a Numpy ndarray, and with few
> exceptions [1] works transparently with all of Numpy's universal functions.
> However, Numpy currently doesn't make it easy to wrap calls to ufuncs, so
> there's some overhead associated with redoing unit conversions on every
> function
> call (and even if Numpy did make this easier, as it will in 1.10,
> performing
> unit conversions repeatedly in a loop will add significant overhead to any
> fitting operation, so better to do the unit conversions first so that all
> quantities are expressed in the same base units and orders of magnitude,
> then as
> Derek suggested pass in the raw arrays (from .value) to high performance
> code.
>
> Having a mechanism to automate such conversions will be useful I think.
>
> Erik
>
>
> [1]
> http://astropy.readthedocs.org/en/stable/known_issues.html#quantity-issues
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>



-- 

* * * * * * * * http://www.mssl.ucl.ac.uk/~npmk/ * * * *
Dr. N.P.M. Kuin      (n.kuin at ucl.ac.uk)
phone +44-(0)1483 (prefix) -204927 (work)
mobile +44(0)7806985366  skype ID: npkuin
Mullard Space Science Laboratory  – University College London  –
Holmbury St Mary – Dorking – Surrey RH5 6NT–  U.K.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20150825/3195e5cf/attachment.html>


More information about the AstroPy mailing list