[AstroPy] Linearity correction by polynomial fitting

[Tom Aldcroft]

On Thu, Apr 18, 2013 at 11:34 AM, Joe Philip Ninan <indiajoe at gmail.com>wrote:

> Hi,
>
> Context:
> I have a data cube, which is from an up-the-ramp readout of a detector.
> (i.e. data cube with the axis X pixel,Y pixel and Time)
> In order to fit a linearity correction function which maps from detector
> counts to actual count, I was planning to fit a straight line on the time
> axis (for each pixel), upto a threshold and then fit a 2 or 3 degree
> polynomial to map the difference between the extrapolated straight line and
> the non-linear counts of the pixel above the threshold (till it hits hard
> saturation).
>
> Now, the issue: [I want to avoid any python loops.]
> Calculating the slope and difference could be easily implemented by numpy
> masked array multiplications and other ndarray arithmetic.
> But I couldn't find anything for fitting a polynomial for every pixel
> along an axis of an ndarray.
> Is there a tool to do this? Without implementing a python loop to loop
> through each pixel and calculate coefficients?
> Worst case scenario, i think i will have to write a C routine to do
> polynomial fit and call it from python. But i would really love if a pure
> python alternative exists.
>

One thing to keep in mind is avoiding premature optimization and using
profiling to determine exactly where things are slow.  You could first code
it up in pure python loops and then see where the time is being spent.

In this case the best tool is line_profiler (
http://pypi.python.org/pypi/line_profiler).  If you worry about speed you
should immediately get proficient with this excellent tool as well as
cProfile.

Note that if your time axis is 100 elements long, dong a 3rd order
polynomial fit with np.polyfit is relatively slow (320 us on my mac).  The
python loops around this will be negligible, so just writing simple nested
loops around this will be fine.  (Apologies if I'm misunderstanding your
exact problem).

Cheers,
Tom


>
> Thanks
> -cheers
> joe
>
>
>
>
>
> --
> /---------------------------------------------------------------
> "GNU/Linux: because a PC is a terrible thing to waste" -  GNU Generation
>
> ************************************************
> Joe Philip Ninan    http://sites.google.com/site/jpninan/
> Research Scholar        /________________\
> DAA,                            | Vadakeparambil |
> TIFR,                           | Pullad P.O.         |
> Mumbai-05, India.      | Kerala, India      |
> Ph: +917738438212   | PIN:689548       |
> ------------------------------\_______________/--------------
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20130419/1e392f5c/attachment.html>