[AstroPy] Image combine

Perry Greenfield perry at stsci.edu
Wed Mar 7 10:17:43 EST 2012


Do you have any good example of code like this is successfully  
distributed to users in the wild? So far, the dependencies on the  
specific GPU hardware seem to make this impractical right now. But  
maybe I'm missing something.

And I'm also a bit confused by the first statement. Are you arguing  
that we should be porting SPP to GPUs?

Perry

On Mar 7, 2012, at 10:11 AM, Tiago Ribeiro de Souza wrote:

> Hi guys,
>
> Even though I understand that people want to be free from using  
> IRAF, I don't agree that only  translating its basic functionality  
> to other programming language is actually useful. If we ought to  
> look to the future I would say that we should start an effort to  
> provide a GPU-enabled suite of image reduction facility ratter than  
> replicating existing code in different languages. Given the level of  
> paralalization of image reduction (say image combine, subtraction,  
> division, source extraction, spectral extraction and more!) GPU  
> programming will definitively be much more important and useful.
>
> For those who are interested, check out this tutorial at http://www.macresearch.org/opencl 
>  and http://developer.nvidia.com/content/september-2009-opencl-public-downloads
>
> Cheers,
> T.
>
> Em 07/03/2012, às 10:52, Perry Greenfield escreveu:
>
>> Hi Neil,
>>
>> That's a bit misleading. The application of the masking operation is
>> certainly done in C. The whole issue is how tightly integrated the
>> masks are with native arrays. For large arrays, it's not likely a
>> significant speed issue per se, but it is perhaps a memory issue  
>> (with
>> consequent speed issues if memory caches are involved).
>>
>> Perry
>>
>> On Mar 7, 2012, at 8:45 AM, Neil Crighton wrote:
>>
>>> Hi Wolfgang,
>>>
>>> The problem is that masked arrays in numpy are not implemented in c,
>>> so they are not c-speed.  There is a proposal to implement a masked
>>> array in c (https://github.com/numpy/numpy/blob/master/doc/neps/missing-data.rst
>>> )
>>> but this hasn't been finalised.
>>>
>>> Cheers, Neil
>>>
>>>
>>> On 7 March 2012 14:41, Wolfgang Kerzendorf <wkerzendorf at gmail.com>
>>> wrote:
>>>> Hello Jose,
>>>>
>>>> What you're trying to do is already half implemented in numpy and
>>>> called
>>>> masked arrays:
>>>>
>>>> Let's say you want to stack images and you have put them together
>>>> in a 3d
>>>> cube (first axis being the images):
>>>>
>>>> myimages = numpy.ma.MaskedArray(rand(5,100,100),
>>>> mask=zeros((5,100,100).astype(bool))
>>>>
>>>> let's do a simple sigma clipping algorithm:
>>>>
>>>> image_mean = mean(myimages)
>>>> image_std = std(myimages)
>>>>
>>>> #now let's adjust the mask to mask the pixels that are more than 1
>>>> sigma out
>>>>
>>>> new_mask = abs(myimages - image_mean) > 1.
>>>>
>>>> myimages.mask = new_mask
>>>>
>>>> #if you do a mean now it will ignore the values where the mask =  
>>>> True
>>>>
>>>> mean(myimages, axis=0)
>>>>
>>>> This will be all near C-speeds (I guess a factor of 100 is easily
>>>> in there).
>>>> ----
>>>>
>>>> As other's have pointed out using i,j in a loop is a very bad idea
>>>> for numpy
>>>> arrays. Numpy arrays are much more easy to handle than in most  
>>>> other
>>>> languages, there is a bit of getting used to required though. I
>>>> recommend
>>>> reviewing the different operations
>>>> on http://www.scipy.org/Tentative_NumPy_Tutorial.
>>>>
>>>> Hope that helps,
>>>>
>>>>   Wolfgang
>>>> On 2012-03-07, at 4:41 AM, Sergio Pascual wrote:
>>>>
>>>> Hello
>>>>
>>>> I have written a image combine implementation for the pipeline of
>>>> EMIR[1], a near infrared instrument for the 10m GTC Telescope. It
>>>> does
>>>> scaling,
>>>> weighting and masking. It's written as a C/C++ extension. It's part
>>>> of
>>>> a larger package, numina, used for other GTC instruments, but I may
>>>> split combine if there is interest in it. The package, called  
>>>> numina,
>>>> its under heavy development, but the combine part is fairly stable.
>>>>
>>>> You can see the code here https://guaix.fis.ucm.es/hg/numina/
>>>>
>>>> The combine part is the module numina.array.combine, fairly all the
>>>> C/C++ code is in src/
>>>>
>>>> Regards, Sergio
>>>>
>>>>
>>>> [1] http://www.gtc.iac.es/pages/instrumentacion/emir.php
>>>> [2] http://www.gtc.iac.es/
>>>>
>>>>
>>>>
>>>> El día 6 de marzo de 2012 12:00, Jose Miguel Ibáñez
>>>> <ppmime at gmail.com>
>>>> escribió:
>>>>
>>>> Hello everyone,
>>>>
>>>>
>>>> does anyone know of an implementation of the iraf.imcombine task in
>>>>
>>>> python+numpy ? (of course, not using pyraf.imcombine call)
>>>>
>>>>
>>>>
>>>> Thanks !
>>>>
>>>> Jose
>>>>
>>>> _______________________________________________
>>>>
>>>> AstroPy mailing list
>>>>
>>>> AstroPy at scipy.org
>>>>
>>>> http://mail.scipy.org/mailman/listinfo/astropy
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Sergio Pascual     http://guaix.fis.ucm.es/~spr    +34 91 394 5018
>>>> gpg fingerprint: 5203 B42D 86A0 5649 410A F4AC A35F D465 F263 BCCC
>>>> Departamento de Astrofísica -- Universidad Complutense de Madrid
>>>> (Spain)
>>>> _______________________________________________
>>>> AstroPy mailing list
>>>> AstroPy at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/astropy
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> AstroPy mailing list
>>>> AstroPy at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/astropy
>>>>
>>> _______________________________________________
>>> AstroPy mailing list
>>> AstroPy at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/astropy
>>
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at scipy.org
>> http://mail.scipy.org/mailman/listinfo/astropy
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy




More information about the AstroPy mailing list