[MATRIX-SIG] Much ado about nothingness.

[Aaron Watters]

I'm not sure I'm following you.  Maybe a simple example would
help.  What do you want to do and what is a slow way to do it
(using lots of python loops if necessary).  Once we see that
someone (likely not me) can suggest how to speed it up using
Numeric (and maybe altering the representation).

Guessing:
Suppose you have a 2d array of prices for securities by time with another 2d array
of same shape that indicate which price listings are invalid/unknown.
Suppose you wish to apply the heuristic of copying the youngest price
into the unknown positions, and further suppose the prices at time 0
are all known.

It seems to me you could pass a "price vector" across the time dimension,
picking out new valid prices and masking in most recent valid prices to
the source array for null positions.  My skills with numeric are such
that I'd have to work on this for a while to get it, so I'll let you think about
it instead.  It probably would amount to about 10 lines if that of NumPy.

It's likely that I suppose wrong however, and I could be off base in
any number of other ways.  --Aaron Watters

----------
> From: Hoon Yoon - IPT Quant <hyoon@nyptsrv1.etsd.ml.com>
> To: matrix-sig@python.org; arw@mh.lucent.com
> Subject: Re: [MATRIX-SIG] Much ado about nothingness.
> Date: Tuesday, July 08, 1997 12:28 PM
> 
> Aaron,
> 
>   This is what I love about working in Python. I had pretty hard time
>   making vendors understand why different nothingness mattered so much.
>   Now I am actually getting educated here. 
>   
>   I don't think I am stretching too much, when I say there isn't a good
>   stat package or DBMS out there that handles missings sufficiently.
>   For my computational needs, I think a mask of nulls will do nicely.
>   
>   Going back to how I am used to handling it.
>   Gauss has something called packer, which packs away any rows that 
>   contains missing value, which is a (.) dot in Gauss. In Python this
>   could be much more generalized. I can get index of non missing or
>   missing by row or column or N-th dim. And do either take or reverse
>   of take. This is bit costly on memory, however. I am actually creating
>   equal dimension (even at at bits) of 1's and zeros, which is not too
>   bad. I also noticed that whenver I used None in my array, I get a complex
>   array. Nothing should be uniform and should not be this expensive.
>   For backward filling, this is always has been tricky business. If I
>   have one time value missing, I can use the above 1/0 array move the
>   original array by 1 dimension backward and mult the result, but what
>   do I do if I have multiple period of missings? I guess a repeat is
>   in order untill I get "None in Target_array" is 0. Of course I am working
>   under premise that the first row of N-th dimention is not itself has 
>   missing. That could be really unfun.
>   Not to make too much fun of Gauss, but msg==msg was 1. None of the
>   stat package I know has more than one kind of nothingness for input.
>   
>   I guess this means that Python does not have naitive Null handler
>   upto now? I wouldn't be terribly surprised, but I think it needs one.
>   I am still getting confused about the N-th dimension thing. I am sure
>   it's good thing, but confused still. 
>   
>   How did everyone dealt with this before?
> Thank you for the reply,
> 
> Hoon,
> 

_______________
MATRIX-SIG  - SIG on Matrix Math for Python

send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________