[MATRIX-SIG] Much ado about nothingness.

[Hoon Yoon - IPT Quant]

Hello fellow snake handlers,

  I am attempting to do some Trading (stocks) related number crunching
  in Python. I am having quite a difficulty trying to implement my old
  Gauss logic to Python. It should be dealt in a different way; however,
  my mind is still in the old 2D complete matrix stage that I am finding
  it difficult to find what I wanna to do in Python.
 =20
  # I have some x period of info into Python lists/arrays. Of course as
  of any financial data there are missing values. So, I stored them as
  None, thinking that None is closest to msg values, but, as some of the
  readers would have realized, none of the nonzero, choose, equal, =
where, etc...
  can be performed on array or list with None values. I get:
  	TypeError: function not supported for these types, and can't=20
  	coerceto supported types
  Now some might say why not just use either zeros or negative values. =
Well,
  if I am just dealing with price and vols that's okay, but I am also =
dealing
  with other info like earnings, which can be easily negative.
  There is at least distinction in mathmatics/databases about missing, =
undefined,
  and not available. This is very thorny issue in numerical computing. =
Is msg =3D=3D
  msg? (This is undefined) What's truly missing vs. undefined? This is =
quite a
  issue. For people brave enogh, I encourage Joe Celko's articles in =
DBMS with
  same title, which can be obtained through DBMS. He really does good =
job of=20
  explaining the issues. There are other issues like +&- infinity NA and =
so on,
  which Joe does not get into, but overall the article serves as good =
introduction
  to the concepts.
  May be all this issues have been considered and Python has way to =
handle all this,
  but I could not find it. I would really like to see find_msg & =
index_msg find of
  functions. The most often used way to deal with missing values are :
    a. Just copy over from last period or do some estimate based on =
others, which is
  bit dangerous due to introduction of serial correlation.
    b. Get rid of the entire row of the observation. Which means throw =
the whole stock
  or the whole day from the model. Less dangerous, but could be =
problematic, espcially
  if you don't have too many observation to play with.
 =20
  find_msg should return matrix of same dimention filled with 0,1. =
index_msg should
  only work on 1 dim and give me back index to an array. I can use =
find_msg for take
  kind of function, or rever of take.
 =20
=04 Hopefully I can do this without loops!

Thanks for reading,

Hoon,

_______________
MATRIX-SIG  - SIG on Matrix Math for Python

send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________