[SciPy-User] pylab

[Joshua Holbrook]

On Tue, Jul 20, 2010 at 3:12 PM, Benjamin Root <ben.root at ou.edu> wrote:
> On Tue, Jul 20, 2010 at 12:24 PM, <PHobson at geosyntec.com> wrote:
>>
>> > > Ben and Pierre,
>> > >
>> > > Thanks for the tips! Glad I chimed in here b/c I've definitely learned
>> > something. I'm often pulling data out from a database, so null records
>> > come back to me as None's. I'll be sure to set a CASE statement now that
>> > fills the NULLs in with an obviously junk value that I can mask from now
>> > on.
>> > >
>> > > -paul
>>
>> > Why not user your None as said junk value?
>> >
>> > --Josh
>>
>> All my existing routines just throw values from the database (Postgres or
>> MS SQL) cursor directly into numpy arrays. As Pierre pointed out, the None
>> values force the array's dtype to object.
>>
>> In [40]: x
>> Out[40]: array([2.5, 12.2, 5, None], dtype=object)
>>
>> I could easily mask the Nones, but switching it over to a value such as
>> -99999 would let it me keep the dtype as float or integer. Probably not a
>> big deal at the moment, but I'm trying to adopt as many best practices as
>> possible so nothing comes back to bite me later on. (Negative values are
>> pretty rare in environmental data, so I think it's a safe bet).
>> -paul
>>
>
> As a general rule, magic values can be very tricky to work with, and it is a
> good idea to think out conventions.  I was just bitten by a bug where a
> previous developer wanted coordinate values of exactly 0.0 (yes, floating
> point) to mean something special, but had no constraints for the domain.
>  So, once in a blue moon, a data point would get processed differently than
> all the others, causing slightly odd analysis results.
>
> So, my personal rule is to use NaNs whenever/where-ever possible because if
> they sneak into a calculation unexpectedly, they will usually propagate such
> that your program breaks -- which is a good thing.  Bad results that appear
> to be good is a bad thing.  When you have to do integers, pick a magic value
> that is impossible (e.g., negative value for temperature in Kelvins), and
> document that magic value.
>
> I hope this is helpful.
>
> Ben Root
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>
>

Hmm! Yeah, NaN would make more sense. >_<