[Numpy-discussion] suggestion for generalizing numpy functions

[Travis E. Oliphant]

Darren Dale wrote:
> On Mon, Mar 9, 2009 at 9:50 AM, Darren Dale <dsdale24 at gmail.com 
> <mailto:dsdale24 at gmail.com>> wrote:
>
>     I spent some time over the weekend fixing a few bugs in numpy that
>     were exposed when attempting to use ufuncs with ndarray
>     subclasses. It got me thinking that, with relatively little work,
>     numpy's functions could be made to be more general. For example,
>     the numpy.ma <http://numpy.ma> module redefines many of the
>     standard ufuncs in order to do some preprocessing before the
>     builtin ufunc is called. Likewise, in the units/quantities package
>     I have been working on, I would like to perform a dimensional
>     analysis to make sure an operation is allowed before I call a
>     ufunc that might change data in place.
>

The suggestions behind this idea are interesting.   It seems related to 
me, to the concept of "contexts" that Eric presented at SciPy a couple 
of years ago that keeps coming up at Enthought.    It may be of benefit 
to solve the problem from that perspective rather than the "sub-class" 
perspective. 

Unfortunately, I don't have time to engage this discussion as it 
deserves, but I wanted to encourage you because I think there are good 
ideas in what you are doing.    The sub-class route may be a decent 
solution, but it also might be worthwhile to think from the perspective 
of contexts as well.

Basically, the context idea is that rather than "sub-class" the ndarray, 
you create a more powerful name-space for code that uses arrays to live 
in.   Because python code can execute using a namespace that is any 
dictionary-like thing, you can create a "namespace" object with more 
powerful getters and setters that intercepts the getting and setting of 
names as the Python code is executing. 

This allows every variable to be "adapted" in a manner analagous to 
"type-maps" in SWIG --- but in a more powerful way.    We have been 
taking advantage of this basic but powerful idea quite a bit.   
Unit-handling is a case where "contexts" and generic functions rather 
than sub-classes appears to be an approach  to solving the problem.

The other important idea about contexts is that you can layer-on 
adapters on getting and setting variables into the namespace which 
provide more hooks for doing some powerful things in easy-to-remember 
ways.  

I apologize if it sounds like I'm hi-jacking your question to promote an 
agenda.   I really like the generality you are trying to reach with your 
suggestions and just wanted to voice the opinion that it might be better 
to look for a solution using the two dimensions of "objects" and 
"namespaces" (o.k.  generic functions are probably another dimension in 
my metaphor) rather than just sub-classes of objects.

-- 

Travis Oliphant
Enthought, Inc.
(512) 536-1057 (office)
(512) 536-1059 (fax)
http://www.enthought.com
oliphant at enthought.com