[Matrix-SIG] Re: a full-blown interactive data analysis environment

Travis Oliphant Oliphant.Travis@mayo.edu
Mon, 8 Feb 1999 16:34:10 -0600 (EST)


I am beginning now to see Joe's Point here better after having some
experience building different types of modules.

We definitely need to talk about what kind of things we expect from the
language.  For example, MATLAB defines only one datatype but NumPy defines
about 10.  For lowlevel operations which are coded in C this makes a
difference.

It seems that we will need to have separate language-independent and
language-dependent code developed.  I think it is a good idea to establish
the expectations of code contributed to the project in terms of having the
interpreted-language specific and non interpreted-language specific code
separated.  I'm beginning to see my released code in this light better.
Most of what I've done is interpreted-language specific wrapping around
already non-specific code (cephes, fftw).  But it is often desirable to
use language-specific features in the interface (like treating cephes
functions as ufunc objects).  I'm not sure that this should be or even can
be generalized enough to eliminate the use (if not the need) for specific
code that really doesn't contribute to the goal Joe has in mind.  

If this all sounds confusing, it is because I am a little confused as to
how to actually make something like Joe suggests work.  Are we talking
about describing a generic interface to specialized code that will adapt
itself to every future language?  Can we guarantee that.  Some languages
will work, some won't with it, right?  

I guess what I understand now that I think is doable is writing packages
with the idea of general usability in mind.  For example, I'm going to
try to change my N-D convolution and order-filtering routines so that the 
core routine assumes little about how the needed data is stored as an
object in the interpreted language.  This is not necessarily going to be
easy (for me) and may not even be possible without sacrificing speed or
usability in Python.  This is the sort of difficulty I'm talking about in
making a generic package that interfaces (well?) with any future scripting
language.

MATLAB defines their extensibility feature in terms of a GATEWAY procedure
and the code.  The GATEWAY handles all the MATLAB specific stuff and the
procedure itself is portable.  I could see doing something like that for
this package.  We would need a procedure for every supported datatype that
way, but it could be done.   This may not be possible with every desired
addition, however (I'm not sure how this idea would change the
cephesmodule I've released --- I guess all I did was write the GATEWAY for
that one)

I'd love to hear thoughts and ideas from people who have written
code and linked it with interpreted languages.  To move this forward we
definitely need some experienced people to help define generic
expectations from the interpreted language (with Python as a
model, of course :-)  ) and then just publish it somewhere and then start
writing code and documentation.

Best,

Travis Oliphant