On 23-Dec-09, at 10:34 AM, Anne Archibald wrote:

It's been a little while since I took a really close look at it, but I'll try to describe the problems I had. Chiefly I had problems with documentation - the only way I could figure out how to build additional gufuncs was monkey-see-monkey-do, just copying an existing one in an existing file and hoping the build system figured it out. It was also not at all clear how to, say, link to LAPACK, let alone decide based on input types which arguments to promote and how to call out to LAPACK.

I tried to create a new generalized ufunc (a logsumexp to go with logaddexp, so as to avoid all the needless exp's and log's that would be incurred by logaddexp.reduce) and had exactly the same problem. I did get it to build but it was misbehaving (returning an array of the same size as the input) and I couldn't figure out quite why. I agree that the documentation is lacking, but I think it's (rightly) a low priority in the midst of the release candidate.

The key idea would be that the "linear algebra dimensions" would always be the last one(s); this is fairly easy to arrange with rollaxis when it isn't already true, would tend to reduce copying on input to LAPACK, and is what the gufunc API wants.

Would it actually reduce copying if you were using default C-ordered arrays? Maybe I'm mistaken but I thought one almost always had to copy in order to translate C to Fortran order except for a few functions that can take row-ordered stuff. Otherwise, +1 all the way. David

On 23-Dec-09, at 2:19 PM, David Goldsmith wrote:

Thanks Anne (and Dave): it may seem to you to be "a bit silly to dream up an API without implementing anything," but I think it's useful to get these things "on the record" so to speak, and as a person charged with being especially concerned w/ the doc, it's particularly important for me to hear when its specific deficiencies are productivity blockers...

In fact, there are gufuncs in the tests that are quite instructive and would form the basis of good documentation, though not enough of them to give a complete picture of what the generalized ufunc architecture can do (I remember looking for an example of a particular supported pattern and coming up short, though I can't for the life of me remember which). The existing documentation, plus source code from the umath_tests module marked up descriptively (what all the parameters do, especially the ones which currently receive magic numbers) would probably be the way to go down the road. David

On Wed, Dec 23, 2009 at 05:30:16PM -0800, David Goldsmith wrote:

On Wed, Dec 23, 2009 at 2:26 PM, David Warde-Farley wrote:

...

On 23-Dec-09, at 2:19 PM, David Goldsmith wrote:

...

Thanks Anne (and Dave): it may seem to you to be "a bit silly to dream up an API without implementing anything," but I think it's useful to get these things "on the record" so to speak, and as a person charged with being especially concerned w/ the doc, it's particularly important for me to hear when its specific deficiencies are productivity blockers...

In fact, there are gufuncs in the tests that are quite instructive and would form the basis of good documentation, though not enough of them to give a complete picture of what the generalized ufunc architecture can do (I remember looking for an example of a particular supported pattern and coming up short,

If you came up short, how/why are you certain that the existing arch would support it?

The existing documentation made the capabilities of generalized ufuncs pretty clear, however not much is demonstrated in terms of the appropriate C API (or code generator) constructs. David

2009/12/23 David Warde-Farley :

On 23-Dec-09, at 10:34 AM, Anne Archibald wrote:

...

The key idea would be that the "linear algebra dimensions" would always be the last one(s); this is fairly easy to arrange with rollaxis when it isn't already true, would tend to reduce copying on input to LAPACK, and is what the gufunc API wants.

Would it actually reduce copying if you were using default C-ordered arrays? Maybe I'm mistaken but I thought one almost always had to copy in order to translate C to Fortran order except for a few functions that can take row-ordered stuff.

That's a good point. But even if you need to do a transpose, it'll be faster to transpose data in a contiguous block than data scattered all over memory. Maybe more to the point, broadcasting adds axes to the beginning, so that (say) two-dimensional arrays can act as "matrix scalars". Anne