Re: [yt-dev] YTArray YTEP

Hi Casey, On Tue, Mar 12, 2013 at 1:20 PM, Casey W. Stark wrote:

Hi all.

I'm writing the YTEP and I came across a technical problem.

For the operation overloading, I realized that we might be able/have to remove the __op__ methods and do everything inside the __array_wrap__ method. Currently if you call `a + b` (where they're YTArray objects), it will check the units and make sure that's valid. But if you call np.add(a, b), it goes straight to __array_wrap__ without checking units!

Maybe we can have two ufunc registries -- one for unit conversion functions and one for unit check functions?

I'll be honest with you: I don't know enough about array wrapping to have an answer to this, and my *guess* is that you and Nathan are the best equipped to figure this one out. This might be a good reason to look at other implementations of array wrapping (there are, I think, even units-ful array wrappings) and maybe even query the numpy list. Sorry not to have the answer, I am just worried we might give the wrong one. -Matt

- Casey

On Mon, Mar 11, 2013 at 11:28 AM, Matthew Turk wrote:

...

Hi Doug,

On Mon, Mar 11, 2013 at 1:40 PM, Douglas Harvey Rudd wrote:

...

Hi Matt,

Let us know when you'd like us to push ARTIO changes to the mainline. It's still being actively worked on, but the main features should work, and we really would like to get the new code into the first 3.0 alpha so we can push our users to start testing.

Awesome! I think I will have time to read and test them in detail probably starting Wednesday, but I suspect that they're probably in a good state already to go in. I'll ping you once I've put in a handful of other changes I'm looking at.

Incidentally, I'm planning to put all this on a blog post about the workshop, but I wanted to take this opportunity to say how cool it was to see over the course of the few days the transition for ARTIO to a frontend based on opaque data objects.

-Matt

...

Douglas Rudd Scientific Computing Consultant Research Computing Center drudd@uchicago.edu

On Mar 11, 2013, at 12:21 PM, Matthew Turk wrote:

...

On Mon, Mar 11, 2013 at 1:15 PM, Nathan Goldbaum wrote:

...

Hi Matt,

I think the steps you outline are a good start. My plan to continue the units work was to try to get the unit tests passing and see what needs to get done as that process proceeds. I'm glad to hear you're excited enough about this to get started on that work.

Absolutely. On the plane I was able to get it down to 193 fails, out of ~2600 or so. The biggest issues seem to be a problem with how the broadcasting is working (as evidenced in the radial velocity fields), things like Baroclinic vorticity disagreeing with how we're changing units, and some misc other problems. I've pushed my changes up to Casey's fork.

...

At the moment the work of subclassing NDArray is probably ~75% complete. To finish we'll need to make sure we cover all of the possible ufuncs and do appropriate operations in all cases. Having a formal spec will help in that effort. There are also a couple of odd issues that still need to be worked out to make the code cleaner, i.e. the issue referenced in fce1e6c.

I definitely think we need to address the ufuncs through design. There are several issues I'm concerned about, specifically those cases where we do *not* want to return a subclass of YTArray. As an example, any of the ufuncs that return logical operators. These could in principle return dimensionless (which is what I have done) but I think I'm more comfortable returning just standard arrays . I've also had to put in what I think is something of a hack in returning scalars when a ufunc drops to a single value.

...

Right now all of this work is going on in Casey's yt-3.0 fork: https://bitbucket.org/caseywstark/yt-3.0. Last week we were just pushing directly to the fork but I think now that we're all in different time zones it's probably a good idea to process further changes via pull requests to that fork. Once everything is ready, Casey will be able to issue a pull request to the yt_analysis/yt-3.0 repository. Does anyone disagree with that idea?

Yes, I think it should live separately for a while -- it's *incredibly* invasive, and I'd like to roll out 3.0a1 on the 15th. I think we should wait until the tagging of 3.0a1 to do this. I'm also a bit concerned that there's heavy development on the ART and ARTIO fronts that has not yet been pulled back in to mainline, and I want to try to pull that in before this change. I'm not trying to put the brakes on the development, just that I think we need to take this a bit carefully since it's quite invasive and we now have a set of people using 3.0 for their work. Maybe we could aim to have the YTEP done, and then see how ready the code base is?

...

Cheers,

Nathan

On Mar 11, 2013, at 9:09 AM, Matthew Turk wrote:

Hi all,

Last week the units stuff really took off -- great work on that front! I was wondering if I might solicit a YTEP? As I have been going through the YTArray changes to get the unit tests passing, I'm realizing just how invasive it is in some ways. I think having an enumeration of the behavior of the YTArray under ufuncs as well as common operations will be pretty essential. That is something we should also probably discuss on this list.

I'll be submitting a few changes for review by Casey and Nathan as well. Do either of you have a feeling for where things stand now and what is needed next?

-Matt _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org