I'd like to publicly announce that I will be leaving my current position at
NCSA at the beginning of May and starting a 12-week batch at the recurse
center in NYC this summer. If you're curious about the recurse center, take
a look at their website for details (https://recurse.com). While I will
still be around and able to participate in the community at a lower level,
I will need to step down from my current role as primary maintainer of yt
when I leave NCSA so I can focus on new projects and opportunities.
Matt has volunteered to take up some of the slack on code review but I'm
hoping that everyone else can also chip in. We also have two postdocs here
at NCSA (Madicken Munk and Jared Coughlin) who have both expressed interest
in stepping up their contributions to yt and helping out with day-to-day
support and maintenance. I also hope we can have a conversation about
updating yt's governance documents and adding rotating members to the
steering committee at the development workshop next week.
I would like to continue maintaining unyt as part of the yt project going
forward as I expect that will take substantially less time than maintaining
yt as a whole. If people are nervous about yt depending on unyt when I'm
not employed to work on yt development that is fair and I'm happy to talk
about the relationship between unyt and yt in light of all of this.
My plan for the next few weeks is to help get yt-4.0 in a state where we
can do a release. That means merging the yt-4.0 branch into master, helping
out with testing and documentation, and triaging and fixing remaining
issues. I don't know if we'll be able to do a yt 4.0 final release by May
but hopefully we'll at least have a much more solid base to work from.
I'm going to try my best to make this transition as smooth as possible. If
you're aware of places where I could take action now to lead off headaches
down the road it would be great to get reminders about those places.
It's been an amazing experience to have so much of my professional life and
achievements influenced by yt and the yt community over the past 8 years or
so. I'm looking forward to seeing all the cool new things you all will
build in the next few years.
With a heavy heart,
I'd like to cut yt 3.5.1 next week. I've issued a PR with some backports
If anyone has anything they'd like to see get included or any objections to
cutting a release please let me know here.
Something that's come up for me a lot lately is hearing from people who
have datasets that are mostly recognized by yt, but have a few extra fields
that aren't normally present. I think that in some cases, these can be read
in, but are returned as dimensionless. Even then, this produces a number of
cumbersome side-effects for using them in analysis that I won't bother to
get into unless someone is interested.
Unless I'm mistaken, the only current solution to this is to modify the
known_other_fields and known_particle_fields tuples in that frontend's
fields.py file. I think there are a lot of benefits to creating a means for
users to "declare" additional on-disk fields for a dataset.
Here are some of the big ones, for me at least:
1. There are certain simulation codes with a large number of slight
variations. Keeping up with them all is probably unfeasible, especially
when some of these will have relatively short life spans.
2. The current solution requires either someone to maintain a separate fork
(or uncommitted changes!) or intervention by experienced developers plus
the delay time associated with propagating changes upstream.
3. Perhaps such a system could make it easier for long-lasting variants to
ultimately become officially supported by providing the user a simpler way
to debug issues relating to new fields (like units, proper aliases, etc.).
In fairness, some potential cons:
1. This gives people an excuse to not add official support for new variants
and results in a separate ecosystem of frontends only supported through
2. It could be quite difficult to do for some datasets.
As for how to do this, my impression is that something needs to be provided
at load time before the field_list is assembled, something like handing
yt.load some sort of FieldDeclaration object or a series of dicts/tuples to
pass into the FieldInfoContainer.
Anyway, I think this could be very helpful to a lot of people, but I'm very
interested to hear any thoughts on whether this is a good idea and how we
might do it.