yt-dev

yt-dev@python.org

1 participants
2508 discussions

by Matthew Turk

Hi all, On Friday I'll be holding "yt office hours" at 9:30AM Central (10:30AM Eastern, 3:30PM GMT). I'll announce the URL on Slack in #general just beforehand, but the stream will *probably* be at twitch.tv/powersoffour . If you have questions about yt, or want to get your feet wet developing or see how code review and whatnot happens, please stop on by! -Matt

3 days, 6 hours

Org-level projects

by Matthew Turk

Hi folks, I've started using org-level projects to collect and organize work. The first one I started up was for stretched grids: https://github.com/orgs/yt-project/projects/2 Please feel free to add items, play with this functionality, and so on. One neat thing about it is that we can link issues across repositories here. -Matt

1 week, 4 days

Steering Committee Update: Project Coordination

by Matthew Turk

Hi all, The steering committee has discussed the current state of yt, where things are and where they are going, and talked through various strategies for moving forward effectively. Where we're at right now is that many of the maintainers have been timesliced quite a bit; this often results in bursts of activity, which is then not followed up, and a long delay between (for instance) pull requests being issued, being looked at, and issues being responded to. Additionally, the thing that seems *really* lacking, which needs to be addressed, is that the decision making process for the community isn't entirely obvious -- not so much in terms of the rules, but of what is being worked on, who's working on things, and how to engage. I've been somewhat less engaged with the project than I'd like lately, and part of that has been situational (teaching, family, pandemic overhead) and some of it has been that I've been trying to engage at the project level without stepping on any toes *and* to try to contribute code. The upshot was that neither was really happening! So, after discussion with the steering committee, we've decided that the best way forward for now is for me to step into a "project manager" role. What this means is that I'll be taking on a much more active daily role in yt. This will include being the first point of contact for new issues and pull requests, being more assertive about "assigning" issues to individuals (with the knowledge that they may decline!), applying labels, editing issues for clarity and so on, and also stewarding and managing the review process. This also means that I'll be more engaged in public planning of projects and utilizing the project management systems provided by Github. We've long thought these were worth using, but where we ended up was that in the absence of a consensus (or a lazy consensus) we tended not to utilize new things for fear that they would go unused or that others would not approve, or whatever. Additionally, we have in place a structure for governance. But, this structure isn't self-enforcing, and it will be in this role that I will work to ensure we're following the guidelines, processes and structures we have enumerated in our governance documents. Finally, I'll be taking on the problem of holding and running "meetings" and drop-in office hours. I have also been thinking about ways of providing weekly updates on activity in the project for folks who may have unsubscribed from the repo or who find the overall communication volume to be not-quite-right. But, all of this text aside, the proof will be in the actual *doing* of things, and I look forward to *doing* things and being more vocal about it. -Matt

1 month, 1 week

Update on yt 4.0.2: planning for what's next ?

by Clément Robert

Hi all, The 4.0.2 milestone is currently complete !https://github.com/yt-project/yt/milestone/18 I’ve switched to using the "backport-yt-4.0.x” label in PRs that are non-blocking, but could be included in the release if we’re able to merge them in time. I believe that our attempt at automating backports with MeeseekBox was a success, and it helped a lot in producing what I think is going to be the biggest bugfix release in yt’s (recent ?) history. Because this tool is most useful and reliable when used continuously (as opposed to firing 30 backport PRs at once as we did when we started using it), I suppose now is a good time to ask what’s next: do we want to go to 4.1.0, or 4.0.3 ? Since there has been no discussion about scheduling yt 4.1.0 yet, I’d suggest that we plan for an additional bugfix release (4.0.3). We don’t have to set a date for it yet, I am mostly proposing that we keep backporting bugfixes to the yt-4.0.x branch until we’re ready for a feature release. What do you think ? Take care Clément

1 month, 1 week

[ANN] yt_astro_analysis 1.1 released

by Britton Smith

Greetings! yt_astro_analysis version 1.1 is released! yt_astro_analysis is an extension of yt for astrophysical analysis. This is the first stable release of yt_astro_analysis to be compatible with yt version 4.0 and later. New Features - The HaloCatalog has been significantly refactored (#58 <https://github.com/yt-project/yt_astro_analysis/pull/58>, #62 <https://github.com/yt-project/yt_astro_analysis/pull/62>) with the following additional improvements: - the interface to the Rockstar halo finder is now compatible with the latest version of Rockstar Galaxies #55 <https://github.com/yt-project/yt_astro_analysis/pull/55> - all halo finders now support being run with time-series of datasets - halo particle ids now savable with FoF and HOP halo finders #52 <https://github.com/yt-project/yt_astro_analysis/pull/52> - looping over halos is done with io chunks instead of ds.all_data for a significant speedup and reduction in memory - Allow more flexibility for specifying rockstar particle mass #84 <https://github.com/yt-project/yt_astro_analysis/pull/84> - Add restart option for rockstar #82 <https://github.com/yt-project/yt_astro_analysis/pull/82> - Adding an outer_radius parameter to the iterative COM callback #34 <https://github.com/yt-project/yt_astro_analysis/pull/34> - Remove the sunyaev_zeldovich analysis module. This is now pyxsim <http://hea-www.cfa.harvard.edu/~jzuhone/pyxsim/>. #79 <https://github.com/yt-project/yt_astro_analysis/pull/79> - Drop support for python 3.6 #100 <https://github.com/yt-project/yt_astro_analysis/pull/100>, #101 <https://github.com/yt-project/yt_astro_analysis/pull/101> Minor Enhancements and Bugfixes - significant project management and ci improvements #89 <https://github.com/yt-project/yt_astro_analysis/pull/89>, #90 <https://github.com/yt-project/yt_astro_analysis/pull/90>, #91 <https://github.com/yt-project/yt_astro_analysis/pull/91>, #92 <https://github.com/yt-project/yt_astro_analysis/pull/92>, #96 <https://github.com/yt-project/yt_astro_analysis/pull/96>, #95 <https://github.com/yt-project/yt_astro_analysis/pull/95>, #97 <https://github.com/yt-project/yt_astro_analysis/pull/97>, #108 <https://github.com/yt-project/yt_astro_analysis/pull/108>, #109 <https://github.com/yt-project/yt_astro_analysis/pull/109> - Add annotate_halos function #98 <https://github.com/yt-project/yt_astro_analysis/pull/98> - only access particle_type field in rockstar if it exists and is needed #111 <https://github.com/yt-project/yt_astro_analysis/pull/111> - fix light cone projections with weight fields #37 <https://github.com/yt-project/yt_astro_analysis/pull/37> - Fix HaloCatalog progress bar #40 <https://github.com/yt-project/yt_astro_analysis/pull/40> - clarify rockstar error message about using the wrong number of MPI processes #42 <https://github.com/yt-project/yt_astro_analysis/pull/42>, #113 <https://github.com/yt-project/yt_astro_analysis/pull/113> - check derived_field_list for base fields #43 <https://github.com/yt-project/yt_astro_analysis/pull/43> - allow cosmology splice from a single dataset #49 <https://github.com/yt-project/yt_astro_analysis/pull/49> - Fix iterator #68 <https://github.com/yt-project/yt_astro_analysis/pull/68> - Support new config file format #65 <https://github.com/yt-project/yt_astro_analysis/pull/65> - Enable circleci testing #44 <https://github.com/yt-project/yt_astro_analysis/pull/44> - Add max_box_fraction to plan_cosmology_splice #76 <https://github.com/yt-project/yt_astro_analysis/pull/76> - Fix HaloCatalog output_dir #81 <https://github.com/yt-project/yt_astro_analysis/pull/81> - remove deprecated dm_only keyword from halo finder #57 <https://github.com/yt-project/yt_astro_analysis/pull/57> - update amr_grid.inp #77 <https://github.com/yt-project/yt_astro_analysis/pull/77> New Contributors - Corentin Cadiou (@cphyc <https://github.com/cphyc>) - Nathan Goldbaum (@ngoldbaum <https://github.com/ngoldbaum>) - Meng Yuan Ho (@willsonho2000 <https://github.com/willsonho2000>) - Cameron Hummels (@chummels <https://github.com/chummels>) - Clément Robert (@neutrinoceros <https://github.com/neutrinoceros>) Full Changelog: yt_astro_analysis-1.0.0...yt_astro_analysis-1.1.0 <https://github.com/yt-project/yt_astro_analysis/compare/yt_astro_analysis-1…> The yt_astro_analysis package lives at: https://github.com/yt-project/yt_astro_analysis Documentation can be found at: https://yt-astro-analysis.readthedocs.io Special thanks to Clément Robert for assembling the machinery to make this release happen! On behalf of the yt-astro development team.

1 month, 1 week

Update on yt 4.0.2: call for (final ?) reviews

by Clément Robert

Hello yt folks ! We’re getting close to the next bugfix release, with 61 PRs merged to the yt-4.0.x branch since yt 4.0.1 was released. There are a couple more that could use a review, most of which should be easy to review. You can get an overview of what’s left in the milestone https://github.com/yt-project/yt/milestone/18 For this one bugfix in particular I would really appreciate a technical review, because it requires a change in the testing framework itself https://github.com/yt-project/yt/issues/3632 This particular PR happens to be important to me because figures with height > width are extremely common when plotting data in spherical coordinates, something my group uses a lot. Beyond these, I’ll defer to you guys, and our release manager in particular, to decide when the actual release should happen. Have a good day! Clément

1 month, 3 weeks

Typing: request for second set of eyes

by Matthew Turk

Hi folks, Can somebody (other than me) take a look at this? https://github.com/yt-project/yt/pull/3546 I think it should be fine to go in, and I don't want to let it linger too long. -Matt

1 month, 3 weeks

Fwd: Supporting arbitrary grid schemes ?

by Clément Robert

Sorry I accidentally responded to Micheal only, here’s my response > Begin forwarded message: > > From: Clément Robert <clement.robert(a)protonmail.com> > Subject: Re: [yt-dev] Supporting arbitrary grid schemes ? > > Date: 20 October 2021 at 17:04:12 CEST > To: Michael Zingale <michael.zingale(a)stonybrook.edu> > > Interesting. I think yt already has some support for these, though I admit I’ve never seen it used outside of (failed) image tests, so here’s one I was able to find on my computer. > Does this fall into the “mapped grid” catagory ? > > Indeed what I want to support in the context of my current team is probably a subset of these “logically rectangular” grids. My teams uses rectilinear grids, but we stil have orthogonality everywhere. > >> On 20 Oct 2021, at 15:41, Michael Zingale <michael.zingale(a)stonybrook.edu> wrote: >> >> here's an example of some simple mapped grids that are used in astro: >> >> http://faculty.washington.edu/rjl/pubs/circles/SIR000723.pdf >> >> it sounds like what you are describing is a restricted version of a general mapped grid. >> >> On Wed, Oct 20, 2021 at 9:31 AM Clément Robert via yt-dev <yt-dev(a)python.org> wrote: >> >>> I don’t know ! Where can I find a definition for these ? >>> >>>> On 20 Oct 2021, at 13:52, Michael Zingale <michael.zingale(a)stonybrook.edu> wrote: >>>> >>>> It's this similar to mapped grids? >>>> >>>> On Wed, Oct 20, 2021, 5:33 AM Clément Robert via yt-dev <yt-dev(a)python.org> wrote: >>>> >>>>> Hi folks >>>>> >>>>> As you may be aware, yt’s grid based index classes offer support for patched-based AMR as well octrees, but it’s currently lacking support for so called “stretched gridding scheme”. This is a known limitation in several existing frontends (AMRVAC, BoxLib, Chombo-Pluto are the ones I know of). >>>>> >>>>> For context, what I refer to as “stretched grids” are grids where cell sizes are not simple powers of 1/2 (with respect to a unitary reference) as in classic AMR and octrees, but may be specified by arbitrary rules (such as geometric series). Some hydro code support having parts of the grid following classic AMR scheme and other blocks to be “stretched” along one or several directions at once. This is the case of AMRVAC and Pluto, for instance. BoxLib also has support for complex refinement rules that are not covered by storing a single `refine_by` integer as we’re currently doing in Index classes (this is https://github.com/yt-project/yt/issues/3481). >>>>> It seems to me that the “easy” way of supporting these data formats would be to generalise yt’s mechanisms for reconstructing cell position and size (currently it’s using cell index, domain edges, refinement level + a `refine_by` multiplier) so that cell edges can be stored as arrays that frontends would need to somehow retrieve from datafile or reconstruct from parameter files. >>>>> >>>>> There’s been some effort in the past to get over this limitation led by John and Matt, and we’ve been hitting this wall from time to time in discussions. >>>>> Now, I understand that this would most likely not be a trivial change, especially because we don’t want to break existing frontends in the process (or performance !), and at the moment, I don’t have a very clear idea of what exact steps are needed to achieve this goal, and in particular I don’t have enough material to draft a YTEP on the topic. >>>>> It seems clear we’d need *some* refactoring/generalisation of Index classes and friend classes and new pixelizer routines would likely be needed. Anything else ? >>>>> >>>>> I however have personal motivation to move forward here since my team is developing an hydro code and running simulations where grid streching is the rule rather than the exception. >>>>> >>>>> What I would like to do to get us rolling is simply collect the existing issues and PRs related to this goal. I know there are a couple, but it’s very easy to loose track of them, and it’s hard to see the big picture. >>>>> Can I create a label (“arbitrary grids” ?) and apply it to existing material touching this topic ? >>>>> Do you know of existing PRs and issues that are relevant to this topic ? >>>>> >>>>> Here are the ones I know: >>>>> - https://github.com/yt-project/yt/issues/3481 >>>>> - https://github.com/yt-project/yt/pull/2998 >>>>> - https://github.com/yt-project/yt/issues/1880 >>>>> - https://github.com/yt-project/yt/issues/385 (from 2012 !!) >>>>> >>>>> Thanks >>>>> Clément >>>>> _______________________________________________ >>>>> yt-dev mailing list -- yt-dev(a)python.org >>>>> To unsubscribe send an email to yt-dev-leave(a)python.org >>>>> https://mail.python.org/mailman3/lists/yt-dev.python.org/ >>>>> Member address: michael.zingale(a)stonybrook.edu >>> >>> _______________________________________________ >>> yt-dev mailing list -- yt-dev(a)python.org >>> To unsubscribe send an email to yt-dev-leave(a)python.org >>> https://mail.python.org/mailman3/lists/yt-dev.python.org/ >>> Member address: michael.zingale(a)stonybrook.edu >> >> -- >> >> Michael Zingale >> Professor of Physics and Astronomy >> >> Dept. of Physics & Astronomy • Stony Brook University • Stony Brook, NY 11794-3800 >> phone: 631-632-8225 >> e-mail: michael.zingale(a)stonybrook.edu >> web: [https://zingale.github.io](https://zingale.github.io/) >> github: [https://github.com/zingale](http://github.com/zingale)

1 month, 4 weeks

Fwd: Astronomical software development workshop, May 16–20, 2022, @ CCA

by Stephanie Tonnesen

Hi yt community! I just heard about this meeting being held this upcoming May 2022 at the CCA about open source development practices. I think many of us might be interested! If so you can sign up to get updates about the meeting. Best, Stephanie Dear colleagues, We are planning a workshop focused on topics in open source software development practices for astronomy, held in person at the Center for Computational Astrophysics in New York City, May 16–20, 2022. The program and scope are still in development, but we expect it to be of interest to astronomers with a wide range of open source experience, and across all career stages. The schedule will include a large fraction of unstructured time for participant-driven discussion, collaboration, and co-working. For more information about this workshop and to sign up to hear more details as they become available, please check out our website at https://code.astrodata.nyc Please share this email with your department lists and/or anyone else who you think might be interested in participating in this event. Best, Dan (on behalf of the organizing committee) -- Dan Foreman-Mackey Associate Research Scientist, Flatiron Institute http://dfm.io

2 months, 1 week

Temporarily moving to fallow fields

by Madicken Munk

Hi yt-dev, I am self-nominating myself to temporarily move to the fallow fields <https://yt-project.github.io/governance/leadership.html#groups-within-the-y…> for my development and community work in the yt-project. I would like to be doing more in the project but I am not able to commit to the level I'd like to, and I'd like to notify the community of my absence. In the interim, I have asked Kacper (@Xarthisius) to be release manager for the project and he has agreed. Happy development everybody. I look forward to positively contributing in the future. Madicken

2 months, 2 weeks

Jump to page:

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

yt-dev