yt-dev

yt-dev@python.org

6 participants
2462 discussions

yt-4: Migration to a new configuration system | action required on your side

by Corentin CADIOU

Dear all, TLDR; We recently migrated yt's configuration system from the yt-3 format to a new and more flexible format in yt-4 (the toml format, see https://github.com/yt-project/yt/pull/2981). *This requires a manual intervention to perform the migration (see details below).* ------------------------------------------------------------------------ *What does it change for users?* The new configuration system is documented online here: https://yt-project.org/docs/dev/reference/configuration.html. *Performing the migration* The change requires a one-time manual intervention on your side to perform the migration automatically. To perform the migration, simply type in your console yt config migrate This should migrate your old configuration to the new format (stored in ~/.config/yt/yt.toml) and store a backup of your old configuration file in ~/.config/yt/ytrc.bak should any error happen. Please report any issue you may encounter in the migration procedure so that we can fix them before releasing officially yt-4. If you get an error about a missing toml package, you may also have to install the toml package manually| | | conda install -c conda-forge toml # if you are using conda pip install toml # otherwise | Per-folder configuration The new configuration system allows you to configure yt per-folder. In particular, if a file named yt.toml is present in your current working folder, this file will override the global configuration. You can easily configure yt from the command line using yt config set yt log_level 40 --global # this will write in ~/.config/yt/yt.toml # or yt config set yt log_level 40 --local # this will write in ./yt.toml Naming convention We have also rationalized the naming convention of configurable keys in the configuration file: all keys now use snake case (i.e. words separated by an underscore). For example loglevel has become log_level. This is handled automatically by the configuration script, but bear that in mind if you modify the configuration file. For reference, all configurable keys can currently be found online https://github.com/yt-project/yt/blob/main/yt/config.py#L12 and the most important ones are documented online (https://yt-project.org/docs/dev/reference/configuration.html). What does it change for developers? The new format should allow more flexible configuration options. In particular, it supports nested configuration blocks and allows to find the most specific key in the configuration file # yt.toml [fields] cmap = "arbre" [fields.gas] cmap = "viridis" [fields.gas.temperature] cmap = "plasma" In Python, you can then query the "most specific" key matching a given path: from yt.config import ytcfg ytcfg.get_most_specific("fields", "gas", "temperature", "cmap") == "plasma" ytcfg.get_most_specific("fields", "gas", "density", "cmap") == "viridis" ytcfg.get_most_specific("fields", "io", "velocity_x", "cmap") == "arbre" This isn't used anywhere /yet/. The other breaking change compared to the old configuration system is that ytcfg.get returns typed quantities (as read by the toml parser) instead of strings. Best, Corentin -- Dr. Corentin Cadiou Post Doctoral Research Assistant Cosmoparticle Initiative Hub, desk 27 University College London (UCL) Gower St, Bloomsbury, London WC1E 6BT mobile: +33.6.43.18.66.83

5 days, 4 hours

Change default branch name

by Matthew Turk

Hi folks, In January, Github is making it easier to change the default branch name with a toggle on the website: https://github.com/github/renaming I know lots of other projects have done this (including maestro) and we could do it ourselves, but I'd like to propose that we do this as soon as GH has the tooling in place to handle it for us. If anyone objects we can dig into this during a team meeting or something, but personally this kind of seems uncontroversial and should be easy to implement. -Matt

1 week

Branch name change

by Matthew Turk

Hi folks, As a quick update, the primary branch for yt-project/yt is now "main", renamed from "master". This should be minimally invasive, but if you run into any troubles, let us know! -Matt

1 week

Auto-fixing linting errors

by Clément Robert

Dear yt-dev I recently found out about https://pre-commit.ci, and I'd like to propose we adopt this tool as part of our CI for linting/auto-fixing. The idea is that this automatically runs $ pre-commit run --all-files on each pull request and commits to the corresponding branch to auto-fix errors with pre-commit hooks (black, isort and flynt in our case). I'm reaching out to you because this would have small repercusions on everyone's workflow. Let me try to give a comprehensive description of gains and costs in the following. what it gets us a simplification of our CI tooling, namely: - `tests/lint_requirements.txt` - `.github/workflows/style-checks.yaml` - https://github.com/yt-project/slash-command-processor all become useless `.github/PULL_REQUEST_TEMPLATE.yaml` will also be simplified as all linting items in `PR Checklist` become redundant. On top of this, new hooks could be seamlessly added in the future without increasing the entry barrier for occasional contributors (as installing pre-commit hooks locally is and remains an opt-in). what it costs us - "zero" configuration: the only requirement is `.pre-commit.yaml`, which is already part of the repo. - minor updates to .pre-commit-config.yaml, namely : - bumping black's version (reason: black's `--exclude` flag is overidden by pre-commit's `--all-files`, but more recent versions of black have a `--force-exclude` flag that overcomes this) this will require a new PR to adopt black's latest (minor) updates in code styling - signing up tohttps://pre-commit.ci (free) what this changes to your workflow PRs will be automatically fixed by the bot, so you'll either need to occasionally run `git pull` or `git push --force` to account for it. Note that this can be avoided completely by installing pre-commit hooks locally which is and will remain an opt-in, see https://yt-project.org/docs/dev/developing/developing.html#pre-commit-hooks. Let me know what you think. Take care, and happy new year ! Cheers, Clément

2 weeks, 2 days

proposal: move new blog format into yt-project org

by Madicken Munk

Hi yt-dev, I have been working on an update to our blog, which is currently on the yt-project.org website here: https://blog.yt-project.org/ as you can see, there haven't been updates to it in quite some time. Matt and I have ported all of the old posts from this blog to a hugo page that has tagging and categorization, so one can filter blog posts based on search interest. Here's what it looks like right now: munkm.github.io/yt-blog/ and here is the repo currently: https://github.com/munkm/yt-blog What do you all think? I think we could get some very cool contributions to the blog this way. -Madicken PS: I presented this blog at the yt user's workshop a few weeks ago! If you want to see it in action you can watch it here: https://www.youtube.com/watch?v=wKh_8AAlFhY

1 month

Proposal: move two repositories to yt-project

by Matthew Turk

Hi folks, For a while, we've had two projects "incubating" in the org data-exp-lab, which is tied to the research group I'm in here at UIUC. These are the "widgyts" repo, which has some jupyter widgets in it, and "yt_idv" which is where the IDV now lives. I'd like to propose moving these to yt-project. I don't see this as being very controversial, but I thought I'd give the opportunity for folks to chime in yay/nay. Thanks, Matt

1 month

Switch to new configuration system

by Corentin CADIOU

Dear yt devs, TLDR: should we change the configuration format from .ini to another one (e.g. toml)? ------------------------------------------------------------------------ Last meeting we discussed what steps were required for the 4.0 release and one of the point was whether we should change how we handle configurations files (e.g. to get https://github.com/yt-project/yt/pull/1931 eventually in). Given that changing the format of yt's configuration is a big breaking change, this would have its place in a yt-3 to 4 transition plan. Amon the possibilities of a new or improved configuration system, users may be able to set some preferences per-field (e.g. have a configuration that implements the user's wish that "velocity_divergence should be displayed using a linear scale, with the seismic colormap"), or to pick their favourite unit system (e.g. cgs for the astro community, mks for the others). I know this has already been discussed earlier (both in PR comments and /via/ email), but let's take a moment to think about it with a fresh perspective. I can see three possible options: 1. Keep the .ini file format and build on top of that. _Pros:_ * no need to write a script to convert the old configuration files to a new format * easy to write for the user _Cons:_ * may become increasingly cumbersome to maintain if the available configuration keys are extended * no support for subgroups within one configuration group, so the configuration file may become cluttered or we'd need to handle this manually in the codebase 2. Switch to toml _Pros:_ * the language support semantically boolean/str/numbers (and parse them accordingly) * language is standardized & similar to .INI * easy to write (but see below) * possible to have nested configs, e.g. to set defaults for a range of different fields: [yt.fields.gas] # Use linear scale by default for gas fields … use_log = false [yt.fields.gas.pressure] # … except for pressure fields use_log = true That would be parsed natively as {'yt': { 'fields': { 'gas': { 'use_log': False, 'pressure':{ 'use_log': True } } } } * the syntax make it explicit what type each value is, so we could check the validity of the config when yt is loaded (for ex. [yt]\nloglevel=nan is invalid because loglevel should be a number). _Cons:_ * users will have to migrate their old config (if any), though we could provide a migration script * yet-another dependency (but we could vendor the parser, there are small implementations) * may be harder to write (how does a user know the expected type of each key? Should [yt]\n loglevel by a string or a number?) * for developers: we'd need to change the way the config is supported in the codebase (currently we're relying on ConfigParser) * there are some gotchas, for example boolean true is written "true" not "True" 3. Switch to yaml or json? What is your opinion? As a starting point, you could think about how you'd implement the example given above to deactivate the log scale for plots made with any field with type "gas", except pressure (is this something we want? If so, what's the best solution to achieve it in a meaningful way for the user?). Regards, Corentin -- Dr. Corentin Cadiou Post Doctoral Research Assistant Cosmoparticle Initiative Hub, desk 27 University College London (UCL) Gower St, Bloomsbury, London WC1E 6BT mobile: +33.6.43.18.66.83

2 months

ANN: yt 3.6.1 release

by Madicken Munk

Hello yt friends! yt 3.6.1 is now out! It includes one backport: matplotlib 3.3.x support. See PR 2754 <https://github.com/yt-project/yt/pull/2754> and the initial bug report at 2752 <https://github.com/yt-project/yt/issues/2752> . For release notes, see https://yt-project.org/doc/reference/changelog.html#version-3-6-1. Binaries for yt 3.6.1 are available via pip and conda. If you installed via the install script or use conda to manage your python installation, you can update yt via: $ conda update -c conda-forge yt And via pip if you manage your python installation with pip: $ pip install -U yt As always, if you have any questions, concerns, or run into any trouble updating please don’t hesitate to send a message to the mailing list or stop by our Slack workspace. Thanks for being in the community! - the yt development team

2 months, 2 weeks

yt 4.0 planning meeting

by Madicken Munk

Hi yt-dev! This week during the yt team meeting we decided to hold a meeting to plan out our next steps with yt-4.0, specifically: - if all PRs labeled with the 4.0 label should be blockers to a release - what else we need to do for the 4.0 release - what our plan is for required 4.0 features (who is doing them, do they need help, etc.?) - moving non-urgent PRs/issues to a later release, if relevant Please mark your availability on the poll if you'd like to join for this meeting! https://www.when2meet.com/?10232055-iXucU Madicken

2 months, 2 weeks

Weighing in on interactive volume rendering PR?

by Matthew Turk

Hi folks, In https://github.com/yt-project/yt/pull/2896 I issued a rewrite for the interactive volume rendering. There's been some discussion, but basically the path is diverging: * Merge in, and review via yt methods * Split into separate package (likely following a null merge, so that the history is preserved *somewhere*) and removing the existing interactive volume rendering If folks have time, chiming in on it over there would be very helpful. Thanks! -Matt

2 months, 2 weeks

Jump to page:

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

yt-dev