Dear all,
TLDR; We recently migrated yt's configuration system from the yt-3
format to a new and more flexible format in yt-4 (the toml format, see
https://github.com/yt-project/yt/pull/2981). *This requires a manual
intervention to perform the migration (see details below).*
------------------------------------------------------------------------
*What does it change for users?*
The new configuration system is documented online here:
https://yt-project.org/docs/dev/reference/configuration.html.
*Performing the migration*
The change requires a one-time manual intervention on your side to
perform the migration automatically. To perform the migration, simply
type in your console
yt config migrate
This should migrate your old configuration to the new format (stored in
~/.config/yt/yt.toml) and store a backup of your old configuration file
in ~/.config/yt/ytrc.bak should any error happen. Please report any
issue you may encounter in the migration procedure so that we can fix
them before releasing officially yt-4.
If you get an error about a missing toml package, you may also have to
install the toml package manually|
|
| conda install -c conda-forge toml # if you are using conda
pip install toml # otherwise
|
Per-folder configuration
The new configuration system allows you to configure yt per-folder. In
particular, if a file named yt.toml is present in your current working
folder, this file will override the global configuration. You can easily
configure yt from the command line using
yt config set yt log_level 40 --global # this will write in
~/.config/yt/yt.toml
# or
yt config set yt log_level 40 --local # this will write in ./yt.toml
Naming convention
We have also rationalized the naming convention of configurable keys in
the configuration file: all keys now use snake case (i.e. words
separated by an underscore). For example loglevel has become log_level.
This is handled automatically by the configuration script, but bear that
in mind if you modify the configuration file. For reference, all
configurable keys can currently be found online
https://github.com/yt-project/yt/blob/main/yt/config.py#L12 and the most
important ones are documented online
(https://yt-project.org/docs/dev/reference/configuration.html).
What does it change for developers?
The new format should allow more flexible configuration options. In
particular, it supports nested configuration blocks and allows to find
the most specific key in the configuration file
# yt.toml
[fields]
cmap = "arbre"
[fields.gas]
cmap = "viridis"
[fields.gas.temperature]
cmap = "plasma"
In Python, you can then query the "most specific" key matching a given path:
from yt.config import ytcfg
ytcfg.get_most_specific("fields", "gas", "temperature", "cmap") == "plasma"
ytcfg.get_most_specific("fields", "gas", "density", "cmap") == "viridis"
ytcfg.get_most_specific("fields", "io", "velocity_x", "cmap") == "arbre"
This isn't used anywhere /yet/. The other breaking change compared to
the old configuration system is that ytcfg.get returns typed quantities
(as read by the toml parser) instead of strings.
Best,
Corentin
--
Dr. Corentin Cadiou
Post Doctoral Research Assistant
Cosmoparticle Initiative Hub, desk 27
University College London (UCL)
Gower St, Bloomsbury, London WC1E 6BT
mobile: +33.6.43.18.66.83
Hi folks,
In January, Github is making it easier to change the default branch name
with a toggle on the website:
https://github.com/github/renaming
I know lots of other projects have done this (including maestro) and we
could do it ourselves, but I'd like to propose that we do this as soon as
GH has the tooling in place to handle it for us.
If anyone objects we can dig into this during a team meeting or something,
but personally this kind of seems uncontroversial and should be easy to
implement.
-Matt
Hi folks,
As a quick update, the primary branch for yt-project/yt is now "main",
renamed from "master". This should be minimally invasive, but if you run
into any troubles, let us know!
-Matt
Dear yt-dev
I recently found out about https://pre-commit.ci, and I'd like to propose we adopt this tool as
part of our CI for linting/auto-fixing.
The idea is that this automatically runs
$ pre-commit run --all-files
on each pull request and commits to the corresponding branch to auto-fix errors with
pre-commit hooks (black, isort and flynt in our case).
I'm reaching out to you because this would have small repercusions on everyone's
workflow. Let me try to give a comprehensive description of gains and costs in the following.
what it gets us
a simplification of our CI tooling, namely:
- `tests/lint_requirements.txt`
- `.github/workflows/style-checks.yaml`
- https://github.com/yt-project/slash-command-processor
all become useless
`.github/PULL_REQUEST_TEMPLATE.yaml` will also be simplified as all linting items in `PR
Checklist` become redundant.
On top of this, new hooks could be seamlessly added in the future without increasing the
entry barrier for occasional contributors (as installing pre-commit hooks locally is and
remains an opt-in).
what it costs us
- "zero" configuration: the only requirement is `.pre-commit.yaml`, which is already part
of the repo.
- minor updates to .pre-commit-config.yaml, namely :
- bumping black's version
(reason: black's `--exclude` flag is overidden by pre-commit's `--all-files`, but more
recent versions of black have a `--force-exclude` flag that overcomes this)
this will require a new PR to adopt black's latest (minor) updates in code styling
- signing up tohttps://pre-commit.ci (free)
what this changes to your workflow
PRs will be automatically fixed by the bot, so you'll either need to occasionally run `git pull` or `git push
--force` to account for it.
Note that this can be avoided completely by installing pre-commit hooks locally which is
and will remain an opt-in, see
https://yt-project.org/docs/dev/developing/developing.html#pre-commit-hooks.
Let me know what you think. Take care, and happy new year !
Cheers, Clément
Hi yt-dev,
I have been working on an update to our blog, which is currently on the
yt-project.org website here:
https://blog.yt-project.org/
as you can see, there haven't been updates to it in quite some time. Matt
and I have ported all of the old posts from this blog to a hugo page that
has tagging and categorization, so one can filter blog posts based on
search interest. Here's what it looks like right now:
munkm.github.io/yt-blog/
and here is the repo currently:
https://github.com/munkm/yt-blog
What do you all think? I think we could get some very cool contributions to
the blog this way.
-Madicken
PS: I presented this blog at the yt user's workshop a few weeks ago! If you
want to see it in action you can watch it here:
https://www.youtube.com/watch?v=wKh_8AAlFhY
Hi folks,
For a while, we've had two projects "incubating" in the org data-exp-lab,
which is tied to the research group I'm in here at UIUC. These are the
"widgyts" repo, which has some jupyter widgets in it, and "yt_idv" which is
where the IDV now lives.
I'd like to propose moving these to yt-project. I don't see this as being
very controversial, but I thought I'd give the opportunity for folks to
chime in yay/nay.
Thanks,
Matt
Dear yt devs,
TLDR: should we change the configuration format from .ini to another one
(e.g. toml)?
------------------------------------------------------------------------
Last meeting we discussed what steps were required for the 4.0 release
and one of the point was whether we should change how we handle
configurations files (e.g. to get
https://github.com/yt-project/yt/pull/1931 eventually in). Given that
changing the format of yt's configuration is a big breaking change, this
would have its place in a yt-3 to 4 transition plan.
Amon the possibilities of a new or improved configuration system, users
may be able to set some preferences per-field (e.g. have a configuration
that implements the user's wish that "velocity_divergence should be
displayed using a linear scale, with the seismic colormap"), or to pick
their favourite unit system (e.g. cgs for the astro community, mks for
the others).
I know this has already been discussed earlier (both in PR comments and
/via/ email), but let's take a moment to think about it with a fresh
perspective. I can see three possible options:
1. Keep the .ini file format and build on top of that.
_Pros:_
* no need to write a script to convert the old configuration files
to a new format
* easy to write for the user
_Cons:_
* may become increasingly cumbersome to maintain if the available
configuration keys are extended
* no support for subgroups within one configuration group, so the
configuration file may become cluttered or we'd need to handle
this manually in the codebase
2. Switch to toml
_Pros:_
* the language support semantically boolean/str/numbers (and parse
them accordingly)
* language is standardized & similar to .INI
* easy to write (but see below)
* possible to have nested configs, e.g. to set defaults for a
range of different fields:
[yt.fields.gas]
# Use linear scale by default for gas fields …
use_log = false
[yt.fields.gas.pressure]
# … except for pressure fields
use_log = true
That would be parsed natively as
{'yt': {
'fields': {
'gas': {
'use_log': False,
'pressure':{
'use_log': True
}
}
}
}
* the syntax make it explicit what type each value is, so we could
check the validity of the config when yt is loaded (for ex.
[yt]\nloglevel=nan is invalid because loglevel should be a number).
_Cons:_
* users will have to migrate their old config (if any), though we
could provide a migration script
* yet-another dependency (but we could vendor the parser, there
are small implementations)
* may be harder to write (how does a user know the expected type
of each key? Should [yt]\n loglevel by a string or a number?)
* for developers: we'd need to change the way the config is
supported in the codebase (currently we're relying on ConfigParser)
* there are some gotchas, for example boolean true is written
"true" not "True"
3. Switch to yaml or json?
What is your opinion? As a starting point, you could think about how
you'd implement the example given above to deactivate the log scale for
plots made with any field with type "gas", except pressure (is this
something we want? If so, what's the best solution to achieve it in a
meaningful way for the user?).
Regards,
Corentin
--
Dr. Corentin Cadiou
Post Doctoral Research Assistant
Cosmoparticle Initiative Hub, desk 27
University College London (UCL)
Gower St, Bloomsbury, London WC1E 6BT
mobile: +33.6.43.18.66.83
Hello yt friends!
yt 3.6.1 is now out!
It includes one backport: matplotlib 3.3.x support. See PR 2754
<https://github.com/yt-project/yt/pull/2754> and the initial bug report at
2752 <https://github.com/yt-project/yt/issues/2752> .
For release notes, see
https://yt-project.org/doc/reference/changelog.html#version-3-6-1.
Binaries for yt 3.6.1 are available via pip and conda. If you installed via
the install script or use conda to manage your python installation, you can
update yt via:
$ conda update -c conda-forge yt
And via pip if you manage your python installation with pip:
$ pip install -U yt
As always, if you have any questions, concerns, or run into any trouble
updating please don’t hesitate to send a message to the mailing list or
stop by our Slack workspace.
Thanks for being in the community!
- the yt development team
Hi yt-dev!
This week during the yt team meeting we decided to hold a meeting to plan
out our next steps with yt-4.0, specifically:
- if all PRs labeled with the 4.0 label should be blockers to a release
- what else we need to do for the 4.0 release
- what our plan is for required 4.0 features (who is doing them, do they
need help, etc.?)
- moving non-urgent PRs/issues to a later release, if relevant
Please mark your availability on the poll if you'd like to join for this
meeting!
https://www.when2meet.com/?10232055-iXucU
Madicken
Hi folks,
In https://github.com/yt-project/yt/pull/2896 I issued a rewrite for the
interactive volume rendering. There's been some discussion, but basically
the path is diverging:
* Merge in, and review via yt methods
* Split into separate package (likely following a null merge, so that the
history is preserved *somewhere*) and removing the existing interactive
volume rendering
If folks have time, chiming in on it over there would be very helpful.
Thanks!
-Matt