[yt-dev] Re: Switch to new configuration system

Hi there, I support the idea of providing a conversion script and printing out a command to run to do the conversion. This needs to be a disruption that takes people less than 60 second to return to their workflow. Britton On Mon, Nov 23, 2020 at 8:53 PM Clément Robert via yt-dev wrote:

Hi, I support changing the format to toml, which feels infinitely less controversial than yaml or json since it’s already so close to ini. Correct me if I’m wrong, but that would allow for the config to be accessed internally as a dictionary instead of through a ConfigParser instance, right ?

I also support Matt’s suggestion for migration strategy, provided we don’t expect silent parsing errors to occur (e.g. True being parsed as a string instead of a boolean. This example is not relevant but you get the idea).

Cheers, Clément

On 23 Nov 2020, at 16:05, Matthew Turk wrote:

Hi Corentin,

This is a great idea.

I was originally opposed to toml. But, I think it's now a transitive dependency for yt anyway, so we should be safe using it, and I think that it's reasonably common enough that it will be usable for folks. I would wager than migration is not going to be too terribly difficult, as we can probably parse a config, see if it matches the defaults, and if not, just bail and say that it should be converted.

On Mon, Nov 23, 2020 at 8:40 AM Corentin CADIOU wrote:

...

Dear yt devs,

TLDR: should we change the configuration format from .ini to another one (e.g. toml)? ------------------------------

Last meeting we discussed what steps were required for the 4.0 release and one of the point was whether we should change how we handle configurations files (e.g. to get https://github.com/yt-project/yt/pull/1931 eventually in). Given that changing the format of yt's configuration is a big breaking change, this would have its place in a yt-3 to 4 transition plan.

Amon the possibilities of a new or improved configuration system, users may be able to set some preferences per-field (e.g. have a configuration that implements the user's wish that "velocity_divergence should be displayed using a linear scale, with the seismic colormap"), or to pick their favourite unit system (e.g. cgs for the astro community, mks for the others).

I know this has already been discussed earlier (both in PR comments and *via* email), but let's take a moment to think about it with a fresh perspective. I can see three possible options:

1. Keep the .ini file format and build on top of that. *Pros:* - no need to write a script to convert the old configuration files to a new format - easy to write for the user *Cons:* - may become increasingly cumbersome to maintain if the available configuration keys are extended - no support for subgroups within one configuration group, so the configuration file may become cluttered or we'd need to handle this manually in the codebase 2. Switch to toml *Pros:* - the language support semantically boolean/str/numbers (and parse them accordingly) - language is standardized & similar to .INI - easy to write (but see below) - possible to have nested configs, e.g. to set defaults for a range of different fields:

[yt.fields.gas] # Use linear scale by default for gas fields … use_log = false

[yt.fields.gas.pressure] # … except for pressure fields use_log = true

That would be parsed natively as {'yt': { 'fields': { 'gas': { 'use_log': False, 'pressure': { 'use_log': True } } } } - the syntax make it explicit what type each value is, so we could check the validity of the config when yt is loaded (for ex. [yt]\nloglevel=nan is invalid because loglevel should be a number). *Cons:* - users will have to migrate their old config (if any), though we could provide a migration script - yet-another dependency (but we could vendor the parser, there are small implementations) - may be harder to write (how does a user know the expected type of each key? Should [yt]\n loglevel by a string or a number?) - for developers: we'd need to change the way the config is supported in the codebase (currently we're relying on ConfigParser) - there are some gotchas, for example boolean true is written "true" not "True" 3. Switch to yaml or json?

What is your opinion? As a starting point, you could think about how you'd implement the example given above to deactivate the log scale for plots made with any field with type "gas", except pressure (is this something we want? If so, what's the best solution to achieve it in a meaningful way for the user?).

Regards, Corentin

-- Dr. Corentin Cadiou Post Doctoral Research Assistant Cosmoparticle Initiative Hub, desk 27 University College London (UCL) Gower St, Bloomsbury, London WC1E 6BT

mobile: +33.6.43.18.66.83

_______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org https://mail.python.org/mailman3/lists/yt-dev.python.org/ Member address: matthewturk@gmail.com

_______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org https://mail.python.org/mailman3/lists/yt-dev.python.org/ Member address: brittonsmith@gmail.com