Hello, As of today, there is no way to configure yt plots per-field. One is stuck with configuration that are yt-wide (e.g. default_colormap). For example, I often find myself typing p.set_cmap('density', 'dusk'). I suggest to add a way to configure the behaviour of yt plotting interface per-field using yt configuration, so that I could set once and for all some defaults for plots. I pushed a first PR on github (https://github.com/yt-project/yt/pull/1931) and opened an issue (https://github.com/yt-project/yt/issues/1888) to discuss the feature. Nathan and I discussed a bit about how this should be implemented in the configuration file. Since ini files are quite restricted (e.g. you cannot have subsections), Nathan suggested at one point to use the toml format (https://github.com/toml-lang/toml). However, it is incompatible with the older .ini format. For example, boolean should be in lowercase (true, false) and not Python-like (True, False), strings have to be surrounded by apostrophes, ... Another suggestion is to add each fields' configuration as a new section in the ini file. For example [config.field_type.field_name] cmap = viridis log = False This is what is implemented in the PR, but maybe some of you have more clever ideas about this? One fairly strong constraint is that the configuration files should be as backward-compatible as possible. If not, we'd have to write a migration script from the old to the new format. We should also discuss what can be configured here. There is already the colormap and log scale implemented. One very neat feature would be to be able to configure the default unit, but it may be hard to implement (e.g. how do we configure the default unit for both projection and slices?) and probably confusing for the user. So here are the questions. Do you think per-field configuration should be part of yt? If so what format should it use? And what are your thoughts on what should and shouldn't be configurable? Corentin -- Corentin Cadiou PhD student Institut d'Astrophysique de Paris (IAP), desk 142b 98 bis boulevard Arago 75014 Paris, France phone: +33.6.43.18.66.83
Hi Corentin, Great discussion. On Tue, Aug 21, 2018 at 4:34 AM Corentin CADIOU <corentin.cadiou@iap.fr> wrote:
Hello,
As of today, there is no way to configure yt plots per-field. One is stuck with configuration that are yt-wide (e.g. default_colormap). For example, I often find myself typing p.set_cmap('density', 'dusk'). I suggest to add a way to configure the behaviour of yt plotting interface per-field using yt configuration, so that I could set once and for all some defaults for plots. I pushed a first PR on github (https://github.com/yt-project/yt/pull/1931) and opened an issue (https://github.com/yt-project/yt/issues/1888) to discuss the feature.
Nathan and I discussed a bit about how this should be implemented in the configuration file. Since ini files are quite restricted (e.g. you cannot have subsections), Nathan suggested at one point to use the toml format (https://github.com/toml-lang/toml). However, it is incompatible with the older .ini format. For example, boolean should be in lowercase (true, false) and not Python-like (True, False), strings have to be surrounded by apostrophes, ...
Taking a step back, I suspect that the following are both true: 1) Folks who have been using yt for a lot longer are more likely to have configuration files scattered about 2) The existing config files don't actually do *that* much We've also migrated in the past, and could probably implement that again. (Kacper did it the first time.) So I don't think that this should be that big of a barrier; we could also consider the idea of having a field configuration toml file that sits next to the ini file. Do we need new dependencies if we have toml?
Another suggestion is to add each fields' configuration as a new section in the ini file. For example
[config.field_type.field_name] cmap = viridis log = False
This is what is implemented in the PR, but maybe some of you have more clever ideas about this? One fairly strong constraint is that the configuration files should be as backward-compatible as possible. If not, we'd have to write a migration script from the old to the new format.
Or have them be in a separate file, which might even make sense -- and would be consistent with the idea that we may eventually want to override these by frontend.
We should also discuss what can be configured here. There is already the colormap and log scale implemented. One very neat feature would be to be able to configure the default unit, but it may be hard to implement (e.g. how do we configure the default unit for both projection and slices?) and probably confusing for the user.
Definitely default unit for non-integrated quantities! I think for integrated quantities like projections what we could do is configure the default integration unit, and that would combine with the default unit to produce the final result.
So here are the questions. Do you think per-field configuration should be part of yt? If so what format should it use? And what are your thoughts on what should and shouldn't be configurable?
1) Yes 2) I don't particularly mind toml, but I'd rather we break hard if we're going to break from INI-style files. A second file would make sense to me.
Corentin
-- Corentin Cadiou PhD student Institut d'Astrophysique de Paris (IAP), desk 142b 98 bis boulevard Arago 75014 Paris, France
phone: +33.6.43.18.66.83
_______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
On Tue, Aug 21, 2018 at 8:12 AM Matthew Turk <matthewturk@gmail.com> wrote:
Hi Corentin,
Great discussion.
On Tue, Aug 21, 2018 at 4:34 AM Corentin CADIOU <corentin.cadiou@iap.fr> wrote:
Hello,
As of today, there is no way to configure yt plots per-field. One is
stuck with configuration that are yt-wide (e.g. default_colormap). For example, I often find myself typing p.set_cmap('density', 'dusk'). I suggest to add a way to configure the behaviour of yt plotting interface per-field using yt configuration, so that I could set once and for all some defaults for plots. I pushed a first PR on github ( https://github.com/yt-project/yt/pull/1931) and opened an issue ( https://github.com/yt-project/yt/issues/1888) to discuss the feature.
Nathan and I discussed a bit about how this should be implemented in the
configuration file. Since ini files are quite restricted (e.g. you cannot have subsections), Nathan suggested at one point to use the toml format ( https://github.com/toml-lang/toml). However, it is incompatible with the older .ini format. For example, boolean should be in lowercase (true, false) and not Python-like (True, False), strings have to be surrounded by apostrophes, ...
Taking a step back, I suspect that the following are both true:
1) Folks who have been using yt for a lot longer are more likely to have configuration files scattered about 2) The existing config files don't actually do *that* much
We've also migrated in the past, and could probably implement that again. (Kacper did it the first time.)
This is referring to "yt config migrate" which you should take a look at. We could definitely overload the existing command for this purpose. I think using the standard library configparser module you could first load the ini-style file, inspect the state of the resulting ConfigParser object, convert the data to a dict, and pass that dict to a toml writer.
So I don't think that this should be that big of a barrier; we could also consider the idea of having a field configuration toml file that sits next to the ini file. Do we need new dependencies if we have toml?
We'd probably want to depend on either this: https://github.com/uiri/toml which seems to be the most popular library or this: https://github.com/avakar/pytoml Which seems to be what pip has vendored and I trust dstufft's opinion on these things... Another option would be to vender one of those libraries but I don't think that's necessary anymore given the state of the packaging ecosystem and that both of these are pure python.
Another suggestion is to add each fields' configuration as a new section
in the ini file. For example
[config.field_type.field_name] cmap = viridis log = False
This is what is implemented in the PR, but maybe some of you have more
clever ideas about this? One fairly strong constraint is that the configuration files should be as backward-compatible as possible. If not, we'd have to write a migration script from the old to the new format.
Or have them be in a separate file, which might even make sense -- and would be consistent with the idea that we may eventually want to override these by frontend.
This is starting to get close to my ideas for field_day. I was also hoping to use toml there... But yes, it would be good to use whatever syntax we come up with here for frontend-specific overrides.
We should also discuss what can be configured here. There is already the
colormap and log scale implemented. One very neat feature would be to be able to configure the default unit, but it may be hard to implement (e.g. how do we configure the default unit for both projection and slices?) and probably confusing for the user. >
Definitely default unit for non-integrated quantities! I think for integrated quantities like projections what we could do is configure the default integration unit, and that would combine with the default unit to produce the final result.
Yup, and we need to be clear to document this use case since it'll come up a lot.
So here are the questions. Do you think per-field configuration should be part of yt? If so what format should it use? And what are your thoughts on what should and shouldn't be configurable?
1) Yes 2) I don't particularly mind toml, but I'd rather we break hard if we're going to break from INI-style files. A second file would make sense to me.
I agree that this is a worthwhile improvement. I would also like to move to a richer configuration format than ini-style files. ConfigParser is also a pain to deal with. That said, unless you also want to systematically update everywhere we use the configuration system in yt (might be possible with some sed magic, so don't discount just updating everything) you'd also need to write an emulation layer or figure out how to generate a ConfigParser object from the toml file.
Corentin
-- Corentin Cadiou PhD student Institut d'Astrophysique de Paris (IAP), desk 142b 98 bis boulevard Arago 75014 Paris, France
phone: +33.6.43.18.66.83
_______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
_______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
As a quick update, I looked at the usages of ytcfg, and it looks like in the 43 files that it shows up, the most common things it checks: * default colormap * communicator information for parallelism * if we're in testing or not The latter two are basically using ytcfg to pass around globals that are set up internally in yt. The others seem related to xray data, test data, etc. My *guess* is that even if it's not sed-able, this wouldn't be an overwhelming thing to change. But, it would require a bit of work. On Tue, Aug 21, 2018 at 9:30 AM Nathan Goldbaum <nathan12343@gmail.com> wrote:
On Tue, Aug 21, 2018 at 8:12 AM Matthew Turk <matthewturk@gmail.com> wrote:
Hi Corentin,
Great discussion.
On Tue, Aug 21, 2018 at 4:34 AM Corentin CADIOU <corentin.cadiou@iap.fr> wrote:
Hello,
As of today, there is no way to configure yt plots per-field. One is stuck with configuration that are yt-wide (e.g. default_colormap). For example, I often find myself typing p.set_cmap('density', 'dusk'). I suggest to add a way to configure the behaviour of yt plotting interface per-field using yt configuration, so that I could set once and for all some defaults for plots. I pushed a first PR on github (https://github.com/yt-project/yt/pull/1931) and opened an issue (https://github.com/yt-project/yt/issues/1888) to discuss the feature.
Nathan and I discussed a bit about how this should be implemented in the configuration file. Since ini files are quite restricted (e.g. you cannot have subsections), Nathan suggested at one point to use the toml format (https://github.com/toml-lang/toml). However, it is incompatible with the older .ini format. For example, boolean should be in lowercase (true, false) and not Python-like (True, False), strings have to be surrounded by apostrophes, ...
Taking a step back, I suspect that the following are both true:
1) Folks who have been using yt for a lot longer are more likely to have configuration files scattered about 2) The existing config files don't actually do *that* much
We've also migrated in the past, and could probably implement that again. (Kacper did it the first time.)
This is referring to "yt config migrate" which you should take a look at. We could definitely overload the existing command for this purpose.
I think using the standard library configparser module you could first load the ini-style file, inspect the state of the resulting ConfigParser object, convert the data to a dict, and pass that dict to a toml writer.
So I don't think that this should be that big of a barrier; we could also consider the idea of having a field configuration toml file that sits next to the ini file. Do we need new dependencies if we have toml?
We'd probably want to depend on either this:
which seems to be the most popular library or this:
https://github.com/avakar/pytoml
Which seems to be what pip has vendored and I trust dstufft's opinion on these things...
Another option would be to vender one of those libraries but I don't think that's necessary anymore given the state of the packaging ecosystem and that both of these are pure python.
Another suggestion is to add each fields' configuration as a new section in the ini file. For example
[config.field_type.field_name] cmap = viridis log = False
This is what is implemented in the PR, but maybe some of you have more clever ideas about this? One fairly strong constraint is that the configuration files should be as backward-compatible as possible. If not, we'd have to write a migration script from the old to the new format.
Or have them be in a separate file, which might even make sense -- and would be consistent with the idea that we may eventually want to override these by frontend.
This is starting to get close to my ideas for field_day. I was also hoping to use toml there...
But yes, it would be good to use whatever syntax we come up with here for frontend-specific overrides.
We should also discuss what can be configured here. There is already the colormap and log scale implemented. One very neat feature would be to be able to configure the default unit, but it may be hard to implement (e.g. how do we configure the default unit for both projection and slices?) and probably confusing for the user.
Definitely default unit for non-integrated quantities! I think for integrated quantities like projections what we could do is configure the default integration unit, and that would combine with the default unit to produce the final result.
Yup, and we need to be clear to document this use case since it'll come up a lot.
So here are the questions. Do you think per-field configuration should be part of yt? If so what format should it use? And what are your thoughts on what should and shouldn't be configurable?
1) Yes 2) I don't particularly mind toml, but I'd rather we break hard if we're going to break from INI-style files. A second file would make sense to me.
I agree that this is a worthwhile improvement. I would also like to move to a richer configuration format than ini-style files. ConfigParser is also a pain to deal with.
That said, unless you also want to systematically update everywhere we use the configuration system in yt (might be possible with some sed magic, so don't discount just updating everything) you'd also need to write an emulation layer or figure out how to generate a ConfigParser object from the toml file.
Corentin
-- Corentin Cadiou PhD student Institut d'Astrophysique de Paris (IAP), desk 142b 98 bis boulevard Arago 75014 Paris, France
phone: +33.6.43.18.66.83
_______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
_______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
_______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
Interesting idea--I'm all for allowing more configurability in the config file on disk. I guess I don't understand the problem with leaving it as a YAML file and adding in additional configuration within that framework? Is it just that it doesn't allow for subsections? Looking forward to seeing how this all turns out! Cameron On Tue, Aug 21, 2018 at 7:37 AM, Matthew Turk <matthewturk@gmail.com> wrote:
As a quick update, I looked at the usages of ytcfg, and it looks like in the 43 files that it shows up, the most common things it checks:
* default colormap * communicator information for parallelism * if we're in testing or not
The latter two are basically using ytcfg to pass around globals that are set up internally in yt. The others seem related to xray data, test data, etc.
My *guess* is that even if it's not sed-able, this wouldn't be an overwhelming thing to change. But, it would require a bit of work. On Tue, Aug 21, 2018 at 9:30 AM Nathan Goldbaum <nathan12343@gmail.com> wrote:
On Tue, Aug 21, 2018 at 8:12 AM Matthew Turk <matthewturk@gmail.com>
Hi Corentin,
Great discussion.
On Tue, Aug 21, 2018 at 4:34 AM Corentin CADIOU <corentin.cadiou@iap.fr>
wrote:
Hello,
As of today, there is no way to configure yt plots per-field. One is
stuck with configuration that are yt-wide (e.g. default_colormap). For example, I often find myself typing p.set_cmap('density', 'dusk'). I suggest to add a way to configure the behaviour of yt plotting interface
Nathan and I discussed a bit about how this should be implemented in
Taking a step back, I suspect that the following are both true:
1) Folks who have been using yt for a lot longer are more likely to have configuration files scattered about 2) The existing config files don't actually do *that* much
We've also migrated in the past, and could probably implement that again. (Kacper did it the first time.)
This is referring to "yt config migrate" which you should take a look at. We could definitely overload the existing command for this purpose.
I think using the standard library configparser module you could first load the ini-style file, inspect the state of the resulting ConfigParser object, convert the data to a dict, and pass that dict to a toml writer.
So I don't think that this should be that big of a barrier; we could also consider the idea of having a field configuration toml file that sits next to the ini file. Do we need new dependencies if we have toml?
We'd probably want to depend on either this:
which seems to be the most popular library or this:
https://github.com/avakar/pytoml
Which seems to be what pip has vendored and I trust dstufft's opinion on
wrote: per-field using yt configuration, so that I could set once and for all some defaults for plots. I pushed a first PR on github (https://github.com/yt- project/yt/pull/1931) and opened an issue (https://github.com/yt- project/yt/issues/1888) to discuss the feature. the configuration file. Since ini files are quite restricted (e.g. you cannot have subsections), Nathan suggested at one point to use the toml format (https://github.com/toml-lang/toml). However, it is incompatible with the older .ini format. For example, boolean should be in lowercase (true, false) and not Python-like (True, False), strings have to be surrounded by apostrophes, ... these things...
Another option would be to vender one of those libraries but I don't
think that's necessary anymore given the state of the packaging ecosystem and that both of these are pure python.
Another suggestion is to add each fields' configuration as a new
[config.field_type.field_name] cmap = viridis log = False
This is what is implemented in the PR, but maybe some of you have
more clever ideas about this? One fairly strong constraint is that the configuration files should be as backward-compatible as possible. If not, we'd have to write a migration script from the old to the new format.
Or have them be in a separate file, which might even make sense -- and would be consistent with the idea that we may eventually want to override these by frontend.
This is starting to get close to my ideas for field_day. I was also hoping to use toml there...
But yes, it would be good to use whatever syntax we come up with here for frontend-specific overrides.
We should also discuss what can be configured here. There is already
Definitely default unit for non-integrated quantities! I think for integrated quantities like projections what we could do is configure the default integration unit, and that would combine with the default unit to produce the final result.
Yup, and we need to be clear to document this use case since it'll come up a lot.
So here are the questions. Do you think per-field configuration
should be part of yt? If so what format should it use? And what are your
1) Yes 2) I don't particularly mind toml, but I'd rather we break hard if we're going to break from INI-style files. A second file would make sense to me.
I agree that this is a worthwhile improvement. I would also like to move to a richer configuration format than ini-style files. ConfigParser is also a pain to deal with.
That said, unless you also want to systematically update everywhere we use the configuration system in yt (might be possible with some sed magic, so don't discount just updating everything) you'd also need to write an emulation layer or figure out how to generate a ConfigParser object from
section in the ini file. For example the colormap and log scale implemented. One very neat feature would be to be able to configure the default unit, but it may be hard to implement (e.g. how do we configure the default unit for both projection and slices?) and probably confusing for the user. thoughts on what should and shouldn't be configurable? the toml file.
Corentin
-- Corentin Cadiou PhD student Institut d'Astrophysique de Paris (IAP), desk 142b 98 bis boulevard Arago 75014 Paris, France
phone: +33.6.43.18.66.83
_______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
_______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
_______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
_______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
-- Cameron Hummels NSF Postdoctoral Fellow Department of Astronomy California Institute of Technology http://chummels.org
On Tue, Aug 21, 2018 at 11:00 AM Cameron Hummels <chummels@gmail.com> wrote:
Interesting idea--I'm all for allowing more configurability in the config file on disk. I guess I don't understand the problem with leaving it as a YAML file and adding in additional configuration within that framework? Is it just that it doesn't allow for subsections?
Not sure what you mean by this. We're not using YAML currently. We *could* switch to YAML instead of TOML (or some other format) but I think that would be a mistake as summarized in this PEP: https://www.python.org/dev/peps/pep-0518/#other-file-formats YAML has lots of issues and corner cases, best to avoid that if we have the freedom to do so.
Looking forward to seeing how this all turns out!
Cameron
On Tue, Aug 21, 2018 at 7:37 AM, Matthew Turk <matthewturk@gmail.com> wrote:
As a quick update, I looked at the usages of ytcfg, and it looks like in the 43 files that it shows up, the most common things it checks:
* default colormap * communicator information for parallelism * if we're in testing or not
The latter two are basically using ytcfg to pass around globals that are set up internally in yt. The others seem related to xray data, test data, etc.
My *guess* is that even if it's not sed-able, this wouldn't be an overwhelming thing to change. But, it would require a bit of work. On Tue, Aug 21, 2018 at 9:30 AM Nathan Goldbaum <nathan12343@gmail.com> wrote:
On Tue, Aug 21, 2018 at 8:12 AM Matthew Turk <matthewturk@gmail.com>
Hi Corentin,
Great discussion.
On Tue, Aug 21, 2018 at 4:34 AM Corentin CADIOU <
corentin.cadiou@iap.fr> wrote:
Hello,
As of today, there is no way to configure yt plots per-field. One is
stuck with configuration that are yt-wide (e.g. default_colormap). For example, I often find myself typing p.set_cmap('density', 'dusk'). I suggest to add a way to configure the behaviour of yt plotting interface
Nathan and I discussed a bit about how this should be implemented in
Taking a step back, I suspect that the following are both true:
1) Folks who have been using yt for a lot longer are more likely to have configuration files scattered about 2) The existing config files don't actually do *that* much
We've also migrated in the past, and could probably implement that again. (Kacper did it the first time.)
This is referring to "yt config migrate" which you should take a look at. We could definitely overload the existing command for this purpose.
I think using the standard library configparser module you could first load the ini-style file, inspect the state of the resulting ConfigParser object, convert the data to a dict, and pass that dict to a toml writer.
So I don't think that this should be that big of a barrier; we could also consider the idea of having a field configuration toml file that sits next to the ini file. Do we need new dependencies if we have toml?
We'd probably want to depend on either this:
which seems to be the most popular library or this:
https://github.com/avakar/pytoml
Which seems to be what pip has vendored and I trust dstufft's opinion on these things...
Another option would be to vender one of those libraries but I don't
wrote: per-field using yt configuration, so that I could set once and for all some defaults for plots. I pushed a first PR on github ( https://github.com/yt-project/yt/pull/1931) and opened an issue ( https://github.com/yt-project/yt/issues/1888) to discuss the feature. the configuration file. Since ini files are quite restricted (e.g. you cannot have subsections), Nathan suggested at one point to use the toml format (https://github.com/toml-lang/toml). However, it is incompatible with the older .ini format. For example, boolean should be in lowercase (true, false) and not Python-like (True, False), strings have to be surrounded by apostrophes, ... think that's necessary anymore given the state of the packaging ecosystem and that both of these are pure python.
Another suggestion is to add each fields' configuration as a new
[config.field_type.field_name] cmap = viridis log = False
This is what is implemented in the PR, but maybe some of you have
more clever ideas about this? One fairly strong constraint is that the configuration files should be as backward-compatible as possible. If not, we'd have to write a migration script from the old to the new format.
Or have them be in a separate file, which might even make sense -- and would be consistent with the idea that we may eventually want to override these by frontend.
This is starting to get close to my ideas for field_day. I was also hoping to use toml there...
But yes, it would be good to use whatever syntax we come up with here for frontend-specific overrides.
We should also discuss what can be configured here. There is already
Definitely default unit for non-integrated quantities! I think for integrated quantities like projections what we could do is configure the default integration unit, and that would combine with the default unit to produce the final result.
Yup, and we need to be clear to document this use case since it'll come up a lot.
So here are the questions. Do you think per-field configuration
should be part of yt? If so what format should it use? And what are your
1) Yes 2) I don't particularly mind toml, but I'd rather we break hard if we're going to break from INI-style files. A second file would make sense to me.
I agree that this is a worthwhile improvement. I would also like to move to a richer configuration format than ini-style files. ConfigParser is also a pain to deal with.
That said, unless you also want to systematically update everywhere we use the configuration system in yt (might be possible with some sed magic, so don't discount just updating everything) you'd also need to write an emulation layer or figure out how to generate a ConfigParser object from
section in the ini file. For example the colormap and log scale implemented. One very neat feature would be to be able to configure the default unit, but it may be hard to implement (e.g. how do we configure the default unit for both projection and slices?) and probably confusing for the user. thoughts on what should and shouldn't be configurable? the toml file.
Corentin
-- Corentin Cadiou PhD student Institut d'Astrophysique de Paris (IAP), desk 142b 98 bis boulevard Arago 75014 Paris, France
phone: +33.6.43.18.66.83
_______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
_______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
_______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
_______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
-- Cameron Hummels NSF Postdoctoral Fellow Department of Astronomy California Institute of Technology http://chummels.org _______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
one more thing: I think we shouldn’t merge this until 3.5 is out the door and we should target this new config file format for the yt 4.0 release. I think we’re too late in the cycle for 3.5 to get this in for the 3.x series without risking major user pain. Best to test it out for at leasy a few months on the master branch before a release. On Tue, Aug 21, 2018 at 11:19 AM Nathan Goldbaum <nathan12343@gmail.com> wrote:
On Tue, Aug 21, 2018 at 11:00 AM Cameron Hummels <chummels@gmail.com> wrote:
Interesting idea--I'm all for allowing more configurability in the config file on disk. I guess I don't understand the problem with leaving it as a YAML file and adding in additional configuration within that framework? Is it just that it doesn't allow for subsections?
Not sure what you mean by this. We're not using YAML currently.
We *could* switch to YAML instead of TOML (or some other format) but I think that would be a mistake as summarized in this PEP:
https://www.python.org/dev/peps/pep-0518/#other-file-formats
YAML has lots of issues and corner cases, best to avoid that if we have the freedom to do so.
Looking forward to seeing how this all turns out!
Cameron
On Tue, Aug 21, 2018 at 7:37 AM, Matthew Turk <matthewturk@gmail.com> wrote:
As a quick update, I looked at the usages of ytcfg, and it looks like in the 43 files that it shows up, the most common things it checks:
* default colormap * communicator information for parallelism * if we're in testing or not
The latter two are basically using ytcfg to pass around globals that are set up internally in yt. The others seem related to xray data, test data, etc.
My *guess* is that even if it's not sed-able, this wouldn't be an overwhelming thing to change. But, it would require a bit of work. On Tue, Aug 21, 2018 at 9:30 AM Nathan Goldbaum <nathan12343@gmail.com> wrote:
On Tue, Aug 21, 2018 at 8:12 AM Matthew Turk <matthewturk@gmail.com>
Hi Corentin,
Great discussion.
On Tue, Aug 21, 2018 at 4:34 AM Corentin CADIOU <
corentin.cadiou@iap.fr> wrote:
Hello,
As of today, there is no way to configure yt plots per-field. One
is stuck with configuration that are yt-wide (e.g. default_colormap). For example, I often find myself typing p.set_cmap('density', 'dusk'). I suggest to add a way to configure the behaviour of yt plotting interface
Nathan and I discussed a bit about how this should be implemented
in the configuration file. Since ini files are quite restricted (e.g. you cannot have subsections), Nathan suggested at one point to use the toml
Taking a step back, I suspect that the following are both true:
1) Folks who have been using yt for a lot longer are more likely to have configuration files scattered about 2) The existing config files don't actually do *that* much
We've also migrated in the past, and could probably implement that again. (Kacper did it the first time.)
This is referring to "yt config migrate" which you should take a look at. We could definitely overload the existing command for this purpose.
I think using the standard library configparser module you could first load the ini-style file, inspect the state of the resulting ConfigParser object, convert the data to a dict, and pass that dict to a toml writer.
So I don't think that this should be that big of a barrier; we could also consider the idea of having a field configuration toml file that sits next to the ini file. Do we need new dependencies if we have toml?
We'd probably want to depend on either this:
which seems to be the most popular library or this:
https://github.com/avakar/pytoml
Which seems to be what pip has vendored and I trust dstufft's opinion on these things...
Another option would be to vender one of those libraries but I don't
wrote: per-field using yt configuration, so that I could set once and for all some defaults for plots. I pushed a first PR on github ( https://github.com/yt-project/yt/pull/1931) and opened an issue ( https://github.com/yt-project/yt/issues/1888) to discuss the feature. format (https://github.com/toml-lang/toml). However, it is incompatible with the older .ini format. For example, boolean should be in lowercase (true, false) and not Python-like (True, False), strings have to be surrounded by apostrophes, ... think that's necessary anymore given the state of the packaging ecosystem and that both of these are pure python.
Another suggestion is to add each fields' configuration as a new
[config.field_type.field_name] cmap = viridis log = False
This is what is implemented in the PR, but maybe some of you have
more clever ideas about this? One fairly strong constraint is that the configuration files should be as backward-compatible as possible. If not, we'd have to write a migration script from the old to the new format.
Or have them be in a separate file, which might even make sense -- and would be consistent with the idea that we may eventually want to override these by frontend.
This is starting to get close to my ideas for field_day. I was also hoping to use toml there...
But yes, it would be good to use whatever syntax we come up with here for frontend-specific overrides.
We should also discuss what can be configured here. There is
already the colormap and log scale implemented. One very neat feature would be to be able to configure the default unit, but it may be hard to implement (e.g. how do we configure the default unit for both projection and slices?) and probably confusing for the user.
Definitely default unit for non-integrated quantities! I think for integrated quantities like projections what we could do is configure the default integration unit, and that would combine with the default unit to produce the final result.
Yup, and we need to be clear to document this use case since it'll come up a lot.
So here are the questions. Do you think per-field configuration
should be part of yt? If so what format should it use? And what are your
1) Yes 2) I don't particularly mind toml, but I'd rather we break hard if we're going to break from INI-style files. A second file would make sense to me.
I agree that this is a worthwhile improvement. I would also like to move to a richer configuration format than ini-style files. ConfigParser is also a pain to deal with.
That said, unless you also want to systematically update everywhere we use the configuration system in yt (might be possible with some sed magic, so don't discount just updating everything) you'd also need to write an emulation layer or figure out how to generate a ConfigParser object from
section in the ini file. For example thoughts on what should and shouldn't be configurable? the toml file.
Corentin
-- Corentin Cadiou PhD student Institut d'Astrophysique de Paris (IAP), desk 142b 98 bis boulevard Arago 75014 Paris, France
phone: +33.6.43.18.66.83
_______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
_______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
_______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
_______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
-- Cameron Hummels NSF Postdoctoral Fellow Department of Astronomy California Institute of Technology http://chummels.org _______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
This discussion all sounds reasonable to me. I really like the idea of per-field units, since our fields are sometimes very problem dependent that adding them all into the frontend would be messy. TOML seems like a good option, with the minor downside that it adds another dependency. The link that Nathan sent comparing all of these formats was very informative. On Wed, Aug 22, 2018 at 9:15 AM Nathan Goldbaum <nathan12343@gmail.com> wrote:
one more thing:
I think we shouldn’t merge this until 3.5 is out the door and we should target this new config file format for the yt 4.0 release. I think we’re too late in the cycle for 3.5 to get this in for the 3.x series without risking major user pain. Best to test it out for at leasy a few months on the master branch before a release.
On Tue, Aug 21, 2018 at 11:19 AM Nathan Goldbaum <nathan12343@gmail.com> wrote:
On Tue, Aug 21, 2018 at 11:00 AM Cameron Hummels <chummels@gmail.com> wrote:
Interesting idea--I'm all for allowing more configurability in the config file on disk. I guess I don't understand the problem with leaving it as a YAML file and adding in additional configuration within that framework? Is it just that it doesn't allow for subsections?
Not sure what you mean by this. We're not using YAML currently.
We *could* switch to YAML instead of TOML (or some other format) but I think that would be a mistake as summarized in this PEP:
https://www.python.org/dev/peps/pep-0518/#other-file-formats
YAML has lots of issues and corner cases, best to avoid that if we have the freedom to do so.
Looking forward to seeing how this all turns out!
Cameron
On Tue, Aug 21, 2018 at 7:37 AM, Matthew Turk <matthewturk@gmail.com> wrote:
As a quick update, I looked at the usages of ytcfg, and it looks like in the 43 files that it shows up, the most common things it checks:
* default colormap * communicator information for parallelism * if we're in testing or not
The latter two are basically using ytcfg to pass around globals that are set up internally in yt. The others seem related to xray data, test data, etc.
My *guess* is that even if it's not sed-able, this wouldn't be an overwhelming thing to change. But, it would require a bit of work. On Tue, Aug 21, 2018 at 9:30 AM Nathan Goldbaum <nathan12343@gmail.com> wrote:
On Tue, Aug 21, 2018 at 8:12 AM Matthew Turk <matthewturk@gmail.com>
Hi Corentin,
Great discussion.
On Tue, Aug 21, 2018 at 4:34 AM Corentin CADIOU <
corentin.cadiou@iap.fr> wrote:
> > Hello, > > As of today, there is no way to configure yt plots per-field. One is stuck with configuration that are yt-wide (e.g. default_colormap). For example, I often find myself typing p.set_cmap('density', 'dusk'). I suggest to add a way to configure the behaviour of yt plotting interface
> > Nathan and I discussed a bit about how this should be implemented in the configuration file. Since ini files are quite restricted (e.g. you cannot have subsections), Nathan suggested at one point to use the toml
Taking a step back, I suspect that the following are both true:
1) Folks who have been using yt for a lot longer are more likely to have configuration files scattered about 2) The existing config files don't actually do *that* much
We've also migrated in the past, and could probably implement that again. (Kacper did it the first time.)
This is referring to "yt config migrate" which you should take a look at. We could definitely overload the existing command for this purpose.
I think using the standard library configparser module you could first load the ini-style file, inspect the state of the resulting ConfigParser object, convert the data to a dict, and pass that dict to a toml writer.
So I don't think that this should be that big of a barrier; we could also consider the idea of having a field configuration toml file that sits next to the ini
file.
Do we need new dependencies if we have toml?
We'd probably want to depend on either this:
which seems to be the most popular library or this:
https://github.com/avakar/pytoml
Which seems to be what pip has vendored and I trust dstufft's opinion on these things...
Another option would be to vender one of those libraries but I don't
wrote: per-field using yt configuration, so that I could set once and for all some defaults for plots. I pushed a first PR on github ( https://github.com/yt-project/yt/pull/1931) and opened an issue ( https://github.com/yt-project/yt/issues/1888) to discuss the feature. format (https://github.com/toml-lang/toml). However, it is incompatible with the older .ini format. For example, boolean should be in lowercase (true, false) and not Python-like (True, False), strings have to be surrounded by apostrophes, ... think that's necessary anymore given the state of the packaging ecosystem and that both of these are pure python.
> > Another suggestion is to add each fields' configuration as a new
> > [config.field_type.field_name] > cmap = viridis > log = False > > This is what is implemented in the PR, but maybe some of you have more clever ideas about this? One fairly strong constraint is that the configuration files should be as backward-compatible as possible. If not, we'd have to write a migration script from the old to the new format.
Or have them be in a separate file, which might even make sense -- and would be consistent with the idea that we may eventually want to override these by frontend.
This is starting to get close to my ideas for field_day. I was also hoping to use toml there...
But yes, it would be good to use whatever syntax we come up with here for frontend-specific overrides.
> > We should also discuss what can be configured here. There is already the colormap and log scale implemented. One very neat feature would be to be able to configure the default unit, but it may be hard to implement (e.g. how do we configure the default unit for both projection and slices?) and probably confusing for the user. >
Definitely default unit for non-integrated quantities! I think for integrated quantities like projections what we could do is configure the default integration unit, and that would combine with the default unit to produce the final result.
Yup, and we need to be clear to document this use case since it'll come up a lot.
> So here are the questions. Do you think per-field configuration
should be part of yt? If so what format should it use? And what are your
1) Yes 2) I don't particularly mind toml, but I'd rather we break hard if we're going to break from INI-style files. A second file would make sense to me.
I agree that this is a worthwhile improvement. I would also like to move to a richer configuration format than ini-style files. ConfigParser is also a pain to deal with.
That said, unless you also want to systematically update everywhere we use the configuration system in yt (might be possible with some sed magic, so don't discount just updating everything) you'd also need to write an emulation layer or figure out how to generate a ConfigParser object from
section in the ini file. For example thoughts on what should and shouldn't be configurable? the toml file.
> > Corentin > > -- > Corentin Cadiou > PhD student > Institut d'Astrophysique de Paris (IAP), desk 142b > 98 bis boulevard Arago > 75014 Paris, France > > phone: +33.6.43.18.66.83 > > _______________________________________________ > yt-dev mailing list -- yt-dev@python.org > To unsubscribe send an email to yt-dev-leave@python.org _______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
_______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
_______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
-- Cameron Hummels NSF Postdoctoral Fellow Department of Astronomy California Institute of Technology http://chummels.org _______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
_______________________________________________
yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
-- Michael Zingale Associate Professor Dept. of Physics & Astronomy • Stony Brook University • Stony Brook, NY 11794-3800 *phone*: 631-632-8225 *e-mail*: Michael.Zingale@stonybrook.edu *web*: http://www.astro.sunysb.edu/mzingale github: http://github.com/zingale
I concur with this. I think the per-field configuration is a great idea!
On Aug 22, 2018, at 9:14 AM, Nathan Goldbaum <nathan12343@gmail.com> wrote:
one more thing:
I think we shouldn’t merge this until 3.5 is out the door and we should target this new config file format for the yt 4.0 release. I think we’re too late in the cycle for 3.5 to get this in for the 3.x series without risking major user pain. Best to test it out for at leasy a few months on the master branch before a release.
On Tue, Aug 21, 2018 at 11:19 AM Nathan Goldbaum <nathan12343@gmail.com <mailto:nathan12343@gmail.com>> wrote:
On Tue, Aug 21, 2018 at 11:00 AM Cameron Hummels <chummels@gmail.com <mailto:chummels@gmail.com>> wrote: Interesting idea--I'm all for allowing more configurability in the config file on disk. I guess I don't understand the problem with leaving it as a YAML file and adding in additional configuration within that framework? Is it just that it doesn't allow for subsections?
Not sure what you mean by this. We're not using YAML currently.
We *could* switch to YAML instead of TOML (or some other format) but I think that would be a mistake as summarized in this PEP:
https://www.python.org/dev/peps/pep-0518/#other-file-formats <https://www.python.org/dev/peps/pep-0518/#other-file-formats>
YAML has lots of issues and corner cases, best to avoid that if we have the freedom to do so.
Looking forward to seeing how this all turns out!
Cameron
On Tue, Aug 21, 2018 at 7:37 AM, Matthew Turk <matthewturk@gmail.com <mailto:matthewturk@gmail.com>> wrote: As a quick update, I looked at the usages of ytcfg, and it looks like in the 43 files that it shows up, the most common things it checks:
* default colormap * communicator information for parallelism * if we're in testing or not
The latter two are basically using ytcfg to pass around globals that are set up internally in yt. The others seem related to xray data, test data, etc.
My *guess* is that even if it's not sed-able, this wouldn't be an overwhelming thing to change. But, it would require a bit of work. On Tue, Aug 21, 2018 at 9:30 AM Nathan Goldbaum <nathan12343@gmail.com <mailto:nathan12343@gmail.com>> wrote:
On Tue, Aug 21, 2018 at 8:12 AM Matthew Turk <matthewturk@gmail.com <mailto:matthewturk@gmail.com>> wrote:
Hi Corentin,
Great discussion.
On Tue, Aug 21, 2018 at 4:34 AM Corentin CADIOU <corentin.cadiou@iap.fr <mailto:corentin.cadiou@iap.fr>> wrote:
Hello,
As of today, there is no way to configure yt plots per-field. One is stuck with configuration that are yt-wide (e.g. default_colormap). For example, I often find myself typing p.set_cmap('density', 'dusk'). I suggest to add a way to configure the behaviour of yt plotting interface per-field using yt configuration, so that I could set once and for all some defaults for plots. I pushed a first PR on github (https://github.com/yt-project/yt/pull/1931 <https://github.com/yt-project/yt/pull/1931>) and opened an issue (https://github.com/yt-project/yt/issues/1888 <https://github.com/yt-project/yt/issues/1888>) to discuss the feature.
Nathan and I discussed a bit about how this should be implemented in the configuration file. Since ini files are quite restricted (e.g. you cannot have subsections), Nathan suggested at one point to use the toml format (https://github.com/toml-lang/toml <https://github.com/toml-lang/toml>). However, it is incompatible with the older .ini format. For example, boolean should be in lowercase (true, false) and not Python-like (True, False), strings have to be surrounded by apostrophes, ...
Taking a step back, I suspect that the following are both true:
1) Folks who have been using yt for a lot longer are more likely to have configuration files scattered about 2) The existing config files don't actually do *that* much
We've also migrated in the past, and could probably implement that again. (Kacper did it the first time.)
This is referring to "yt config migrate" which you should take a look at. We could definitely overload the existing command for this purpose.
I think using the standard library configparser module you could first load the ini-style file, inspect the state of the resulting ConfigParser object, convert the data to a dict, and pass that dict to a toml writer.
So I don't think that this should be that big of a barrier; we could also consider the idea of having a field configuration toml file that sits next to the ini file. Do we need new dependencies if we have toml?
We'd probably want to depend on either this:
https://github.com/uiri/toml <https://github.com/uiri/toml>
which seems to be the most popular library or this:
https://github.com/avakar/pytoml <https://github.com/avakar/pytoml>
Which seems to be what pip has vendored and I trust dstufft's opinion on these things...
Another option would be to vender one of those libraries but I don't think that's necessary anymore given the state of the packaging ecosystem and that both of these are pure python.
Another suggestion is to add each fields' configuration as a new section in the ini file. For example
[config.field_type.field_name] cmap = viridis log = False
This is what is implemented in the PR, but maybe some of you have more clever ideas about this? One fairly strong constraint is that the configuration files should be as backward-compatible as possible. If not, we'd have to write a migration script from the old to the new format.
Or have them be in a separate file, which might even make sense -- and would be consistent with the idea that we may eventually want to override these by frontend.
This is starting to get close to my ideas for field_day. I was also hoping to use toml there...
But yes, it would be good to use whatever syntax we come up with here for frontend-specific overrides.
We should also discuss what can be configured here. There is already the colormap and log scale implemented. One very neat feature would be to be able to configure the default unit, but it may be hard to implement (e.g. how do we configure the default unit for both projection and slices?) and probably confusing for the user.
Definitely default unit for non-integrated quantities! I think for integrated quantities like projections what we could do is configure the default integration unit, and that would combine with the default unit to produce the final result.
Yup, and we need to be clear to document this use case since it'll come up a lot.
So here are the questions. Do you think per-field configuration should be part of yt? If so what format should it use? And what are your thoughts on what should and shouldn't be configurable?
1) Yes 2) I don't particularly mind toml, but I'd rather we break hard if we're going to break from INI-style files. A second file would make sense to me.
I agree that this is a worthwhile improvement. I would also like to move to a richer configuration format than ini-style files. ConfigParser is also a pain to deal with.
That said, unless you also want to systematically update everywhere we use the configuration system in yt (might be possible with some sed magic, so don't discount just updating everything) you'd also need to write an emulation layer or figure out how to generate a ConfigParser object from the toml file.
Corentin
-- Corentin Cadiou PhD student Institut d'Astrophysique de Paris (IAP), desk 142b 98 bis boulevard Arago 75014 Paris, France
phone: +33.6.43.18.66.83
_______________________________________________ yt-dev mailing list -- yt-dev@python.org <mailto:yt-dev@python.org> To unsubscribe send an email to yt-dev-leave@python.org <mailto:yt-dev-leave@python.org>
_______________________________________________ yt-dev mailing list -- yt-dev@python.org <mailto:yt-dev@python.org> To unsubscribe send an email to yt-dev-leave@python.org <mailto:yt-dev-leave@python.org>
_______________________________________________ yt-dev mailing list -- yt-dev@python.org <mailto:yt-dev@python.org> To unsubscribe send an email to yt-dev-leave@python.org <mailto:yt-dev-leave@python.org>
_______________________________________________ yt-dev mailing list -- yt-dev@python.org <mailto:yt-dev@python.org> To unsubscribe send an email to yt-dev-leave@python.org <mailto:yt-dev-leave@python.org>
-- Cameron Hummels NSF Postdoctoral Fellow Department of Astronomy California Institute of Technology http://chummels.org <http://chummels.org/> _______________________________________________ yt-dev mailing list -- yt-dev@python.org <mailto:yt-dev@python.org> To unsubscribe send an email to yt-dev-leave@python.org <mailto:yt-dev-leave@python.org> _______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
Thank you all for your valuable feedback. I agree with Nathan that we should keep it on the master branch for the time being, at least until the feature is stable. I've got a working version for python 3 (see https://github.com/yt-project/yt/pull/1931) but not for python 2 (because of annoying string issues) that includes your comments. Don't hesitate to kick-in and suggest improvement / comments. Corentin On 22/08/2018 15.18, John ZuHone wrote:
I concur with this. I think the per-field configuration is a great idea!
On Aug 22, 2018, at 9:14 AM, Nathan Goldbaum <nathan12343@gmail.com <mailto:nathan12343@gmail.com>> wrote:
one more thing:
I think we shouldn’t merge this until 3.5 is out the door and we should target this new config file format for the yt 4.0 release. I think we’re too late in the cycle for 3.5 to get this in for the 3.x series without risking major user pain. Best to test it out for at leasy a few months on the master branch before a release.
On Tue, Aug 21, 2018 at 11:19 AM Nathan Goldbaum <nathan12343@gmail.com <mailto:nathan12343@gmail.com>> wrote:
On Tue, Aug 21, 2018 at 11:00 AM Cameron Hummels <chummels@gmail.com <mailto:chummels@gmail.com>> wrote:
Interesting idea--I'm all for allowing more configurability in the config file on disk. I guess I don't understand the problem with leaving it as a YAML file and adding in additional configuration within that framework? Is it just that it doesn't allow for subsections?
Not sure what you mean by this. We're not using YAML currently.
We *could* switch to YAML instead of TOML (or some other format) but I think that would be a mistake as summarized in this PEP:
https://www.python.org/dev/peps/pep-0518/#other-file-formats
YAML has lots of issues and corner cases, best to avoid that if we have the freedom to do so.
Looking forward to seeing how this all turns out!
Cameron
On Tue, Aug 21, 2018 at 7:37 AM, Matthew Turk <matthewturk@gmail.com <mailto:matthewturk@gmail.com>> wrote:
As a quick update, I looked at the usages of ytcfg, and it looks like in the 43 files that it shows up, the most common things it checks:
* default colormap * communicator information for parallelism * if we're in testing or not
The latter two are basically using ytcfg to pass around globals that are set up internally in yt. The others seem related to xray data, test data, etc.
My *guess* is that even if it's not sed-able, this wouldn't be an overwhelming thing to change. But, it would require a bit of work. On Tue, Aug 21, 2018 at 9:30 AM Nathan Goldbaum <nathan12343@gmail.com <mailto:nathan12343@gmail.com>> wrote: > > > > On Tue, Aug 21, 2018 at 8:12 AM Matthew Turk <matthewturk@gmail.com <mailto:matthewturk@gmail.com>> wrote: >> >> Hi Corentin, >> >> Great discussion. >> >> On Tue, Aug 21, 2018 at 4:34 AM Corentin CADIOU <corentin.cadiou@iap.fr <mailto:corentin.cadiou@iap.fr>> wrote: >> > >> > Hello, >> > >> > As of today, there is no way to configure yt plots per-field. One is stuck with configuration that are yt-wide (e.g. default_colormap). For example, I often find myself typing p.set_cmap('density', 'dusk'). I suggest to add a way to configure the behaviour of yt plotting interface per-field using yt configuration, so that I could set once and for all some defaults for plots. I pushed a first PR on github (https://github.com/yt-project/yt/pull/1931) and opened an issue (https://github.com/yt-project/yt/issues/1888) to discuss the feature. >> > >> > Nathan and I discussed a bit about how this should be implemented in the configuration file. Since ini files are quite restricted (e.g. you cannot have subsections), Nathan suggested at one point to use the toml format (https://github.com/toml-lang/toml). However, it is incompatible with the older .ini format. For example, boolean should be in lowercase (true, false) and not Python-like (True, False), strings have to be surrounded by apostrophes, ... >> >> Taking a step back, I suspect that the following are both true: >> >> 1) Folks who have been using yt for a lot longer are more likely to >> have configuration files scattered about >> 2) The existing config files don't actually do *that* much >> >> We've also migrated in the past, and could probably implement that >> again. (Kacper did it the first time.) > > > This is referring to "yt config migrate" which you should take a look at. We could definitely overload the existing command for this purpose. > > I think using the standard library configparser module you could first load the ini-style file, inspect the state of the resulting ConfigParser object, convert the data to a dict, and pass that dict to a toml writer. > >> >> So I don't think that this >> should be that big of a barrier; we could also consider the idea of >> having a field configuration toml file that sits next to the ini file. >> Do we need new dependencies if we have toml? > > > We'd probably want to depend on either this: > > https://github.com/uiri/toml > > which seems to be the most popular library or this: > > https://github.com/avakar/pytoml > > Which seems to be what pip has vendored and I trust dstufft's opinion on these things... > > Another option would be to vender one of those libraries but I don't think that's necessary anymore given the state of the packaging ecosystem and that both of these are pure python. > >> >> > >> > Another suggestion is to add each fields' configuration as a new section in the ini file. For example >> > >> > [config.field_type.field_name] >> > cmap = viridis >> > log = False >> > >> > This is what is implemented in the PR, but maybe some of you have more clever ideas about this? One fairly strong constraint is that the configuration files should be as backward-compatible as possible. If not, we'd have to write a migration script from the old to the new format. >> >> Or have them be in a separate file, which might even make sense -- and >> would be consistent with the idea that we may eventually want to >> override these by frontend. > > > This is starting to get close to my ideas for field_day. I was also hoping to use toml there... > > But yes, it would be good to use whatever syntax we come up with here for frontend-specific overrides. > >> > >> > We should also discuss what can be configured here. There is already the colormap and log scale implemented. One very neat feature would be to be able to configure the default unit, but it may be hard to implement (e.g. how do we configure the default unit for both projection and slices?) and probably confusing for the user. >> > >> >> Definitely default unit for non-integrated quantities! I think for >> integrated quantities like projections what we could do is configure >> the default integration unit, and that would combine with the default >> unit to produce the final result. > > > Yup, and we need to be clear to document this use case since it'll come up a lot. > >> >> >> > So here are the questions. Do you think per-field configuration should be part of yt? If so what format should it use? And what are your thoughts on what should and shouldn't be configurable? >> >> 1) Yes >> 2) I don't particularly mind toml, but I'd rather we break hard if >> we're going to break from INI-style files. A second file would make >> sense to me. > > > I agree that this is a worthwhile improvement. I would also like to move to a richer configuration format than ini-style files. ConfigParser is also a pain to deal with. > > That said, unless you also want to systematically update everywhere we use the configuration system in yt (might be possible with some sed magic, so don't discount just updating everything) you'd also need to write an emulation layer or figure out how to generate a ConfigParser object from the toml file. > >> >> >> > >> > Corentin >> > >> > -- >> > Corentin Cadiou >> > PhD student >> > Institut d'Astrophysique de Paris (IAP), desk 142b >> > 98 bis boulevard Arago >> > 75014 Paris, France >> > >> > phone: +33.6.43.18.66.83 >> > >> > _______________________________________________ >> > yt-dev mailing list -- yt-dev@python.org <mailto:yt-dev@python.org> >> > To unsubscribe send an email to yt-dev-leave@python.org <mailto:yt-dev-leave@python.org> >> _______________________________________________ >> yt-dev mailing list -- yt-dev@python.org <mailto:yt-dev@python.org> >> To unsubscribe send an email to yt-dev-leave@python.org <mailto:yt-dev-leave@python.org> > > _______________________________________________ > yt-dev mailing list -- yt-dev@python.org <mailto:yt-dev@python.org> > To unsubscribe send an email to yt-dev-leave@python.org <mailto:yt-dev-leave@python.org> _______________________________________________ yt-dev mailing list -- yt-dev@python.org <mailto:yt-dev@python.org> To unsubscribe send an email to yt-dev-leave@python.org <mailto:yt-dev-leave@python.org>
-- Cameron Hummels NSF Postdoctoral Fellow Department of Astronomy California Institute of Technology http://chummels.org <http://chummels.org/> _______________________________________________ yt-dev mailing list -- yt-dev@python.org <mailto:yt-dev@python.org> To unsubscribe send an email to yt-dev-leave@python.org <mailto:yt-dev-leave@python.org>
_______________________________________________ yt-dev mailing list -- yt-dev@python.org <mailto:yt-dev@python.org> To unsubscribe send an email to yt-dev-leave@python.org <mailto:yt-dev-leave@python.org>
_______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
-- Corentin Cadiou PhD student Institut d'Astrophysique de Paris (IAP), desk 142b 98 bis boulevard Arago 75014 Paris, France phone: +33.6.43.18.66.83
On Wed, Aug 22, 2018 at 9:13 AM Corentin CADIOU <corentin.cadiou@iap.fr> wrote:
Thank you all for your valuable feedback.
I agree with Nathan that we should keep it on the master branch for the time being, at least until the feature is stable. I've got a working version for python 3 (see https://github.com/yt-project/yt/pull/1931) but not for python 2 (because of annoying string issues) that includes your comments. Don't hesitate to kick-in and suggest improvement / comments.
Thankfully the yt-4.0 branch will only target python3 on release. So far I've kept testing on python2.7 going to make backporting easier but it's starting to cause issues for developers who need to hack things to get them to work on python2.7. In fact, how about today I issue a pull request to turn off testing on python2 on that branch? :)
Corentin On 22/08/2018 15.18, John ZuHone wrote:
I concur with this. I think the per-field configuration is a great idea!
On Aug 22, 2018, at 9:14 AM, Nathan Goldbaum <nathan12343@gmail.com> wrote:
one more thing:
I think we shouldn’t merge this until 3.5 is out the door and we should target this new config file format for the yt 4.0 release. I think we’re too late in the cycle for 3.5 to get this in for the 3.x series without risking major user pain. Best to test it out for at leasy a few months on the master branch before a release.
On Tue, Aug 21, 2018 at 11:19 AM Nathan Goldbaum <nathan12343@gmail.com> wrote:
On Tue, Aug 21, 2018 at 11:00 AM Cameron Hummels <chummels@gmail.com> wrote:
Interesting idea--I'm all for allowing more configurability in the config file on disk. I guess I don't understand the problem with leaving it as a YAML file and adding in additional configuration within that framework? Is it just that it doesn't allow for subsections?
Not sure what you mean by this. We're not using YAML currently.
We *could* switch to YAML instead of TOML (or some other format) but I think that would be a mistake as summarized in this PEP:
https://www.python.org/dev/peps/pep-0518/#other-file-formats
YAML has lots of issues and corner cases, best to avoid that if we have the freedom to do so.
Looking forward to seeing how this all turns out!
Cameron
On Tue, Aug 21, 2018 at 7:37 AM, Matthew Turk <matthewturk@gmail.com> wrote:
As a quick update, I looked at the usages of ytcfg, and it looks like in the 43 files that it shows up, the most common things it checks:
* default colormap * communicator information for parallelism * if we're in testing or not
The latter two are basically using ytcfg to pass around globals that are set up internally in yt. The others seem related to xray data, test data, etc.
My *guess* is that even if it's not sed-able, this wouldn't be an overwhelming thing to change. But, it would require a bit of work. On Tue, Aug 21, 2018 at 9:30 AM Nathan Goldbaum <nathan12343@gmail.com> wrote:
On Tue, Aug 21, 2018 at 8:12 AM Matthew Turk <matthewturk@gmail.com>
Hi Corentin,
Great discussion.
On Tue, Aug 21, 2018 at 4:34 AM Corentin CADIOU <
corentin.cadiou@iap.fr> wrote:
> > Hello, > > As of today, there is no way to configure yt plots per-field. One is stuck with configuration that are yt-wide (e.g. default_colormap). For example, I often find myself typing p.set_cmap('density', 'dusk'). I suggest to add a way to configure the behaviour of yt plotting interface
> > Nathan and I discussed a bit about how this should be implemented in the configuration file. Since ini files are quite restricted (e.g. you cannot have subsections), Nathan suggested at one point to use the toml
Taking a step back, I suspect that the following are both true:
1) Folks who have been using yt for a lot longer are more likely to have configuration files scattered about 2) The existing config files don't actually do *that* much
We've also migrated in the past, and could probably implement that again. (Kacper did it the first time.)
This is referring to "yt config migrate" which you should take a look at. We could definitely overload the existing command for this purpose.
I think using the standard library configparser module you could first load the ini-style file, inspect the state of the resulting ConfigParser object, convert the data to a dict, and pass that dict to a toml writer.
So I don't think that this should be that big of a barrier; we could also consider the idea of having a field configuration toml file that sits next to the ini
file.
Do we need new dependencies if we have toml?
We'd probably want to depend on either this:
which seems to be the most popular library or this:
https://github.com/avakar/pytoml
Which seems to be what pip has vendored and I trust dstufft's opinion on these things...
Another option would be to vender one of those libraries but I don't
wrote: per-field using yt configuration, so that I could set once and for all some defaults for plots. I pushed a first PR on github ( https://github.com/yt-project/yt/pull/1931) and opened an issue ( https://github.com/yt-project/yt/issues/1888) to discuss the feature. format (https://github.com/toml-lang/toml). However, it is incompatible with the older .ini format. For example, boolean should be in lowercase (true, false) and not Python-like (True, False), strings have to be surrounded by apostrophes, ... think that's necessary anymore given the state of the packaging ecosystem and that both of these are pure python.
> > Another suggestion is to add each fields' configuration as a new
> > [config.field_type.field_name] > cmap = viridis > log = False > > This is what is implemented in the PR, but maybe some of you have more clever ideas about this? One fairly strong constraint is that the configuration files should be as backward-compatible as possible. If not, we'd have to write a migration script from the old to the new format.
Or have them be in a separate file, which might even make sense -- and would be consistent with the idea that we may eventually want to override these by frontend.
This is starting to get close to my ideas for field_day. I was also hoping to use toml there...
But yes, it would be good to use whatever syntax we come up with here for frontend-specific overrides.
> > We should also discuss what can be configured here. There is already the colormap and log scale implemented. One very neat feature would be to be able to configure the default unit, but it may be hard to implement (e.g. how do we configure the default unit for both projection and slices?) and probably confusing for the user. >
Definitely default unit for non-integrated quantities! I think for integrated quantities like projections what we could do is configure the default integration unit, and that would combine with the default unit to produce the final result.
Yup, and we need to be clear to document this use case since it'll come up a lot.
> So here are the questions. Do you think per-field configuration
should be part of yt? If so what format should it use? And what are your
1) Yes 2) I don't particularly mind toml, but I'd rather we break hard if we're going to break from INI-style files. A second file would make sense to me.
I agree that this is a worthwhile improvement. I would also like to move to a richer configuration format than ini-style files. ConfigParser is also a pain to deal with.
That said, unless you also want to systematically update everywhere we use the configuration system in yt (might be possible with some sed magic, so don't discount just updating everything) you'd also need to write an emulation layer or figure out how to generate a ConfigParser object from
section in the ini file. For example thoughts on what should and shouldn't be configurable? the toml file.
> > Corentin > > -- > Corentin Cadiou > PhD student > Institut d'Astrophysique de Paris (IAP), desk 142b > 98 bis boulevard Arago > 75014 Paris, France > > phone: +33.6.43.18.66.83 > > _______________________________________________ > yt-dev mailing list -- yt-dev@python.org > To unsubscribe send an email to yt-dev-leave@python.org _______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
_______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
_______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
-- Cameron Hummels NSF Postdoctoral Fellow Department of Astronomy California Institute of Technology http://chummels.org _______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
_______________________________________________
yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
_______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
-- Corentin Cadiou PhD student Institut d'Astrophysique de Paris (IAP), desk 142b 98 bis boulevard Arago 75014 Paris, France
phone: +33.6.43.18.66.83
_______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
+1 to this! I'm still having some python2 compatibility issues. On 22/08/18 15:19, Nathan Goldbaum wrote:
On Wed, Aug 22, 2018 at 9:13 AM Corentin CADIOU <corentin.cadiou@iap.fr <mailto:corentin.cadiou@iap.fr>> wrote:
Thank you all for your valuable feedback.
I agree with Nathan that we should keep it on the master branch for the time being, at least until the feature is stable. I've got a working version for python 3 (see https://github.com/yt-project/yt/pull/1931) but not for python 2 (because of annoying string issues) that includes your comments. Don't hesitate to kick-in and suggest improvement / comments.
Thankfully the yt-4.0 branch will only target python3 on release. So far I've kept testing on python2.7 going to make backporting easier but it's starting to cause issues for developers who need to hack things to get them to work on python2.7.
In fact, how about today I issue a pull request to turn off testing on python2 on that branch? :)
Corentin
On 22/08/2018 15.18, John ZuHone wrote:
I concur with this. I think the per-field configuration is a great idea!
On Aug 22, 2018, at 9:14 AM, Nathan Goldbaum <nathan12343@gmail.com <mailto:nathan12343@gmail.com>> wrote:
one more thing:
I think we shouldn’t merge this until 3.5 is out the door and we should target this new config file format for the yt 4.0 release. I think we’re too late in the cycle for 3.5 to get this in for the 3.x series without risking major user pain. Best to test it out for at leasy a few months on the master branch before a release.
On Tue, Aug 21, 2018 at 11:19 AM Nathan Goldbaum <nathan12343@gmail.com <mailto:nathan12343@gmail.com>> wrote:
On Tue, Aug 21, 2018 at 11:00 AM Cameron Hummels <chummels@gmail.com <mailto:chummels@gmail.com>> wrote:
Interesting idea--I'm all for allowing more configurability in the config file on disk. I guess I don't understand the problem with leaving it as a YAML file and adding in additional configuration within that framework? Is it just that it doesn't allow for subsections?
Not sure what you mean by this. We're not using YAML currently.
We *could* switch to YAML instead of TOML (or some other format) but I think that would be a mistake as summarized in this PEP:
https://www.python.org/dev/peps/pep-0518/#other-file-formats
YAML has lots of issues and corner cases, best to avoid that if we have the freedom to do so.
Looking forward to seeing how this all turns out!
Cameron
On Tue, Aug 21, 2018 at 7:37 AM, Matthew Turk <matthewturk@gmail.com <mailto:matthewturk@gmail.com>> wrote:
As a quick update, I looked at the usages of ytcfg, and it looks like in the 43 files that it shows up, the most common things it checks:
* default colormap * communicator information for parallelism * if we're in testing or not
The latter two are basically using ytcfg to pass around globals that are set up internally in yt. The others seem related to xray data, test data, etc.
My *guess* is that even if it's not sed-able, this wouldn't be an overwhelming thing to change. But, it would require a bit of work. On Tue, Aug 21, 2018 at 9:30 AM Nathan Goldbaum <nathan12343@gmail.com <mailto:nathan12343@gmail.com>> wrote: > > > > On Tue, Aug 21, 2018 at 8:12 AM Matthew Turk <matthewturk@gmail.com <mailto:matthewturk@gmail.com>> wrote: >> >> Hi Corentin, >> >> Great discussion. >> >> On Tue, Aug 21, 2018 at 4:34 AM Corentin CADIOU <corentin.cadiou@iap.fr <mailto:corentin.cadiou@iap.fr>> wrote: >> > >> > Hello, >> > >> > As of today, there is no way to configure yt plots per-field. One is stuck with configuration that are yt-wide (e.g. default_colormap). For example, I often find myself typing p.set_cmap('density', 'dusk'). I suggest to add a way to configure the behaviour of yt plotting interface per-field using yt configuration, so that I could set once and for all some defaults for plots. I pushed a first PR on github (https://github.com/yt-project/yt/pull/1931) and opened an issue (https://github.com/yt-project/yt/issues/1888) to discuss the feature. >> > >> > Nathan and I discussed a bit about how this should be implemented in the configuration file. Since ini files are quite restricted (e.g. you cannot have subsections), Nathan suggested at one point to use the toml format (https://github.com/toml-lang/toml). However, it is incompatible with the older .ini format. For example, boolean should be in lowercase (true, false) and not Python-like (True, False), strings have to be surrounded by apostrophes, ... >> >> Taking a step back, I suspect that the following are both true: >> >> 1) Folks who have been using yt for a lot longer are more likely to >> have configuration files scattered about >> 2) The existing config files don't actually do *that* much >> >> We've also migrated in the past, and could probably implement that >> again. (Kacper did it the first time.) > > > This is referring to "yt config migrate" which you should take a look at. We could definitely overload the existing command for this purpose. > > I think using the standard library configparser module you could first load the ini-style file, inspect the state of the resulting ConfigParser object, convert the data to a dict, and pass that dict to a toml writer. > >> >> So I don't think that this >> should be that big of a barrier; we could also consider the idea of >> having a field configuration toml file that sits next to the ini file. >> Do we need new dependencies if we have toml? > > > We'd probably want to depend on either this: > > https://github.com/uiri/toml > > which seems to be the most popular library or this: > > https://github.com/avakar/pytoml > > Which seems to be what pip has vendored and I trust dstufft's opinion on these things... > > Another option would be to vender one of those libraries but I don't think that's necessary anymore given the state of the packaging ecosystem and that both of these are pure python. > >> >> > >> > Another suggestion is to add each fields' configuration as a new section in the ini file. For example >> > >> > [config.field_type.field_name] >> > cmap = viridis >> > log = False >> > >> > This is what is implemented in the PR, but maybe some of you have more clever ideas about this? One fairly strong constraint is that the configuration files should be as backward-compatible as possible. If not, we'd have to write a migration script from the old to the new format. >> >> Or have them be in a separate file, which might even make sense -- and >> would be consistent with the idea that we may eventually want to >> override these by frontend. > > > This is starting to get close to my ideas for field_day. I was also hoping to use toml there... > > But yes, it would be good to use whatever syntax we come up with here for frontend-specific overrides. > >> > >> > We should also discuss what can be configured here. There is already the colormap and log scale implemented. One very neat feature would be to be able to configure the default unit, but it may be hard to implement (e.g. how do we configure the default unit for both projection and slices?) and probably confusing for the user. >> > >> >> Definitely default unit for non-integrated quantities! I think for >> integrated quantities like projections what we could do is configure >> the default integration unit, and that would combine with the default >> unit to produce the final result. > > > Yup, and we need to be clear to document this use case since it'll come up a lot. > >> >> >> > So here are the questions. Do you think per-field configuration should be part of yt? If so what format should it use? And what are your thoughts on what should and shouldn't be configurable? >> >> 1) Yes >> 2) I don't particularly mind toml, but I'd rather we break hard if >> we're going to break from INI-style files. A second file would make >> sense to me. > > > I agree that this is a worthwhile improvement. I would also like to move to a richer configuration format than ini-style files. ConfigParser is also a pain to deal with. > > That said, unless you also want to systematically update everywhere we use the configuration system in yt (might be possible with some sed magic, so don't discount just updating everything) you'd also need to write an emulation layer or figure out how to generate a ConfigParser object from the toml file. > >> >> >> > >> > Corentin >> > >> > -- >> > Corentin Cadiou >> > PhD student >> > Institut d'Astrophysique de Paris (IAP), desk 142b >> > 98 bis boulevard Arago >> > 75014 Paris, France >> > >> > phone: +33.6.43.18.66.83 >> > >> > _______________________________________________ >> > yt-dev mailing list -- yt-dev@python.org <mailto:yt-dev@python.org> >> > To unsubscribe send an email to yt-dev-leave@python.org <mailto:yt-dev-leave@python.org> >> _______________________________________________ >> yt-dev mailing list -- yt-dev@python.org <mailto:yt-dev@python.org> >> To unsubscribe send an email to yt-dev-leave@python.org <mailto:yt-dev-leave@python.org> > > _______________________________________________ > yt-dev mailing list -- yt-dev@python.org <mailto:yt-dev@python.org> > To unsubscribe send an email to yt-dev-leave@python.org <mailto:yt-dev-leave@python.org> _______________________________________________ yt-dev mailing list -- yt-dev@python.org <mailto:yt-dev@python.org> To unsubscribe send an email to yt-dev-leave@python.org <mailto:yt-dev-leave@python.org>
-- Cameron Hummels NSF Postdoctoral Fellow Department of Astronomy California Institute of Technology http://chummels.org <http://chummels.org/> _______________________________________________ yt-dev mailing list -- yt-dev@python.org <mailto:yt-dev@python.org> To unsubscribe send an email to yt-dev-leave@python.org <mailto:yt-dev-leave@python.org>
_______________________________________________ yt-dev mailing list -- yt-dev@python.org <mailto:yt-dev@python.org> To unsubscribe send an email to yt-dev-leave@python.org <mailto:yt-dev-leave@python.org>
_______________________________________________ yt-dev mailing list --yt-dev@python.org <mailto:yt-dev@python.org> To unsubscribe send an email toyt-dev-leave@python.org <mailto:yt-dev-leave@python.org>
-- Corentin Cadiou PhD student Institut d'Astrophysique de Paris (IAP), desk 142b 98 bis boulevard Arago 75014 Paris, France
phone: +33.6.43.18.66.83
_______________________________________________ yt-dev mailing list -- yt-dev@python.org <mailto:yt-dev@python.org> To unsubscribe send an email to yt-dev-leave@python.org <mailto:yt-dev-leave@python.org>
_______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
On Wed, Aug 22, 2018 at 8:15 AM Nathan Goldbaum <nathan12343@gmail.com> wrote:
one more thing:
I think we shouldn’t merge this until 3.5 is out the door and we should target this new config file format for the yt 4.0 release. I think we’re too late in the cycle for 3.5 to get this in for the 3.x series without risking major user pain. Best to test it out for at leasy a few months on the master branch before a release.
Sure, that sounds right to me.
On Tue, Aug 21, 2018 at 11:19 AM Nathan Goldbaum <nathan12343@gmail.com> wrote:
On Tue, Aug 21, 2018 at 11:00 AM Cameron Hummels <chummels@gmail.com> wrote:
Interesting idea--I'm all for allowing more configurability in the config file on disk. I guess I don't understand the problem with leaving it as a YAML file and adding in additional configuration within that framework? Is it just that it doesn't allow for subsections?
Not sure what you mean by this. We're not using YAML currently.
We *could* switch to YAML instead of TOML (or some other format) but I think that would be a mistake as summarized in this PEP:
https://www.python.org/dev/peps/pep-0518/#other-file-formats
YAML has lots of issues and corner cases, best to avoid that if we have the freedom to do so.
Looking forward to seeing how this all turns out!
Cameron
On Tue, Aug 21, 2018 at 7:37 AM, Matthew Turk <matthewturk@gmail.com> wrote:
As a quick update, I looked at the usages of ytcfg, and it looks like in the 43 files that it shows up, the most common things it checks:
* default colormap * communicator information for parallelism * if we're in testing or not
The latter two are basically using ytcfg to pass around globals that are set up internally in yt. The others seem related to xray data, test data, etc.
My *guess* is that even if it's not sed-able, this wouldn't be an overwhelming thing to change. But, it would require a bit of work. On Tue, Aug 21, 2018 at 9:30 AM Nathan Goldbaum <nathan12343@gmail.com> wrote:
On Tue, Aug 21, 2018 at 8:12 AM Matthew Turk <matthewturk@gmail.com> wrote:
Hi Corentin,
Great discussion.
On Tue, Aug 21, 2018 at 4:34 AM Corentin CADIOU <corentin.cadiou@iap.fr> wrote: > > Hello, > > As of today, there is no way to configure yt plots per-field. One is stuck with configuration that are yt-wide (e.g. default_colormap). For example, I often find myself typing p.set_cmap('density', 'dusk'). I suggest to add a way to configure the behaviour of yt plotting interface per-field using yt configuration, so that I could set once and for all some defaults for plots. I pushed a first PR on github (https://github.com/yt-project/yt/pull/1931) and opened an issue (https://github.com/yt-project/yt/issues/1888) to discuss the feature. > > Nathan and I discussed a bit about how this should be implemented in the configuration file. Since ini files are quite restricted (e.g. you cannot have subsections), Nathan suggested at one point to use the toml format (https://github.com/toml-lang/toml). However, it is incompatible with the older .ini format. For example, boolean should be in lowercase (true, false) and not Python-like (True, False), strings have to be surrounded by apostrophes, ...
Taking a step back, I suspect that the following are both true:
1) Folks who have been using yt for a lot longer are more likely to have configuration files scattered about 2) The existing config files don't actually do *that* much
We've also migrated in the past, and could probably implement that again. (Kacper did it the first time.)
This is referring to "yt config migrate" which you should take a look at. We could definitely overload the existing command for this purpose.
I think using the standard library configparser module you could first load the ini-style file, inspect the state of the resulting ConfigParser object, convert the data to a dict, and pass that dict to a toml writer.
So I don't think that this should be that big of a barrier; we could also consider the idea of having a field configuration toml file that sits next to the ini file. Do we need new dependencies if we have toml?
We'd probably want to depend on either this:
which seems to be the most popular library or this:
https://github.com/avakar/pytoml
Which seems to be what pip has vendored and I trust dstufft's opinion on these things...
Another option would be to vender one of those libraries but I don't think that's necessary anymore given the state of the packaging ecosystem and that both of these are pure python.
> > Another suggestion is to add each fields' configuration as a new section in the ini file. For example > > [config.field_type.field_name] > cmap = viridis > log = False > > This is what is implemented in the PR, but maybe some of you have more clever ideas about this? One fairly strong constraint is that the configuration files should be as backward-compatible as possible. If not, we'd have to write a migration script from the old to the new format.
Or have them be in a separate file, which might even make sense -- and would be consistent with the idea that we may eventually want to override these by frontend.
This is starting to get close to my ideas for field_day. I was also hoping to use toml there...
But yes, it would be good to use whatever syntax we come up with here for frontend-specific overrides.
> > We should also discuss what can be configured here. There is already the colormap and log scale implemented. One very neat feature would be to be able to configure the default unit, but it may be hard to implement (e.g. how do we configure the default unit for both projection and slices?) and probably confusing for the user. >
Definitely default unit for non-integrated quantities! I think for integrated quantities like projections what we could do is configure the default integration unit, and that would combine with the default unit to produce the final result.
Yup, and we need to be clear to document this use case since it'll come up a lot.
> So here are the questions. Do you think per-field configuration should be part of yt? If so what format should it use? And what are your thoughts on what should and shouldn't be configurable?
1) Yes 2) I don't particularly mind toml, but I'd rather we break hard if we're going to break from INI-style files. A second file would make sense to me.
I agree that this is a worthwhile improvement. I would also like to move to a richer configuration format than ini-style files. ConfigParser is also a pain to deal with.
That said, unless you also want to systematically update everywhere we use the configuration system in yt (might be possible with some sed magic, so don't discount just updating everything) you'd also need to write an emulation layer or figure out how to generate a ConfigParser object from the toml file.
> > Corentin > > -- > Corentin Cadiou > PhD student > Institut d'Astrophysique de Paris (IAP), desk 142b > 98 bis boulevard Arago > 75014 Paris, France > > phone: +33.6.43.18.66.83 > > _______________________________________________ > yt-dev mailing list -- yt-dev@python.org > To unsubscribe send an email to yt-dev-leave@python.org _______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
_______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
_______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
-- Cameron Hummels NSF Postdoctoral Fellow Department of Astronomy California Institute of Technology http://chummels.org _______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
_______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
participants (7)
-
Ashley Kelly -
Cameron Hummels -
Corentin CADIOU -
John ZuHone -
Matthew Turk -
Michael Zingale -
Nathan Goldbaum