soliciting thoughts on implementing evolving frontend
Hi everyone, I'm working on adding/updating yt's support for Enzo-P data. If you're unfamiliar, Enzo-P is a totally different data format than Enzo. Enzo-P is still under heavy development, but is approaching the point where it can be used for science. I added an initial frontend last year to support grid data and this open PR <https://github.com/yt-project/yt/pull/1490> adds support for particles and cosmological data sets. This new PR also accommodates changes to the field naming conventions and the ability to get simulation parameters from a libconfig formatted file. Support for these changes is currently implemented a way that respects backward compatibility, which is mostly for the initial sample datasets that accompanied the addition of this frontend. It is likely that the file format will continue to evolve over the next months as we converge on the best design. Since this code is still mostly in the development stage, I would prefer to not have to maintain backward compatibility with prior stages of this evolution as there is almost assuredly no data existing in those forms other than the sample data in yt-project.org/data. Trying to do that will only introduce more if statements and confusion in the code. What I am asking is if I can be allowed to hit the reset button for this frontend and support only the data being produced by the current tip of Enzo-P. This would mean replacing existing sample data, but that would allow me to already make considerable simplifications to the frontend. In addition, I would like to be able to do this once or twice again in the near future as things get finalized, but before the code starts seeing wider public use. I realize this is a break from our pledge to be backward compatible as much as possible, which is why I'm bringing it up here. If periodically altering the frontend in this way is off limits, then an alternative could be moving Enzo-P support into an external yt extension package until the format is converged. I'm happy to do that, too. I would prefer if the open PR not sit and sit, either being accepted sooner than later with the understanding that things may change or being declined so I can put that functionality elsewhere. Any thoughts would be appreciated. Britton
Hi Britton, I personally think it would be fine to hit the reset button; the risk of actually breaking a workflow is probably quite low here. The other option might be to version it independently as a frontend extension, but that seems like a nice easy way not to get any feedback at *all*, which wouldn't be that great. -Matt On Tue, May 22, 2018 at 12:51 PM Britton Smith <brittonsmith@gmail.com> wrote:
Hi everyone,
I'm working on adding/updating yt's support for Enzo-P data. If you're unfamiliar, Enzo-P is a totally different data format than Enzo. Enzo-P is still under heavy development, but is approaching the point where it can be used for science. I added an initial frontend last year to support grid data and this open PR <https://github.com/yt-project/yt/pull/1490> adds support for particles and cosmological data sets. This new PR also accommodates changes to the field naming conventions and the ability to get simulation parameters from a libconfig formatted file. Support for these changes is currently implemented a way that respects backward compatibility, which is mostly for the initial sample datasets that accompanied the addition of this frontend.
It is likely that the file format will continue to evolve over the next months as we converge on the best design. Since this code is still mostly in the development stage, I would prefer to not have to maintain backward compatibility with prior stages of this evolution as there is almost assuredly no data existing in those forms other than the sample data in yt-project.org/data. Trying to do that will only introduce more if statements and confusion in the code.
What I am asking is if I can be allowed to hit the reset button for this frontend and support only the data being produced by the current tip of Enzo-P. This would mean replacing existing sample data, but that would allow me to already make considerable simplifications to the frontend. In addition, I would like to be able to do this once or twice again in the near future as things get finalized, but before the code starts seeing wider public use.
I realize this is a break from our pledge to be backward compatible as much as possible, which is why I'm bringing it up here. If periodically altering the frontend in this way is off limits, then an alternative could be moving Enzo-P support into an external yt extension package until the format is converged. I'm happy to do that, too. I would prefer if the open PR not sit and sit, either being accepted sooner than later with the understanding that things may change or being declined so I can put that functionality elsewhere.
Any thoughts would be appreciated.
Britton _______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
I think it’s ok. It would be good to make that explicit to users though, even ones that don’t read the docs. Maybe when loading Enzo-p data, yt could issue a user-visible warning saying that the current Enzo-p output format is not final and yt may not be able to load the file currently being loaded in the future. It might also be a good idea to have some sort of version number written to disk by Enzo-p so yt could quickly discard data that it can no longer read, along with a messsge suggesting to update Enzo-p. I think with if we do that it will be clear what our guarantees are. In the future we can remove the warnings and special handling when the Enzo-p output format stabilizes. From then on though I think we’d need to maintain backward compatibility if any changes happen to the Enzo-p output format after it officially stabilizes. On Tue, May 22, 2018 at 12:51 PM Britton Smith <brittonsmith@gmail.com> wrote:
Hi everyone,
I'm working on adding/updating yt's support for Enzo-P data. If you're unfamiliar, Enzo-P is a totally different data format than Enzo. Enzo-P is still under heavy development, but is approaching the point where it can be used for science. I added an initial frontend last year to support grid data and this open PR <https://github.com/yt-project/yt/pull/1490> adds support for particles and cosmological data sets. This new PR also accommodates changes to the field naming conventions and the ability to get simulation parameters from a libconfig formatted file. Support for these changes is currently implemented a way that respects backward compatibility, which is mostly for the initial sample datasets that accompanied the addition of this frontend.
It is likely that the file format will continue to evolve over the next months as we converge on the best design. Since this code is still mostly in the development stage, I would prefer to not have to maintain backward compatibility with prior stages of this evolution as there is almost assuredly no data existing in those forms other than the sample data in yt-project.org/data. Trying to do that will only introduce more if statements and confusion in the code.
What I am asking is if I can be allowed to hit the reset button for this frontend and support only the data being produced by the current tip of Enzo-P. This would mean replacing existing sample data, but that would allow me to already make considerable simplifications to the frontend. In addition, I would like to be able to do this once or twice again in the near future as things get finalized, but before the code starts seeing wider public use.
I realize this is a break from our pledge to be backward compatible as much as possible, which is why I'm bringing it up here. If periodically altering the frontend in this way is off limits, then an alternative could be moving Enzo-P support into an external yt extension package until the format is converged. I'm happy to do that, too. I would prefer if the open PR not sit and sit, either being accepted sooner than later with the understanding that things may change or being declined so I can put that functionality elsewhere.
Any thoughts would be appreciated.
Britton _______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
I like Nathan's suggestion of giving a user-visible warning that the enzo-p frontend is under development and might change. Other than that, I'm fine with a reset. Cameron On Tue, May 22, 2018 at 11:02 AM, Nathan Goldbaum <nathan12343@gmail.com> wrote:
I think it’s ok. It would be good to make that explicit to users though, even ones that don’t read the docs. Maybe when loading Enzo-p data, yt could issue a user-visible warning saying that the current Enzo-p output format is not final and yt may not be able to load the file currently being loaded in the future.
It might also be a good idea to have some sort of version number written to disk by Enzo-p so yt could quickly discard data that it can no longer read, along with a messsge suggesting to update Enzo-p.
I think with if we do that it will be clear what our guarantees are. In the future we can remove the warnings and special handling when the Enzo-p output format stabilizes. From then on though I think we’d need to maintain backward compatibility if any changes happen to the Enzo-p output format after it officially stabilizes.
On Tue, May 22, 2018 at 12:51 PM Britton Smith <brittonsmith@gmail.com> wrote:
Hi everyone,
I'm working on adding/updating yt's support for Enzo-P data. If you're unfamiliar, Enzo-P is a totally different data format than Enzo. Enzo-P is still under heavy development, but is approaching the point where it can be used for science. I added an initial frontend last year to support grid data and this open PR <https://github.com/yt-project/yt/pull/1490> adds support for particles and cosmological data sets. This new PR also accommodates changes to the field naming conventions and the ability to get simulation parameters from a libconfig formatted file. Support for these changes is currently implemented a way that respects backward compatibility, which is mostly for the initial sample datasets that accompanied the addition of this frontend.
It is likely that the file format will continue to evolve over the next months as we converge on the best design. Since this code is still mostly in the development stage, I would prefer to not have to maintain backward compatibility with prior stages of this evolution as there is almost assuredly no data existing in those forms other than the sample data in yt-project.org/data. Trying to do that will only introduce more if statements and confusion in the code.
What I am asking is if I can be allowed to hit the reset button for this frontend and support only the data being produced by the current tip of Enzo-P. This would mean replacing existing sample data, but that would allow me to already make considerable simplifications to the frontend. In addition, I would like to be able to do this once or twice again in the near future as things get finalized, but before the code starts seeing wider public use.
I realize this is a break from our pledge to be backward compatible as much as possible, which is why I'm bringing it up here. If periodically altering the frontend in this way is off limits, then an alternative could be moving Enzo-P support into an external yt extension package until the format is converged. I'm happy to do that, too. I would prefer if the open PR not sit and sit, either being accepted sooner than later with the understanding that things may change or being declined so I can put that functionality elsewhere.
Any thoughts would be appreciated.
Britton _______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
_______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
-- Cameron Hummels NSF Postdoctoral Fellow Department of Astronomy California Institute of Technology http://chummels.org
Thanks everyone for your feedback. I certainly agree that at some point the file format should be considered stable and backward compatibility maintained thereafter. Until that time, I agree with issuing a visible warning at load time. Even then, I will do my best to keep compatibility breaks to a minimum. I'll also see what can be done about adding versioning information. Thanks, Britton On Tue, May 22, 2018 at 2:09 PM, Cameron Hummels <chummels@gmail.com> wrote:
I like Nathan's suggestion of giving a user-visible warning that the enzo-p frontend is under development and might change. Other than that, I'm fine with a reset.
Cameron
On Tue, May 22, 2018 at 11:02 AM, Nathan Goldbaum <nathan12343@gmail.com> wrote:
I think it’s ok. It would be good to make that explicit to users though, even ones that don’t read the docs. Maybe when loading Enzo-p data, yt could issue a user-visible warning saying that the current Enzo-p output format is not final and yt may not be able to load the file currently being loaded in the future.
It might also be a good idea to have some sort of version number written to disk by Enzo-p so yt could quickly discard data that it can no longer read, along with a messsge suggesting to update Enzo-p.
I think with if we do that it will be clear what our guarantees are. In the future we can remove the warnings and special handling when the Enzo-p output format stabilizes. From then on though I think we’d need to maintain backward compatibility if any changes happen to the Enzo-p output format after it officially stabilizes.
On Tue, May 22, 2018 at 12:51 PM Britton Smith <brittonsmith@gmail.com> wrote:
Hi everyone,
I'm working on adding/updating yt's support for Enzo-P data. If you're unfamiliar, Enzo-P is a totally different data format than Enzo. Enzo-P is still under heavy development, but is approaching the point where it can be used for science. I added an initial frontend last year to support grid data and this open PR <https://github.com/yt-project/yt/pull/1490> adds support for particles and cosmological data sets. This new PR also accommodates changes to the field naming conventions and the ability to get simulation parameters from a libconfig formatted file. Support for these changes is currently implemented a way that respects backward compatibility, which is mostly for the initial sample datasets that accompanied the addition of this frontend.
It is likely that the file format will continue to evolve over the next months as we converge on the best design. Since this code is still mostly in the development stage, I would prefer to not have to maintain backward compatibility with prior stages of this evolution as there is almost assuredly no data existing in those forms other than the sample data in yt-project.org/data. Trying to do that will only introduce more if statements and confusion in the code.
What I am asking is if I can be allowed to hit the reset button for this frontend and support only the data being produced by the current tip of Enzo-P. This would mean replacing existing sample data, but that would allow me to already make considerable simplifications to the frontend. In addition, I would like to be able to do this once or twice again in the near future as things get finalized, but before the code starts seeing wider public use.
I realize this is a break from our pledge to be backward compatible as much as possible, which is why I'm bringing it up here. If periodically altering the frontend in this way is off limits, then an alternative could be moving Enzo-P support into an external yt extension package until the format is converged. I'm happy to do that, too. I would prefer if the open PR not sit and sit, either being accepted sooner than later with the understanding that things may change or being declined so I can put that functionality elsewhere.
Any thoughts would be appreciated.
Britton _______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
_______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
-- Cameron Hummels NSF Postdoctoral Fellow Department of Astronomy California Institute of Technology http://chummels.org
_______________________________________________ yt-dev mailing list -- yt-dev@python.org To unsubscribe send an email to yt-dev-leave@python.org
participants (4)
-
Britton Smith -
Cameron Hummels -
Matthew Turk -
Nathan Goldbaum