Mailman 3 the future of analysis_modules - yt-dev

newer
Issue #1301: Dimensions for energy...

the future of analysis_modules

Britton Smith

23 Nov 2016 23 Nov '16

1:37 a.m.

Greetings, I would like to open a discussion on the idea of moving most of yt’s analysis modules into an external yt extensions package. For ease of reading, I will separate this email into what this would mean for the code, what I see are the pros, cons, logistics, and open questions. I would very much appreciate comment on this. What this means If we did this, most of the contents of yt/analysis_modules would be moved into a repository named something like yt_astro_analysis. Installing this would be an option in the install script and would likely also be pip installable. Imports would change from from yt.analysis_modules.halo_analysis.api import HaloCatalog to from yt.extensions.astro_analysis.halo_analysis.api import HaloCatalog After creating yt_astro_analysis, there would be a period where the old analysis_modules would still exist but be deprecated before being removed at some point down the road. Pros - Almost all of the current analysis modules are specific to astrophysics. As we continue to make the core functionality of yt less astro specific, it’s not clear how to make room for non-astro analysis modules. Putting everything together under analysis_modules will make navigation and documentation messy and confusing. This would also significantly slim down the yt codebase. - Many of the tools in analysis_modules are very old and are in need of API-breaking refactor. Some of these, like two_point_functions, did not make the jump from yt-2 to yt-3 and are no longer usable. Many tools no longer have a champion, someone interested in using, maintaining, and updating them as yt’s core functionality develops and changes. Moving analysis_modules from yt decouples them from yt’s release cycle, allowing interested parties to make updates and releases on a separate, likely shorter timescale. Some analysis_modules may even be better suited to be moved into other extensions that are actively developed, such as the case of the AbsorptionSpectrum with the Trident project. - Similar to the point above, yt releases would not be slowed by the need to update all of the championless modules. Individual analysis modules can be tied to specific stable releases of yt and so assured to work there. Cons - This will take a non-zero amount of work. See below for a summary of the primary tasks. - There are some outstanding logistical questions. See below. - Not having yt and analysis modules explicitly tied to the same codebase/releases could result in analysis tools falling behind and out of date. - The disruption and need to alter scripts could irritate and alienate users. Proposed progression This is roughly how this would happen. Here is a table with all existing analysis modules, their status, and potential future: https://goo.gl/HZykQA 1. Create yt_astro_analysis repo with all analysis modules that are to be moved. Add an entry to the extensions page on yt-project.org. Make it installed by default in the install script, at least at first. 2. Deprecate all moved modules in yt. 3. After some time, remove deprecated modules from yt. Open issues Here are some logistics and questions that still need to be worked out. - Can we set up separate testing for yt_astro_analysis? Would maintaining this be a pain? - How/where would the documentation be hosted? - How would we move the analysis modules source code and maintain its revision history? Questions to yt-dev - Are you +/-1 on this? Any other comments? - Changes to the analysis_modules spreadsheet (https://goo.gl/HZykQA)? - Interested in helping out with this? If this happens, I propose anyone interested meets for a hangout to discuss how to proceed. Thanks for reading! Britton

Attachments:

attachment.html (text/html — 14.9 KB)

Show replies by date

Matthew Turk

23 Nov 23 Nov

1:42 a.m.

Hi Britton, I like this idea a lot. I think it would open us up to a much better development cycle, and even put more credibility in the notion of an extensions ecosystem. It would certainly help us position yt as a lower-level library out of which analysis can be built, which I think is probably the most useful way to position it at this time. I could go on with reasons I like this, but I don't think they'll be news to anyone. I would be willing to help with this. It's probably worth spinning up testing and docs for this on existing infrastructure, and I also think we might be able to get yt.amods to import from the extension, if that's helpful. Getting the source code in is probably doable with something like hg convert. Strong, strong +1. (Sidenote: it's a good thing we don't do "price is right" scoring on votes, or else everyone would be doing 1.0+1e-16 all the time.) -Matt On Tue, Nov 22, 2016 at 7:37 PM, Britton Smith wrote:

...

Greetings,

I would like to open a discussion on the idea of moving most of yt’s analysis modules into an external yt extensions package. For ease of reading, I will separate this email into what this would mean for the code, what I see are the pros, cons, logistics, and open questions. I would very much appreciate comment on this.

What this means

If we did this, most of the contents of yt/analysis_modules would be moved into a repository named something like yt_astro_analysis. Installing this would be an option in the install script and would likely also be pip installable. Imports would change from

from yt.analysis_modules.halo_analysis.api import HaloCatalog

to

from yt.extensions.astro_analysis.halo_analysis.api import HaloCatalog

After creating yt_astro_analysis, there would be a period where the old analysis_modules would still exist but be deprecated before being removed at some point down the road.

Pros

-

Almost all of the current analysis modules are specific to astrophysics. As we continue to make the core functionality of yt less astro specific, it’s not clear how to make room for non-astro analysis modules. Putting everything together under analysis_modules will make navigation and documentation messy and confusing. This would also significantly slim down the yt codebase. -

Many of the tools in analysis_modules are very old and are in need of API-breaking refactor. Some of these, like two_point_functions, did not make the jump from yt-2 to yt-3 and are no longer usable. Many tools no longer have a champion, someone interested in using, maintaining, and updating them as yt’s core functionality develops and changes. Moving analysis_modules from yt decouples them from yt’s release cycle, allowing interested parties to make updates and releases on a separate, likely shorter timescale. Some analysis_modules may even be better suited to be moved into other extensions that are actively developed, such as the case of the AbsorptionSpectrum with the Trident project. -

Similar to the point above, yt releases would not be slowed by the need to update all of the championless modules. Individual analysis modules can be tied to specific stable releases of yt and so assured to work there.

Cons

-

This will take a non-zero amount of work. See below for a summary of the primary tasks. -

There are some outstanding logistical questions. See below. -

Not having yt and analysis modules explicitly tied to the same codebase/releases could result in analysis tools falling behind and out of date. -

The disruption and need to alter scripts could irritate and alienate users.

Proposed progression

This is roughly how this would happen. Here is a table with all existing analysis modules, their status, and potential future: https://goo.gl/HZykQA

1.

Create yt_astro_analysis repo with all analysis modules that are to be moved. Add an entry to the extensions page on yt-project.org. Make it installed by default in the install script, at least at first. 2.

Deprecate all moved modules in yt. 3.

After some time, remove deprecated modules from yt.

Open issues

Here are some logistics and questions that still need to be worked out.

-

Can we set up separate testing for yt_astro_analysis? Would maintaining this be a pain? -

How/where would the documentation be hosted? -

How would we move the analysis modules source code and maintain its revision history?

Questions to yt-dev

-

Are you +/-1 on this? Any other comments? -

Changes to the analysis_modules spreadsheet (https://goo.gl/HZykQA)? -

Interested in helping out with this? If this happens, I propose anyone interested meets for a hangout to discuss how to proceed.

Thanks for reading!

Britton

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

Nathan Goldbaum

1:54 a.m.

Overall I think this is a good idea. I have a couple critiques about the mechanics that I'll reply inline about. I think it makes sense to separate out all of the astro-specific analysis modules. On Tue, Nov 22, 2016 at 8:37 PM, Britton Smith wrote:

...

Greetings,

I would like to open a discussion on the idea of moving most of yt’s analysis modules into an external yt extensions package. For ease of reading, I will separate this email into what this would mean for the code, what I see are the pros, cons, logistics, and open questions. I would very much appreciate comment on this.

What this means

If we did this, most of the contents of yt/analysis_modules would be moved into a repository named something like yt_astro_analysis. Installing this would be an option in the install script and would likely also be pip installable. Imports would change from

from yt.analysis_modules.halo_analysis.api import HaloCatalog

to

from yt.extensions.astro_analysis.halo_analysis.api import HaloCatalog

After creating yt_astro_analysis, there would be a period where the old analysis_modules would still exist but be deprecated before being removed at some point down the road.

Pros

-

Almost all of the current analysis modules are specific to astrophysics. As we continue to make the core functionality of yt less astro specific, it’s not clear how to make room for non-astro analysis modules. Putting everything together under analysis_modules will make navigation and documentation messy and confusing. This would also significantly slim down the yt codebase. -

Many of the tools in analysis_modules are very old and are in need of API-breaking refactor. Some of these, like two_point_functions, did not make the jump from yt-2 to yt-3 and are no longer usable. Many tools no longer have a champion, someone interested in using, maintaining, and updating them as yt’s core functionality develops and changes. Moving analysis_modules from yt decouples them from yt’s release cycle, allowing interested parties to make updates and releases on a separate, likely shorter timescale. Some analysis_modules may even be better suited to be moved into other extensions that are actively developed, such as the case of the AbsorptionSpectrum with the Trident project. -

Similar to the point above, yt releases would not be slowed by the need to update all of the championless modules. Individual analysis modules can be tied to specific stable releases of yt and so assured to work there.

Cons

-

This will take a non-zero amount of work. See below for a summary of the primary tasks. -

There are some outstanding logistical questions. See below. -

Not having yt and analysis modules explicitly tied to the same codebase/releases could result in analysis tools falling behind and out of date. -

The disruption and need to alter scripts could irritate and alienate users.

Proposed progression

This is roughly how this would happen. Here is a table with all existing analysis modules, their status, and potential future: https://goo.gl/HZykQA

1.

Create yt_astro_analysis repo with all analysis modules that are to be moved. Add an entry to the extensions page on yt-project.org. Make it installed by default in the install script, at least at first.

Should there only be one repo? If we're going to do this, it might help future maintainability to have one repo per analysis module. That way maintainers that only care about one analysis module don't need to worry about changes to other modules. I also don't think the clump finder in particular should be split out into its own package. That one isn't astro-specific and it might make sense to try to integrate it more deeply with the core of yt.

...

1.

Deprecate all moved modules in yt. 2.

After some time, remove deprecated modules from yt.

Open issues

Here are some logistics and questions that still need to be worked out.

-

Can we set up separate testing for yt_astro_analysis? Would maintaining this be a pain?

If the tests don't require large datasets and are relatively quick, this might be a good opportunity to explore bitbucket pipelines. There's also drone.io or we can use the yt testing infrastructure. Since I'm not the maintainer of the yt testing infra I can't speak to whether it's ok to expand it to more packages.

...

-

How/where would the documentation be hosted?

Readthedocs will likely be sufficient. If we care about inline code examples or whatnot, then we can explore using the yt testing infrastructure, with the same caveats as above.

...

-

How would we move the analysis modules source code and maintain its revision history?

This can be done relatively straightforwardly using the convert extension for mercurial and a filemap.

...

Questions to yt-dev

-

Are you +/-1 on this? Any other comments?

...

-

Changes to the analysis_modules spreadsheet (https://goo.gl/HZykQA)? -

Interested in helping out with this? If this happens, I propose anyone interested meets for a hangout to discuss how to proceed.

I'm happy to help.

...

Thanks for reading!

Britton

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

Bili Dong

6:32 a.m.

I’m +1 to the general idea and Nathan’s idea to have one repo per analysis module. And I’m willing to help. Bili

...

On Nov 22, 2016, at 5:54 PM, Nathan Goldbaum wrote:

Overall I think this is a good idea. I have a couple critiques about the mechanics that I'll reply inline about. I think it makes sense to separate out all of the astro-specific analysis modules.

On Tue, Nov 22, 2016 at 8:37 PM, Britton Smith mailto:brittonsmith@gmail.com> wrote: Greetings,

I would like to open a discussion on the idea of moving most of yt’s analysis modules into an external yt extensions package. For ease of reading, I will separate this email into what this would mean for the code, what I see are the pros, cons, logistics, and open questions. I would very much appreciate comment on this.

What this means If we did this, most of the contents of yt/analysis_modules would be moved into a repository named something like yt_astro_analysis. Installing this would be an option in the install script and would likely also be pip installable. Imports would change from from yt.analysis_modules.halo_analysis.api import HaloCatalog to from yt.extensions.astro_analysis.halo_analysis.api import HaloCatalog

After creating yt_astro_analysis, there would be a period where the old analysis_modules would still exist but be deprecated before being removed at some point down the road.

Pros Almost all of the current analysis modules are specific to astrophysics. As we continue to make the core functionality of yt less astro specific, it’s not clear how to make room for non-astro analysis modules. Putting everything together under analysis_modules will make navigation and documentation messy and confusing. This would also significantly slim down the yt codebase. Many of the tools in analysis_modules are very old and are in need of API-breaking refactor. Some of these, like two_point_functions, did not make the jump from yt-2 to yt-3 and are no longer usable. Many tools no longer have a champion, someone interested in using, maintaining, and updating them as yt’s core functionality develops and changes. Moving analysis_modules from yt decouples them from yt’s release cycle, allowing interested parties to make updates and releases on a separate, likely shorter timescale. Some analysis_modules may even be better suited to be moved into other extensions that are actively developed, such as the case of the AbsorptionSpectrum with the Trident project. Similar to the point above, yt releases would not be slowed by the need to update all of the championless modules. Individual analysis modules can be tied to specific stable releases of yt and so assured to work there.

Cons This will take a non-zero amount of work. See below for a summary of the primary tasks. There are some outstanding logistical questions. See below. Not having yt and analysis modules explicitly tied to the same codebase/releases could result in analysis tools falling behind and out of date. The disruption and need to alter scripts could irritate and alienate users.

Proposed progression This is roughly how this would happen. Here is a table with all existing analysis modules, their status, and potential future: https://goo.gl/HZykQA https://goo.gl/HZykQA Create yt_astro_analysis repo with all analysis modules that are to be moved. Add an entry to the extensions page on yt-project.org http://yt-project.org/. Make it installed by default in the install script, at least at first.

Should there only be one repo? If we're going to do this, it might help future maintainability to have one repo per analysis module. That way maintainers that only care about one analysis module don't need to worry about changes to other modules.

I also don't think the clump finder in particular should be split out into its own package. That one isn't astro-specific and it might make sense to try to integrate it more deeply with the core of yt.

Deprecate all moved modules in yt. After some time, remove deprecated modules from yt.

Open issues Here are some logistics and questions that still need to be worked out. Can we set up separate testing for yt_astro_analysis? Would maintaining this be a pain?

If the tests don't require large datasets and are relatively quick, this might be a good opportunity to explore bitbucket pipelines. There's also drone.io http://drone.io/ or we can use the yt testing infrastructure. Since I'm not the maintainer of the yt testing infra I can't speak to whether it's ok to expand it to more packages.

How/where would the documentation be hosted?

Readthedocs will likely be sufficient. If we care about inline code examples or whatnot, then we can explore using the yt testing infrastructure, with the same caveats as above.

How would we move the analysis modules source code and maintain its revision history?

This can be done relatively straightforwardly using the convert extension for mercurial and a filemap.

Questions to yt-dev Are you +/-1 on this? Any other comments?

+1

Changes to the analysis_modules spreadsheet (https://goo.gl/HZykQA https://goo.gl/HZykQA)? Interested in helping out with this? If this happens, I propose anyone interested meets for a hangout to discuss how to proceed.

I'm happy to help.

Thanks for reading!

Britton

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org mailto:yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org mailto:yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

John ZuHone

4:19 p.m.

I’m +1 on this and I agree with Nathan’s comments. In particular, I think that we shouldn’t bundle them all into one package but probably split them off into separate packages. This requires more work on the behalf of the maintainers of those modules, of course. In addition to keeping in clump_finder, particle_trajectories should also probably be left in, and perhaps be turned into some kind of time series object or something else. I’m happy to help out—in particular I can split off sunyaev_zeldovich and ppv_cube and spin them off as separate packages. photon_simulator is already deprecated in favor of pyXSIM. All that spectral_integrator does is generate fields, so perhaps its functionality needs to be merged into the astro fields part somehow and left inside yt itself.

...

On Nov 22, 2016, at 8:54 PM, Nathan Goldbaum wrote:

Overall I think this is a good idea. I have a couple critiques about the mechanics that I'll reply inline about. I think it makes sense to separate out all of the astro-specific analysis modules.

On Tue, Nov 22, 2016 at 8:37 PM, Britton Smith mailto:brittonsmith@gmail.com> wrote: Greetings,

I would like to open a discussion on the idea of moving most of yt’s analysis modules into an external yt extensions package. For ease of reading, I will separate this email into what this would mean for the code, what I see are the pros, cons, logistics, and open questions. I would very much appreciate comment on this.

What this means If we did this, most of the contents of yt/analysis_modules would be moved into a repository named something like yt_astro_analysis. Installing this would be an option in the install script and would likely also be pip installable. Imports would change from from yt.analysis_modules.halo_analysis.api import HaloCatalog to from yt.extensions.astro_analysis.halo_analysis.api import HaloCatalog

After creating yt_astro_analysis, there would be a period where the old analysis_modules would still exist but be deprecated before being removed at some point down the road.

Pros Almost all of the current analysis modules are specific to astrophysics. As we continue to make the core functionality of yt less astro specific, it’s not clear how to make room for non-astro analysis modules. Putting everything together under analysis_modules will make navigation and documentation messy and confusing. This would also significantly slim down the yt codebase. Many of the tools in analysis_modules are very old and are in need of API-breaking refactor. Some of these, like two_point_functions, did not make the jump from yt-2 to yt-3 and are no longer usable. Many tools no longer have a champion, someone interested in using, maintaining, and updating them as yt’s core functionality develops and changes. Moving analysis_modules from yt decouples them from yt’s release cycle, allowing interested parties to make updates and releases on a separate, likely shorter timescale. Some analysis_modules may even be better suited to be moved into other extensions that are actively developed, such as the case of the AbsorptionSpectrum with the Trident project. Similar to the point above, yt releases would not be slowed by the need to update all of the championless modules. Individual analysis modules can be tied to specific stable releases of yt and so assured to work there.

Cons This will take a non-zero amount of work. See below for a summary of the primary tasks. There are some outstanding logistical questions. See below. Not having yt and analysis modules explicitly tied to the same codebase/releases could result in analysis tools falling behind and out of date. The disruption and need to alter scripts could irritate and alienate users.

Proposed progression This is roughly how this would happen. Here is a table with all existing analysis modules, their status, and potential future: https://goo.gl/HZykQA https://goo.gl/HZykQA Create yt_astro_analysis repo with all analysis modules that are to be moved. Add an entry to the extensions page on yt-project.org http://yt-project.org/. Make it installed by default in the install script, at least at first.

Should there only be one repo? If we're going to do this, it might help future maintainability to have one repo per analysis module. That way maintainers that only care about one analysis module don't need to worry about changes to other modules.

I also don't think the clump finder in particular should be split out into its own package. That one isn't astro-specific and it might make sense to try to integrate it more deeply with the core of yt.

Deprecate all moved modules in yt. After some time, remove deprecated modules from yt.

Open issues Here are some logistics and questions that still need to be worked out. Can we set up separate testing for yt_astro_analysis? Would maintaining this be a pain?

If the tests don't require large datasets and are relatively quick, this might be a good opportunity to explore bitbucket pipelines. There's also drone.io http://drone.io/ or we can use the yt testing infrastructure. Since I'm not the maintainer of the yt testing infra I can't speak to whether it's ok to expand it to more packages.

How/where would the documentation be hosted?

Readthedocs will likely be sufficient. If we care about inline code examples or whatnot, then we can explore using the yt testing infrastructure, with the same caveats as above.

How would we move the analysis modules source code and maintain its revision history?

This can be done relatively straightforwardly using the convert extension for mercurial and a filemap.

Questions to yt-dev Are you +/-1 on this? Any other comments?

+1

Changes to the analysis_modules spreadsheet (https://goo.gl/HZykQA https://goo.gl/HZykQA)? Interested in helping out with this? If this happens, I propose anyone interested meets for a hangout to discuss how to proceed.

I'm happy to help.

Thanks for reading!

Britton

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org mailto:yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org mailto:yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

Britton Smith

5:48 p.m.

Hey all, Thanks to everyone for their feedback. This has been a good discussion so far. With Thanksgiving tomorrow, let's give people until the end of Monday to contribute comments. It looks like this idea is getting a good amount of support, so let's plan of having a live discussion about this in the near future. After Monday, I will organize a doodle poll inviting everyone who has responded so far. If you don't have anything to add to the discussion but would like to be involved, just drop me a note and I'll include you on the poll. Thanks again everyone and have a good weekend. Britton On Wed, Nov 23, 2016 at 8:19 AM, John ZuHone wrote:

...

I’m +1 on this and I agree with Nathan’s comments. In particular, I think that we shouldn’t bundle them all into one package but probably split them off into separate packages. This requires more work on the behalf of the maintainers of those modules, of course.

In addition to keeping in clump_finder, particle_trajectories should also probably be left in, and perhaps be turned into some kind of time series object or something else.

I’m happy to help out—in particular I can split off sunyaev_zeldovich and ppv_cube and spin them off as separate packages. photon_simulator is already deprecated in favor of pyXSIM.

All that spectral_integrator does is generate fields, so perhaps its functionality needs to be merged into the astro fields part somehow and left inside yt itself.

On Nov 22, 2016, at 8:54 PM, Nathan Goldbaum wrote:

Overall I think this is a good idea. I have a couple critiques about the mechanics that I'll reply inline about. I think it makes sense to separate out all of the astro-specific analysis modules.

On Tue, Nov 22, 2016 at 8:37 PM, Britton Smith wrote:

...
Greetings,

I would like to open a discussion on the idea of moving most of yt’s analysis modules into an external yt extensions package. For ease of reading, I will separate this email into what this would mean for the code, what I see are the pros, cons, logistics, and open questions. I would very much appreciate comment on this.

What this means If we did this, most of the contents of yt/analysis_modules would be moved into a repository named something like yt_astro_analysis. Installing this would be an option in the install script and would likely also be pip installable. Imports would change from from yt.analysis_modules.halo_analysis.api import HaloCatalog to from yt.extensions.astro_analysis.halo_analysis.api import HaloCatalog

After creating yt_astro_analysis, there would be a period where the old analysis_modules would still exist but be deprecated before being removed at some point down the road.

Pros

- Almost all of the current analysis modules are specific to astrophysics. As we continue to make the core functionality of yt less astro specific, it’s not clear how to make room for non-astro analysis modules. Putting everything together under analysis_modules will make navigation and documentation messy and confusing. This would also significantly slim down the yt codebase. - Many of the tools in analysis_modules are very old and are in need of API-breaking refactor. Some of these, like two_point_functions, did not make the jump from yt-2 to yt-3 and are no longer usable. Many tools no longer have a champion, someone interested in using, maintaining, and updating them as yt’s core functionality develops and changes. Moving analysis_modules from yt decouples them from yt’s release cycle, allowing interested parties to make updates and releases on a separate, likely shorter timescale. Some analysis_modules may even be better suited to be moved into other extensions that are actively developed, such as the case of the AbsorptionSpectrum with the Trident project. - Similar to the point above, yt releases would not be slowed by the need to update all of the championless modules. Individual analysis modules can be tied to specific stable releases of yt and so assured to work there.

Cons

- This will take a non-zero amount of work. See below for a summary of the primary tasks. - There are some outstanding logistical questions. See below. - Not having yt and analysis modules explicitly tied to the same codebase/releases could result in analysis tools falling behind and out of date. - The disruption and need to alter scripts could irritate and alienate users.

Proposed progression This is roughly how this would happen. Here is a table with all existing analysis modules, their status, and potential future: https://goo.gl/HZykQA

1. Create yt_astro_analysis repo with all analysis modules that are to be moved. Add an entry to the extensions page on yt-project.org. Make it installed by default in the install script, at least at first.

Should there only be one repo? If we're going to do this, it might help future maintainability to have one repo per analysis module. That way maintainers that only care about one analysis module don't need to worry about changes to other modules.

I also don't think the clump finder in particular should be split out into its own package. That one isn't astro-specific and it might make sense to try to integrate it more deeply with the core of yt.

...
1. Deprecate all moved modules in yt. 2. After some time, remove deprecated modules from yt.

Open issues Here are some logistics and questions that still need to be worked out.

- Can we set up separate testing for yt_astro_analysis? Would maintaining this be a pain?

If the tests don't require large datasets and are relatively quick, this might be a good opportunity to explore bitbucket pipelines. There's also drone.io or we can use the yt testing infrastructure. Since I'm not the maintainer of the yt testing infra I can't speak to whether it's ok to expand it to more packages.

...
- How/where would the documentation be hosted?

Readthedocs will likely be sufficient. If we care about inline code examples or whatnot, then we can explore using the yt testing infrastructure, with the same caveats as above.

...
- How would we move the analysis modules source code and maintain its revision history?

This can be done relatively straightforwardly using the convert extension for mercurial and a filemap.

...
Questions to yt-dev

- Are you +/-1 on this? Any other comments?

+1

...
- Changes to the analysis_modules spreadsheet (https://goo.gl/HZykQA)? - Interested in helping out with this? If this happens, I propose anyone interested meets for a hangout to discuss how to proceed.

I'm happy to help.

...
Thanks for reading!

Britton

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

Britton Smith

29 Nov 29 Nov

6:10 p.m.

Hi everyone, We will have a hangout discussion about moving forward with this on Thursday, Dec 1 at 2pm EST. If you weren't on the poll and would like to be invited to the hangout, send me an email. Britton On Wed, Nov 23, 2016 at 9:48 AM, Britton Smith wrote:

...

Hey all,

Thanks to everyone for their feedback. This has been a good discussion so far. With Thanksgiving tomorrow, let's give people until the end of Monday to contribute comments.

It looks like this idea is getting a good amount of support, so let's plan of having a live discussion about this in the near future. After Monday, I will organize a doodle poll inviting everyone who has responded so far. If you don't have anything to add to the discussion but would like to be involved, just drop me a note and I'll include you on the poll.

Thanks again everyone and have a good weekend.

Britton

On Wed, Nov 23, 2016 at 8:19 AM, John ZuHone wrote:

...
I’m +1 on this and I agree with Nathan’s comments. In particular, I think that we shouldn’t bundle them all into one package but probably split them off into separate packages. This requires more work on the behalf of the maintainers of those modules, of course.

In addition to keeping in clump_finder, particle_trajectories should also probably be left in, and perhaps be turned into some kind of time series object or something else.

I’m happy to help out—in particular I can split off sunyaev_zeldovich and ppv_cube and spin them off as separate packages. photon_simulator is already deprecated in favor of pyXSIM.

All that spectral_integrator does is generate fields, so perhaps its functionality needs to be merged into the astro fields part somehow and left inside yt itself.

On Nov 22, 2016, at 8:54 PM, Nathan Goldbaum wrote:

Overall I think this is a good idea. I have a couple critiques about the mechanics that I'll reply inline about. I think it makes sense to separate out all of the astro-specific analysis modules.

On Tue, Nov 22, 2016 at 8:37 PM, Britton Smith wrote:

...
Greetings,

I would like to open a discussion on the idea of moving most of yt’s analysis modules into an external yt extensions package. For ease of reading, I will separate this email into what this would mean for the code, what I see are the pros, cons, logistics, and open questions. I would very much appreciate comment on this.

What this means If we did this, most of the contents of yt/analysis_modules would be moved into a repository named something like yt_astro_analysis. Installing this would be an option in the install script and would likely also be pip installable. Imports would change from from yt.analysis_modules.halo_analysis.api import HaloCatalog to from yt.extensions.astro_analysis.halo_analysis.api import HaloCatalog

After creating yt_astro_analysis, there would be a period where the old analysis_modules would still exist but be deprecated before being removed at some point down the road.

Pros

- Almost all of the current analysis modules are specific to astrophysics. As we continue to make the core functionality of yt less astro specific, it’s not clear how to make room for non-astro analysis modules. Putting everything together under analysis_modules will make navigation and documentation messy and confusing. This would also significantly slim down the yt codebase. - Many of the tools in analysis_modules are very old and are in need of API-breaking refactor. Some of these, like two_point_functions, did not make the jump from yt-2 to yt-3 and are no longer usable. Many tools no longer have a champion, someone interested in using, maintaining, and updating them as yt’s core functionality develops and changes. Moving analysis_modules from yt decouples them from yt’s release cycle, allowing interested parties to make updates and releases on a separate, likely shorter timescale. Some analysis_modules may even be better suited to be moved into other extensions that are actively developed, such as the case of the AbsorptionSpectrum with the Trident project. - Similar to the point above, yt releases would not be slowed by the need to update all of the championless modules. Individual analysis modules can be tied to specific stable releases of yt and so assured to work there.

Cons

- This will take a non-zero amount of work. See below for a summary of the primary tasks. - There are some outstanding logistical questions. See below. - Not having yt and analysis modules explicitly tied to the same codebase/releases could result in analysis tools falling behind and out of date. - The disruption and need to alter scripts could irritate and alienate users.

Proposed progression This is roughly how this would happen. Here is a table with all existing analysis modules, their status, and potential future: https://goo.gl/HZykQA

1. Create yt_astro_analysis repo with all analysis modules that are to be moved. Add an entry to the extensions page on yt-project.org. Make it installed by default in the install script, at least at first.

Should there only be one repo? If we're going to do this, it might help future maintainability to have one repo per analysis module. That way maintainers that only care about one analysis module don't need to worry about changes to other modules.

I also don't think the clump finder in particular should be split out into its own package. That one isn't astro-specific and it might make sense to try to integrate it more deeply with the core of yt.

...
1. Deprecate all moved modules in yt. 2. After some time, remove deprecated modules from yt.

Open issues Here are some logistics and questions that still need to be worked out.

- Can we set up separate testing for yt_astro_analysis? Would maintaining this be a pain?

If the tests don't require large datasets and are relatively quick, this might be a good opportunity to explore bitbucket pipelines. There's also drone.io or we can use the yt testing infrastructure. Since I'm not the maintainer of the yt testing infra I can't speak to whether it's ok to expand it to more packages.

...
- How/where would the documentation be hosted?

Readthedocs will likely be sufficient. If we care about inline code examples or whatnot, then we can explore using the yt testing infrastructure, with the same caveats as above.

...
- How would we move the analysis modules source code and maintain its revision history?

This can be done relatively straightforwardly using the convert extension for mercurial and a filemap.

...
Questions to yt-dev

- Are you +/-1 on this? Any other comments?

+1

...
- Changes to the analysis_modules spreadsheet (https://goo.gl/HZykQA )? - Interested in helping out with this? If this happens, I propose anyone interested meets for a hangout to discuss how to proceed.

I'm happy to help.

...
Thanks for reading!

Britton

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

John ZuHone

6:12 p.m.

Thanks for doing this, Britton! ================================ John ZuHone, Chandra/ACIS Operations Harvard-Smithsonian Center for Astrophysics 60 Garden St., MS-67 (w) 617-496-1816 Cambridge, MA 02138. (m) 781-708-5004 john.zuhone@cfa.harvard.edu http://hea-www.cfa.harvard.edu/~jzuhone ================================

...

On Nov 29, 2016, at 1:10 PM, Britton Smith wrote:

Hi everyone,

We will have a hangout discussion about moving forward with this on Thursday, Dec 1 at 2pm EST. If you weren't on the poll and would like to be invited to the hangout, send me an email.

Britton

...
On Wed, Nov 23, 2016 at 9:48 AM, Britton Smith wrote: Hey all,

Thanks to everyone for their feedback. This has been a good discussion so far. With Thanksgiving tomorrow, let's give people until the end of Monday to contribute comments.

It looks like this idea is getting a good amount of support, so let's plan of having a live discussion about this in the near future. After Monday, I will organize a doodle poll inviting everyone who has responded so far. If you don't have anything to add to the discussion but would like to be involved, just drop me a note and I'll include you on the poll.

Thanks again everyone and have a good weekend.

Britton

...
On Wed, Nov 23, 2016 at 8:19 AM, John ZuHone wrote: I’m +1 on this and I agree with Nathan’s comments. In particular, I think that we shouldn’t bundle them all into one package but probably split them off into separate packages. This requires more work on the behalf of the maintainers of those modules, of course.

In addition to keeping in clump_finder, particle_trajectories should also probably be left in, and perhaps be turned into some kind of time series object or something else.

I’m happy to help out—in particular I can split off sunyaev_zeldovich and ppv_cube and spin them off as separate packages. photon_simulator is already deprecated in favor of pyXSIM.

All that spectral_integrator does is generate fields, so perhaps its functionality needs to be merged into the astro fields part somehow and left inside yt itself.

...
On Nov 22, 2016, at 8:54 PM, Nathan Goldbaum wrote:

Overall I think this is a good idea. I have a couple critiques about the mechanics that I'll reply inline about. I think it makes sense to separate out all of the astro-specific analysis modules.

On Tue, Nov 22, 2016 at 8:37 PM, Britton Smith wrote:

...
Greetings,

I would like to open a discussion on the idea of moving most of yt’s analysis modules into an external yt extensions package. For ease of reading, I will separate this email into what this would mean for the code, what I see are the pros, cons, logistics, and open questions. I would very much appreciate comment on this.

What this means If we did this, most of the contents of yt/analysis_modules would be moved into a repository named something like yt_astro_analysis. Installing this would be an option in the install script and would likely also be pip installable. Imports would change from from yt.analysis_modules.halo_analysis.api import HaloCatalog to from yt.extensions.astro_analysis.halo_analysis.api import HaloCatalog

After creating yt_astro_analysis, there would be a period where the old analysis_modules would still exist but be deprecated before being removed at some point down the road.

Pros Almost all of the current analysis modules are specific to astrophysics. As we continue to make the core functionality of yt less astro specific, it’s not clear how to make room for non-astro analysis modules. Putting everything together under analysis_modules will make navigation and documentation messy and confusing. This would also significantly slim down the yt codebase. Many of the tools in analysis_modules are very old and are in need of API-breaking refactor. Some of these, like two_point_functions, did not make the jump from yt-2 to yt-3 and are no longer usable. Many tools no longer have a champion, someone interested in using, maintaining, and updating them as yt’s core functionality develops and changes. Moving analysis_modules from yt decouples them from yt’s release cycle, allowing interested parties to make updates and releases on a separate, likely shorter timescale. Some analysis_modules may even be better suited to be moved into other extensions that are actively developed, such as the case of the AbsorptionSpectrum with the Trident project. Similar to the point above, yt releases would not be slowed by the need to update all of the championless modules. Individual analysis modules can be tied to specific stable releases of yt and so assured to work there.

Cons This will take a non-zero amount of work. See below for a summary of the primary tasks. There are some outstanding logistical questions. See below. Not having yt and analysis modules explicitly tied to the same codebase/releases could result in analysis tools falling behind and out of date. The disruption and need to alter scripts could irritate and alienate users.

Proposed progression This is roughly how this would happen. Here is a table with all existing analysis modules, their status, and potential future: https://goo.gl/HZykQA Create yt_astro_analysis repo with all analysis modules that are to be moved. Add an entry to the extensions page on yt-project.org. Make it installed by default in the install script, at least at first.

Should there only be one repo? If we're going to do this, it might help future maintainability to have one repo per analysis module. That way maintainers that only care about one analysis module don't need to worry about changes to other modules.

I also don't think the clump finder in particular should be split out into its own package. That one isn't astro-specific and it might make sense to try to integrate it more deeply with the core of yt.

...
Deprecate all moved modules in yt. After some time, remove deprecated modules from yt.

Open issues Here are some logistics and questions that still need to be worked out. Can we set up separate testing for yt_astro_analysis? Would maintaining this be a pain?

If the tests don't require large datasets and are relatively quick, this might be a good opportunity to explore bitbucket pipelines. There's also drone.io or we can use the yt testing infrastructure. Since I'm not the maintainer of the yt testing infra I can't speak to whether it's ok to expand it to more packages.

...
How/where would the documentation be hosted?

Readthedocs will likely be sufficient. If we care about inline code examples or whatnot, then we can explore using the yt testing infrastructure, with the same caveats as above.

...
How would we move the analysis modules source code and maintain its revision history?

This can be done relatively straightforwardly using the convert extension for mercurial and a filemap.

...
Questions to yt-dev Are you +/-1 on this? Any other comments?

+1

...
Changes to the analysis_modules spreadsheet (https://goo.gl/HZykQA)? Interested in helping out with this? If this happens, I propose anyone interested meets for a hangout to discuss how to proceed.

I'm happy to help.

...
Thanks for reading!

Britton

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

Britton Smith

5 Dec 5 Dec

6:20 p.m.

Hi again everyone, Last week, a few of us had a hangout to discuss breaking most the analysis modules off into external, pip-installable packages. Below is a summary of this discussion. If anyone has any questions, please feel free to post them here. Britton *What will happen:* - Most of the contents of analysis_modules will be moved to individual external packages under the yt_analysis user on bitbucket or on the yt-project organization on github. Putting packages on github will enable integration with testing services like travis, circleci and appveyor. Some will be move to yt/data_objects. - Modules without an interested maintainer will be collected into an attic repository under yt_analysis. - Status of each module can be seen here https://docs.google.com/spreadsheets/d/1uMGD5fjV7eHjNGWek1uGA-GP-xrfmWRMMN54..., with names of people volunteering to relocate them. *Logistics* - Modules to be renamed yt_<module name> so as to be importable from yt.extensions. - Docs: individual docs for each module can be hosted on readthedocs.io. - Testing: continue to use in-house yt testing infrastructure or free CI services like travis, circleci, and appveyor (on github) or drone.io (on bitbucket). Kacper will help set this up. Exporting version history: use the hg convert extension. Nathan to create some instructions for this. - Modules will be marked as deprecated with message to pip install external package. Deprecated modules will be removed for yt-4.0 release. - An extension page to be added to docs. *Procedure* 1. Create new repo under yt_analysis and export source and docs. 2. Create readthedocs page. 3. Set up testing, work with Kacper. 4. Add entry to extensions page on yt-projects.org. 5. Issue PR to yt marking analysis module as deprecated. On Tue, Nov 29, 2016 at 10:10 AM, Britton Smith wrote:

...

Hi everyone,

We will have a hangout discussion about moving forward with this on Thursday, Dec 1 at 2pm EST. If you weren't on the poll and would like to be invited to the hangout, send me an email.

Britton

On Wed, Nov 23, 2016 at 9:48 AM, Britton Smith wrote:

...
Hey all,

Thanks to everyone for their feedback. This has been a good discussion so far. With Thanksgiving tomorrow, let's give people until the end of Monday to contribute comments.

It looks like this idea is getting a good amount of support, so let's plan of having a live discussion about this in the near future. After Monday, I will organize a doodle poll inviting everyone who has responded so far. If you don't have anything to add to the discussion but would like to be involved, just drop me a note and I'll include you on the poll.

Thanks again everyone and have a good weekend.

Britton

On Wed, Nov 23, 2016 at 8:19 AM, John ZuHone wrote:

...
I’m +1 on this and I agree with Nathan’s comments. In particular, I think that we shouldn’t bundle them all into one package but probably split them off into separate packages. This requires more work on the behalf of the maintainers of those modules, of course.

In addition to keeping in clump_finder, particle_trajectories should also probably be left in, and perhaps be turned into some kind of time series object or something else.

I’m happy to help out—in particular I can split off sunyaev_zeldovich and ppv_cube and spin them off as separate packages. photon_simulator is already deprecated in favor of pyXSIM.

All that spectral_integrator does is generate fields, so perhaps its functionality needs to be merged into the astro fields part somehow and left inside yt itself.

On Nov 22, 2016, at 8:54 PM, Nathan Goldbaum wrote:

Overall I think this is a good idea. I have a couple critiques about the mechanics that I'll reply inline about. I think it makes sense to separate out all of the astro-specific analysis modules.

On Tue, Nov 22, 2016 at 8:37 PM, Britton Smith wrote:

...
Greetings,

I would like to open a discussion on the idea of moving most of yt’s analysis modules into an external yt extensions package. For ease of reading, I will separate this email into what this would mean for the code, what I see are the pros, cons, logistics, and open questions. I would very much appreciate comment on this.

What this means If we did this, most of the contents of yt/analysis_modules would be moved into a repository named something like yt_astro_analysis. Installing this would be an option in the install script and would likely also be pip installable. Imports would change from from yt.analysis_modules.halo_analysis.api import HaloCatalog to from yt.extensions.astro_analysis.halo_analysis.api import HaloCatalog

After creating yt_astro_analysis, there would be a period where the old analysis_modules would still exist but be deprecated before being removed at some point down the road.

Pros

- Almost all of the current analysis modules are specific to astrophysics. As we continue to make the core functionality of yt less astro specific, it’s not clear how to make room for non-astro analysis modules. Putting everything together under analysis_modules will make navigation and documentation messy and confusing. This would also significantly slim down the yt codebase. - Many of the tools in analysis_modules are very old and are in need of API-breaking refactor. Some of these, like two_point_functions, did not make the jump from yt-2 to yt-3 and are no longer usable. Many tools no longer have a champion, someone interested in using, maintaining, and updating them as yt’s core functionality develops and changes. Moving analysis_modules from yt decouples them from yt’s release cycle, allowing interested parties to make updates and releases on a separate, likely shorter timescale. Some analysis_modules may even be better suited to be moved into other extensions that are actively developed, such as the case of the AbsorptionSpectrum with the Trident project. - Similar to the point above, yt releases would not be slowed by the need to update all of the championless modules. Individual analysis modules can be tied to specific stable releases of yt and so assured to work there.

Cons

- This will take a non-zero amount of work. See below for a summary of the primary tasks. - There are some outstanding logistical questions. See below. - Not having yt and analysis modules explicitly tied to the same codebase/releases could result in analysis tools falling behind and out of date. - The disruption and need to alter scripts could irritate and alienate users.

Proposed progression This is roughly how this would happen. Here is a table with all existing analysis modules, their status, and potential future: https://goo.gl/HZykQA

1. Create yt_astro_analysis repo with all analysis modules that are to be moved. Add an entry to the extensions page on yt-project.org. Make it installed by default in the install script, at least at first.

Should there only be one repo? If we're going to do this, it might help future maintainability to have one repo per analysis module. That way maintainers that only care about one analysis module don't need to worry about changes to other modules.

I also don't think the clump finder in particular should be split out into its own package. That one isn't astro-specific and it might make sense to try to integrate it more deeply with the core of yt.

...
1. Deprecate all moved modules in yt. 2. After some time, remove deprecated modules from yt.

Open issues Here are some logistics and questions that still need to be worked out.

- Can we set up separate testing for yt_astro_analysis? Would maintaining this be a pain?

If the tests don't require large datasets and are relatively quick, this might be a good opportunity to explore bitbucket pipelines. There's also drone.io or we can use the yt testing infrastructure. Since I'm not the maintainer of the yt testing infra I can't speak to whether it's ok to expand it to more packages.

...
- How/where would the documentation be hosted?

Readthedocs will likely be sufficient. If we care about inline code examples or whatnot, then we can explore using the yt testing infrastructure, with the same caveats as above.

...
- How would we move the analysis modules source code and maintain its revision history?

This can be done relatively straightforwardly using the convert extension for mercurial and a filemap.

...
Questions to yt-dev

- Are you +/-1 on this? Any other comments?

+1

...
- Changes to the analysis_modules spreadsheet (https://goo.gl/HZykQA )? - Interested in helping out with this? If this happens, I propose anyone interested meets for a hangout to discuss how to proceed.

I'm happy to help.

...
Thanks for reading!

Britton

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

Kacper Kowalik

23 Nov 23 Nov

4:52 p.m.

On 11/22/2016 07:54 PM, Nathan Goldbaum wrote:

...

Overall I think this is a good idea. I have a couple critiques about the mechanics that I'll reply inline about. I think it makes sense to separate out all of the astro-specific analysis modules.

On Tue, Nov 22, 2016 at 8:37 PM, Britton Smith wrote:

...
Greetings,

I would like to open a discussion on the idea of moving most of yt’s analysis modules into an external yt extensions package. For ease of reading, I will separate this email into what this would mean for the code, what I see are the pros, cons, logistics, and open questions. I would very much appreciate comment on this.

What this means

If we did this, most of the contents of yt/analysis_modules would be moved into a repository named something like yt_astro_analysis. Installing this would be an option in the install script and would likely also be pip installable. Imports would change from

from yt.analysis_modules.halo_analysis.api import HaloCatalog

to

from yt.extensions.astro_analysis.halo_analysis.api import HaloCatalog

After creating yt_astro_analysis, there would be a period where the old analysis_modules would still exist but be deprecated before being removed at some point down the road.

Pros

-

Almost all of the current analysis modules are specific to astrophysics. As we continue to make the core functionality of yt less astro specific, it’s not clear how to make room for non-astro analysis modules. Putting everything together under analysis_modules will make navigation and documentation messy and confusing. This would also significantly slim down the yt codebase. -

Many of the tools in analysis_modules are very old and are in need of API-breaking refactor. Some of these, like two_point_functions, did not make the jump from yt-2 to yt-3 and are no longer usable. Many tools no longer have a champion, someone interested in using, maintaining, and updating them as yt’s core functionality develops and changes. Moving analysis_modules from yt decouples them from yt’s release cycle, allowing interested parties to make updates and releases on a separate, likely shorter timescale. Some analysis_modules may even be better suited to be moved into other extensions that are actively developed, such as the case of the AbsorptionSpectrum with the Trident project. -

Similar to the point above, yt releases would not be slowed by the need to update all of the championless modules. Individual analysis modules can be tied to specific stable releases of yt and so assured to work there.

Cons

-

This will take a non-zero amount of work. See below for a summary of the primary tasks. -

There are some outstanding logistical questions. See below. -

Not having yt and analysis modules explicitly tied to the same codebase/releases could result in analysis tools falling behind and out of date. -

The disruption and need to alter scripts could irritate and alienate users.

Proposed progression

This is roughly how this would happen. Here is a table with all existing analysis modules, their status, and potential future: https://goo.gl/HZykQA

1.

Create yt_astro_analysis repo with all analysis modules that are to be moved. Add an entry to the extensions page on yt-project.org. Make it installed by default in the install script, at least at first.

Should there only be one repo? If we're going to do this, it might help future maintainability to have one repo per analysis module. That way maintainers that only care about one analysis module don't need to worry about changes to other modules.

I also don't think the clump finder in particular should be split out into its own package. That one isn't astro-specific and it might make sense to try to integrate it more deeply with the core of yt.

...
1.

Deprecate all moved modules in yt. 2.

After some time, remove deprecated modules from yt.

Open issues

Here are some logistics and questions that still need to be worked out.

-

Can we set up separate testing for yt_astro_analysis? Would maintaining this be a pain?

If the tests don't require large datasets and are relatively quick, this might be a good opportunity to explore bitbucket pipelines. There's also drone.io or we can use the yt testing infrastructure. Since I'm not the maintainer of the yt testing infra I can't speak to whether it's ok to expand it to more packages.

It's certainly possible, not sure how easy it'd be with current setup though. However if it's too hard or it requires too much interaction with the said maintainer it's a bug and we should fix it.

...

...
-

How/where would the documentation be hosted?

Readthedocs will likely be sufficient. If we care about inline code examples or whatnot, then we can explore using the yt testing infrastructure, with the same caveats as above.

Same as above :)

...

...
-

How would we move the analysis modules source code and maintain its revision history?

This can be done relatively straightforwardly using the convert extension for mercurial and a filemap.

...
Questions to yt-dev

-

Are you +/-1 on this? Any other comments?

+1

ditto

...

...
-

Changes to the analysis_modules spreadsheet (https://goo.gl/HZykQA)? -

Interested in helping out with this? If this happens, I propose anyone interested meets for a hangout to discuss how to proceed.

I'm happy to help.

...
Thanks for reading!

Britton

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

2698

Age (days ago)

2710

Last active (days ago)

List overview

Download

9 comments

6 participants

participants (6)

Bili Dong
Britton Smith
John ZuHone
Kacper Kowalik
Matthew Turk
Nathan Goldbaum

the future of analysis_modules

tags

participants (6)