Is my program appropriate for Scikit-Image?
I am a developer at Johns Hopkins University's Neurodata Lab and am writing a pure-python package called ARDENT that does fairly sophisticated 3D image registration. The core algorithm is based on this paper that our group produced in 2005, which has over 1200 citations and has become a standard in the field of medical image registration: Computing Large Deformation Metric Mappings via Geodesic Flows of Diffeomorphisms, available at https://link.springer.com/article/10.1023/B:VISI.0000043755.93987.aa ARDENT works across multiple modalities (images generated by different methods, e.g. fMRI, light microscopy, etc.), and although it has convenient presets written with neuroscience applications in mind, it is fully general to all manner of image registration tasks. I would ideally like to start a conversation but I have a few specific questions. -- Do you think Scikit-Image is the place for a program like the one I have described? -- Would it make more sense for our package to be incorporated into an existing Scikit-Image framework like the Transform module or included as an insular, standalone component? It seems like there is some functional overlap with Transform.estimate_transform so I suspect this might be the most logical place for ARDENT. -- Would Scikit-Image would be comfortable with PyTorch as a dependency or as an optional dependency? The core LDDMM implementation that does ARDENT's heavy lifting utilizes PyTorch for GPU acceleration, but it could feasibly be rewritten without PyTorch. Thank you very much, Devin Crowley JHU Research Assistant
Hi Devin, I know Juan has recently expressed interest in expanding registration capabilities in scikit-image and it sounds like a well established method. How different is this algorithm from the symmetric diffeomorphic normalization (SyN) registration implemented in ANTs? That one currently has a python (+cython) based implementation in the DiPy software package, although it does not have the GPU support you mentioned. I would take a look at https://github.com/scikit-image/scikit-image/issues/3810 for some initial discussion related to this type of thing. Another primary concern for scikit-image is the feasibility of long-term maintenance, so we are reluctant to add things that add new dependencies or substantially complicate the build process. It would probably be most feasible to adopt a CPU-only variant that does not introduce new dependencies to scikit-image. Perhaps a GPU-specific backend could reside in a separate repository? It may be that at some point in the future, with things like NumPy's NEP-18, that GPU array support will become more feasible within scikit-image itself, but I don't think we are quite there yet. - Greg On Tue, Jul 2, 2019 at 12:00 PM Devin Crowley <devin.g.crowley@gmail.com> wrote:
I am a developer at Johns Hopkins University's Neurodata Lab and am writing a pure-python package called ARDENT that does fairly sophisticated 3D image registration. The core algorithm is based on this paper that our group produced in 2005, which has over 1200 citations and has become a standard in the field of medical image registration: Computing Large Deformation Metric Mappings via Geodesic Flows of Diffeomorphisms, available at https://link.springer.com/article/10.1023/B:VISI.0000043755.93987.aa
ARDENT works across multiple modalities (images generated by different methods, e.g. fMRI, light microscopy, etc.), and although it has convenient presets written with neuroscience applications in mind, it is fully general to all manner of image registration tasks.
I would ideally like to start a conversation but I have a few specific questions.
-- Do you think Scikit-Image is the place for a program like the one I have described?
-- Would it make more sense for our package to be incorporated into an existing Scikit-Image framework like the Transform module or included as an insular, standalone component? It seems like there is some functional overlap with Transform.estimate_transform so I suspect this might be the most logical place for ARDENT.
-- Would Scikit-Image would be comfortable with PyTorch as a dependency or as an optional dependency? The core LDDMM implementation that does ARDENT's heavy lifting utilizes PyTorch for GPU acceleration, but it could feasibly be rewritten without PyTorch.
Thank you very much, Devin Crowley JHU Research Assistant _______________________________________________ scikit-image mailing list -- scikit-image@python.org To unsubscribe send an email to scikit-image-leave@python.org
Hi Devin, and thanks for getting in touch about this! To firm up what Greg said, I don't think there's any chance that we'll take on PyTorch as a dependency in the near future. We are actually aiming to *reduce* our dependencies at this point! Having said that, we are definitely 100% interested in expanding scikit-image's registration capabilities, as also pointed out by Greg! And this algorithm, being standard, is definitely a good fit for the library. I'd recommend that you comment on the issue [1]. In addition to code maintenance, another concern we have is to provide a SciPy-thonic API, which means generally functions with simple inputs and outputs (arrays, tuples, lists, dicts, and strings), rather than long-lived, public-facing objects. The more sophisticated the method, the more challenging it is to develop such an API. However, I think it could live quite comfortably in the `geometric_transform` API proposed in #3810. Looking forward to hearing more about this! Juan. .. [1] https://github.com/scikit-image/scikit-image/issues/3810 On Wed, 3 Jul 2019, at 3:30 AM, Gregory Lee wrote:
Hi Devin,
I know Juan has recently expressed interest in expanding registration capabilities in scikit-image and it sounds like a well established method. How different is this algorithm from the symmetric diffeomorphic normalization (SyN) registration implemented in ANTs? That one currently has a python (+cython) based implementation in the DiPy software package, although it does not have the GPU support you mentioned.
I would take a look at https://github.com/scikit-image/scikit-image/issues/3810 for some initial discussion related to this type of thing.
Another primary concern for scikit-image is the feasibility of long-term maintenance, so we are reluctant to add things that add new dependencies or substantially complicate the build process. It would probably be most feasible to adopt a CPU-only variant that does not introduce new dependencies to scikit-image. Perhaps a GPU-specific backend could reside in a separate repository? It may be that at some point in the future, with things like NumPy's NEP-18, that GPU array support will become more feasible within scikit-image itself, but I don't think we are quite there yet.
- Greg
On Tue, Jul 2, 2019 at 12:00 PM Devin Crowley <devin.g.crowley@gmail.com> wrote:
I am a developer at Johns Hopkins University's Neurodata Lab and am writing a pure-python package called ARDENT that does fairly sophisticated 3D image registration. The core algorithm is based on this paper that our group produced in 2005, which has over 1200 citations and has become a standard in the field of medical image registration: Computing Large Deformation Metric Mappings via Geodesic Flows of Diffeomorphisms, available at https://link.springer.com/article/10.1023/B:VISI.0000043755.93987.aa
ARDENT works across multiple modalities (images generated by different methods, e.g. fMRI, light microscopy, etc.), and although it has convenient presets written with neuroscience applications in mind, it is fully general to all manner of image registration tasks.
I would ideally like to start a conversation but I have a few specific questions.
-- Do you think Scikit-Image is the place for a program like the one I have described?
-- Would it make more sense for our package to be incorporated into an existing Scikit-Image framework like the Transform module or included as an insular, standalone component? It seems like there is some functional overlap with Transform.estimate_transform so I suspect this might be the most logical place for ARDENT.
-- Would Scikit-Image would be comfortable with PyTorch as a dependency or as an optional dependency? The core LDDMM implementation that does ARDENT's heavy lifting utilizes PyTorch for GPU acceleration, but it could feasibly be rewritten without PyTorch.
Thank you very much, Devin Crowley JHU Research Assistant _______________________________________________ scikit-image mailing list -- scikit-image@python.org To unsubscribe send an email to scikit-image-leave@python.org
scikit-image mailing list -- scikit-image@python.org To unsubscribe send an email to scikit-image-leave@python.org
The algorithm is related, in that it uses a transformation generated from integrating a velocity flow field. But it is different in that it is not symmetric, which allows (for example) a target image to be noisy or damaged relative to an atlas image. The Ants Syn paper ( https://doi.org/10.1016/j.media.2007.06.004) is based on the formulation we referenced (LDDMM), meaning LDDMM is a bit simpler. We investigated this difference in the presence of noise here (https://doi.org/10.1090/qam/1527) and demonstrated that the symmetric version has poorer performance. The second thing is that our method does not require ITK, a large and difficult codebase. All substantial computations are either interpolation or FFT. Because of this, we don't rely on any functionality from pytorch other than FFT and interpolation, which are both available in scipy. Pytorch is only used because it makes GPU easy. We'd like to contribute, and we may have a small team of undergraduate biomedical engineering students who can contribute starting in the Fall. Would reaching out to Juan directly be the best way to get started? Thanks for getting back to me! Devin P.S. Hope you enjoyed your 4th of July! On Tue, Jul 2, 2019 at 1:30 PM Gregory Lee <grlee77@gmail.com> wrote:
Hi Devin,
I know Juan has recently expressed interest in expanding registration capabilities in scikit-image and it sounds like a well established method. How different is this algorithm from the symmetric diffeomorphic normalization (SyN) registration implemented in ANTs? That one currently has a python (+cython) based implementation in the DiPy software package, although it does not have the GPU support you mentioned.
I would take a look at https://github.com/scikit-image/scikit-image/issues/3810 for some initial discussion related to this type of thing.
Another primary concern for scikit-image is the feasibility of long-term maintenance, so we are reluctant to add things that add new dependencies or substantially complicate the build process. It would probably be most feasible to adopt a CPU-only variant that does not introduce new dependencies to scikit-image. Perhaps a GPU-specific backend could reside in a separate repository? It may be that at some point in the future, with things like NumPy's NEP-18, that GPU array support will become more feasible within scikit-image itself, but I don't think we are quite there yet.
- Greg
On Tue, Jul 2, 2019 at 12:00 PM Devin Crowley <devin.g.crowley@gmail.com> wrote:
I am a developer at Johns Hopkins University's Neurodata Lab and am writing a pure-python package called ARDENT that does fairly sophisticated 3D image registration. The core algorithm is based on this paper that our group produced in 2005, which has over 1200 citations and has become a standard in the field of medical image registration: Computing Large Deformation Metric Mappings via Geodesic Flows of Diffeomorphisms, available at https://link.springer.com/article/10.1023/B:VISI.0000043755.93987.aa
ARDENT works across multiple modalities (images generated by different methods, e.g. fMRI, light microscopy, etc.), and although it has convenient presets written with neuroscience applications in mind, it is fully general to all manner of image registration tasks.
I would ideally like to start a conversation but I have a few specific questions.
-- Do you think Scikit-Image is the place for a program like the one I have described?
-- Would it make more sense for our package to be incorporated into an existing Scikit-Image framework like the Transform module or included as an insular, standalone component? It seems like there is some functional overlap with Transform.estimate_transform so I suspect this might be the most logical place for ARDENT.
-- Would Scikit-Image would be comfortable with PyTorch as a dependency or as an optional dependency? The core LDDMM implementation that does ARDENT's heavy lifting utilizes PyTorch for GPU acceleration, but it could feasibly be rewritten without PyTorch.
Thank you very much, Devin Crowley JHU Research Assistant _______________________________________________ scikit-image mailing list -- scikit-image@python.org To unsubscribe send an email to scikit-image-leave@python.org
_______________________________________________ scikit-image mailing list -- scikit-image@python.org To unsubscribe send an email to scikit-image-leave@python.org
Hi Devin, On Fri, 05 Jul 2019 10:53:50 -0400, Devin Crowley wrote:
The second thing is that our method does not require ITK, a large and difficult codebase. All substantial computations are either interpolation or FFT. Because of this, we don't rely on any functionality from pytorch other than FFT and interpolation, which are both available in scipy. Pytorch is only used because it makes GPU easy.
We'd like to contribute, and we may have a small team of undergraduate biomedical engineering students who can contribute starting in the Fall. Would reaching out to Juan directly be the best way to get started?
Fantastic! As others mentioned, we are very interested in this functionality, and would enjoy working with your team to integrate your contribution. You can read more about contributing to scikit-image here: https://scikit-image.org/docs/stable/contribute.html The long and short of it is that you can proceed by making one or more pull requests to the repository, where it will be reviewed by the development team. It sometimes takes a while of iterative development and conversation to get everything into shape, but hopefully since you already have working code this won't take too long. Please take a look at the references given by Juan also: we would appreciate feedback on the registration interfaces proposed. Best regards, Stéfan
Excellent, it sounds like this will be a good home for our code. Some broad strokes moving forward: Our plan is to first complete the ARDENT package for a neurodata workshop starting on August 19. For the rest of August we will be refining it based on feedback. At the beginning of September we will begin refactoring our implementation of the underlying algorithm to be in compliance with Scikit-Image dependencies. We may have a team of biomedical engineering students contributing at the semester's onset that month. Once this is complete, we will reach out again for input on refactoring the interface to adhere to the Scikit-Image style. Looking forward to working with you! - Devin On Fri, Jul 5, 2019 at 3:40 PM Stefan van der Walt <stefanv@berkeley.edu> wrote:
Hi Devin,
The second thing is that our method does not require ITK, a large and difficult codebase. All substantial computations are either interpolation or FFT. Because of this, we don't rely on any functionality from pytorch other
On Fri, 05 Jul 2019 10:53:50 -0400, Devin Crowley wrote: than
FFT and interpolation, which are both available in scipy. Pytorch is only used because it makes GPU easy.
We'd like to contribute, and we may have a small team of undergraduate biomedical engineering students who can contribute starting in the Fall. Would reaching out to Juan directly be the best way to get started?
Fantastic! As others mentioned, we are very interested in this functionality, and would enjoy working with your team to integrate your contribution.
You can read more about contributing to scikit-image here: https://scikit-image.org/docs/stable/contribute.html
The long and short of it is that you can proceed by making one or more pull requests to the repository, where it will be reviewed by the development team. It sometimes takes a while of iterative development and conversation to get everything into shape, but hopefully since you already have working code this won't take too long.
Please take a look at the references given by Juan also: we would appreciate feedback on the registration interfaces proposed.
Best regards, Stéfan _______________________________________________ scikit-image mailing list -- scikit-image@python.org To unsubscribe send an email to scikit-image-leave@python.org
participants (4)
-
Devin Crowley -
Gregory Lee -
Juan Nunez-Iglesias -
Stefan van der Walt