[Neuroimaging] Planning for data formats - upcoming journal club

Wed Nov 10 11:34:09 EST 2021

Dear all

I am new to writing to this list but I simply wanted to bring a couple of
pointers to related efforts in microscopy. Apologies if this is redundant.

Open Microscopy Environment. They started with a data model, in XML. The
data model covers data and metadata so initially you could write data and
metadata in pure XML. However, a more practical solution involved TIFF with
a metadata encoded jn XML embedded in the headers. Now, and this is the
interesting part, a larger consortium (and here is the interesting part) is
working on a Next Generation File Format, that does not use HDF5 due to
parallel read/write problems, but Zarr. My point being that choosing HDF5
now might has some implications on data access down the road.

The more the communities connect the more we can benefit from each other’s
previous work.

https://ngff.openmicroscopy.org/latest/

https://forum.image.sc/tag/ome-ngff

https://quarep.org/working-groups/wg-7-metadata/

Sincerely
Thomas

Thomas

On Wed, Nov 10, 2021 at 5:41 AM Guillaume Flandin <g.flandin at ucl.ac.uk>
wrote:

> Hi All,
>
> Also interested in attending, time permitting. There has been many
> discussions on this topic over the years across several platforms - one
> of them is captured in Anderson's document but I could also remember
> this one:
>   https://github.com/bids-standard/bids-specification/issues/197
> Compression was also discussed in the early days of BIDS but it seems it
> was a group email exchange with no public archive.
>
> Best,
> Guillaume.
>
> On 09/11/2021 18:32, Ariel Rokem wrote:
> > ⚠ Caution: External sender
> >
> >
> > Hi Matthew and all,
> >
> > Sounds interesting! At some point, we might want to discuss Francois
> > Rheault's (and others') efforts to update the formats for tractography
> > data (https://github.com/frheault/tractography_file_format/).
> >
> > Thanks for organizing!
> > Ariel
> > On Tue, Nov 9, 2021 at 3:32 AM Matthew Brett <matthew.brett at gmail.com
> > <mailto:matthew.brett at gmail.com>> wrote:
> >
> >     Hi,
> >
> >     Summary:  we are planning to do a journal club to discuss new
> >     neuroimaging data formats.  Please let me (Matthew) know if you want
> >     to be included in the scheduling for the journal club.  Of course,
> >     once we've got a time, anyone can join the journal club Zoom.
> >
> >     Detail:
> >
> >     As part of the Chan-Zuckerberg grant work, we are thinking about new
> >     formats for using and sharing neuroimaging data.
> >
> >     As you can see in the grant documents:
> >
> >     https://github.com/matthew-brett/czi-nibabel
> >
> >     we've been thinking about HDF5, but further research (e.g. [1]) has
> >     given us some pause for reflection.  Meanwhile, the astronomy
> >     community has done some serious thinking about this, and written this
> >     up in a paper proposing a new astronomy format:
> >
> >     Greenfield, M. Droettboom, E. Bray,
> >     ASDF: A new data format for astronomy,
> >     Astronomy and Computing,
> >     Volume 12,
> >     2015,
> >     Pages 240-251,
> >     ISSN 2213-1337,
> >     https://doi.org/10.1016/j.ascom.2015.06.004
> >     <
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoi.org%2F10.1016%2Fj.ascom.2015.06.004&data=04%7C01%7Cg.flandin%40ucl.ac.uk%7C5ed1afff99ca460e7dc008d9a3b04a6c%7C1faf88fea9984c5b93c9210a11d9a5c2%7C0%7C0%7C637720800862344941%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Z%2FHoOKl4tljjjzQ5IWnHjwMH65eunYlTNIdbIK6CTXI%3D&reserved=0
> >.
> >     (https://www.sciencedirect.com/science/article/pii/S2213133715000645
> >     <
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.sciencedirect.com%2Fscience%2Farticle%2Fpii%2FS2213133715000645&data=04%7C01%7Cg.flandin%40ucl.ac.uk%7C5ed1afff99ca460e7dc008d9a3b04a6c%7C1faf88fea9984c5b93c9210a11d9a5c2%7C0%7C0%7C637720800862354932%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=ialaMuABzvWaOXvlBj0KcpzRYYmXrjyLvW8Lm2CnoHc%3D&reserved=0
> >)
> >
> >     Both the discussion and the format seem like a very good starting
> >     point for our own thinking about data formats, so we'd like to have a
> >     journal club to discuss it.
> >
> >     I (Matthew) am going to organize this via Calendly, and I've made a
> >     Caldendly account for Nipy-related stuff, but, if you want to have a
> >     say in when we do the journal club, please let me know so I can add
> >     you to that account.  I'll publish the Calendly link in a day or so.
> >
> >     Cheers,
> >
> >     Matthew
> >
> >     [1] https://cyrille.rossant.net/moving-away-hdf5/
> >     <
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcyrille.rossant.net%2Fmoving-away-hdf5%2F&data=04%7C01%7Cg.flandin%40ucl.ac.uk%7C5ed1afff99ca460e7dc008d9a3b04a6c%7C1faf88fea9984c5b93c9210a11d9a5c2%7C0%7C0%7C637720800862354932%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=afzzkSku7KmL1XetF6NaTsmxDDQ8LR5ebFe0q1oov5c%3D&reserved=0
> >
> >     _______________________________________________
> >     Neuroimaging mailing list
> >     Neuroimaging at python.org <mailto:Neuroimaging at python.org>
> >     https://mail.python.org/mailman/listinfo/neuroimaging
> >     <
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.python.org%2Fmailman%2Flistinfo%2Fneuroimaging&data=04%7C01%7Cg.flandin%40ucl.ac.uk%7C5ed1afff99ca460e7dc008d9a3b04a6c%7C1faf88fea9984c5b93c9210a11d9a5c2%7C0%7C0%7C637720800862364927%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=54jpAtVghfeD5eVfeGJvpcGwgn84E5l1rtvk4CXXV6U%3D&reserved=0
> >
> >
> >
> > _______________________________________________
> > Neuroimaging mailing list
> > Neuroimaging at python.org
> > https://mail.python.org/mailman/listinfo/neuroimaging
> >
> _______________________________________________
> Neuroimaging mailing list
> Neuroimaging at python.org
> https://mail.python.org/mailman/listinfo/neuroimaging
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/neuroimaging/attachments/20211110/21ccd8ce/attachment-0001.html>