Reading Array Data
Hi All, I'm trying to get YT to read a vector which is stored as part of particle structure in Enzo (a vector in the active particle structure). The data looks OK when I view the raw HDF5 data using hdfview. I find that in some snapshots YT can read the data but not in others. If there is more than one particle things get worse for sure. It could be that when writing to the HDF5 file I haven't put the data in a format that is compatible with YT. I've attached a simple script that will fall over when trying to read the data and a link to the enzo output (size = 12 GBs). The active particles are in files DD0120.cpu0022 (Grid 3072), DD0120.cpu0023 (Grid 3076) and DD0014 (Grid 3080). YT falls over in the io_handler but it's not obvious to me whether I need to modify that code or the format that I write the data to the HDF5 file in. Any help/pointers would be much appreciated. Cheers, John Link to dataset: https://www.dropbox.com/sh/o6acuvia1sqyp7y/AAC0lEI5lZ6bGa0tP2-G1jLea?dl=0
On Thu, Jan 11, 2018 at 3:43 AM, John Regan <johnanthonyregan@gmail.com> wrote:
Hi All,
I'm trying to get YT to read a vector which is stored as part of particle structure in Enzo (a vector in the active particle structure). The data looks OK when I view the raw HDF5 data using hdfview. I find that in some snapshots YT can read the data but not in others. If there is more than one particle things get worse for sure.
It could be that when writing to the HDF5 file I haven't put the data in a format that is compatible with YT.
I've attached a simple script that will fall over when trying to read the data and a link to the enzo output (size = 12 GBs). The active particles are in files DD0120.cpu0022 (Grid 3072), DD0120.cpu0023 (Grid 3076) and DD0014 (Grid 3080). YT falls over in the io_handler but it's not obvious to me whether I need to modify that code or the format that I write the data to the HDF5 file in.
It's hard to say if there's an issue on the Enzo side or the yt side without a copy of the traceback you are seeing or a way to reproduce the issue. I'm also not sure what your data format looks like offhand. If I remember correctly, you added support for vector active particle attributes to your fork of Enzo and it's entirely possible that there needs to be updates on the yt side to support reading the data from your fork. I suspect the place you need to modify in yt is here: https://github.com/yt-project/yt/blob/eb3358c622bc047200b9bc6a954dde e8f59a0e07/yt/frontends/enzo/io.py#L92-L113
Any help/pointers would be much appreciated.
Cheers, John
Link to dataset: https://www.dropbox.com/sh/o6acuvia1sqyp7y/AAC0lEI5lZ6bGa0tP2-G1jLea?dl=0
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
Hi Nathan , Thanks for the email. If I sent you the exact hdf5 file (i.e. the .cpuXXXX file) would that would and the traceback? You could see the format the array gets written then.... It wasn't clear to me from reading through the YT reader would work. Cheers, John On Thu, Jan 11, 2018 at 3:43 PM, Nathan Goldbaum <nathan12343@gmail.com> wrote:
On Thu, Jan 11, 2018 at 3:43 AM, John Regan <johnanthonyregan@gmail.com> wrote:
Hi All,
I'm trying to get YT to read a vector which is stored as part of particle structure in Enzo (a vector in the active particle structure). The data looks OK when I view the raw HDF5 data using hdfview. I find that in some snapshots YT can read the data but not in others. If there is more than one particle things get worse for sure.
It could be that when writing to the HDF5 file I haven't put the data in a format that is compatible with YT.
I've attached a simple script that will fall over when trying to read the data and a link to the enzo output (size = 12 GBs). The active particles are in files DD0120.cpu0022 (Grid 3072), DD0120.cpu0023 (Grid 3076) and DD0014 (Grid 3080). YT falls over in the io_handler but it's not obvious to me whether I need to modify that code or the format that I write the data to the HDF5 file in.
It's hard to say if there's an issue on the Enzo side or the yt side without a copy of the traceback you are seeing or a way to reproduce the issue. I'm also not sure what your data format looks like offhand. If I remember correctly, you added support for vector active particle attributes to your fork of Enzo and it's entirely possible that there needs to be updates on the yt side to support reading the data from your fork.
I suspect the place you need to modify in yt is here:
https://github.com/yt-project/yt/blob/eb3358c622bc047200b9bc 6a954ddee8f59a0e07/yt/frontends/enzo/io.py#L92-L113
Any help/pointers would be much appreciated.
Cheers, John
Link to dataset: https://www.dropbox.com/sh/o6acuvia1sqyp7y/AAC0lEI5lZ6bGa0tP2-G1jLea?dl=0
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
It would probably be best to share the full output. I might just need to see the traceback too, it's hard to say what will help without actually being able to see it :) Is there any chance you can make a smaller output with similar active particle fields? I can also download the 12 GB if you make it publicly visible online somewhere. On Thu, Jan 11, 2018 at 12:42 PM, John Regan <johnanthonyregan@gmail.com> wrote:
Hi Nathan ,
Thanks for the email. If I sent you the exact hdf5 file (i.e. the .cpuXXXX file) would that would and the traceback? You could see the format the array gets written then....
It wasn't clear to me from reading through the YT reader would work.
Cheers, John
On Thu, Jan 11, 2018 at 3:43 PM, Nathan Goldbaum <nathan12343@gmail.com> wrote:
On Thu, Jan 11, 2018 at 3:43 AM, John Regan <johnanthonyregan@gmail.com> wrote:
Hi All,
I'm trying to get YT to read a vector which is stored as part of particle structure in Enzo (a vector in the active particle structure). The data looks OK when I view the raw HDF5 data using hdfview. I find that in some snapshots YT can read the data but not in others. If there is more than one particle things get worse for sure.
It could be that when writing to the HDF5 file I haven't put the data in a format that is compatible with YT.
I've attached a simple script that will fall over when trying to read the data and a link to the enzo output (size = 12 GBs). The active particles are in files DD0120.cpu0022 (Grid 3072), DD0120.cpu0023 (Grid 3076) and DD0014 (Grid 3080). YT falls over in the io_handler but it's not obvious to me whether I need to modify that code or the format that I write the data to the HDF5 file in.
It's hard to say if there's an issue on the Enzo side or the yt side without a copy of the traceback you are seeing or a way to reproduce the issue. I'm also not sure what your data format looks like offhand. If I remember correctly, you added support for vector active particle attributes to your fork of Enzo and it's entirely possible that there needs to be updates on the yt side to support reading the data from your fork.
I suspect the place you need to modify in yt is here:
https://github.com/yt-project/yt/blob/eb3358c622bc047200b9bc 6a954ddee8f59a0e07/yt/frontends/enzo/io.py#L92-L113
Any help/pointers would be much appreciated.
Cheers, John
Link to dataset: https://www.dropbox.com/sh/o6acuvia1sqyp7y/AAC0lEI5lZ6bGa0tP 2-G1jLea?dl=0
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
Hi Nathan, The 12 GB folder should be available at the link I sent in the first email. Here it is: https://www.dropbox.com/sh/o6acuvia1sqyp7y/AAC0lEI5lZ6bGa0tP 2-G1jLea?dl=0 If you run the script I sent it will fail pretty quickly with a traceback. Let me know if that works. On Thu, Jan 11, 2018 at 6:50 PM, Nathan Goldbaum <nathan12343@gmail.com> wrote:
It would probably be best to share the full output. I might just need to see the traceback too, it's hard to say what will help without actually being able to see it :)
Is there any chance you can make a smaller output with similar active particle fields? I can also download the 12 GB if you make it publicly visible online somewhere.
On Thu, Jan 11, 2018 at 12:42 PM, John Regan <johnanthonyregan@gmail.com> wrote:
Hi Nathan ,
Thanks for the email. If I sent you the exact hdf5 file (i.e. the .cpuXXXX file) would that would and the traceback? You could see the format the array gets written then....
It wasn't clear to me from reading through the YT reader would work.
Cheers, John
On Thu, Jan 11, 2018 at 3:43 PM, Nathan Goldbaum <nathan12343@gmail.com> wrote:
On Thu, Jan 11, 2018 at 3:43 AM, John Regan <johnanthonyregan@gmail.com> wrote:
Hi All,
I'm trying to get YT to read a vector which is stored as part of particle structure in Enzo (a vector in the active particle structure). The data looks OK when I view the raw HDF5 data using hdfview. I find that in some snapshots YT can read the data but not in others. If there is more than one particle things get worse for sure.
It could be that when writing to the HDF5 file I haven't put the data in a format that is compatible with YT.
I've attached a simple script that will fall over when trying to read the data and a link to the enzo output (size = 12 GBs). The active particles are in files DD0120.cpu0022 (Grid 3072), DD0120.cpu0023 (Grid 3076) and DD0014 (Grid 3080). YT falls over in the io_handler but it's not obvious to me whether I need to modify that code or the format that I write the data to the HDF5 file in.
It's hard to say if there's an issue on the Enzo side or the yt side without a copy of the traceback you are seeing or a way to reproduce the issue. I'm also not sure what your data format looks like offhand. If I remember correctly, you added support for vector active particle attributes to your fork of Enzo and it's entirely possible that there needs to be updates on the yt side to support reading the data from your fork.
I suspect the place you need to modify in yt is here:
https://github.com/yt-project/yt/blob/eb3358c622bc047200b9bc 6a954ddee8f59a0e07/yt/frontends/enzo/io.py#L92-L113
Any help/pointers would be much appreciated.
Cheers, John
Link to dataset: https://www.dropbox.com/sh/o6acuvia1sqyp7y/AAC0lEI5lZ6bGa0tP 2-G1jLea?dl=0
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
Ah, apologies, missed that in your first e-mail. On Thu, Jan 11, 2018 at 1:24 PM, John Regan <johnanthonyregan@gmail.com> wrote:
Hi Nathan,
The 12 GB folder should be available at the link I sent in the first email. Here it is: https://www.dropbox.com/sh/o6a cuvia1sqyp7y/AAC0lEI5lZ6bGa0tP2-G1jLea?dl=0
If you run the script I sent it will fail pretty quickly with a traceback. Let me know if that works.
On Thu, Jan 11, 2018 at 6:50 PM, Nathan Goldbaum <nathan12343@gmail.com> wrote:
It would probably be best to share the full output. I might just need to see the traceback too, it's hard to say what will help without actually being able to see it :)
Is there any chance you can make a smaller output with similar active particle fields? I can also download the 12 GB if you make it publicly visible online somewhere.
On Thu, Jan 11, 2018 at 12:42 PM, John Regan <johnanthonyregan@gmail.com> wrote:
Hi Nathan ,
Thanks for the email. If I sent you the exact hdf5 file (i.e. the .cpuXXXX file) would that would and the traceback? You could see the format the array gets written then....
It wasn't clear to me from reading through the YT reader would work.
Cheers, John
On Thu, Jan 11, 2018 at 3:43 PM, Nathan Goldbaum <nathan12343@gmail.com> wrote:
On Thu, Jan 11, 2018 at 3:43 AM, John Regan <johnanthonyregan@gmail.com
wrote:
Hi All,
I'm trying to get YT to read a vector which is stored as part of particle structure in Enzo (a vector in the active particle structure). The data looks OK when I view the raw HDF5 data using hdfview. I find that in some snapshots YT can read the data but not in others. If there is more than one particle things get worse for sure.
It could be that when writing to the HDF5 file I haven't put the data in a format that is compatible with YT.
I've attached a simple script that will fall over when trying to read the data and a link to the enzo output (size = 12 GBs). The active particles are in files DD0120.cpu0022 (Grid 3072), DD0120.cpu0023 (Grid 3076) and DD0014 (Grid 3080). YT falls over in the io_handler but it's not obvious to me whether I need to modify that code or the format that I write the data to the HDF5 file in.
It's hard to say if there's an issue on the Enzo side or the yt side without a copy of the traceback you are seeing or a way to reproduce the issue. I'm also not sure what your data format looks like offhand. If I remember correctly, you added support for vector active particle attributes to your fork of Enzo and it's entirely possible that there needs to be updates on the yt side to support reading the data from your fork.
I suspect the place you need to modify in yt is here:
https://github.com/yt-project/yt/blob/eb3358c622bc047200b9bc 6a954ddee8f59a0e07/yt/frontends/enzo/io.py#L92-L113
Any help/pointers would be much appreciated.
Cheers, John
Link to dataset: https://www.dropbox.com/sh/o6acuvia1sqyp7y/AAC0lEI5lZ6bGa0tP 2-G1jLea?dl=0
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
No worries at all. Really appreciate you looking at this. On Thu, Jan 11, 2018 at 7:26 PM, Nathan Goldbaum <nathan12343@gmail.com> wrote:
Ah, apologies, missed that in your first e-mail.
On Thu, Jan 11, 2018 at 1:24 PM, John Regan <johnanthonyregan@gmail.com> wrote:
Hi Nathan,
The 12 GB folder should be available at the link I sent in the first email. Here it is: https://www.dropbox.com/sh/o6a cuvia1sqyp7y/AAC0lEI5lZ6bGa0tP2-G1jLea?dl=0
If you run the script I sent it will fail pretty quickly with a traceback. Let me know if that works.
On Thu, Jan 11, 2018 at 6:50 PM, Nathan Goldbaum <nathan12343@gmail.com> wrote:
It would probably be best to share the full output. I might just need to see the traceback too, it's hard to say what will help without actually being able to see it :)
Is there any chance you can make a smaller output with similar active particle fields? I can also download the 12 GB if you make it publicly visible online somewhere.
On Thu, Jan 11, 2018 at 12:42 PM, John Regan <johnanthonyregan@gmail.com
wrote:
Hi Nathan ,
Thanks for the email. If I sent you the exact hdf5 file (i.e. the .cpuXXXX file) would that would and the traceback? You could see the format the array gets written then....
It wasn't clear to me from reading through the YT reader would work.
Cheers, John
On Thu, Jan 11, 2018 at 3:43 PM, Nathan Goldbaum <nathan12343@gmail.com
wrote:
On Thu, Jan 11, 2018 at 3:43 AM, John Regan < johnanthonyregan@gmail.com> wrote:
Hi All,
I'm trying to get YT to read a vector which is stored as part of particle structure in Enzo (a vector in the active particle structure). The data looks OK when I view the raw HDF5 data using hdfview. I find that in some snapshots YT can read the data but not in others. If there is more than one particle things get worse for sure.
It could be that when writing to the HDF5 file I haven't put the data in a format that is compatible with YT.
I've attached a simple script that will fall over when trying to read the data and a link to the enzo output (size = 12 GBs). The active particles are in files DD0120.cpu0022 (Grid 3072), DD0120.cpu0023 (Grid 3076) and DD0014 (Grid 3080). YT falls over in the io_handler but it's not obvious to me whether I need to modify that code or the format that I write the data to the HDF5 file in.
It's hard to say if there's an issue on the Enzo side or the yt side without a copy of the traceback you are seeing or a way to reproduce the issue. I'm also not sure what your data format looks like offhand. If I remember correctly, you added support for vector active particle attributes to your fork of Enzo and it's entirely possible that there needs to be updates on the yt side to support reading the data from your fork.
I suspect the place you need to modify in yt is here:
https://github.com/yt-project/yt/blob/eb3358c622bc047200b9bc 6a954ddee8f59a0e07/yt/frontends/enzo/io.py#L92-L113
Any help/pointers would be much appreciated.
Cheers, John
Link to dataset: https://www.dropbox.com/sh/o6acuvia1sqyp7y/AAC0lEI5lZ6bGa0tP 2-G1jLea?dl=0
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
I've opened a pull request with a fix: https://github.com/yt-project/yt/pull/1665 On Thu, Jan 11, 2018 at 1:29 PM, John Regan <johnanthonyregan@gmail.com> wrote:
No worries at all. Really appreciate you looking at this.
On Thu, Jan 11, 2018 at 7:26 PM, Nathan Goldbaum <nathan12343@gmail.com> wrote:
Ah, apologies, missed that in your first e-mail.
On Thu, Jan 11, 2018 at 1:24 PM, John Regan <johnanthonyregan@gmail.com> wrote:
Hi Nathan,
The 12 GB folder should be available at the link I sent in the first email. Here it is: https://www.dropbox.com/sh/o6a cuvia1sqyp7y/AAC0lEI5lZ6bGa0tP2-G1jLea?dl=0
If you run the script I sent it will fail pretty quickly with a traceback. Let me know if that works.
On Thu, Jan 11, 2018 at 6:50 PM, Nathan Goldbaum <nathan12343@gmail.com> wrote:
It would probably be best to share the full output. I might just need to see the traceback too, it's hard to say what will help without actually being able to see it :)
Is there any chance you can make a smaller output with similar active particle fields? I can also download the 12 GB if you make it publicly visible online somewhere.
On Thu, Jan 11, 2018 at 12:42 PM, John Regan < johnanthonyregan@gmail.com> wrote:
Hi Nathan ,
Thanks for the email. If I sent you the exact hdf5 file (i.e. the .cpuXXXX file) would that would and the traceback? You could see the format the array gets written then....
It wasn't clear to me from reading through the YT reader would work.
Cheers, John
On Thu, Jan 11, 2018 at 3:43 PM, Nathan Goldbaum < nathan12343@gmail.com> wrote:
On Thu, Jan 11, 2018 at 3:43 AM, John Regan < johnanthonyregan@gmail.com> wrote:
> Hi All, > > I'm trying to get YT to read a vector which is stored as part of > particle structure in Enzo (a vector in the active particle structure). The > data looks OK when I view the raw HDF5 data using hdfview. I find that in > some snapshots YT can read the data but not in others. If there is more > than one particle things get worse for sure. > > It could be that when writing to the HDF5 file I haven't put the > data in a format that is compatible with YT. > > I've attached a simple script that will fall over when trying to > read the data and a link to the enzo output (size = 12 GBs). The active > particles are in files DD0120.cpu0022 (Grid 3072), DD0120.cpu0023 (Grid > 3076) and DD0014 (Grid 3080). YT falls over in the io_handler but it's not > obvious to me whether I need to modify that code or the format that I write > the data to the HDF5 file in. >
It's hard to say if there's an issue on the Enzo side or the yt side without a copy of the traceback you are seeing or a way to reproduce the issue. I'm also not sure what your data format looks like offhand. If I remember correctly, you added support for vector active particle attributes to your fork of Enzo and it's entirely possible that there needs to be updates on the yt side to support reading the data from your fork.
I suspect the place you need to modify in yt is here:
https://github.com/yt-project/yt/blob/eb3358c622bc047200b9bc 6a954ddee8f59a0e07/yt/frontends/enzo/io.py#L92-L113
> > Any help/pointers would be much appreciated. > > Cheers, > John > > Link to dataset: > https://www.dropbox.com/sh/o6acuvia1sqyp7y/AAC0lEI5lZ6bGa0tP > 2-G1jLea?dl=0 > > _______________________________________________ > yt-users mailing list > yt-users@lists.spacepope.org > http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org > >
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
Top man! I'll pull that down in the morning. What was the issue? On Thu, Jan 11, 2018 at 7:57 PM, Nathan Goldbaum <nathan12343@gmail.com> wrote:
I've opened a pull request with a fix:
https://github.com/yt-project/yt/pull/1665
On Thu, Jan 11, 2018 at 1:29 PM, John Regan <johnanthonyregan@gmail.com> wrote:
No worries at all. Really appreciate you looking at this.
On Thu, Jan 11, 2018 at 7:26 PM, Nathan Goldbaum <nathan12343@gmail.com> wrote:
Ah, apologies, missed that in your first e-mail.
On Thu, Jan 11, 2018 at 1:24 PM, John Regan <johnanthonyregan@gmail.com> wrote:
Hi Nathan,
The 12 GB folder should be available at the link I sent in the first email. Here it is: https://www.dropbox.com/sh/o6a cuvia1sqyp7y/AAC0lEI5lZ6bGa0tP2-G1jLea?dl=0
If you run the script I sent it will fail pretty quickly with a traceback. Let me know if that works.
On Thu, Jan 11, 2018 at 6:50 PM, Nathan Goldbaum <nathan12343@gmail.com
wrote:
It would probably be best to share the full output. I might just need to see the traceback too, it's hard to say what will help without actually being able to see it :)
Is there any chance you can make a smaller output with similar active particle fields? I can also download the 12 GB if you make it publicly visible online somewhere.
On Thu, Jan 11, 2018 at 12:42 PM, John Regan < johnanthonyregan@gmail.com> wrote:
Hi Nathan ,
Thanks for the email. If I sent you the exact hdf5 file (i.e. the .cpuXXXX file) would that would and the traceback? You could see the format the array gets written then....
It wasn't clear to me from reading through the YT reader would work.
Cheers, John
On Thu, Jan 11, 2018 at 3:43 PM, Nathan Goldbaum < nathan12343@gmail.com> wrote:
> > > On Thu, Jan 11, 2018 at 3:43 AM, John Regan < > johnanthonyregan@gmail.com> wrote: > >> Hi All, >> >> I'm trying to get YT to read a vector which is stored as part of >> particle structure in Enzo (a vector in the active particle structure). The >> data looks OK when I view the raw HDF5 data using hdfview. I find that in >> some snapshots YT can read the data but not in others. If there is more >> than one particle things get worse for sure. >> >> It could be that when writing to the HDF5 file I haven't put the >> data in a format that is compatible with YT. >> >> I've attached a simple script that will fall over when trying to >> read the data and a link to the enzo output (size = 12 GBs). The active >> particles are in files DD0120.cpu0022 (Grid 3072), DD0120.cpu0023 (Grid >> 3076) and DD0014 (Grid 3080). YT falls over in the io_handler but it's not >> obvious to me whether I need to modify that code or the format that I write >> the data to the HDF5 file in. >> > > It's hard to say if there's an issue on the Enzo side or the yt side > without a copy of the traceback you are seeing or a way to reproduce the > issue. I'm also not sure what your data format looks like offhand. If I > remember correctly, you added support for vector active particle attributes > to your fork of Enzo and it's entirely possible that there needs to be > updates on the yt side to support reading the data from your fork. > > I suspect the place you need to modify in yt is here: > > https://github.com/yt-project/yt/blob/eb3358c622bc047200b9bc > 6a954ddee8f59a0e07/yt/frontends/enzo/io.py#L92-L113 > > >> >> Any help/pointers would be much appreciated. >> >> Cheers, >> John >> >> Link to dataset: >> https://www.dropbox.com/sh/o6acuvia1sqyp7y/AAC0lEI5lZ6bGa0tP >> 2-G1jLea?dl=0 >> >> _______________________________________________ >> yt-users mailing list >> yt-users@lists.spacepope.org >> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org >> >> > > _______________________________________________ > yt-users mailing list > yt-users@lists.spacepope.org > http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org > >
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
We support array particle fields already, we just needed to teach the Enzo frontend that some active particle fields can be array fields and how to detect that. On Thu, Jan 11, 2018 at 2:10 PM, John Regan <johnanthonyregan@gmail.com> wrote:
Top man! I'll pull that down in the morning. What was the issue?
On Thu, Jan 11, 2018 at 7:57 PM, Nathan Goldbaum <nathan12343@gmail.com> wrote:
I've opened a pull request with a fix:
https://github.com/yt-project/yt/pull/1665
On Thu, Jan 11, 2018 at 1:29 PM, John Regan <johnanthonyregan@gmail.com> wrote:
No worries at all. Really appreciate you looking at this.
On Thu, Jan 11, 2018 at 7:26 PM, Nathan Goldbaum <nathan12343@gmail.com> wrote:
Ah, apologies, missed that in your first e-mail.
On Thu, Jan 11, 2018 at 1:24 PM, John Regan <johnanthonyregan@gmail.com
wrote:
Hi Nathan,
The 12 GB folder should be available at the link I sent in the first email. Here it is: https://www.dropbox.com/sh/o6a cuvia1sqyp7y/AAC0lEI5lZ6bGa0tP2-G1jLea?dl=0
If you run the script I sent it will fail pretty quickly with a traceback. Let me know if that works.
On Thu, Jan 11, 2018 at 6:50 PM, Nathan Goldbaum < nathan12343@gmail.com> wrote:
It would probably be best to share the full output. I might just need to see the traceback too, it's hard to say what will help without actually being able to see it :)
Is there any chance you can make a smaller output with similar active particle fields? I can also download the 12 GB if you make it publicly visible online somewhere.
On Thu, Jan 11, 2018 at 12:42 PM, John Regan < johnanthonyregan@gmail.com> wrote:
> Hi Nathan , > > Thanks for the email. If I sent you the exact hdf5 file (i.e. the > .cpuXXXX file) would that would and the traceback? You could see the format > the array gets written then.... > > It wasn't clear to me from reading through the YT reader would work. > > Cheers, > John > > > On Thu, Jan 11, 2018 at 3:43 PM, Nathan Goldbaum < > nathan12343@gmail.com> wrote: > >> >> >> On Thu, Jan 11, 2018 at 3:43 AM, John Regan < >> johnanthonyregan@gmail.com> wrote: >> >>> Hi All, >>> >>> I'm trying to get YT to read a vector which is stored as part of >>> particle structure in Enzo (a vector in the active particle structure). The >>> data looks OK when I view the raw HDF5 data using hdfview. I find that in >>> some snapshots YT can read the data but not in others. If there is more >>> than one particle things get worse for sure. >>> >>> It could be that when writing to the HDF5 file I haven't put the >>> data in a format that is compatible with YT. >>> >>> I've attached a simple script that will fall over when trying to >>> read the data and a link to the enzo output (size = 12 GBs). The active >>> particles are in files DD0120.cpu0022 (Grid 3072), DD0120.cpu0023 (Grid >>> 3076) and DD0014 (Grid 3080). YT falls over in the io_handler but it's not >>> obvious to me whether I need to modify that code or the format that I write >>> the data to the HDF5 file in. >>> >> >> It's hard to say if there's an issue on the Enzo side or the yt >> side without a copy of the traceback you are seeing or a way to reproduce >> the issue. I'm also not sure what your data format looks like offhand. If I >> remember correctly, you added support for vector active particle attributes >> to your fork of Enzo and it's entirely possible that there needs to be >> updates on the yt side to support reading the data from your fork. >> >> I suspect the place you need to modify in yt is here: >> >> https://github.com/yt-project/yt/blob/eb3358c622bc047200b9bc >> 6a954ddee8f59a0e07/yt/frontends/enzo/io.py#L92-L113 >> >> >>> >>> Any help/pointers would be much appreciated. >>> >>> Cheers, >>> John >>> >>> Link to dataset: >>> https://www.dropbox.com/sh/o6acuvia1sqyp7y/AAC0lEI5lZ6bGa0tP >>> 2-G1jLea?dl=0 >>> >>> _______________________________________________ >>> yt-users mailing list >>> yt-users@lists.spacepope.org >>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org >>> >>> >> >> _______________________________________________ >> yt-users mailing list >> yt-users@lists.spacepope.org >> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org >> >> > > _______________________________________________ > yt-users mailing list > yt-users@lists.spacepope.org > http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org > >
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
participants (2)
-
John Regan -
Nathan Goldbaum