Issue loading data with yt
Hi yt-users, Let’s try this again. The file I sent the first time was too large. I’m using yt version 3.3.1, and I’ve recently encountered an issue loading data correctly. I’m running SPH simulations of white dwarf mergers and want to know what the mass fractions of different elements are at different times. I have two files that contain the data: 1) a tipsy file, 2) and a text file that contains the mass fraction values of helium-4. The tipsy file loads just fine and I’m able to access the information from the helium file once the tipsy file is loaded. For some reason though, when I have more than 262144 particles in my simulation, yt does not load the helium mass fraction values correctly anymore. I’ve initialized all particles to have a 4% helium mass fraction, and all of the particles up to and including the particle indexed by 262143 have this value, but after that the particles all have 0% according to yt. I’ve checked the helium files directly and all values are at 4%. I’m confused as to why yt would stop loading values correctly after a certain point. I’ve included two tipsy files as well as their corresponding helium mass fraction files. I have also written a script called visualization_test_script.py that makes scatter plots of the white dwarfs showing particles that have correct and incorrect values of helium. I can send more tipsy files if necessary, but they are too large to send any more here. Thanks, Matt
Hi Matt, This sounds similar to a bug that I (thought) I had fixed a while back. I'll take a look at your issue, see if I can reproduce it and put in a fix. Ben On Mon, Oct 31, 2016 at 11:06 AM, Matthew Robert Brinson <mbrinson@uwm.edu> wrote:
Hi yt-users,
Let’s try this again. The file I sent the first time was too large.
I’m using yt version 3.3.1, and I’ve recently encountered an issue loading data correctly. I’m running SPH simulations of white dwarf mergers and want to know what the mass fractions of different elements are at different times. I have two files that contain the data: 1) a tipsy file, 2) and a text file that contains the mass fraction values of helium-4. The tipsy file loads just fine and I’m able to access the information from the helium file once the tipsy file is loaded. For some reason though, when I have more than 262144 particles in my simulation, yt does not load the helium mass fraction values correctly anymore. I’ve initialized all particles to have a 4% helium mass fraction, and all of the particles up to and including the particle indexed by 262143 have this value, but after that the particles all have 0% according to yt. I’ve checked the helium files directly and all values are at 4%. I’m confused as to why yt would stop loading values correctly after a certain point.
I’ve included two tipsy files as well as their corresponding helium mass fraction files. I have also written a script called visualization_test_script.py that makes scatter plots of the white dwarfs showing particles that have correct and incorrect values of helium. I can send more tipsy files if necessary, but they are too large to send any more here.
Thanks, Matt
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
Hi Matthew, First of all, thank you for the detailed report. That made it much easier to track down the issue. Just in case you run into issues in the future and want to share larger data files, you can use the yt curldrop for that: https://docs.hub.yt/services.html#curldrop As Ben mentioned, your issue is eerily reminiscent of Issue 1084: https://bitbucket.org/yt_analysis/yt/issues/1084 This was fixed last year by Ben Keller in PR 1735: https://bitbucket.org/yt_analysis/yt/pull-requests/1735 However, it turns out that the implementation in PR 1735 had a bug that your test script revealed. I think there's a one-liner fix: https://bitbucket.org/yt_analysis/yt/pull-requests/2430 If you want to test out that fix, that would be very much appreciated. The easiest way to do so is to modify your current yt installation by hand (you can find it's location by looking at yt.__file__). That said, if you have time I'd recommend compiling yt from source: $ # get rid of old yt installation via conda or pip $ pip uninstall yt $ conda uninstall yt $ hg clone https://bitbucket.org/yt_analysis/yt ./yt-hg $ cd yt-hg $ # pull bugfix from Nathan's fork of yt and update to bugfix changeset $ hg pull -u -r 1a290fab65e4 https://bitbucket.org/ngoldbaum/yt $ # compile yt $ python setup.py develop This will build yt from source. It may take a few minutes because it will need to compile yt's C extensions. Once that's done, try rerunning your test script. -Nathan On Mon, Oct 31, 2016 at 10:27 AM, B.W. Keller <kellerbw@mcmaster.ca> wrote:
Hi Matt,
This sounds similar to a bug that I (thought) I had fixed a while back. I'll take a look at your issue, see if I can reproduce it and put in a fix.
Ben
On Mon, Oct 31, 2016 at 11:06 AM, Matthew Robert Brinson <mbrinson@uwm.edu
wrote:
Hi yt-users,
Let’s try this again. The file I sent the first time was too large.
I’m using yt version 3.3.1, and I’ve recently encountered an issue loading data correctly. I’m running SPH simulations of white dwarf mergers and want to know what the mass fractions of different elements are at different times. I have two files that contain the data: 1) a tipsy file, 2) and a text file that contains the mass fraction values of helium-4. The tipsy file loads just fine and I’m able to access the information from the helium file once the tipsy file is loaded. For some reason though, when I have more than 262144 particles in my simulation, yt does not load the helium mass fraction values correctly anymore. I’ve initialized all particles to have a 4% helium mass fraction, and all of the particles up to and including the particle indexed by 262143 have this value, but after that the particles all have 0% according to yt. I’ve checked the helium files directly and all values are at 4%. I’m confused as to why yt would stop loading values correctly after a certain point.
I’ve included two tipsy files as well as their corresponding helium mass fraction files. I have also written a script called visualization_test_script.py that makes scatter plots of the white dwarfs showing particles that have correct and incorrect values of helium. I can send more tipsy files if necessary, but they are too large to send any more here.
Thanks, Matt
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
Thanks for your help! That one-liner fixed it. Matt On Oct 31, 2016, at 2:17 PM, Nathan Goldbaum <nathan12343@gmail.com<mailto:nathan12343@gmail.com>> wrote: Hi Matthew, First of all, thank you for the detailed report. That made it much easier to track down the issue. Just in case you run into issues in the future and want to share larger data files, you can use the yt curldrop for that: https://docs.hub.yt/services.html#curldrop As Ben mentioned, your issue is eerily reminiscent of Issue 1084: https://bitbucket.org/yt_analysis/yt/issues/1084 This was fixed last year by Ben Keller in PR 1735: https://bitbucket.org/yt_analysis/yt/pull-requests/1735 However, it turns out that the implementation in PR 1735 had a bug that your test script revealed. I think there's a one-liner fix: https://bitbucket.org/yt_analysis/yt/pull-requests/2430 If you want to test out that fix, that would be very much appreciated. The easiest way to do so is to modify your current yt installation by hand (you can find it's location by looking at yt.__file__). That said, if you have time I'd recommend compiling yt from source: $ # get rid of old yt installation via conda or pip $ pip uninstall yt $ conda uninstall yt $ hg clone https://bitbucket.org/yt_analysis/yt ./yt-hg $ cd yt-hg $ # pull bugfix from Nathan's fork of yt and update to bugfix changeset $ hg pull -u -r 1a290fab65e4 https://bitbucket.org/ngoldbaum/yt $ # compile yt $ python setup.py develop This will build yt from source. It may take a few minutes because it will need to compile yt's C extensions. Once that's done, try rerunning your test script. -Nathan On Mon, Oct 31, 2016 at 10:27 AM, B.W. Keller <kellerbw@mcmaster.ca<mailto:kellerbw@mcmaster.ca>> wrote: Hi Matt, This sounds similar to a bug that I (thought) I had fixed a while back. I'll take a look at your issue, see if I can reproduce it and put in a fix. Ben On Mon, Oct 31, 2016 at 11:06 AM, Matthew Robert Brinson <mbrinson@uwm.edu<mailto:mbrinson@uwm.edu>> wrote: Hi yt-users, Let’s try this again. The file I sent the first time was too large. I’m using yt version 3.3.1, and I’ve recently encountered an issue loading data correctly. I’m running SPH simulations of white dwarf mergers and want to know what the mass fractions of different elements are at different times. I have two files that contain the data: 1) a tipsy file, 2) and a text file that contains the mass fraction values of helium-4. The tipsy file loads just fine and I’m able to access the information from the helium file once the tipsy file is loaded. For some reason though, when I have more than 262144 particles in my simulation, yt does not load the helium mass fraction values correctly anymore. I’ve initialized all particles to have a 4% helium mass fraction, and all of the particles up to and including the particle indexed by 262143 have this value, but after that the particles all have 0% according to yt. I’ve checked the helium files directly and all values are at 4%. I’m confused as to why yt would stop loading values correctly after a certain point. I’ve included two tipsy files as well as their corresponding helium mass fraction files. I have also written a script called visualization_test_script.py that makes scatter plots of the white dwarfs showing particles that have correct and incorrect values of helium. I can send more tipsy files if necessary, but they are too large to send any more here. Thanks, Matt _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org<mailto:yt-users@lists.spacepope.org> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org<mailto:yt-users@lists.spacepope.org> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org<mailto:yt-users@lists.spacepope.org> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
Hi, I sent this the other day but I’m not sure if it went through, so apologies if you get this twice. The one-liner that you gave me fixed the problem. Thanks so much for your help, Matt On Oct 31, 2016, at 2:17 PM, Nathan Goldbaum <nathan12343@gmail.com<mailto:nathan12343@gmail.com>> wrote: Hi Matthew, First of all, thank you for the detailed report. That made it much easier to track down the issue. Just in case you run into issues in the future and want to share larger data files, you can use the yt curldrop for that: https://docs.hub.yt/services.html#curldrop As Ben mentioned, your issue is eerily reminiscent of Issue 1084: https://bitbucket.org/yt_analysis/yt/issues/1084 This was fixed last year by Ben Keller in PR 1735: https://bitbucket.org/yt_analysis/yt/pull-requests/1735 However, it turns out that the implementation in PR 1735 had a bug that your test script revealed. I think there's a one-liner fix: https://bitbucket.org/yt_analysis/yt/pull-requests/2430 If you want to test out that fix, that would be very much appreciated. The easiest way to do so is to modify your current yt installation by hand (you can find it's location by looking at yt.__file__). That said, if you have time I'd recommend compiling yt from source: $ # get rid of old yt installation via conda or pip $ pip uninstall yt $ conda uninstall yt $ hg clone https://bitbucket.org/yt_analysis/yt ./yt-hg $ cd yt-hg $ # pull bugfix from Nathan's fork of yt and update to bugfix changeset $ hg pull -u -r 1a290fab65e4 https://bitbucket.org/ngoldbaum/yt $ # compile yt $ python setup.py develop This will build yt from source. It may take a few minutes because it will need to compile yt's C extensions. Once that's done, try rerunning your test script. -Nathan On Mon, Oct 31, 2016 at 10:27 AM, B.W. Keller <kellerbw@mcmaster.ca<mailto:kellerbw@mcmaster.ca>> wrote: Hi Matt, This sounds similar to a bug that I (thought) I had fixed a while back. I'll take a look at your issue, see if I can reproduce it and put in a fix. Ben On Mon, Oct 31, 2016 at 11:06 AM, Matthew Robert Brinson <mbrinson@uwm.edu<mailto:mbrinson@uwm.edu>> wrote: Hi yt-users, Let’s try this again. The file I sent the first time was too large. I’m using yt version 3.3.1, and I’ve recently encountered an issue loading data correctly. I’m running SPH simulations of white dwarf mergers and want to know what the mass fractions of different elements are at different times. I have two files that contain the data: 1) a tipsy file, 2) and a text file that contains the mass fraction values of helium-4. The tipsy file loads just fine and I’m able to access the information from the helium file once the tipsy file is loaded. For some reason though, when I have more than 262144 particles in my simulation, yt does not load the helium mass fraction values correctly anymore. I’ve initialized all particles to have a 4% helium mass fraction, and all of the particles up to and including the particle indexed by 262143 have this value, but after that the particles all have 0% according to yt. I’ve checked the helium files directly and all values are at 4%. I’m confused as to why yt would stop loading values correctly after a certain point. I’ve included two tipsy files as well as their corresponding helium mass fraction files. I have also written a script called visualization_test_script.py that makes scatter plots of the white dwarfs showing particles that have correct and incorrect values of helium. I can send more tipsy files if necessary, but they are too large to send any more here. Thanks, Matt _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org<mailto:yt-users@lists.spacepope.org> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org<mailto:yt-users@lists.spacepope.org> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org<mailto:yt-users@lists.spacepope.org> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
I got it, glad it helped! On Wed, Nov 2, 2016 at 3:06 PM, Matthew Robert Brinson <mbrinson@uwm.edu> wrote:
Hi,
I sent this the other day but I’m not sure if it went through, so apologies if you get this twice. The one-liner that you gave me fixed the problem.
Thanks so much for your help, Matt
On Oct 31, 2016, at 2:17 PM, Nathan Goldbaum <nathan12343@gmail.com> wrote:
Hi Matthew,
First of all, thank you for the detailed report. That made it much easier to track down the issue. Just in case you run into issues in the future and want to share larger data files, you can use the yt curldrop for that:
https://docs.hub.yt/services.html#curldrop
As Ben mentioned, your issue is eerily reminiscent of Issue 1084:
https://bitbucket.org/yt_analysis/yt/issues/1084
This was fixed last year by Ben Keller in PR 1735:
https://bitbucket.org/yt_analysis/yt/pull-requests/1735
However, it turns out that the implementation in PR 1735 had a bug that your test script revealed. I think there's a one-liner fix:
https://bitbucket.org/yt_analysis/yt/pull-requests/2430
If you want to test out that fix, that would be very much appreciated. The easiest way to do so is to modify your current yt installation by hand (you can find it's location by looking at yt.__file__). That said, if you have time I'd recommend compiling yt from source:
$ # get rid of old yt installation via conda or pip $ pip uninstall yt $ conda uninstall yt
$ hg clone https://bitbucket.org/yt_analysis/yt ./yt-hg $ cd yt-hg
$ # pull bugfix from Nathan's fork of yt and update to bugfix changeset $ hg pull -u -r 1a290fab65e4 https://bitbucket.org/ngoldbaum/yt
$ # compile yt $ python setup.py develop
This will build yt from source. It may take a few minutes because it will need to compile yt's C extensions. Once that's done, try rerunning your test script.
-Nathan
On Mon, Oct 31, 2016 at 10:27 AM, B.W. Keller <kellerbw@mcmaster.ca> wrote:
Hi Matt,
This sounds similar to a bug that I (thought) I had fixed a while back. I'll take a look at your issue, see if I can reproduce it and put in a fix.
Ben
On Mon, Oct 31, 2016 at 11:06 AM, Matthew Robert Brinson < mbrinson@uwm.edu> wrote:
Hi yt-users,
Let’s try this again. The file I sent the first time was too large.
I’m using yt version 3.3.1, and I’ve recently encountered an issue loading data correctly. I’m running SPH simulations of white dwarf mergers and want to know what the mass fractions of different elements are at different times. I have two files that contain the data: 1) a tipsy file, 2) and a text file that contains the mass fraction values of helium-4. The tipsy file loads just fine and I’m able to access the information from the helium file once the tipsy file is loaded. For some reason though, when I have more than 262144 particles in my simulation, yt does not load the helium mass fraction values correctly anymore. I’ve initialized all particles to have a 4% helium mass fraction, and all of the particles up to and including the particle indexed by 262143 have this value, but after that the particles all have 0% according to yt. I’ve checked the helium files directly and all values are at 4%. I’m confused as to why yt would stop loading values correctly after a certain point.
I’ve included two tipsy files as well as their corresponding helium mass fraction files. I have also written a script called visualization_test_script.py that makes scatter plots of the white dwarfs showing particles that have correct and incorrect values of helium. I can send more tipsy files if necessary, but they are too large to send any more here.
Thanks, Matt
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
participants (3)
-
B.W. Keller -
Matthew Robert Brinson -
Nathan Goldbaum