Send yt-users mailing list submissions to
yt-users@lists.spacepope.org
To subscribe or unsubscribe via the World Wide Web, visit
http://lists.spacepope.org/listinfo.cgi/yt-users- spacepope.org
or, via email, send a message with subject or body 'help' to
yt-users-request@lists.spacepope.org
You can reach the person managing the list at
yt-users-owner@lists.spacepope.org
When replying, please edit your Subject line so it is more specific
than "Re: Contents of yt-users digest..."
Today's Topics:
1. Re: FLASH Time Series Analysis with Particle Files (Alex Hill)
2. Re: FLASH Time Series Analysis with Particle Files
(Jason Galyardt)
3. Re: FLASH Time Series Analysis with Particle Files
(Nathan Goldbaum)
---------- Forwarded message ----------
From: Alex Hill <ashill@haverford.edu>
To: Discussion of the yt analysis package <yt-users@lists.spacepope.org>
Cc:
Date: Wed, 30 Nov 2016 13:59:19 -0500
Subject: Re: [yt-users] FLASH Time Series Analysis with Particle Files
Hi Jason,
Yes, I use Flash files with particles routinely now. As John alludes to, there are issues when the plot files and particle files don’t match in both file name (including the plot/particle file number) and timestamp. I now avoid those issues by ensuring that the names are the same; I don’t recall if the issue with different names/numbers was solved but could look into it again if it would help.
-Alex
> On Nov 30, 2016, at 11:32, John Zuhone <jzuhone@gmail.com> wrote:
>
> Yes, my understanding is that this is working, provided that your files have the same basename and the times of the files are the same.
>
>> On Nov 30, 2016, at 10:32 AM, Nathan Goldbaum <nathan12343@gmail.com> wrote:
>>
>> Hi Jason,
>>
>> As far as I'm aware this should be working now. Are you having trouble?
>>
>> -Nathan
>>
>> On Wed, Nov 30, 2016 at 9:16 AM, Jason Galyardt <jason.galyardt@gmail.com> wrote:
>> Dear yt-users,
>>
>> In May 2015, there was some traffic on this list concerning errors in time series analysis on FLASH simulations that include both plot files and particle files. Does anyone have an update on the status of this issue?
>>
>> Thanks,
>> Jason
>>
>> ------
>> Jason Galyardt
>> University of Georgia
>>
>>
>> _______________________________________________
>> yt-users mailing list
>> yt-users@lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-users- spacepope.org
>>
>>
>> _______________________________________________
>> yt-users mailing list
>> yt-users@lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-users- spacepope.org
>
> _______________________________________________
> yt-users mailing list
> yt-users@lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users- spacepope.org
---------- Forwarded message ----------
From: Jason Galyardt <jason.galyardt@gmail.com>
To: Discussion of the yt analysis package <yt-users@lists.spacepope.org>
Cc:
Date: Wed, 30 Nov 2016 14:47:06 -0500
Subject: Re: [yt-users] FLASH Time Series Analysis with Particle FilesHi John & Nathan,Yes, I'm having trouble using the time series analysis machinery with my FLASH simulations. I've checked that the plot files and particle files all have the same current_time variable, so there's no issue there. I'm using a development version of yt with a changeset ID of 5cf30893afdf. This changeset includes Nathan's changes supporting cut_region functionality for FLASH plot files. Here's what I see when I try to do a time series analysis:
###########################
In [9]: ts = yt.load("mysim_hdf5_plt_cnt_*",
particle_filename="mysim_hdf5_part_*")
In [10]: for ds in ts:
print ds.current_time
....:
------------------------------------------------------------ ---------------
IOErrorTraceback (most recent call last)
<ipython-input-10-fccd01bb5065> in <module>()
----> 1 for ds in ts:
2 print ds.current_time
3
/home/emrys/yt/yt/data_objects/time_series.pyc in __iter__(self)
179 for o in self._pre_outputs:
180 if isinstance(o, string_types):
--> 181 ds = self._load(o, **self.kwargs)
182 self._setup_function(ds)
183 yield ds
/home/emrys/yt/yt/data_objects/time_series.pyc in _load(self, output_fn, **kwargs)
396 elif self._mixed_dataset_types:
397 return load(output_fn, **kwargs)
--> 398 ds = load(output_fn, **kwargs)
399 self._dataset_cls = ds.__class__
400 return ds
/home/emrys/yt/yt/convenience.pyc in load(*args, **kwargs)
84 candidates = find_lowest_subclasses(candidates)
85 if len(candidates) == 1:
---> 86 return candidates[0](*args, **kwargs)
87 if len(candidates) == 0:
88 if ytcfg.get("yt", "enzo_db") != '' \
/home/emrys/yt/yt/frontends/flash/data_structures.pyc in __init__(self, filename, dataset_type, storage_filename, particle_filename, units_override, unit_system)
209 self._particle_handle = HDF5FileHandler(self.particle_filename)
210 except:
--> 211 raise IOError(self.particle_filename)
212 # Check if the particle file has the same time
213 if self._particle_handle != self._handle:
IOError: mysim_hdf5_part_*###########################It looks to me like HDF5FileHandler() just does not like the wildcard. I've also tried, unsuccessfully, to feed a list of particle files to yt.load(). I thought perhaps that I should be using the DatasetSeries framework directly, but I'm not clear on how to pass in the list of particle files, as DatasetSeries does not understand the particle_filename keyword argument.Any ideas?Thanks,JasonOn Wed, Nov 30, 2016 at 1:57 PM, <yt-users-request@lists.spacepope.org > wrote:Send yt-users mailing list submissions to
yt-users@lists.spacepope.org
To subscribe or unsubscribe via the World Wide Web, visit
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope. org
or, via email, send a message with subject or body 'help' to
yt-users-request@lists.spacepope.org
You can reach the person managing the list at
yt-users-owner@lists.spacepope.org
When replying, please edit your Subject line so it is more specific
than "Re: Contents of yt-users digest..."
Today's Topics:
1. FLASH Time Series Analysis with Particle Files (Jason Galyardt)
2. Re: FLASH Time Series Analysis with Particle Files
(Nathan Goldbaum)
3. Re: FLASH Time Series Analysis with Particle Files (John Zuhone)
---------- Forwarded message ----------
From: Jason Galyardt <jason.galyardt@gmail.com>
To: Discussion of the yt analysis package <yt-users@lists.spacepope.org>
Cc:
Date: Wed, 30 Nov 2016 10:16:44 -0500
Subject: [yt-users] FLASH Time Series Analysis with Particle FilesDear yt-users,In May 2015, there was some traffic on this list concerning errors in time series analysis on FLASH simulations that include both plot files and particle files. Does anyone have an update on the status of this issue?Thanks,Jason
------Jason GalyardtUniversity of Georgia
---------- Forwarded message ----------
From: Nathan Goldbaum <nathan12343@gmail.com>
To: Discussion of the yt analysis package <yt-users@lists.spacepope.org>
Cc:
Date: Wed, 30 Nov 2016 09:32:37 -0600
Subject: Re: [yt-users] FLASH Time Series Analysis with Particle FilesHi Jason,As far as I'm aware this should be working now. Are you having trouble?-NathanOn Wed, Nov 30, 2016 at 9:16 AM, Jason Galyardt <jason.galyardt@gmail.com> wrote:Dear yt-users,In May 2015, there was some traffic on this list concerning errors in time series analysis on FLASH simulations that include both plot files and particle files. Does anyone have an update on the status of this issue?Thanks,Jason
------Jason GalyardtUniversity of Georgia
_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.o rg
---------- Forwarded message ----------
From: John Zuhone <jzuhone@gmail.com>
To: Discussion of the yt analysis package <yt-users@lists.spacepope.org>
Cc:
Date: Wed, 30 Nov 2016 11:32:09 -0500
Subject: Re: [yt-users] FLASH Time Series Analysis with Particle FilesYes, my understanding is that this is working, provided that your files have the same basename and the times of the files are the same.On Nov 30, 2016, at 10:32 AM, Nathan Goldbaum <nathan12343@gmail.com> wrote:Hi Jason,As far as I'm aware this should be working now. Are you having trouble?-Nathan______________________________On Wed, Nov 30, 2016 at 9:16 AM, Jason Galyardt <jason.galyardt@gmail.com> wrote:Dear yt-users,In May 2015, there was some traffic on this list concerning errors in time series analysis on FLASH simulations that include both plot files and particle files. Does anyone have an update on the status of this issue?Thanks,Jason
------Jason GalyardtUniversity of Georgia
_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.o rg
_________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope. org
_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope. org
---------- Forwarded message ----------
From: Nathan Goldbaum <nathan12343@gmail.com>
To: Discussion of the yt analysis package <yt-users@lists.spacepope.org>
Cc:
Date: Thu, 1 Dec 2016 09:46:54 -0600
Subject: Re: [yt-users] FLASH Time Series Analysis with Particle FilesHi Jason,Thanks for those details.The FLASH frontend has a capability to automatically detect the presence of an accompanying particle file for a given plotfile. I have a test dataset (that I will be uploading to yt-project.org/data shortly for future questions about flash datasets that contain both particle and mesh fields) that I've written the following test script for:import ytts = yt.load('fiducial_1to1_b0_hdf5_plt_cnt_*') for ds in ts:print(ds.particle_type_counts)This time series contains five outputs, with a plotfile and a particlefile for each output:$ lsfiducial_1to1_b0_hdf5_part_0000 fiducial_1to1_b0_hdf5_plt_ cnt_0001 fiducial_1to1_b0_hdf5_part_0001 fiducial_1to1_b0_hdf5_plt_ cnt_0002 fiducial_1to1_b0_hdf5_part_0002 fiducial_1to1_b0_hdf5_plt_ cnt_0003 fiducial_1to1_b0_hdf5_part_0003 fiducial_1to1_b0_hdf5_plt_ cnt_0004 fiducial_1to1_b0_hdf5_part_0004 test.py fiducial_1to1_b0_hdf5_plt_cnt_0000 When I run the script, it outputs:{'io': 9999998}{'io': 9999998}{'io': 9999998}{'io': 9999998}{'io': 9999998}As you can see, my script only explicitly loads the plotfile but yt still recognizes that there are about 10 million particles in each ouput, since the particle file for each plotfile is automatically associated with the plotfile. You can see how this functionality is setup in the FLASH frontend here:This requires that the particle file associated with a plotfile has the same output number. We used to strictly enforce that the particle file has the same simulation time as the plotifle, but this ended up being too strict for real-world use cases so now we only print a warning (see around like 217 in the file I linked to above).Is this sufficient for your use cases or do you need to explicitly specify the name of the particle file every time you load the dataset? If so, one hacky way around this would be to manually associate the particle file with each plotfile using the `setup_function` keyword argument for the DatasetSeries object. I could also look into making it so that if you pass a glob pattern as the `particle_filename` keyword argument to the time series object, it does the right thing, but I want to make sure that the existing functionality will not work for you before trying that.Hope that clears things up,NathanOn Wed, Nov 30, 2016 at 1:47 PM, Jason Galyardt <jason.galyardt@gmail.com> wrote:Hi John & Nathan,Yes, I'm having trouble using the time series analysis machinery with my FLASH simulations. I've checked that the plot files and particle files all have the same current_time variable, so there's no issue there. I'm using a development version of yt with a changeset ID of 5cf30893afdf. This changeset includes Nathan's changes supporting cut_region functionality for FLASH plot files. Here's what I see when I try to do a time series analysis:
###########################
In [9]: ts = yt.load("mysim_hdf5_plt_cnt_*",
particle_filename="mysim_hdf5_part_*")
In [10]: for ds in ts:
print ds.current_time
....:
------------------------------------------------------------ ---------------
IOErrorTraceback (most recent call last)
<ipython-input-10-fccd01bb5065> in <module>()
----> 1 for ds in ts:
2 print ds.current_time
3
/home/emrys/yt/yt/data_objects/time_series.pyc in __iter__(self)
179 for o in self._pre_outputs:
180 if isinstance(o, string_types):
--> 181 ds = self._load(o, **self.kwargs)
182 self._setup_function(ds)
183 yield ds
/home/emrys/yt/yt/data_objects/time_series.pyc in _load(self, output_fn, **kwargs)
396 elif self._mixed_dataset_types:
397 return load(output_fn, **kwargs)
--> 398 ds = load(output_fn, **kwargs)
399 self._dataset_cls = ds.__class__
400 return ds
/home/emrys/yt/yt/convenience.pyc in load(*args, **kwargs)
84 candidates = find_lowest_subclasses(candidates)
85 if len(candidates) == 1:
---> 86 return candidates[0](*args, **kwargs)
87 if len(candidates) == 0:
88 if ytcfg.get("yt", "enzo_db") != '' \
/home/emrys/yt/yt/frontends/flash/data_structures.pyc in __init__(self, filename, dataset_type, storage_filename, particle_filename, units_override, unit_system)
209 self._particle_handle = HDF5FileHandler(self.particle_filename)
210 except:
--> 211 raise IOError(self.particle_filename)
212 # Check if the particle file has the same time
213 if self._particle_handle != self._handle:
IOError: mysim_hdf5_part_*###########################It looks to me like HDF5FileHandler() just does not like the wildcard. I've also tried, unsuccessfully, to feed a list of particle files to yt.load(). I thought perhaps that I should be using the DatasetSeries framework directly, but I'm not clear on how to pass in the list of particle files, as DatasetSeries does not understand the particle_filename keyword argument.Any ideas?Thanks,JasonOn Wed, Nov 30, 2016 at 1:57 PM, <yt-users-request@lists.spacepope.org > wrote:Send yt-users mailing list submissions to
yt-users@lists.spacepope.org
To subscribe or unsubscribe via the World Wide Web, visit
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.o rg
or, via email, send a message with subject or body 'help' to
yt-users-request@lists.spacepope.org
You can reach the person managing the list at
yt-users-owner@lists.spacepope.org
When replying, please edit your Subject line so it is more specific
than "Re: Contents of yt-users digest..."
Today's Topics:
1. FLASH Time Series Analysis with Particle Files (Jason Galyardt)
2. Re: FLASH Time Series Analysis with Particle Files
(Nathan Goldbaum)
3. Re: FLASH Time Series Analysis with Particle Files (John Zuhone)
---------- Forwarded message ----------
From: Jason Galyardt <jason.galyardt@gmail.com>
To: Discussion of the yt analysis package <yt-users@lists.spacepope.org>
Cc:
Date: Wed, 30 Nov 2016 10:16:44 -0500
Subject: [yt-users] FLASH Time Series Analysis with Particle FilesDear yt-users,In May 2015, there was some traffic on this list concerning errors in time series analysis on FLASH simulations that include both plot files and particle files. Does anyone have an update on the status of this issue?Thanks,Jason
------Jason GalyardtUniversity of Georgia
---------- Forwarded message ----------
From: Nathan Goldbaum <nathan12343@gmail.com>
To: Discussion of the yt analysis package <yt-users@lists.spacepope.org>
Cc:
Date: Wed, 30 Nov 2016 09:32:37 -0600
Subject: Re: [yt-users] FLASH Time Series Analysis with Particle FilesHi Jason,As far as I'm aware this should be working now. Are you having trouble?-NathanOn Wed, Nov 30, 2016 at 9:16 AM, Jason Galyardt <jason.galyardt@gmail.com> wrote:Dear yt-users,In May 2015, there was some traffic on this list concerning errors in time series analysis on FLASH simulations that include both plot files and particle files. Does anyone have an update on the status of this issue?Thanks,Jason
------Jason GalyardtUniversity of Georgia
_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.o rg
---------- Forwarded message ----------
From: John Zuhone <jzuhone@gmail.com>
To: Discussion of the yt analysis package <yt-users@lists.spacepope.org>
Cc:
Date: Wed, 30 Nov 2016 11:32:09 -0500
Subject: Re: [yt-users] FLASH Time Series Analysis with Particle FilesYes, my understanding is that this is working, provided that your files have the same basename and the times of the files are the same.On Nov 30, 2016, at 10:32 AM, Nathan Goldbaum <nathan12343@gmail.com> wrote:Hi Jason,As far as I'm aware this should be working now. Are you having trouble?-Nathan______________________________On Wed, Nov 30, 2016 at 9:16 AM, Jason Galyardt <jason.galyardt@gmail.com> wrote:Dear yt-users,In May 2015, there was some traffic on this list concerning errors in time series analysis on FLASH simulations that include both plot files and particle files. Does anyone have an update on the status of this issue?Thanks,Jason
------Jason GalyardtUniversity of Georgia
_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.o rg
_________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.o rg
_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.o rg
_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope. org
_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users- spacepope.org