Hi Hilary,

Fist, thank you very much for your help, I really appreciate it. 

Sorry for the delay, the dataset took a while to upload.
Here is a dropbok link to the  dataset I'm using, its a TIPSY binary file, you can also find the the most fundamental version of my code there:

 https://www.dropbox.com/sh/tgtd3b57jkpvyjc/AACnx4QFO99TxUpF2qadXMDWa?dl=0

I think that the problem is that I'm not setting the mass and length conversion factors for the TipsyDataset correctly (I haven't figured out how to do this).

Again, thanks a lot for your help, let me know if I can help with something else.

-Bruno







2014-09-18 15:07 GMT-05:00 Hilary Egan <hilaryye@gmail.com>:
Hi Bruno,

I'm not able to recreate you issue with any of my test datasets, so having access to some of your data might make debugging a little bit easier. If you could share a test dataset (via dropbox, google drive, etc), I'll take a look at this!

-Hilary

On Thu, Sep 18, 2014 at 1:22 PM, Bruno Villasenor <br1villasen@gmail.com> wrote:
Hi,

I just tested the change and it works fine, this is the new output

########################################
P000 yt : [INFO     ] 2014-09-18 13:55:50,652 http://adsabs.harvard.edu/abs/2013ApJ...762..109B
P002 yt : [INFO     ] 2014-09-18 13:55:50,652 Rockstar is using 1 readers and 1 writers
P001 yt : [INFO     ] 2014-09-18 13:55:50,652 Rockstar is using 1 readers and 1 writers
P000 yt : [INFO     ] 2014-09-18 13:55:50,652 Rockstar is using 1 readers and 1 writers
P001 yt : [WARNING  ] 2014-09-18 13:55:50,719 Total Particle Count: 3.154e+05
[     0s] Accepting connections...
[     1s] Accepted all reader / writer connections.
[     1s] Verified all reader / writer connections.
[     1s] Transmitted all client connection info.
[     1s] Reading 1 blocks for snapshot 0...
Exception yt.utilities.exceptions.YTFieldNotFound: YTFieldNotFound() in 'yt.analysis_modules.halo_finding.rockstar.rockstar_interface.rh_read_particles' ignored
[     2s] Transferring particles to writers...
[     2s] Analyzing for FoF groups...
[     2s] Transferring boundary particles between writers...
[     2s] Linking boundary particles...
[     2s] Analyzing for halos / subhalos...
[     3s] Loading merger tree information...
[     3s] Constructing merger tree...
[     3s] [Success] Done with snapshot 0.
[     3s] [Finished]
P001 yt : [INFO     ] 2014-09-18 13:55:53,597 Parameters: current_time              = 2.44801700503e+17 s
P001 yt : [INFO     ] 2014-09-18 13:55:53,597 Parameters: domain_dimensions         = [2 2 2]
P001 yt : [INFO     ] 2014-09-18 13:55:53,597 Parameters: domain_left_edge          = [ 0.  0.  0.]
P001 yt : [INFO     ] 2014-09-18 13:55:53,598 Parameters: domain_right_edge         = [  1.12666939e+23   1.12666939e+23   1.12666939e+23]
P001 yt : [INFO     ] 2014-09-18 13:55:53,598 Parameters: cosmological_simulation   = 1
P001 yt : [INFO     ] 2014-09-18 13:55:53,598 Parameters: current_redshift          = 0.6
P001 yt : [INFO     ] 2014-09-18 13:55:53,598 Parameters: omega_lambda              = 0.699999988079
P001 yt : [INFO     ] 2014-09-18 13:55:53,598 Parameters: omega_matter              = 0.300000011921
P001 yt : [INFO     ] 2014-09-18 13:55:53,599 Parameters: hubble_constant           = 0.699999988079
P000 yt : [INFO     ] 2014-09-18 13:55:53,603 Parameters: current_time              = 2.44801700503e+17 s
P000 yt : [INFO     ] 2014-09-18 13:55:53,604 Parameters: domain_dimensions         = [2 2 2]
P000 yt : [INFO     ] 2014-09-18 13:55:53,604 Parameters: domain_left_edge          = [ 0.  0.  0.]
P000 yt : [INFO     ] 2014-09-18 13:55:53,605 Parameters: domain_right_edge         = [  1.12666939e+23   1.12666939e+23   1.12666939e+23]
P000 yt : [INFO     ] 2014-09-18 13:55:53,605 Parameters: cosmological_simulation   = 1
P000 yt : [INFO     ] 2014-09-18 13:55:53,605 Parameters: current_redshift          = 0.6
P000 yt : [INFO     ] 2014-09-18 13:55:53,605 Parameters: omega_lambda              = 0.699999988079
P000 yt : [INFO     ] 2014-09-18 13:55:53,605 Parameters: omega_matter              = 0.300000011921
P000 yt : [INFO     ] 2014-09-18 13:55:53,605 Parameters: hubble_constant           = 0.699999988079
P000 yt : [INFO     ] 2014-09-18 13:55:53,614 Allocating for 0.000e+00 particles
P002 yt : [INFO     ] 2014-09-18 13:55:53,615 Allocating for 0.000e+00 particles
P001 yt : [INFO     ] 2014-09-18 13:55:53,615 Allocating for 0.000e+00 particles
P000 yt : [WARNING  ] 2014-09-18 13:55:53,618 No halos were found for galaxy.00300
P002 yt : [WARNING  ] 2014-09-18 13:55:53,619 No halos were found for galaxy.00300
P000 yt : [INFO     ] 2014-09-18 13:55:53,619 Saving halo catalog (0 halos) to halo_catalogs/catalog/catalog.
P002 yt : [INFO     ] 2014-09-18 13:55:53,619 Saving halo catalog (0 halos) to halo_catalogs/catalog/catalog.
P001 yt : [WARNING  ] 2014-09-18 13:55:53,623 No halos were found for galaxy.00300
P001 yt : [INFO     ] 2014-09-18 13:55:53,623 Saving halo catalog (0 halos) to halo_catalogs/catalog/catalog.
###################################################


Thanks a lot for the quick fix.

There's something else, I don't understand why Rockstar is not finding any halos. (when I use "hop finder_method" it finds a bunch of them),
does the exception message below means that the finder is looking for something that I should be creating for a TipsyDataset?
########################################
Exception yt.utilities.exceptions.YTFieldNotFound: YTFieldNotFound() in 'yt.analysis_modules.halo_finding.rockstar.rockstar_interface.rh_read_particles' ignored
########################################

I'm an undergraduate student who just very recently started using yt, so if someone has any suggestion I will really appreciate it.

Thanks in advance.

Best Regards
Bruno



2014-09-18 11:40 GMT-05:00 Nathan Goldbaum <nathan12343@gmail.com>:

Looks like Hillary just opened a PR to fix this:


Bruno if you could test this, that would be great.  Let us know if you have any questions about how to locally test a pull request.


On Thursday, September 18, 2014, Matthew Turk <matthewturk@gmail.com> wrote:
Hi Bruno,

On Mon, Sep 15, 2014 at 7:58 PM, Bruno Villasenor <br1villasen@gmail.com> wrote:
> Hi all,
>
> I'm trying to run the RockstarHaloFinder via the HaloCatalog function on a
> TIPSY file.
> When I run the HaloCatalog function I get the following rockstar output:
>
> ########################################################################
> P002 yt : [INFO     ] 2014-09-15 19:36:11,382 Rockstar is using 1 readers
> and 1 writers
> P000 yt : [INFO     ] 2014-09-15 19:36:11,382 The citation for the Rockstar
> halo finder can be found at
> P001 yt : [INFO     ] 2014-09-15 19:36:11,382 Rockstar is using 1 readers
> and 1 writers
> P000 yt : [INFO     ] 2014-09-15 19:36:11,382
> http://adsabs.harvard.edu/abs/2013ApJ...762..109B
> P000 yt : [INFO     ] 2014-09-15 19:36:11,383 Rockstar is using 1 readers
> and 1 writers
> P001 yt : [WARNING  ] 2014-09-15 19:36:41,582 Total Particle Count:
> 4.862e+07
> [     0s] Accepting connections...
> [     1s] Accepted all reader / writer connections.
> [     1s] Verified all reader / writer connections.
> [     1s] Transmitted all client connection info.
> [     2s] Reading 1 blocks for snapshot 0...
> Exception yt.utilities.exceptions.YTFieldNotFound: YTFieldNotFound() in
> 'yt.analysis_modules.halo_finding.rockstar.rockstar_interface.rh_read_particles'
> ignored
> [     2s] Transferring particles to writers...
> [     2s] Analyzing for FoF groups...
> [     2s] Transferring boundary particles between writers...
> [     2s] Linking boundary particles...
> [     3s] Analyzing for halos / subhalos...
> [     3s] Loading merger tree information...
> [     3s] Constructing merger tree...
> [     3s] [Success] Done with snapshot 0.
> [     3s] [Finished]
> #######################################################################
>
> followed by the next  YT output
>
> #######################################################################
> P001 yt : [INFO     ] 2014-09-15 19:36:44,986 Parameters: current_time
> = 2.44801700503e+17 s
> P001 yt : [INFO     ] 2014-09-15 19:36:44,987 Parameters: domain_dimensions
> = [2 2 2]
> P000 yt : [INFO     ] 2014-09-15 19:36:44,994 Parameters: current_time
> = 2.44801700503e+17 s
> P000 yt : [INFO     ] 2014-09-15 19:36:44,994 Parameters: domain_dimensions
> = [2 2 2]
> P000 yt : [INFO     ] 2014-09-15 19:36:44,994 Parameters: domain_left_edge
> = [ 0.  0.  0.]
> P000 yt : [INFO     ] 2014-09-15 19:36:45,012 Parameters: domain_right_edge
> = [ 0.00207136  0.00207136  0.00207136]
> P000 yt : [INFO     ] 2014-09-15 19:36:45,013 Parameters:
> cosmological_simulation   = 1
> P000 yt : [INFO     ] 2014-09-15 19:36:45,014 Parameters: current_redshift
> = 0.6
> P001 yt : [INFO     ] 2014-09-15 19:36:44,987 Parameters: domain_left_edge
> = [ 0.  0.  0.]
> P000 yt : [INFO     ] 2014-09-15 19:36:45,014 Parameters: omega_lambda
> = 0.699999988079
> P000 yt : [INFO     ] 2014-09-15 19:36:45,014 Parameters: omega_matter
> = 0.300000011921
> P000 yt : [INFO     ] 2014-09-15 19:36:45,014 Parameters: hubble_constant
> = 0.699999988079
> P001 yt : [INFO     ] 2014-09-15 19:36:45,014 Parameters: domain_right_edge
> = [ 0.00207136  0.00207136  0.00207136]
> P001 yt : [INFO     ] 2014-09-15 19:36:45,014 Parameters:
> cosmological_simulation   = 1
> P001 yt : [INFO     ] 2014-09-15 19:36:45,015 Parameters: current_redshift
> = 0.6
> P001 yt : [INFO     ] 2014-09-15 19:36:45,015 Parameters: omega_lambda
> = 0.699999988079
> P001 yt : [INFO     ] 2014-09-15 19:36:45,015 Parameters: omega_matter
> = 0.300000011921
> P001 yt : [INFO     ] 2014-09-15 19:36:45,015 Parameters: hubble_constant
> = 0.699999988079
> P000 yt : [INFO     ] 2014-09-15 19:36:45,070 Allocating for 0.000e+00
> particles
> P001 yt : [INFO     ] 2014-09-15 19:36:45,070 Allocating for 0.000e+00
> particles
> P002 yt : [INFO     ] 2014-09-15 19:36:45,070 Allocating for 0.000e+00
> particles
> Traceback (most recent call last):
>   File "simple_rockstar.py", line 44, in <module>
> Traceback (most recent call last):
>   File "simple_rockstar.py", line 44, in <module>
> Traceback (most recent call last):
>   File "simple_rockstar.py", line 44, in <module>
>     hc.create()
>   File
> "/home/bruno/apps/yt-x86_64/src/yt-hg/yt/analysis_modules/halo_analysis/halo_catalog.py",
> line 297, in create
>     hc.create()
>   File
> "/home/bruno/apps/yt-x86_64/src/yt-hg/yt/analysis_modules/halo_analysis/halo_catalog.py",
> line 297, in create
>     hc.create()
>   File
> "/home/bruno/apps/yt-x86_64/src/yt-hg/yt/analysis_modules/halo_analysis/halo_catalog.py",
> line 297, in create
>     self._run(save_halos, save_catalog, njobs=njobs, dynamic=dynamic)
>   File
> "/home/bruno/apps/yt-x86_64/src/yt-hg/yt/utilities/parallel_tools/parallel_analysis_interface.py",
> line 248, in barrierize
>     self._run(save_halos, save_catalog, njobs=njobs, dynamic=dynamic)
>     self._run(save_halos, save_catalog, njobs=njobs, dynamic=dynamic)
>   File
> "/home/bruno/apps/yt-x86_64/src/yt-hg/yt/utilities/parallel_tools/parallel_analysis_interface.py",
> line 248, in barrierize
>   File
> "/home/bruno/apps/yt-x86_64/src/yt-hg/yt/utilities/parallel_tools/parallel_analysis_interface.py",
> line 248, in barrierize
>     retval = func(*args, **kwargs)
>   File
> "/home/bruno/apps/yt-x86_64/src/yt-hg/yt/analysis_modules/halo_analysis/halo_catalog.py",
> line 367, in _run
>     self.halos_ds.index
> AttributeError: 'NoneType' object has no attribute 'index'
>     retval = func(*args, **kwargs)
>   File
> "/home/bruno/apps/yt-x86_64/src/yt-hg/yt/analysis_modules/halo_analysis/halo_catalog.py",
> line 367, in _run
>     self.halos_ds.index
>     retval = func(*args, **kwargs)
> AttributeError: 'NoneType' object has no attribute 'index'
>   File
> "/home/bruno/apps/yt-x86_64/src/yt-hg/yt/analysis_modules/halo_analysis/halo_catalog.py",
> line 367, in _run
>     self.halos_ds.index
> AttributeError: 'NoneType' object has no attribute 'index'

This does indeed look like an internal error.  It looks to me like the
halos_ds is not being created -- even if R* finds something, I think
this should be created.  Britton and/or Hilary, have you seen this
error before?

-Matt

> #######################################################################################
>
> It seems to me like rockstar is not finding anything (Exception message) and
> then when yt load and save to disk it fails.
> The simplest working version of my code is the next one:
>
> #######################################################################################
> import yt
> from yt.mods import *
> from yt.analysis_modules.halo_analysis.api import HaloCatalog
> from yt.data_objects.particle_filters import add_particle_filter
> from mpi4py import MPI
> import numpy as np
>
> dataDir = "/home/bruno/Desktop/data/yt/"
> dataFile = 'DM.Only.tipsy/dwf1.6144.01280'
> ts = DatasetSeries([dataDir + dataFile])
> data_ds = ts[0]
> #data_ds = yt.load( dataDir + dataFile )
> data_ds.set_units( )
>
> #Add symbol "Mpccm"
> data_ds.unit_registry.add( 'Mpccm',
>  float(units.Mpc.in_cgs().v),
>  units.dimensions.length)
>
> #Create particle filter to filter to filter a single mass particle type
> def low_mass(pfilter, data):
>     filter = data["all", "Mass"] < 1.3e-12 * units.Msun
>     return filter
> add_particle_filter("low_mass", function=low_mass,
>                     filtered_type='all',
>                     requires=["Mass"])
> data_ds.add_particle_filter("low_mass")
>
> #Set cosmology parameters
> data_ds.hubble_constant = 0.7
> data_ds.omega_matter = 0.3
> data_ds.current_redshift = 0.6
> data_ds.omega_lambda = 0.7
>
> #Arguments for RockstarHoloFinder
> rockstar_kwargs = {
> 'particle_type': 'low_mass',
> 'force_res': 1e-4  #100pc
> }
>
> hc = HaloCatalog(data_ds=data_ds,
> finder_method='rockstar',
> finder_kwargs = rockstar_kwargs)
> hc.create()
> ##################################################################
>
> If someone has any suggestion  I would really appreciate it.
>
> Thanks,
> Bruno
>
> _______________________________________________
> yt-users mailing list
> yt-users@lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>
_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org



_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org



_______________________________________________
yt-users mailing list
yt-users@lists.spacepope.org
http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org