Greetings, I have just finished an extensive overhaul of the HaloProfiler that includes a bunch of new features, as well as completely changes the running syntax. Other than the addition of various new features, the main goals of this overhaul were: 1) to make the tool more general (mostly by removing hard-coded filtering by virial quantities), essentially allowing someone to profile a random list of points that may or may not be actual halos Easy filtering of halos by virial quantities remains an option, but is now simply a specific instance of something more general and far more powerful: a halo filtering device that allow the user to create their own filter functions to comb through radial profile data and decide whether a halo meets certain criteria. 2) to remove the dependence on an additional parameter file for the various options. The parameter file is now gone. Most of the HaloProfiler parameters have been turned into instantiation keyword args. The keyword list is now considerably longer, but the benefit is that the number of files needed to run this thing has been reduced from 2 (running script and par file) to just 1 (running script). There are a large number of keywords options that are specific to either the profile or projection routines that are taken in at instantiation and stored as attributes. This was done to keep those function calls simple. I'm curious to know peoples' thoughts on whether these keyword args should stay put or move to the individual function calls. Adding fields for profiling and projections have been moved to functions addProfile and addProjection. Here is a brief list of the new features that I can remember: - the halo list read routine can be easily customized to read in columned ascii data of varying formats through the use of a dictionary (see attribute self.halo_list_format) - ability to change the function call, args, and kwargs for the halo finder Profiles: - filter halo list with user-written filter functions (see new file HaloFilters.py for an example of a function to filter based on virial quantities) - extract and output scalar quantities from halos using the filter functions - pre-filter halos based on values in the initial halo list (skipping the profiling altogether and saving a load of time) Projections: - choose the axes to be projected (instead of hard-coded to all three) - easily select the list of halos to be projected (the total list, filtered list, a new file, or an actual list) Before I commit all this, I'd like a little feedback, mostly on the migration of the par file parameters to keyword args. I put three files in the pastebin for people to download. I chose not to submit a diff since the changes were so sweeping. The new HaloProfiler.py: http://paste.enzotools.org/show/178/ HaloFilters.py (should go in yt/extensions with HaloProfiler.py) http://paste.enzotools.org/show/179/ runHaloProfiler.py (an example script to run the new HaloProfiler) http://paste.enzotools.org/show/180/ I will try my best to write up full documentation for this as soon as possible, perhaps even today. Please let me know what you think. Britton
Hi Britton, This is awesome! I just have a couple comments. * You can't include {} in a function dec -- it is instantiated at parse-time, so it persists and any modifications made are kept. Usually what's done is to default to None, and do an if something is None: something = {} * lists in function declaration must be copied with [:] ; in fact, I'd say avoid DeepCopy unless you know for a fact it will be something other than strings (which I don't think is the case for these objects) * addProfile and addProjection should be add_profile and add_projection; I'd prefer *all* code follow PEP-8 ( http://www.python.org/dev/peps/pep-0008/ ) but I'd definitely like to keep a consistent lowercase-underscore mechanism for methods on objects. (I also think we should try to follow the function declaration guidelines with respect to spaces and commas.) * na.array(range(something)) can just be na.arange(something) * if type(something) is list should be isinstance(something, types.ListType) (I think) * I'm inclined to think we don't want a second verbosity check, and instead put stuff into mylog.debug, but I'll leave that up to you... All in all, this looks like a great addition, and a great set of extensions. Thanks so much for doing all this! -Matt On Mon, Aug 24, 2009 at 12:51 PM, Britton Smith<brittonsmith@gmail.com> wrote:
Greetings,
I have just finished an extensive overhaul of the HaloProfiler that includes a bunch of new features, as well as completely changes the running syntax. Other than the addition of various new features, the main goals of this overhaul were: 1) to make the tool more general (mostly by removing hard-coded filtering by virial quantities), essentially allowing someone to profile a random list of points that may or may not be actual halos Easy filtering of halos by virial quantities remains an option, but is now simply a specific instance of something more general and far more powerful: a halo filtering device that allow the user to create their own filter functions to comb through radial profile data and decide whether a halo meets certain criteria.
2) to remove the dependence on an additional parameter file for the various options. The parameter file is now gone. Most of the HaloProfiler parameters have been turned into instantiation keyword args. The keyword list is now considerably longer, but the benefit is that the number of files needed to run this thing has been reduced from 2 (running script and par file) to just 1 (running script). There are a large number of keywords options that are specific to either the profile or projection routines that are taken in at instantiation and stored as attributes. This was done to keep those function calls simple. I'm curious to know peoples' thoughts on whether these keyword args should stay put or move to the individual function calls. Adding fields for profiling and projections have been moved to functions addProfile and addProjection.
Here is a brief list of the new features that I can remember: - the halo list read routine can be easily customized to read in columned ascii data of varying formats through the use of a dictionary (see attribute self.halo_list_format) - ability to change the function call, args, and kwargs for the halo finder Profiles: - filter halo list with user-written filter functions (see new file HaloFilters.py for an example of a function to filter based on virial quantities) - extract and output scalar quantities from halos using the filter functions - pre-filter halos based on values in the initial halo list (skipping the profiling altogether and saving a load of time) Projections: - choose the axes to be projected (instead of hard-coded to all three) - easily select the list of halos to be projected (the total list, filtered list, a new file, or an actual list)
Before I commit all this, I'd like a little feedback, mostly on the migration of the par file parameters to keyword args. I put three files in the pastebin for people to download. I chose not to submit a diff since the changes were so sweeping. The new HaloProfiler.py: http://paste.enzotools.org/show/178/ HaloFilters.py (should go in yt/extensions with HaloProfiler.py) http://paste.enzotools.org/show/179/ runHaloProfiler.py (an example script to run the new HaloProfiler) http://paste.enzotools.org/show/180/
I will try my best to write up full documentation for this as soon as possible, perhaps even today.
Please let me know what you think.
Britton
_______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Hi everyone, I've made the changes that Matt suggested, and fixed a bug causing an error when no halos survive the filtering process. Here are the new files. Feel free to play around with them and let me know if you have any more suggestions. It seems that most of the people who actually use this thing are on this list, so there's no hurry in getting it out. HaloProfiler.py - http://paste.enzotools.org/show/181/ HaloFilters.py - http://paste.enzotools.org/show/182/ runHaloProfiler.py - http://paste.enzotools.org/show/183/ Britton On Mon, Aug 24, 2009 at 2:10 PM, Matthew Turk <matthewturk@gmail.com> wrote:
Hi Britton,
This is awesome! I just have a couple comments.
* You can't include {} in a function dec -- it is instantiated at parse-time, so it persists and any modifications made are kept. Usually what's done is to default to None, and do an if something is None: something = {} * lists in function declaration must be copied with [:] ; in fact, I'd say avoid DeepCopy unless you know for a fact it will be something other than strings (which I don't think is the case for these objects) * addProfile and addProjection should be add_profile and add_projection; I'd prefer *all* code follow PEP-8 ( http://www.python.org/dev/peps/pep-0008/ ) but I'd definitely like to keep a consistent lowercase-underscore mechanism for methods on objects. (I also think we should try to follow the function declaration guidelines with respect to spaces and commas.) * na.array(range(something)) can just be na.arange(something) * if type(something) is list should be isinstance(something, types.ListType) (I think) * I'm inclined to think we don't want a second verbosity check, and instead put stuff into mylog.debug, but I'll leave that up to you...
All in all, this looks like a great addition, and a great set of extensions. Thanks so much for doing all this!
-Matt
Greetings,
I have just finished an extensive overhaul of the HaloProfiler that includes a bunch of new features, as well as completely changes the running syntax. Other than the addition of various new features, the main goals of this overhaul were: 1) to make the tool more general (mostly by removing hard-coded filtering by virial quantities), essentially allowing someone to profile a random list of points that may or may not be actual halos Easy filtering of halos by virial quantities remains an option, but is now simply a specific instance of something more general and far more
On Mon, Aug 24, 2009 at 12:51 PM, Britton Smith<brittonsmith@gmail.com> wrote: powerful:
a halo filtering device that allow the user to create their own filter functions to comb through radial profile data and decide whether a halo meets certain criteria.
2) to remove the dependence on an additional parameter file for the various options. The parameter file is now gone. Most of the HaloProfiler parameters have been turned into instantiation keyword args. The keyword list is now considerably longer, but the benefit is that the number of files needed to run this thing has been reduced from 2 (running script and par file) to just 1 (running script). There are a large number of keywords options that are specific to either the profile or projection routines that are taken in at instantiation and stored as attributes. This was done to keep those function calls simple. I'm curious to know peoples' thoughts on whether these keyword args should stay put or move to the individual function calls. Adding fields for profiling and projections have been moved to functions addProfile and addProjection.
Here is a brief list of the new features that I can remember: - the halo list read routine can be easily customized to read in
columned > ascii data of varying formats through the use of a dictionary (see attribute > self.halo_list_format) > - ability to change the function call, args, and kwargs for the halo finder > Profiles: > - filter halo list with user-written filter functions (see new file > HaloFilters.py for an example of a function to filter based on virial > quantities) > - extract and output scalar quantities from halos using the filter > functions > - pre-filter halos based on values in the initial halo list (skipping the > profiling altogether and saving a load of time) > Projections: > - choose the axes to be projected (instead of hard-coded to all three) > - easily select the list of halos to be projected (the total list, filtered > list, a new file, or an actual list) > > Before I commit all this, I'd like a little feedback, mostly on the > migration of the par file parameters to keyword args. I put three files in > the pastebin for people to download. I chose not to submit a diff since the > changes were so sweeping. > The new HaloProfiler.py: http://paste.enzotools.org/show/178/ > HaloFilters.py (should go in yt/extensions with HaloProfiler.py) > http://paste.enzotools.org/show/179/ > runHaloProfiler.py (an example script to run the new HaloProfiler) > http://paste.enzotools.org/show/180/ > > I will try my best to write up full documentation for this as soon as > possible, perhaps even today. > > Please let me know what you think. > > Britton > > _______________________________________________ > Yt-dev mailing list > Yt-dev@lists.spacepope.org > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org > > _______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Hi Britton,
I've made the changes that Matt suggested, and fixed a bug causing an error when no halos survive the filtering process. Here are the new files. Feel free to play around with them and let me know if you have any more suggestions. It seems that most of the people who actually use this thing are on this list, so there's no hurry in getting it out. HaloProfiler.py - http://paste.enzotools.org/show/181/ HaloFilters.py - http://paste.enzotools.org/show/182/ runHaloProfiler.py - http://paste.enzotools.org/show/183/
Did you commit these changes? If so, where (hg or svn-trunk)? If you did, I wasn't paying attention, sorry! Thanks! _______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________
Ok. So I'm lazy, this must be it: http://yt.enzotools.org/changeset/1416 Sorry for the inbox filler! _______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student ________________________________(_)_\(_)_______________
Yeah, I think that's it. Either way, all that stuff should be in there. On Mon, Oct 5, 2009 at 12:21 PM, Stephen Skory <stephenskory@yahoo.com>wrote:
Ok. So I'm lazy, this must be it:
http://yt.enzotools.org/changeset/1416
Sorry for the inbox filler!
_______________________________________________________ sskory@physics.ucsd.edu o__ Stephen Skory http://physics.ucsd.edu/~sskory/ <http://physics.ucsd.edu/%7Esskory/> _.>/ _Graduate Student ________________________________(_)_\(_)_______________ _______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
participants (3)
-
Britton Smith -
Matthew Turk -
Stephen Skory