[yt-users] Re: clump finder error with maximum density set

4 Oct 2018

      Okay, so I was trying to figure out why I am getting strange results and
went back to the IsolatedGalaxies cookbook case.  I am attaching the
cut_region verion here.  This may be a cut_region problem, not a clump
problem, though!

First, I ran the plain old cookbook case and got the same very lovely
clumps.

[image: clumps_reloaded_Projection_z_density_cookbook.png]

Then I changed c_max to 2e-24 based on some of the density values I was
getting, and got this:

[image: clumps_reloaded_Projection_z_density_max2en24.png]
Okay, fine, I just didn't set my limit high enough.  So when I run with max
= 7e-24:

[image: clumps_reloaded_Projection_z_density_max7en24.png]
But, when I select the same (or slightly larger) region with cut_region and
the same c_min and c_max (or just using the standard max and min
definitions) I cannot find any clumps!

[image: clumps_reloaded_Projection_z_density.png]

Do you think this is a cut_region or a clump_finder problem?

Thanks!
Stephanie

--
Dr. Stephanie Tonnesen
Associate Research Scientist
CCA, Flatiron Institute
New York, NY

stonnes@gmail.com

On Wed, Oct 3, 2018 at 3:29 PM Britton Smith  wrote:
...
It's my fault. I forgot to remove the imports
from ./yt/analysis_modules/level_sets/api.py.  If you change the import to
"from yt.data_objects.level_sets.api import ..." then it will work.  I'll
update the PR now.
On Wed, Oct 3, 2018 at 12:10 PM Stephanie Tonnesen 
wrote:
...
Okay, my git skills are not so good, so definitely not *sure*.  Just
tried to check by doing this:
cd yt-conda/yt
(root) -bash-4.2$ git pull https://github.com/brittonsmith/yt.git clurmps
From https://github.com/brittonsmith/yt
 * branch                clurmps    -> FETCH_HEAD
Already up-to-date.
so...I think yes?
I also just did the same thing in my script and got the same output.
--
Dr. Stephanie Tonnesen
Associate Research Scientist
CCA, Flatiron Institute
New York, NY
stonnes@gmail.com
On Wed, Oct 3, 2018 at 2:56 PM Nathan Goldbaum 
wrote:
...
Hi Stephanie,
Are you *sure* you pulled britton's pull request? He deleted that entry
in yt/analysis_modules/level_sets/api.py, so it looks like you're using a
version of that file from before his pull request.
-Nathan
On Wed, Oct 3, 2018 at 1:44 PM Stephanie Tonnesen 
wrote:
...
Go it, thanks!
I think I have pulled your update, Britton, and now my code crashes
almost immediately with this error.
yt_clump_fingert.py:4: VisibleDeprecationWarning: The level_sets module
has been moved to yt.data_objects.level_sets.This import is deprecated and
will be
removed in a future release.Please, change the import in your scripts
from 'from yt.analysis_modules.level_sets' to 'from
yt.data_objects.level_sets.'.
  from yt.analysis_modules.level_sets.api import *
Traceback (most recent call last):
  File "yt_clump_fingert.py", line 4, in <module>
    from yt.analysis_modules.level_sets.api import *
  File
"/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/api.py",
line 28, in <module>
    from yt.data_objects.level_sets.clump_handling import \
ImportError: cannot import name 'write_clump_index'
I am running:
yt version
yt module located at:
    /mnt/home/stonnesen/yt-conda/yt
The current version of yt is:
---
Version = 3.5.dev0
Changeset = ff956e8708c5
---
This installation CAN be automatically updated.
I am guessing that maybe I didn't include some other necessary update?
Thanks!!
Stephanie
--
Dr. Stephanie Tonnesen
Associate Research Scientist
CCA, Flatiron Institute
New York, NY
stonnes@gmail.com
On Tue, Oct 2, 2018 at 6:54 PM Britton Smith 
wrote:
...
Hi Stephanie,
While looking into this issue, I found some low hanging refactoring
fruit, so I've issued a pull request that cleans things up and takes care
of this as well. You can have a look here:
https://github.com/yt-project/yt/pull/2045
In running the example you attached, I found that the calculation
resulted in a single clump comprised of the entire domain, so the clump
annotation may be effectively invisible as it will draw a box around the
entire figure. If I used a slightly smaller smaller data container, I was
able to see the annotation, so I think things are ok. Let me know if it
still doesn't seem right.
Britton
On Tue, Oct 2, 2018 at 1:05 PM Stephanie Tonnesen 
wrote:
...
Hi John,
Thanks!  I *think* it works--here is my only confusion.  If I set
c_max to some value less than the max density (say, 1e-24 in the code I
attached previously), it doesn't crash anymore but it doesn't actually plot
a clump contour either.  If I set c_max to the max density setting (or I
guess a bit above, 10**np.floor(np.log10(data_source[field]).max()+1) ),
then I get a contour plotted.  Does this seem right to you?
Also, they both give me this message
"/mnt/xfs1/home/stonnesen/yt-conda/lib/python3.6/site-packages/matplotlib/contour.py:1180:
UserWarning: No contour levels w
ere found within the data range.
  warnings.warn("No contour levels were found"
"
Even though I seem to have one central contour in the c_max =
10**np.floor(np.log10(data_source[field]).max()+1) case.
Thanks!
Stephanie
--
Dr. Stephanie Tonnesen
Associate Research Scientist
CCA, Flatiron Institute
New York, NY
stonnes@gmail.com
On Tue, Oct 2, 2018 at 7:38 AM Wise, John H 
wrote:
> Hi Stephanie,
>
> I actually ran into this bug in the past week.  I fixed in for my
> case
> by making this change in clump_handling.py
>
> -            elif (child._validate()):
> +            elif ((child.children is not None) and
> child._validate()):
>
> I haven't had the time to submit a PR. If this works for you, let me
> know, and I can I create one this week.
>
> Thanks,
> John
>
>
> On 10/01/2018 09:48 PM, Stephanie Tonnesen wrote:
> > Hi,
> >
> > Sorry about the crazy delay, here is a script that reproduces the
> error
> > with the IsolatedGalaxy dataset!
> >
> > The script is attached.  I am happy to help someone look into it,
> or
> > look more myself with a bit of extra guidance.
> >
> > Thanks!!
> >
> > Stephanie
> >
> > --
> > Dr. Stephanie Tonnesen
> > Associate Research Scientist
> > CCA, Flatiron Institute
> > New York, NY
> >
> > stonnes@gmail.com mailto:stonnes@gmail.com
> >
> >
> > On Fri, Sep 14, 2018 at 6:51 AM Stephanie Tonnesen <
> stonnes@gmail.com
> > mailto:stonnes@gmail.com> wrote:
> >
> >     Hi Britton!
> >
> >     No, I have not resolved the issue, unfortunately.  I will try
> and
> >     get a script that reproduces the problem up soon.
> >
> >     Best,
> >     Stephanie
> >
> >     On Thu, Sep 13, 2018 at 7:45 PM Britton Smith
> >     mailto:brittonsmith@gmail.com>
> wrote:
> >
> >         HI Stephanie,
> >
> >         Have you managed to resolve this issue? If so, would you be
> >         willing to submit a pull request to get this fix in the
> main
> >         repo? If not, could you post a script that could be used to
> >         reproduce this error with some of the public data?
> >
> >         Britton
> >
> >         On Fri, Sep 7, 2018 at 1:05 PM Nathan Goldbaum
> >         mailto:nathan12343@gmail.com>
> wrote:
> >
> >             Generally an error like this is due to a bug in yt. I
> don't
> >             know enough about the code you're using to provide
> useful
> >             advice, but perhaps we should be checking if
> `children` is
> >             `None` and doing something else. The "workaround" is
> likely
> >             to fix whatever bug you've run into.
> >
> >             On Fri, Sep 7, 2018 at 2:56 PM, Stephanie Tonnesen
> >             mailto:stonnes@gmail.com> wrote:
> >
> >                 Hi yt-users!
> >
> >                 I have been fiddling some more with yt's clump
> finder
> >                 and have another question.  I want to use density
> to
> >                 find clumps, but want my maximum threshold value
> to be
> >                 below the fiducial setting--instead of above the
> maximum
> >                 possible density in the code I just want the
> finder to
> >                 stop searching at, say 3.2e-24.
> >
> >                 The code was working but running out of memory
> until I
> >                 set the max threshold, and I now get this error:
> >
> >                 ....lots of pleasant looking output messages....
> >                 Updating joins ... :   0%|          | 0/7
> [00:00 >                 Updating joins ... : 100%|██████████| 7/7
> [00:00<00:00,
> >                 48933.55it/s]
> >                 yt : [INFO     ] 2018-09-07 14:08:55,007
> Investigating 2
> >                 children.
> >                 yt : [INFO     ] 2018-09-07 14:08:55,007 Finding
> clumps:
> >                 min: 3.200000e-24, max: 3.200000e-24, step:
> 2.000000
> >                 yt : [INFO     ] 2018-09-07 14:08:55,041 Finding
> clumps:
> >                 min: 3.200000e-24, max: 3.200000e-24, step:
> 2.000000
> >                 yt : [INFO     ] 2018-09-07 14:08:55,042
> Eliminating
> >                 invalid, childless clump with 2 cells.
> >                 yt : [INFO     ] 2018-09-07 14:08:55,042 1 of 2
> children
> >                 survived, linking its children to parent.
> >                 Traceback (most recent call last):
> >                    File "yt_clump_fingert.py", line 59, in <module>
> >                      find_clumps(master_clump, c_min, c_max, step)
> >                    File
> >
>  "/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/clump_handling.py",
> >                 line 444, in find_clumps
> >                      find_clumps(child, min_val*d_clump, max_val,
> d_clump)
> >                    File
> >
>  "/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/clump_handling.py",
> >                 line 438, in find_clumps
> >                      find_clumps(clump, min_val*d_clump, max_val,
> d_clump)
> >                    File
> >
>  "/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/clump_handling.py",
> >                 line 438, in find_clumps
> >                      find_clumps(clump, min_val*d_clump, max_val,
> d_clump)
> >                    File
> >
>  "/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/clump_handling.py",
> >                 line 438, in find_clumps
> >                      find_clumps(clump, min_val*d_clump, max_val,
> d_clump)
> >                    File
> >
>  "/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/clump_handling.py",
> >                 line 461, in find_clumps
> >                      for child in clump.children:
> >                 TypeError: 'NoneType' object is not iterable
> >
> >                 I tried to circumvent this error by changing my
> max to
> >                 2e-24 (not a multiple of 2 of 1e-25), and got the
> same
> >                 error when the min was 3.2e-24 and the max was
> 2e-24 (!).
> >
> >                 Is there any way I can get around this error and
> have
> >                 the clumpfinder stop looking once it has clump
> edges at
> >                 my maximum density?
> >
> >                 Thanks!
> >                 Stephanie
> >                 --
> >                 Dr. Stephanie Tonnesen
> >                 Associate Research Scientist
> >                 CCA, Flatiron Institute
> >                 New York, NY
> >
> >                 stonnes@gmail.com mailto:stonnes@gmail.com
> >
> >                 _______________________________________________
> >                 yt-users mailing list -- yt-users@python.org
> >                 mailto:yt-users@python.org
> >                 To unsubscribe send an email to
> >                 yt-users-leave@python.org  yt-users-leave@python.org>
> >
> >
> >             _______________________________________________
> >             yt-users mailing list -- yt-users@python.org
> >             mailto:yt-users@python.org
> >             To unsubscribe send an email to
> yt-users-leave@python.org
> >             mailto:yt-users-leave@python.org
> >
> >         _______________________________________________
> >         yt-users mailing list -- yt-users@python.org
> >         mailto:yt-users@python.org
> >         To unsubscribe send an email to yt-users-leave@python.org
> >         mailto:yt-users-leave@python.org
> >
> >     --
> >     --
> >     Dr. Stephanie Tonnesen
> >     Associate Research Scientist
> >     CCA, Flatiron Institute
> >     New York, NY
> >
> >     stonnes@gmail.com mailto:stonnes@gmail.com
> >
> >
> >
> > _______________________________________________
> > yt-users mailing list -- yt-users@python.org
> > To unsubscribe send an email to yt-users-leave@python.org
> >
>
> --
> John Wise
> Associate Professor of Physics
> Center for Relativistic Astrophysics, Georgia Tech
> http://cosmo.gatech.edu
> _______________________________________________
> yt-users mailing list -- yt-users@python.org
> To unsubscribe send an email to yt-users-leave@python.org
>
_______________________________________________
yt-users mailing list -- yt-users@python.org
To unsubscribe send an email to yt-users-leave@python.org
_______________________________________________
yt-users mailing list -- yt-users@python.org
To unsubscribe send an email to yt-users-leave@python.org
_______________________________________________
yt-users mailing list -- yt-users@python.org
To unsubscribe send an email to yt-users-leave@python.org
_______________________________________________
yt-users mailing list -- yt-users@python.org
To unsubscribe send an email to yt-users-leave@python.org
_______________________________________________
yt-users mailing list -- yt-users@python.org
To unsubscribe send an email to yt-users-leave@python.org
_______________________________________________
yt-users mailing list -- yt-users@python.org
To unsubscribe send an email to yt-users-leave@python.org