Mailman 3 clump finder error with maximum density set - yt-users

clump finder error with maximum density set

Stephanie Tonnesen

8 Sep 2018 8 Sep '18

1:26 a.m.

Hi yt-users! I have been fiddling some more with yt's clump finder and have another question. I want to use density to find clumps, but want my maximum threshold value to be below the fiducial setting--instead of above the maximum possible density in the code I just want the finder to stop searching at, say 3.2e-24. The code was working but running out of memory until I set the max threshold, and I now get this error: ....lots of pleasant looking output messages.... Updating joins ... : 0%| | 0/7 [00:00

Attachments:

attachment.htm (text/html — 3.5 KB)

Show replies by date

Nathan Goldbaum

8 Sep 8 Sep

1:33 a.m.

Generally an error like this is due to a bug in yt. I don't know enough about the code you're using to provide useful advice, but perhaps we should be checking if `children` is `None` and doing something else. The "workaround" is likely to fix whatever bug you've run into. On Fri, Sep 7, 2018 at 2:56 PM, Stephanie Tonnesen wrote:

...

Hi yt-users!

I have been fiddling some more with yt's clump finder and have another question. I want to use density to find clumps, but want my maximum threshold value to be below the fiducial setting--instead of above the maximum possible density in the code I just want the finder to stop searching at, say 3.2e-24.

The code was working but running out of memory until I set the max threshold, and I now get this error:

....lots of pleasant looking output messages.... Updating joins ... : 0%| | 0/7 [00:00
I tried to circumvent this error by changing my max to 2e-24 (not a multiple of 2 of 1e-25), and got the same error when the min was 3.2e-24 and the max was 2e-24 (!).

Is there any way I can get around this error and have the clumpfinder stop looking once it has clump edges at my maximum density?

Thanks! Stephanie -- Dr. Stephanie Tonnesen Associate Research Scientist CCA, Flatiron Institute New York, NY

stonnes@gmail.com

_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org

Britton Smith

14 Sep 14 Sep

5:14 a.m.

HI Stephanie, Have you managed to resolve this issue? If so, would you be willing to submit a pull request to get this fix in the main repo? If not, could you post a script that could be used to reproduce this error with some of the public data? Britton On Fri, Sep 7, 2018 at 1:05 PM Nathan Goldbaum wrote:

...

Generally an error like this is due to a bug in yt. I don't know enough about the code you're using to provide useful advice, but perhaps we should be checking if `children` is `None` and doing something else. The "workaround" is likely to fix whatever bug you've run into.

On Fri, Sep 7, 2018 at 2:56 PM, Stephanie Tonnesen wrote:

...
Hi yt-users!

I have been fiddling some more with yt's clump finder and have another question. I want to use density to find clumps, but want my maximum threshold value to be below the fiducial setting--instead of above the maximum possible density in the code I just want the finder to stop searching at, say 3.2e-24.

The code was working but running out of memory until I set the max threshold, and I now get this error:

....lots of pleasant looking output messages.... Updating joins ... : 0%| | 0/7 [00:00
I tried to circumvent this error by changing my max to 2e-24 (not a multiple of 2 of 1e-25), and got the same error when the min was 3.2e-24 and the max was 2e-24 (!).

Is there any way I can get around this error and have the clumpfinder stop looking once it has clump edges at my maximum density?

Thanks! Stephanie -- Dr. Stephanie Tonnesen Associate Research Scientist CCA, Flatiron Institute New York, NY

stonnes@gmail.com

_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org

_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org

Stephanie Tonnesen

4:21 p.m.

Hi Britton! No, I have not resolved the issue, unfortunately. I will try and get a script that reproduces the problem up soon. Best, Stephanie On Thu, Sep 13, 2018 at 7:45 PM Britton Smith wrote:

...

HI Stephanie,

Have you managed to resolve this issue? If so, would you be willing to submit a pull request to get this fix in the main repo? If not, could you post a script that could be used to reproduce this error with some of the public data?

Britton

On Fri, Sep 7, 2018 at 1:05 PM Nathan Goldbaum wrote:

...
Generally an error like this is due to a bug in yt. I don't know enough about the code you're using to provide useful advice, but perhaps we should be checking if `children` is `None` and doing something else. The "workaround" is likely to fix whatever bug you've run into.

On Fri, Sep 7, 2018 at 2:56 PM, Stephanie Tonnesen wrote:

...
Hi yt-users!

I have been fiddling some more with yt's clump finder and have another question. I want to use density to find clumps, but want my maximum threshold value to be below the fiducial setting--instead of above the maximum possible density in the code I just want the finder to stop searching at, say 3.2e-24.

The code was working but running out of memory until I set the max threshold, and I now get this error:

....lots of pleasant looking output messages.... Updating joins ... : 0%| | 0/7 [00:00
I tried to circumvent this error by changing my max to 2e-24 (not a multiple of 2 of 1e-25), and got the same error when the min was 3.2e-24 and the max was 2e-24 (!).

Is there any way I can get around this error and have the clumpfinder stop looking once it has clump edges at my maximum density?

Thanks! Stephanie -- Dr. Stephanie Tonnesen Associate Research Scientist CCA, Flatiron Institute New York, NY

stonnes@gmail.com

_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org

_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org

_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org

-- -- Dr. Stephanie Tonnesen Associate Research Scientist CCA, Flatiron Institute New York, NY stonnes@gmail.com

Stephanie Tonnesen

2 Oct 2 Oct

7:18 a.m.

Hi, Sorry about the crazy delay, here is a script that reproduces the error with the IsolatedGalaxy dataset! The script is attached. I am happy to help someone look into it, or look more myself with a bit of extra guidance. Thanks!! Stephanie -- Dr. Stephanie Tonnesen Associate Research Scientist CCA, Flatiron Institute New York, NY stonnes@gmail.com On Fri, Sep 14, 2018 at 6:51 AM Stephanie Tonnesen wrote:

...

Hi Britton!

No, I have not resolved the issue, unfortunately. I will try and get a script that reproduces the problem up soon.

Best, Stephanie

On Thu, Sep 13, 2018 at 7:45 PM Britton Smith wrote:

...
HI Stephanie,

Have you managed to resolve this issue? If so, would you be willing to submit a pull request to get this fix in the main repo? If not, could you post a script that could be used to reproduce this error with some of the public data?

Britton

On Fri, Sep 7, 2018 at 1:05 PM Nathan Goldbaum wrote:

...
Generally an error like this is due to a bug in yt. I don't know enough about the code you're using to provide useful advice, but perhaps we should be checking if `children` is `None` and doing something else. The "workaround" is likely to fix whatever bug you've run into.

On Fri, Sep 7, 2018 at 2:56 PM, Stephanie Tonnesen wrote:

...
Hi yt-users!

I have been fiddling some more with yt's clump finder and have another question. I want to use density to find clumps, but want my maximum threshold value to be below the fiducial setting--instead of above the maximum possible density in the code I just want the finder to stop searching at, say 3.2e-24.

The code was working but running out of memory until I set the max threshold, and I now get this error:

....lots of pleasant looking output messages.... Updating joins ... : 0%| | 0/7 [00:00
I tried to circumvent this error by changing my max to 2e-24 (not a multiple of 2 of 1e-25), and got the same error when the min was 3.2e-24 and the max was 2e-24 (!).

Is there any way I can get around this error and have the clumpfinder stop looking once it has clump edges at my maximum density?

Thanks! Stephanie -- Dr. Stephanie Tonnesen Associate Research Scientist CCA, Flatiron Institute New York, NY

stonnes@gmail.com

_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org

_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org

_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org

-- -- Dr. Stephanie Tonnesen Associate Research Scientist CCA, Flatiron Institute New York, NY

stonnes@gmail.com

Wise, John H

5:06 p.m.

Hi Stephanie, I actually ran into this bug in the past week. I fixed in for my case by making this change in clump_handling.py - elif (child._validate()): + elif ((child.children is not None) and child._validate()): I haven't had the time to submit a PR. If this works for you, let me know, and I can I create one this week. Thanks, John On 10/01/2018 09:48 PM, Stephanie Tonnesen wrote:

...

Hi,

Sorry about the crazy delay, here is a script that reproduces the error with the IsolatedGalaxy dataset!

The script is attached. I am happy to help someone look into it, or look more myself with a bit of extra guidance.

Thanks!!

Stephanie

-- Dr. Stephanie Tonnesen Associate Research Scientist CCA, Flatiron Institute New York, NY

stonnes@gmail.com mailto:stonnes@gmail.com

On Fri, Sep 14, 2018 at 6:51 AM Stephanie Tonnesen mailto:stonnes@gmail.com> wrote:

Hi Britton!

No, I have not resolved the issue, unfortunately. I will try and get a script that reproduces the problem up soon.

Best, Stephanie

On Thu, Sep 13, 2018 at 7:45 PM Britton Smith mailto:brittonsmith@gmail.com> wrote:

HI Stephanie,

Have you managed to resolve this issue? If so, would you be willing to submit a pull request to get this fix in the main repo? If not, could you post a script that could be used to reproduce this error with some of the public data?

Britton

On Fri, Sep 7, 2018 at 1:05 PM Nathan Goldbaum mailto:nathan12343@gmail.com> wrote:

Generally an error like this is due to a bug in yt. I don't know enough about the code you're using to provide useful advice, but perhaps we should be checking if `children` is `None` and doing something else. The "workaround" is likely to fix whatever bug you've run into.

On Fri, Sep 7, 2018 at 2:56 PM, Stephanie Tonnesen mailto:stonnes@gmail.com> wrote:

Hi yt-users!

I have been fiddling some more with yt's clump finder and have another question. I want to use density to find clumps, but want my maximum threshold value to be below the fiducial setting--instead of above the maximum possible density in the code I just want the finder to stop searching at, say 3.2e-24.

The code was working but running out of memory until I set the max threshold, and I now get this error:

....lots of pleasant looking output messages.... Updating joins ... : 0%| | 0/7 [00:00
I tried to circumvent this error by changing my max to 2e-24 (not a multiple of 2 of 1e-25), and got the same error when the min was 3.2e-24 and the max was 2e-24 (!).

Is there any way I can get around this error and have the clumpfinder stop looking once it has clump edges at my maximum density?

Thanks! Stephanie -- Dr. Stephanie Tonnesen Associate Research Scientist CCA, Flatiron Institute New York, NY

stonnes@gmail.com mailto:stonnes@gmail.com

_______________________________________________ yt-users mailing list -- yt-users@python.org mailto:yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org mailto:yt-users-leave@python.org

_______________________________________________ yt-users mailing list -- yt-users@python.org mailto:yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org mailto:yt-users-leave@python.org

_______________________________________________ yt-users mailing list -- yt-users@python.org mailto:yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org mailto:yt-users-leave@python.org

-- -- Dr. Stephanie Tonnesen Associate Research Scientist CCA, Flatiron Institute New York, NY

stonnes@gmail.com mailto:stonnes@gmail.com

_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org

-- John Wise Associate Professor of Physics Center for Relativistic Astrophysics, Georgia Tech http://cosmo.gatech.edu

Stephanie Tonnesen

3 Oct 3 Oct

1:33 a.m.

Hi John, Thanks! I *think* it works--here is my only confusion. If I set c_max to some value less than the max density (say, 1e-24 in the code I attached previously), it doesn't crash anymore but it doesn't actually plot a clump contour either. If I set c_max to the max density setting (or I guess a bit above, 10**np.floor(np.log10(data_source[field]).max()+1) ), then I get a contour plotted. Does this seem right to you? Also, they both give me this message "/mnt/xfs1/home/stonnesen/yt-conda/lib/python3.6/site-packages/matplotlib/contour.py:1180: UserWarning: No contour levels w ere found within the data range. warnings.warn("No contour levels were found" " Even though I seem to have one central contour in the c_max = 10**np.floor(np.log10(data_source[field]).max()+1) case. Thanks! Stephanie -- Dr. Stephanie Tonnesen Associate Research Scientist CCA, Flatiron Institute New York, NY stonnes@gmail.com On Tue, Oct 2, 2018 at 7:38 AM Wise, John H wrote:

...

Hi Stephanie,

I actually ran into this bug in the past week. I fixed in for my case by making this change in clump_handling.py

- elif (child._validate()): + elif ((child.children is not None) and child._validate()):

I haven't had the time to submit a PR. If this works for you, let me know, and I can I create one this week.

Thanks, John

On 10/01/2018 09:48 PM, Stephanie Tonnesen wrote:

...
Hi,

Sorry about the crazy delay, here is a script that reproduces the error with the IsolatedGalaxy dataset!

The script is attached. I am happy to help someone look into it, or look more myself with a bit of extra guidance.

Thanks!!

Stephanie

-- Dr. Stephanie Tonnesen Associate Research Scientist CCA, Flatiron Institute New York, NY

stonnes@gmail.com mailto:stonnes@gmail.com

On Fri, Sep 14, 2018 at 6:51 AM Stephanie Tonnesen mailto:stonnes@gmail.com> wrote:

Hi Britton!

No, I have not resolved the issue, unfortunately. I will try and get a script that reproduces the problem up soon.

Best, Stephanie

On Thu, Sep 13, 2018 at 7:45 PM Britton Smith mailto:brittonsmith@gmail.com> wrote:

HI Stephanie,

Have you managed to resolve this issue? If so, would you be willing to submit a pull request to get this fix in the main repo? If not, could you post a script that could be used to reproduce this error with some of the public data?

Britton

On Fri, Sep 7, 2018 at 1:05 PM Nathan Goldbaum mailto:nathan12343@gmail.com> wrote:

Generally an error like this is due to a bug in yt. I don't know enough about the code you're using to provide useful advice, but perhaps we should be checking if `children` is `None` and doing something else. The "workaround" is likely to fix whatever bug you've run into.

On Fri, Sep 7, 2018 at 2:56 PM, Stephanie Tonnesen mailto:stonnes@gmail.com> wrote:

Hi yt-users!

I have been fiddling some more with yt's clump finder and have another question. I want to use density to find clumps, but want my maximum threshold value to be below the fiducial setting--instead of above the maximum possible density in the code I just want the finder to stop searching at, say 3.2e-24.

The code was working but running out of memory until I set the max threshold, and I now get this error:

....lots of pleasant looking output messages.... Updating joins ... : 0%| | 0/7 [00:00
"/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/clump_handling.py",

...
line 444, in find_clumps find_clumps(child, min_val*d_clump, max_val,

d_clump)

...
File

"/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/clump_handling.py",

...
line 438, in find_clumps find_clumps(clump, min_val*d_clump, max_val,

d_clump)

...
File

"/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/clump_handling.py",

...
line 438, in find_clumps find_clumps(clump, min_val*d_clump, max_val,

d_clump)

...
File

"/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/clump_handling.py",

...
line 438, in find_clumps find_clumps(clump, min_val*d_clump, max_val,

d_clump)

...
File

"/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/clump_handling.py",

...
line 461, in find_clumps for child in clump.children: TypeError: 'NoneType' object is not iterable

I tried to circumvent this error by changing my max to 2e-24 (not a multiple of 2 of 1e-25), and got the same error when the min was 3.2e-24 and the max was 2e-24 (!).

Is there any way I can get around this error and have the clumpfinder stop looking once it has clump edges at my maximum density?

Thanks! Stephanie -- Dr. Stephanie Tonnesen Associate Research Scientist CCA, Flatiron Institute New York, NY

stonnes@gmail.com mailto:stonnes@gmail.com

_______________________________________________ yt-users mailing list -- yt-users@python.org mailto:yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org
yt-users-leave@python.org>

...
_______________________________________________ yt-users mailing list -- yt-users@python.org mailto:yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org mailto:yt-users-leave@python.org

_______________________________________________ yt-users mailing list -- yt-users@python.org mailto:yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org mailto:yt-users-leave@python.org

-- -- Dr. Stephanie Tonnesen Associate Research Scientist CCA, Flatiron Institute New York, NY

stonnes@gmail.com mailto:stonnes@gmail.com

_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org

-- John Wise Associate Professor of Physics Center for Relativistic Astrophysics, Georgia Tech http://cosmo.gatech.edu _______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org

Britton Smith

4:21 a.m.

Hi Stephanie, While looking into this issue, I found some low hanging refactoring fruit, so I've issued a pull request that cleans things up and takes care of this as well. You can have a look here: https://github.com/yt-project/yt/pull/2045 In running the example you attached, I found that the calculation resulted in a single clump comprised of the entire domain, so the clump annotation may be effectively invisible as it will draw a box around the entire figure. If I used a slightly smaller smaller data container, I was able to see the annotation, so I think things are ok. Let me know if it still doesn't seem right. Britton On Tue, Oct 2, 2018 at 1:05 PM Stephanie Tonnesen wrote:

...

Hi John,

Thanks! I *think* it works--here is my only confusion. If I set c_max to some value less than the max density (say, 1e-24 in the code I attached previously), it doesn't crash anymore but it doesn't actually plot a clump contour either. If I set c_max to the max density setting (or I guess a bit above, 10**np.floor(np.log10(data_source[field]).max()+1) ), then I get a contour plotted. Does this seem right to you?

Also, they both give me this message "/mnt/xfs1/home/stonnesen/yt-conda/lib/python3.6/site-packages/matplotlib/contour.py:1180: UserWarning: No contour levels w ere found within the data range. warnings.warn("No contour levels were found" " Even though I seem to have one central contour in the c_max = 10**np.floor(np.log10(data_source[field]).max()+1) case.

Thanks! Stephanie

-- Dr. Stephanie Tonnesen Associate Research Scientist CCA, Flatiron Institute New York, NY

stonnes@gmail.com

On Tue, Oct 2, 2018 at 7:38 AM Wise, John H wrote:

...
Hi Stephanie,

I actually ran into this bug in the past week. I fixed in for my case by making this change in clump_handling.py

- elif (child._validate()): + elif ((child.children is not None) and child._validate()):

I haven't had the time to submit a PR. If this works for you, let me know, and I can I create one this week.

Thanks, John

On 10/01/2018 09:48 PM, Stephanie Tonnesen wrote:

...
Hi,

Sorry about the crazy delay, here is a script that reproduces the error with the IsolatedGalaxy dataset!

The script is attached. I am happy to help someone look into it, or look more myself with a bit of extra guidance.

Thanks!!

Stephanie

-- Dr. Stephanie Tonnesen Associate Research Scientist CCA, Flatiron Institute New York, NY

stonnes@gmail.com mailto:stonnes@gmail.com

On Fri, Sep 14, 2018 at 6:51 AM Stephanie Tonnesen mailto:stonnes@gmail.com> wrote:

Hi Britton!

No, I have not resolved the issue, unfortunately. I will try and get a script that reproduces the problem up soon.

Best, Stephanie

On Thu, Sep 13, 2018 at 7:45 PM Britton Smith mailto:brittonsmith@gmail.com> wrote:

HI Stephanie,

Have you managed to resolve this issue? If so, would you be willing to submit a pull request to get this fix in the main repo? If not, could you post a script that could be used to reproduce this error with some of the public data?

Britton

On Fri, Sep 7, 2018 at 1:05 PM Nathan Goldbaum mailto:nathan12343@gmail.com> wrote:

Generally an error like this is due to a bug in yt. I don't know enough about the code you're using to provide useful advice, but perhaps we should be checking if `children` is `None` and doing something else. The "workaround" is likely to fix whatever bug you've run into.

On Fri, Sep 7, 2018 at 2:56 PM, Stephanie Tonnesen mailto:stonnes@gmail.com> wrote:

Hi yt-users!

I have been fiddling some more with yt's clump finder and have another question. I want to use density to find clumps, but want my maximum threshold value to be below the fiducial setting--instead of above the maximum possible density in the code I just want the finder to stop searching at, say 3.2e-24.

The code was working but running out of memory until I set the max threshold, and I now get this error:

....lots of pleasant looking output messages.... Updating joins ... : 0%| | 0/7 [00:00
"/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/clump_handling.py",

...
line 444, in find_clumps find_clumps(child, min_val*d_clump, max_val,

d_clump)

...
File

"/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/clump_handling.py",

...
line 438, in find_clumps find_clumps(clump, min_val*d_clump, max_val,

d_clump)

...
File

"/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/clump_handling.py",

...
line 438, in find_clumps find_clumps(clump, min_val*d_clump, max_val,

d_clump)

...
File

"/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/clump_handling.py",

...
line 438, in find_clumps find_clumps(clump, min_val*d_clump, max_val,

d_clump)

...
File

"/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/clump_handling.py",

...
line 461, in find_clumps for child in clump.children: TypeError: 'NoneType' object is not iterable

I tried to circumvent this error by changing my max to 2e-24 (not a multiple of 2 of 1e-25), and got the same error when the min was 3.2e-24 and the max was 2e-24

(!).

...
Is there any way I can get around this error and have the clumpfinder stop looking once it has clump edges at my maximum density?

Thanks! Stephanie -- Dr. Stephanie Tonnesen Associate Research Scientist CCA, Flatiron Institute New York, NY

stonnes@gmail.com mailto:stonnes@gmail.com

_______________________________________________ yt-users mailing list -- yt-users@python.org mailto:yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org
yt-users-leave@python.org>

...
_______________________________________________ yt-users mailing list -- yt-users@python.org mailto:yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org mailto:yt-users-leave@python.org

_______________________________________________ yt-users mailing list -- yt-users@python.org mailto:yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org mailto:yt-users-leave@python.org

-- -- Dr. Stephanie Tonnesen Associate Research Scientist CCA, Flatiron Institute New York, NY

stonnes@gmail.com mailto:stonnes@gmail.com

_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org

-- John Wise Associate Professor of Physics Center for Relativistic Astrophysics, Georgia Tech http://cosmo.gatech.edu _______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org

_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org

Stephanie Tonnesen

4 Oct 4 Oct

12:11 a.m.

Go it, thanks! I think I have pulled your update, Britton, and now my code crashes almost immediately with this error. yt_clump_fingert.py:4: VisibleDeprecationWarning: The level_sets module has been moved to yt.data_objects.level_sets.This import is deprecated and will be removed in a future release.Please, change the import in your scripts from 'from yt.analysis_modules.level_sets' to 'from yt.data_objects.level_sets.'. from yt.analysis_modules.level_sets.api import * Traceback (most recent call last): File "yt_clump_fingert.py", line 4, in <module> from yt.analysis_modules.level_sets.api import * File "/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/api.py", line 28, in <module> from yt.data_objects.level_sets.clump_handling import \ ImportError: cannot import name 'write_clump_index' I am running: yt version yt module located at: /mnt/home/stonnesen/yt-conda/yt The current version of yt is: --- Version = 3.5.dev0 Changeset = ff956e8708c5 --- This installation CAN be automatically updated. I am guessing that maybe I didn't include some other necessary update? Thanks!! Stephanie -- Dr. Stephanie Tonnesen Associate Research Scientist CCA, Flatiron Institute New York, NY stonnes@gmail.com On Tue, Oct 2, 2018 at 6:54 PM Britton Smith wrote:

...

Hi Stephanie,

While looking into this issue, I found some low hanging refactoring fruit, so I've issued a pull request that cleans things up and takes care of this as well. You can have a look here: https://github.com/yt-project/yt/pull/2045

In running the example you attached, I found that the calculation resulted in a single clump comprised of the entire domain, so the clump annotation may be effectively invisible as it will draw a box around the entire figure. If I used a slightly smaller smaller data container, I was able to see the annotation, so I think things are ok. Let me know if it still doesn't seem right.

Britton

On Tue, Oct 2, 2018 at 1:05 PM Stephanie Tonnesen wrote:

...
Hi John,

Thanks! I *think* it works--here is my only confusion. If I set c_max to some value less than the max density (say, 1e-24 in the code I attached previously), it doesn't crash anymore but it doesn't actually plot a clump contour either. If I set c_max to the max density setting (or I guess a bit above, 10**np.floor(np.log10(data_source[field]).max()+1) ), then I get a contour plotted. Does this seem right to you?

Also, they both give me this message "/mnt/xfs1/home/stonnesen/yt-conda/lib/python3.6/site-packages/matplotlib/contour.py:1180: UserWarning: No contour levels w ere found within the data range. warnings.warn("No contour levels were found" " Even though I seem to have one central contour in the c_max = 10**np.floor(np.log10(data_source[field]).max()+1) case.

Thanks! Stephanie

-- Dr. Stephanie Tonnesen Associate Research Scientist CCA, Flatiron Institute New York, NY

stonnes@gmail.com

On Tue, Oct 2, 2018 at 7:38 AM Wise, John H wrote:

...
Hi Stephanie,

I actually ran into this bug in the past week. I fixed in for my case by making this change in clump_handling.py

- elif (child._validate()): + elif ((child.children is not None) and child._validate()):

I haven't had the time to submit a PR. If this works for you, let me know, and I can I create one this week.

Thanks, John

On 10/01/2018 09:48 PM, Stephanie Tonnesen wrote:

...
Hi,

Sorry about the crazy delay, here is a script that reproduces the error with the IsolatedGalaxy dataset!

The script is attached. I am happy to help someone look into it, or look more myself with a bit of extra guidance.

Thanks!!

Stephanie

-- Dr. Stephanie Tonnesen Associate Research Scientist CCA, Flatiron Institute New York, NY

stonnes@gmail.com mailto:stonnes@gmail.com

On Fri, Sep 14, 2018 at 6:51 AM Stephanie Tonnesen mailto:stonnes@gmail.com> wrote:

Hi Britton!

No, I have not resolved the issue, unfortunately. I will try and get a script that reproduces the problem up soon.

Best, Stephanie

On Thu, Sep 13, 2018 at 7:45 PM Britton Smith mailto:brittonsmith@gmail.com> wrote:

HI Stephanie,

Have you managed to resolve this issue? If so, would you be willing to submit a pull request to get this fix in the main repo? If not, could you post a script that could be used to reproduce this error with some of the public data?

Britton

On Fri, Sep 7, 2018 at 1:05 PM Nathan Goldbaum mailto:nathan12343@gmail.com> wrote:

Generally an error like this is due to a bug in yt. I don't know enough about the code you're using to provide useful advice, but perhaps we should be checking if `children` is `None` and doing something else. The "workaround" is likely to fix whatever bug you've run into.

On Fri, Sep 7, 2018 at 2:56 PM, Stephanie Tonnesen mailto:stonnes@gmail.com> wrote:

Hi yt-users!

I have been fiddling some more with yt's clump finder and have another question. I want to use density to find clumps, but want my maximum threshold value to be below the fiducial setting--instead of above the maximum possible density in the code I just want the finder to stop searching at, say 3.2e-24.

The code was working but running out of memory until I set the max threshold, and I now get this error:

....lots of pleasant looking output messages.... Updating joins ... : 0%| | 0/7 [00:00
"/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/clump_handling.py",

...
line 444, in find_clumps find_clumps(child, min_val*d_clump, max_val,

d_clump)

...
File

"/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/clump_handling.py",

...
line 438, in find_clumps find_clumps(clump, min_val*d_clump, max_val,

d_clump)

...
File

"/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/clump_handling.py",

...
line 438, in find_clumps find_clumps(clump, min_val*d_clump, max_val,

d_clump)

...
File

"/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/clump_handling.py",

...
line 438, in find_clumps find_clumps(clump, min_val*d_clump, max_val,

d_clump)

...
File

"/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/clump_handling.py",

...
line 461, in find_clumps for child in clump.children: TypeError: 'NoneType' object is not iterable

I tried to circumvent this error by changing my max to 2e-24 (not a multiple of 2 of 1e-25), and got the same error when the min was 3.2e-24 and the max was 2e-24

(!).

...
Is there any way I can get around this error and have the clumpfinder stop looking once it has clump edges at my maximum density?

Thanks! Stephanie -- Dr. Stephanie Tonnesen Associate Research Scientist CCA, Flatiron Institute New York, NY

stonnes@gmail.com mailto:stonnes@gmail.com

_______________________________________________ yt-users mailing list -- yt-users@python.org mailto:yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org
yt-users-leave@python.org>

...
_______________________________________________ yt-users mailing list -- yt-users@python.org mailto:yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org mailto:yt-users-leave@python.org

_______________________________________________ yt-users mailing list -- yt-users@python.org mailto:yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org mailto:yt-users-leave@python.org

-- -- Dr. Stephanie Tonnesen Associate Research Scientist CCA, Flatiron Institute New York, NY

stonnes@gmail.com mailto:stonnes@gmail.com

_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org

-- John Wise Associate Professor of Physics Center for Relativistic Astrophysics, Georgia Tech http://cosmo.gatech.edu _______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org

_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org

_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org

Nathan Goldbaum

12:22 a.m.

Hi Stephanie, Are you *sure* you pulled britton's pull request? He deleted that entry in yt/analysis_modules/level_sets/api.py, so it looks like you're using a version of that file from before his pull request. -Nathan On Wed, Oct 3, 2018 at 1:44 PM Stephanie Tonnesen wrote:

...

Go it, thanks!

I think I have pulled your update, Britton, and now my code crashes almost immediately with this error.

yt_clump_fingert.py:4: VisibleDeprecationWarning: The level_sets module has been moved to yt.data_objects.level_sets.This import is deprecated and will be removed in a future release.Please, change the import in your scripts from 'from yt.analysis_modules.level_sets' to 'from yt.data_objects.level_sets.'. from yt.analysis_modules.level_sets.api import * Traceback (most recent call last): File "yt_clump_fingert.py", line 4, in <module> from yt.analysis_modules.level_sets.api import * File "/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/api.py", line 28, in <module> from yt.data_objects.level_sets.clump_handling import \ ImportError: cannot import name 'write_clump_index'

I am running: yt version

yt module located at: /mnt/home/stonnesen/yt-conda/yt

The current version of yt is:

--- Version = 3.5.dev0 Changeset = ff956e8708c5 --- This installation CAN be automatically updated.

I am guessing that maybe I didn't include some other necessary update?

Thanks!! Stephanie -- Dr. Stephanie Tonnesen Associate Research Scientist CCA, Flatiron Institute New York, NY

stonnes@gmail.com

On Tue, Oct 2, 2018 at 6:54 PM Britton Smith wrote:

...
Hi Stephanie,

While looking into this issue, I found some low hanging refactoring fruit, so I've issued a pull request that cleans things up and takes care of this as well. You can have a look here: https://github.com/yt-project/yt/pull/2045

In running the example you attached, I found that the calculation resulted in a single clump comprised of the entire domain, so the clump annotation may be effectively invisible as it will draw a box around the entire figure. If I used a slightly smaller smaller data container, I was able to see the annotation, so I think things are ok. Let me know if it still doesn't seem right.

Britton

On Tue, Oct 2, 2018 at 1:05 PM Stephanie Tonnesen wrote:

...
Hi John,

Thanks! I *think* it works--here is my only confusion. If I set c_max to some value less than the max density (say, 1e-24 in the code I attached previously), it doesn't crash anymore but it doesn't actually plot a clump contour either. If I set c_max to the max density setting (or I guess a bit above, 10**np.floor(np.log10(data_source[field]).max()+1) ), then I get a contour plotted. Does this seem right to you?

Also, they both give me this message "/mnt/xfs1/home/stonnesen/yt-conda/lib/python3.6/site-packages/matplotlib/contour.py:1180: UserWarning: No contour levels w ere found within the data range. warnings.warn("No contour levels were found" " Even though I seem to have one central contour in the c_max = 10**np.floor(np.log10(data_source[field]).max()+1) case.

Thanks! Stephanie

-- Dr. Stephanie Tonnesen Associate Research Scientist CCA, Flatiron Institute New York, NY

stonnes@gmail.com

On Tue, Oct 2, 2018 at 7:38 AM Wise, John H wrote:

...
Hi Stephanie,

I actually ran into this bug in the past week. I fixed in for my case by making this change in clump_handling.py

- elif (child._validate()): + elif ((child.children is not None) and child._validate()):

I haven't had the time to submit a PR. If this works for you, let me know, and I can I create one this week.

Thanks, John

...
Hi,

Sorry about the crazy delay, here is a script that reproduces the error with the IsolatedGalaxy dataset!

The script is attached. I am happy to help someone look into it, or look more myself with a bit of extra guidance.

Thanks!!

Stephanie

-- Dr. Stephanie Tonnesen Associate Research Scientist CCA, Flatiron Institute New York, NY

stonnes@gmail.com mailto:stonnes@gmail.com

On Fri, Sep 14, 2018 at 6:51 AM Stephanie Tonnesen mailto:stonnes@gmail.com> wrote:

Hi Britton!

No, I have not resolved the issue, unfortunately. I will try and get a script that reproduces the problem up soon.

Best, Stephanie

On Thu, Sep 13, 2018 at 7:45 PM Britton Smith mailto:brittonsmith@gmail.com> wrote:

HI Stephanie,

Have you managed to resolve this issue? If so, would you be willing to submit a pull request to get this fix in the main repo? If not, could you post a script that could be used to reproduce this error with some of the public data?

Britton

On Fri, Sep 7, 2018 at 1:05 PM Nathan Goldbaum mailto:nathan12343@gmail.com> wrote:

Generally an error like this is due to a bug in yt. I don't know enough about the code you're using to provide useful advice, but perhaps we should be checking if `children` is `None` and doing something else. The "workaround" is

On 10/01/2018 09:48 PM, Stephanie Tonnesen wrote: likely

...
to fix whatever bug you've run into.

On Fri, Sep 7, 2018 at 2:56 PM, Stephanie Tonnesen mailto:stonnes@gmail.com> wrote:

Hi yt-users!

I have been fiddling some more with yt's clump finder and have another question. I want to use density to find clumps, but want my maximum threshold value to be below the fiducial setting--instead of above the

maximum

...
possible density in the code I just want the finder to stop searching at, say 3.2e-24.

The code was working but running out of memory until I set the max threshold, and I now get this error:

....lots of pleasant looking output messages.... Updating joins ... : 0%| | 0/7 [00:00
?it/s]

...
Updating joins ... : 100%|██████████| 7/7

[00:00<00:00,

...
48933.55it/s] yt : [INFO ] 2018-09-07 14:08:55,007

Investigating 2

...
children. yt : [INFO ] 2018-09-07 14:08:55,007 Finding

clumps:

...
min: 3.200000e-24, max: 3.200000e-24, step: 2.000000 yt : [INFO ] 2018-09-07 14:08:55,041 Finding

clumps:

...
min: 3.200000e-24, max: 3.200000e-24, step: 2.000000 yt : [INFO ] 2018-09-07 14:08:55,042 Eliminating invalid, childless clump with 2 cells. yt : [INFO ] 2018-09-07 14:08:55,042 1 of 2

children

...
survived, linking its children to parent. Traceback (most recent call last): File "yt_clump_fingert.py", line 59, in <module> find_clumps(master_clump, c_min, c_max, step) File

"/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/clump_handling.py",

...
line 444, in find_clumps find_clumps(child, min_val*d_clump, max_val,

d_clump)

...
File

"/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/clump_handling.py",

...
line 438, in find_clumps find_clumps(clump, min_val*d_clump, max_val,

d_clump)

...
File

"/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/clump_handling.py",

...
line 438, in find_clumps find_clumps(clump, min_val*d_clump, max_val,

d_clump)

...
File

"/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/clump_handling.py",

...
line 438, in find_clumps find_clumps(clump, min_val*d_clump, max_val,

d_clump)

...
File

"/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/clump_handling.py",

...
line 461, in find_clumps for child in clump.children: TypeError: 'NoneType' object is not iterable

I tried to circumvent this error by changing my max to 2e-24 (not a multiple of 2 of 1e-25), and got the same error when the min was 3.2e-24 and the max was 2e-24

(!).

...
Is there any way I can get around this error and have the clumpfinder stop looking once it has clump edges

at

...
my maximum density?

Thanks! Stephanie -- Dr. Stephanie Tonnesen Associate Research Scientist CCA, Flatiron Institute New York, NY

stonnes@gmail.com mailto:stonnes@gmail.com

_______________________________________________ yt-users mailing list -- yt-users@python.org mailto:yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org
yt-users-leave@python.org>

...
_______________________________________________ yt-users mailing list -- yt-users@python.org mailto:yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org mailto:yt-users-leave@python.org

_______________________________________________ yt-users mailing list -- yt-users@python.org mailto:yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org mailto:yt-users-leave@python.org

-- -- Dr. Stephanie Tonnesen Associate Research Scientist CCA, Flatiron Institute New York, NY

stonnes@gmail.com mailto:stonnes@gmail.com

_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org

-- John Wise Associate Professor of Physics Center for Relativistic Astrophysics, Georgia Tech http://cosmo.gatech.edu _______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org

_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org

_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org

_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org

Stephanie Tonnesen

12:37 a.m.

Okay, my git skills are not so good, so definitely not *sure*. Just tried to check by doing this: cd yt-conda/yt (root) -bash-4.2$ git pull https://github.com/brittonsmith/yt.git clurmps From https://github.com/brittonsmith/yt * branch clurmps -> FETCH_HEAD Already up-to-date. so...I think yes? I also just did the same thing in my script and got the same output. -- Dr. Stephanie Tonnesen Associate Research Scientist CCA, Flatiron Institute New York, NY stonnes@gmail.com On Wed, Oct 3, 2018 at 2:56 PM Nathan Goldbaum wrote:

...

Hi Stephanie,

Are you *sure* you pulled britton's pull request? He deleted that entry in yt/analysis_modules/level_sets/api.py, so it looks like you're using a version of that file from before his pull request.

-Nathan

On Wed, Oct 3, 2018 at 1:44 PM Stephanie Tonnesen wrote:

...
Go it, thanks!

I think I have pulled your update, Britton, and now my code crashes almost immediately with this error.

yt_clump_fingert.py:4: VisibleDeprecationWarning: The level_sets module has been moved to yt.data_objects.level_sets.This import is deprecated and will be removed in a future release.Please, change the import in your scripts from 'from yt.analysis_modules.level_sets' to 'from yt.data_objects.level_sets.'. from yt.analysis_modules.level_sets.api import * Traceback (most recent call last): File "yt_clump_fingert.py", line 4, in <module> from yt.analysis_modules.level_sets.api import * File "/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/api.py", line 28, in <module> from yt.data_objects.level_sets.clump_handling import \ ImportError: cannot import name 'write_clump_index'

I am running: yt version

yt module located at: /mnt/home/stonnesen/yt-conda/yt

The current version of yt is:

--- Version = 3.5.dev0 Changeset = ff956e8708c5 --- This installation CAN be automatically updated.

I am guessing that maybe I didn't include some other necessary update?

Thanks!! Stephanie -- Dr. Stephanie Tonnesen Associate Research Scientist CCA, Flatiron Institute New York, NY

stonnes@gmail.com

On Tue, Oct 2, 2018 at 6:54 PM Britton Smith wrote:

...
Hi Stephanie,

While looking into this issue, I found some low hanging refactoring fruit, so I've issued a pull request that cleans things up and takes care of this as well. You can have a look here: https://github.com/yt-project/yt/pull/2045

In running the example you attached, I found that the calculation resulted in a single clump comprised of the entire domain, so the clump annotation may be effectively invisible as it will draw a box around the entire figure. If I used a slightly smaller smaller data container, I was able to see the annotation, so I think things are ok. Let me know if it still doesn't seem right.

Britton

On Tue, Oct 2, 2018 at 1:05 PM Stephanie Tonnesen wrote:

...
Hi John,

Thanks! I *think* it works--here is my only confusion. If I set c_max to some value less than the max density (say, 1e-24 in the code I attached previously), it doesn't crash anymore but it doesn't actually plot a clump contour either. If I set c_max to the max density setting (or I guess a bit above, 10**np.floor(np.log10(data_source[field]).max()+1) ), then I get a contour plotted. Does this seem right to you?

Also, they both give me this message "/mnt/xfs1/home/stonnesen/yt-conda/lib/python3.6/site-packages/matplotlib/contour.py:1180: UserWarning: No contour levels w ere found within the data range. warnings.warn("No contour levels were found" " Even though I seem to have one central contour in the c_max = 10**np.floor(np.log10(data_source[field]).max()+1) case.

Thanks! Stephanie

-- Dr. Stephanie Tonnesen Associate Research Scientist CCA, Flatiron Institute New York, NY

stonnes@gmail.com

On Tue, Oct 2, 2018 at 7:38 AM Wise, John H wrote:

...
Hi Stephanie,

I actually ran into this bug in the past week. I fixed in for my case by making this change in clump_handling.py

- elif (child._validate()): + elif ((child.children is not None) and child._validate()):

I haven't had the time to submit a PR. If this works for you, let me know, and I can I create one this week.

Thanks, John

...
Hi,

Sorry about the crazy delay, here is a script that reproduces the error with the IsolatedGalaxy dataset!

The script is attached. I am happy to help someone look into it, or look more myself with a bit of extra guidance.

Thanks!!

Stephanie

-- Dr. Stephanie Tonnesen Associate Research Scientist CCA, Flatiron Institute New York, NY

stonnes@gmail.com mailto:stonnes@gmail.com

On Fri, Sep 14, 2018 at 6:51 AM Stephanie Tonnesen < stonnes@gmail.com mailto:stonnes@gmail.com> wrote:

Hi Britton!

No, I have not resolved the issue, unfortunately. I will try and get a script that reproduces the problem up soon.

Best, Stephanie

On Thu, Sep 13, 2018 at 7:45 PM Britton Smith mailto:brittonsmith@gmail.com> wrote:

HI Stephanie,

Have you managed to resolve this issue? If so, would you be willing to submit a pull request to get this fix in the main repo? If not, could you post a script that could be used to reproduce this error with some of the public data?

Britton

On Fri, Sep 7, 2018 at 1:05 PM Nathan Goldbaum mailto:nathan12343@gmail.com> wrote:

Generally an error like this is due to a bug in yt. I don't know enough about the code you're using to provide useful advice, but perhaps we should be checking if `children` is `None` and doing something else. The "workaround" is

On 10/01/2018 09:48 PM, Stephanie Tonnesen wrote: likely

...
to fix whatever bug you've run into.

On Fri, Sep 7, 2018 at 2:56 PM, Stephanie Tonnesen mailto:stonnes@gmail.com> wrote:

Hi yt-users!

I have been fiddling some more with yt's clump finder and have another question. I want to use density to find clumps, but want my maximum threshold value to

be

...
below the fiducial setting--instead of above the

maximum

...
possible density in the code I just want the finder

to

...
stop searching at, say 3.2e-24.

The code was working but running out of memory until

I

...
set the max threshold, and I now get this error:

....lots of pleasant looking output messages.... Updating joins ... : 0%| | 0/7 [00:00
?it/s]

...
Updating joins ... : 100%|██████████| 7/7

[00:00<00:00,

...
48933.55it/s] yt : [INFO ] 2018-09-07 14:08:55,007

Investigating 2

...
children. yt : [INFO ] 2018-09-07 14:08:55,007 Finding

clumps:

...
min: 3.200000e-24, max: 3.200000e-24, step: 2.000000 yt : [INFO ] 2018-09-07 14:08:55,041 Finding

clumps:

...
min: 3.200000e-24, max: 3.200000e-24, step: 2.000000 yt : [INFO ] 2018-09-07 14:08:55,042 Eliminating invalid, childless clump with 2 cells. yt : [INFO ] 2018-09-07 14:08:55,042 1 of 2

children

...
survived, linking its children to parent. Traceback (most recent call last): File "yt_clump_fingert.py", line 59, in <module> find_clumps(master_clump, c_min, c_max, step) File

"/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/clump_handling.py",

...
line 444, in find_clumps find_clumps(child, min_val*d_clump, max_val,

d_clump)

...
File

"/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/clump_handling.py",

...
line 438, in find_clumps find_clumps(clump, min_val*d_clump, max_val,

d_clump)

...
File

"/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/clump_handling.py",

...
line 438, in find_clumps find_clumps(clump, min_val*d_clump, max_val,

d_clump)

...
File

"/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/clump_handling.py",

...
line 438, in find_clumps find_clumps(clump, min_val*d_clump, max_val,

d_clump)

...
File

"/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/clump_handling.py",

...
line 461, in find_clumps for child in clump.children: TypeError: 'NoneType' object is not iterable

I tried to circumvent this error by changing my max

to

...
2e-24 (not a multiple of 2 of 1e-25), and got the

same

...
error when the min was 3.2e-24 and the max was 2e-24

(!).

...
Is there any way I can get around this error and have the clumpfinder stop looking once it has clump edges

at

...
my maximum density?

Thanks! Stephanie -- Dr. Stephanie Tonnesen Associate Research Scientist CCA, Flatiron Institute New York, NY

stonnes@gmail.com mailto:stonnes@gmail.com

_______________________________________________ yt-users mailing list -- yt-users@python.org mailto:yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org
yt-users-leave@python.org>

...
_______________________________________________ yt-users mailing list -- yt-users@python.org mailto:yt-users@python.org To unsubscribe send an email to

yt-users-leave@python.org

...
mailto:yt-users-leave@python.org

_______________________________________________ yt-users mailing list -- yt-users@python.org mailto:yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org mailto:yt-users-leave@python.org

-- -- Dr. Stephanie Tonnesen Associate Research Scientist CCA, Flatiron Institute New York, NY

stonnes@gmail.com mailto:stonnes@gmail.com

_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org

-- John Wise Associate Professor of Physics Center for Relativistic Astrophysics, Georgia Tech http://cosmo.gatech.edu _______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org

_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org

_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org

_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org

_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org

Britton Smith

12:56 a.m.

It's my fault. I forgot to remove the imports from ./yt/analysis_modules/level_sets/api.py. If you change the import to "from yt.data_objects.level_sets.api import ..." then it will work. I'll update the PR now. On Wed, Oct 3, 2018 at 12:10 PM Stephanie Tonnesen wrote:

...

Okay, my git skills are not so good, so definitely not *sure*. Just tried to check by doing this:

cd yt-conda/yt (root) -bash-4.2$ git pull https://github.com/brittonsmith/yt.git clurmps From https://github.com/brittonsmith/yt * branch clurmps -> FETCH_HEAD Already up-to-date.

so...I think yes? I also just did the same thing in my script and got the same output. -- Dr. Stephanie Tonnesen Associate Research Scientist CCA, Flatiron Institute New York, NY

stonnes@gmail.com

On Wed, Oct 3, 2018 at 2:56 PM Nathan Goldbaum wrote:

...
Hi Stephanie,

Are you *sure* you pulled britton's pull request? He deleted that entry in yt/analysis_modules/level_sets/api.py, so it looks like you're using a version of that file from before his pull request.

-Nathan

On Wed, Oct 3, 2018 at 1:44 PM Stephanie Tonnesen wrote:

...
Go it, thanks!

I think I have pulled your update, Britton, and now my code crashes almost immediately with this error.

yt_clump_fingert.py:4: VisibleDeprecationWarning: The level_sets module has been moved to yt.data_objects.level_sets.This import is deprecated and will be removed in a future release.Please, change the import in your scripts from 'from yt.analysis_modules.level_sets' to 'from yt.data_objects.level_sets.'. from yt.analysis_modules.level_sets.api import * Traceback (most recent call last): File "yt_clump_fingert.py", line 4, in <module> from yt.analysis_modules.level_sets.api import * File "/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/api.py", line 28, in <module> from yt.data_objects.level_sets.clump_handling import \ ImportError: cannot import name 'write_clump_index'

I am running: yt version

yt module located at: /mnt/home/stonnesen/yt-conda/yt

The current version of yt is:

--- Version = 3.5.dev0 Changeset = ff956e8708c5 --- This installation CAN be automatically updated.

I am guessing that maybe I didn't include some other necessary update?

Thanks!! Stephanie -- Dr. Stephanie Tonnesen Associate Research Scientist CCA, Flatiron Institute New York, NY

stonnes@gmail.com

On Tue, Oct 2, 2018 at 6:54 PM Britton Smith wrote:

...
Hi Stephanie,

While looking into this issue, I found some low hanging refactoring fruit, so I've issued a pull request that cleans things up and takes care of this as well. You can have a look here: https://github.com/yt-project/yt/pull/2045

In running the example you attached, I found that the calculation resulted in a single clump comprised of the entire domain, so the clump annotation may be effectively invisible as it will draw a box around the entire figure. If I used a slightly smaller smaller data container, I was able to see the annotation, so I think things are ok. Let me know if it still doesn't seem right.

Britton

On Tue, Oct 2, 2018 at 1:05 PM Stephanie Tonnesen wrote:

...
Hi John,

Thanks! I *think* it works--here is my only confusion. If I set c_max to some value less than the max density (say, 1e-24 in the code I attached previously), it doesn't crash anymore but it doesn't actually plot a clump contour either. If I set c_max to the max density setting (or I guess a bit above, 10**np.floor(np.log10(data_source[field]).max()+1) ), then I get a contour plotted. Does this seem right to you?

Also, they both give me this message "/mnt/xfs1/home/stonnesen/yt-conda/lib/python3.6/site-packages/matplotlib/contour.py:1180: UserWarning: No contour levels w ere found within the data range. warnings.warn("No contour levels were found" " Even though I seem to have one central contour in the c_max = 10**np.floor(np.log10(data_source[field]).max()+1) case.

Thanks! Stephanie

-- Dr. Stephanie Tonnesen Associate Research Scientist CCA, Flatiron Institute New York, NY

stonnes@gmail.com

On Tue, Oct 2, 2018 at 7:38 AM Wise, John H wrote:

...
Hi Stephanie,

I actually ran into this bug in the past week. I fixed in for my case by making this change in clump_handling.py

- elif (child._validate()): + elif ((child.children is not None) and child._validate()):

I haven't had the time to submit a PR. If this works for you, let me know, and I can I create one this week.

Thanks, John

On 10/01/2018 09:48 PM, Stephanie Tonnesen wrote: > Hi, > > Sorry about the crazy delay, here is a script that reproduces the error > with the IsolatedGalaxy dataset! > > The script is attached. I am happy to help someone look into it, or > look more myself with a bit of extra guidance. > > Thanks!! > > Stephanie > > -- > Dr. Stephanie Tonnesen > Associate Research Scientist > CCA, Flatiron Institute > New York, NY > > stonnes@gmail.com mailto:stonnes@gmail.com > > > On Fri, Sep 14, 2018 at 6:51 AM Stephanie Tonnesen < stonnes@gmail.com > mailto:stonnes@gmail.com> wrote: > > Hi Britton! > > No, I have not resolved the issue, unfortunately. I will try and > get a script that reproduces the problem up soon. > > Best, > Stephanie > > On Thu, Sep 13, 2018 at 7:45 PM Britton Smith > mailto:brittonsmith@gmail.com> wrote: > > HI Stephanie, > > Have you managed to resolve this issue? If so, would you be > willing to submit a pull request to get this fix in the main > repo? If not, could you post a script that could be used to > reproduce this error with some of the public data? > > Britton > > On Fri, Sep 7, 2018 at 1:05 PM Nathan Goldbaum > mailto:nathan12343@gmail.com> wrote: > > Generally an error like this is due to a bug in yt. I don't > know enough about the code you're using to provide useful > advice, but perhaps we should be checking if `children` is > `None` and doing something else. The "workaround" is likely > to fix whatever bug you've run into. > > On Fri, Sep 7, 2018 at 2:56 PM, Stephanie Tonnesen > mailto:stonnes@gmail.com> wrote: > > Hi yt-users! > > I have been fiddling some more with yt's clump finder > and have another question. I want to use density to > find clumps, but want my maximum threshold value to be > below the fiducial setting--instead of above the maximum > possible density in the code I just want the finder to > stop searching at, say 3.2e-24. > > The code was working but running out of memory until I > set the max threshold, and I now get this error: > > ....lots of pleasant looking output messages.... > Updating joins ... : 0%| | 0/7 [00:00 Updating joins ... : 100%|██████████| 7/7 [00:00<00:00, > 48933.55it/s] > yt : [INFO ] 2018-09-07 14:08:55,007 Investigating 2 > children. > yt : [INFO ] 2018-09-07 14:08:55,007 Finding clumps: > min: 3.200000e-24, max: 3.200000e-24, step: 2.000000 > yt : [INFO ] 2018-09-07 14:08:55,041 Finding clumps: > min: 3.200000e-24, max: 3.200000e-24, step: 2.000000 > yt : [INFO ] 2018-09-07 14:08:55,042 Eliminating > invalid, childless clump with 2 cells. > yt : [INFO ] 2018-09-07 14:08:55,042 1 of 2 children > survived, linking its children to parent. > Traceback (most recent call last): > File "yt_clump_fingert.py", line 59, in <module> > find_clumps(master_clump, c_min, c_max, step) > File > "/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/clump_handling.py", > line 444, in find_clumps > find_clumps(child, min_val*d_clump, max_val, d_clump) > File > "/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/clump_handling.py", > line 438, in find_clumps > find_clumps(clump, min_val*d_clump, max_val, d_clump) > File > "/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/clump_handling.py", > line 438, in find_clumps > find_clumps(clump, min_val*d_clump, max_val, d_clump) > File > "/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/clump_handling.py", > line 438, in find_clumps > find_clumps(clump, min_val*d_clump, max_val, d_clump) > File > "/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/clump_handling.py", > line 461, in find_clumps > for child in clump.children: > TypeError: 'NoneType' object is not iterable > > I tried to circumvent this error by changing my max to > 2e-24 (not a multiple of 2 of 1e-25), and got the same > error when the min was 3.2e-24 and the max was 2e-24 (!). > > Is there any way I can get around this error and have > the clumpfinder stop looking once it has clump edges at > my maximum density? > > Thanks! > Stephanie > -- > Dr. Stephanie Tonnesen > Associate Research Scientist > CCA, Flatiron Institute > New York, NY > > stonnes@gmail.com mailto:stonnes@gmail.com > > _______________________________________________ > yt-users mailing list -- yt-users@python.org > mailto:yt-users@python.org > To unsubscribe send an email to > yt-users-leave@python.org > > > _______________________________________________ > yt-users mailing list -- yt-users@python.org > mailto:yt-users@python.org > To unsubscribe send an email to yt-users-leave@python.org > mailto:yt-users-leave@python.org > > _______________________________________________ > yt-users mailing list -- yt-users@python.org > mailto:yt-users@python.org > To unsubscribe send an email to yt-users-leave@python.org > mailto:yt-users-leave@python.org > > -- > -- > Dr. Stephanie Tonnesen > Associate Research Scientist > CCA, Flatiron Institute > New York, NY > > stonnes@gmail.com mailto:stonnes@gmail.com > > > > _______________________________________________ > yt-users mailing list -- yt-users@python.org > To unsubscribe send an email to yt-users-leave@python.org >

-- John Wise Associate Professor of Physics Center for Relativistic Astrophysics, Georgia Tech http://cosmo.gatech.edu _______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org

_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org

_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org

_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org

_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org

_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org

Stephanie Tonnesen

9:51 p.m.

Okay, so I was trying to figure out why I am getting strange results and went back to the IsolatedGalaxies cookbook case. I am attaching the cut_region verion here. This may be a cut_region problem, not a clump problem, though! First, I ran the plain old cookbook case and got the same very lovely clumps. [image: clumps_reloaded_Projection_z_density_cookbook.png] Then I changed c_max to 2e-24 based on some of the density values I was getting, and got this: [image: clumps_reloaded_Projection_z_density_max2en24.png] Okay, fine, I just didn't set my limit high enough. So when I run with max = 7e-24: [image: clumps_reloaded_Projection_z_density_max7en24.png] But, when I select the same (or slightly larger) region with cut_region and the same c_min and c_max (or just using the standard max and min definitions) I cannot find any clumps! [image: clumps_reloaded_Projection_z_density.png] Do you think this is a cut_region or a clump_finder problem? Thanks! Stephanie -- Dr. Stephanie Tonnesen Associate Research Scientist CCA, Flatiron Institute New York, NY stonnes@gmail.com On Wed, Oct 3, 2018 at 3:29 PM Britton Smith wrote:

...

It's my fault. I forgot to remove the imports from ./yt/analysis_modules/level_sets/api.py. If you change the import to "from yt.data_objects.level_sets.api import ..." then it will work. I'll update the PR now.

On Wed, Oct 3, 2018 at 12:10 PM Stephanie Tonnesen wrote:

...
Okay, my git skills are not so good, so definitely not *sure*. Just tried to check by doing this:

cd yt-conda/yt (root) -bash-4.2$ git pull https://github.com/brittonsmith/yt.git clurmps From https://github.com/brittonsmith/yt * branch clurmps -> FETCH_HEAD Already up-to-date.

so...I think yes? I also just did the same thing in my script and got the same output. -- Dr. Stephanie Tonnesen Associate Research Scientist CCA, Flatiron Institute New York, NY

stonnes@gmail.com

On Wed, Oct 3, 2018 at 2:56 PM Nathan Goldbaum wrote:

...
Hi Stephanie,

Are you *sure* you pulled britton's pull request? He deleted that entry in yt/analysis_modules/level_sets/api.py, so it looks like you're using a version of that file from before his pull request.

-Nathan

On Wed, Oct 3, 2018 at 1:44 PM Stephanie Tonnesen wrote:

...
Go it, thanks!

I think I have pulled your update, Britton, and now my code crashes almost immediately with this error.

yt_clump_fingert.py:4: VisibleDeprecationWarning: The level_sets module has been moved to yt.data_objects.level_sets.This import is deprecated and will be removed in a future release.Please, change the import in your scripts from 'from yt.analysis_modules.level_sets' to 'from yt.data_objects.level_sets.'. from yt.analysis_modules.level_sets.api import * Traceback (most recent call last): File "yt_clump_fingert.py", line 4, in <module> from yt.analysis_modules.level_sets.api import * File "/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/api.py", line 28, in <module> from yt.data_objects.level_sets.clump_handling import \ ImportError: cannot import name 'write_clump_index'

I am running: yt version

yt module located at: /mnt/home/stonnesen/yt-conda/yt

The current version of yt is:

--- Version = 3.5.dev0 Changeset = ff956e8708c5 --- This installation CAN be automatically updated.

I am guessing that maybe I didn't include some other necessary update?

Thanks!! Stephanie -- Dr. Stephanie Tonnesen Associate Research Scientist CCA, Flatiron Institute New York, NY

stonnes@gmail.com

On Tue, Oct 2, 2018 at 6:54 PM Britton Smith wrote:

...
Hi Stephanie,

While looking into this issue, I found some low hanging refactoring fruit, so I've issued a pull request that cleans things up and takes care of this as well. You can have a look here: https://github.com/yt-project/yt/pull/2045

In running the example you attached, I found that the calculation resulted in a single clump comprised of the entire domain, so the clump annotation may be effectively invisible as it will draw a box around the entire figure. If I used a slightly smaller smaller data container, I was able to see the annotation, so I think things are ok. Let me know if it still doesn't seem right.

Britton

On Tue, Oct 2, 2018 at 1:05 PM Stephanie Tonnesen wrote:

...
Hi John,

Thanks! I *think* it works--here is my only confusion. If I set c_max to some value less than the max density (say, 1e-24 in the code I attached previously), it doesn't crash anymore but it doesn't actually plot a clump contour either. If I set c_max to the max density setting (or I guess a bit above, 10**np.floor(np.log10(data_source[field]).max()+1) ), then I get a contour plotted. Does this seem right to you?

Also, they both give me this message "/mnt/xfs1/home/stonnesen/yt-conda/lib/python3.6/site-packages/matplotlib/contour.py:1180: UserWarning: No contour levels w ere found within the data range. warnings.warn("No contour levels were found" " Even though I seem to have one central contour in the c_max = 10**np.floor(np.log10(data_source[field]).max()+1) case.

Thanks! Stephanie

-- Dr. Stephanie Tonnesen Associate Research Scientist CCA, Flatiron Institute New York, NY

stonnes@gmail.com

On Tue, Oct 2, 2018 at 7:38 AM Wise, John H wrote:

> Hi Stephanie, > > I actually ran into this bug in the past week. I fixed in for my > case > by making this change in clump_handling.py > > - elif (child._validate()): > + elif ((child.children is not None) and > child._validate()): > > I haven't had the time to submit a PR. If this works for you, let me > know, and I can I create one this week. > > Thanks, > John > > > On 10/01/2018 09:48 PM, Stephanie Tonnesen wrote: > > Hi, > > > > Sorry about the crazy delay, here is a script that reproduces the > error > > with the IsolatedGalaxy dataset! > > > > The script is attached. I am happy to help someone look into it, > or > > look more myself with a bit of extra guidance. > > > > Thanks!! > > > > Stephanie > > > > -- > > Dr. Stephanie Tonnesen > > Associate Research Scientist > > CCA, Flatiron Institute > > New York, NY > > > > stonnes@gmail.com mailto:stonnes@gmail.com > > > > > > On Fri, Sep 14, 2018 at 6:51 AM Stephanie Tonnesen < > stonnes@gmail.com > > mailto:stonnes@gmail.com> wrote: > > > > Hi Britton! > > > > No, I have not resolved the issue, unfortunately. I will try > and > > get a script that reproduces the problem up soon. > > > > Best, > > Stephanie > > > > On Thu, Sep 13, 2018 at 7:45 PM Britton Smith > > mailto:brittonsmith@gmail.com> > wrote: > > > > HI Stephanie, > > > > Have you managed to resolve this issue? If so, would you be > > willing to submit a pull request to get this fix in the > main > > repo? If not, could you post a script that could be used to > > reproduce this error with some of the public data? > > > > Britton > > > > On Fri, Sep 7, 2018 at 1:05 PM Nathan Goldbaum > > mailto:nathan12343@gmail.com> > wrote: > > > > Generally an error like this is due to a bug in yt. I > don't > > know enough about the code you're using to provide > useful > > advice, but perhaps we should be checking if > `children` is > > `None` and doing something else. The "workaround" is > likely > > to fix whatever bug you've run into. > > > > On Fri, Sep 7, 2018 at 2:56 PM, Stephanie Tonnesen > > mailto:stonnes@gmail.com> wrote: > > > > Hi yt-users! > > > > I have been fiddling some more with yt's clump > finder > > and have another question. I want to use density > to > > find clumps, but want my maximum threshold value > to be > > below the fiducial setting--instead of above the > maximum > > possible density in the code I just want the > finder to > > stop searching at, say 3.2e-24. > > > > The code was working but running out of memory > until I > > set the max threshold, and I now get this error: > > > > ....lots of pleasant looking output messages.... > > Updating joins ... : 0%| | 0/7 > [00:00 > Updating joins ... : 100%|██████████| 7/7 > [00:00<00:00, > > 48933.55it/s] > > yt : [INFO ] 2018-09-07 14:08:55,007 > Investigating 2 > > children. > > yt : [INFO ] 2018-09-07 14:08:55,007 Finding > clumps: > > min: 3.200000e-24, max: 3.200000e-24, step: > 2.000000 > > yt : [INFO ] 2018-09-07 14:08:55,041 Finding > clumps: > > min: 3.200000e-24, max: 3.200000e-24, step: > 2.000000 > > yt : [INFO ] 2018-09-07 14:08:55,042 > Eliminating > > invalid, childless clump with 2 cells. > > yt : [INFO ] 2018-09-07 14:08:55,042 1 of 2 > children > > survived, linking its children to parent. > > Traceback (most recent call last): > > File "yt_clump_fingert.py", line 59, in <module> > > find_clumps(master_clump, c_min, c_max, step) > > File > > > "/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/clump_handling.py", > > line 444, in find_clumps > > find_clumps(child, min_val*d_clump, max_val, > d_clump) > > File > > > "/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/clump_handling.py", > > line 438, in find_clumps > > find_clumps(clump, min_val*d_clump, max_val, > d_clump) > > File > > > "/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/clump_handling.py", > > line 438, in find_clumps > > find_clumps(clump, min_val*d_clump, max_val, > d_clump) > > File > > > "/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/clump_handling.py", > > line 438, in find_clumps > > find_clumps(clump, min_val*d_clump, max_val, > d_clump) > > File > > > "/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/clump_handling.py", > > line 461, in find_clumps > > for child in clump.children: > > TypeError: 'NoneType' object is not iterable > > > > I tried to circumvent this error by changing my > max to > > 2e-24 (not a multiple of 2 of 1e-25), and got the > same > > error when the min was 3.2e-24 and the max was > 2e-24 (!). > > > > Is there any way I can get around this error and > have > > the clumpfinder stop looking once it has clump > edges at > > my maximum density? > > > > Thanks! > > Stephanie > > -- > > Dr. Stephanie Tonnesen > > Associate Research Scientist > > CCA, Flatiron Institute > > New York, NY > > > > stonnes@gmail.com mailto:stonnes@gmail.com > > > > _______________________________________________ > > yt-users mailing list -- yt-users@python.org > > mailto:yt-users@python.org > > To unsubscribe send an email to > > yt-users-leave@python.org yt-users-leave@python.org> > > > > > > _______________________________________________ > > yt-users mailing list -- yt-users@python.org > > mailto:yt-users@python.org > > To unsubscribe send an email to > yt-users-leave@python.org > > mailto:yt-users-leave@python.org > > > > _______________________________________________ > > yt-users mailing list -- yt-users@python.org > > mailto:yt-users@python.org > > To unsubscribe send an email to yt-users-leave@python.org > > mailto:yt-users-leave@python.org > > > > -- > > -- > > Dr. Stephanie Tonnesen > > Associate Research Scientist > > CCA, Flatiron Institute > > New York, NY > > > > stonnes@gmail.com mailto:stonnes@gmail.com > > > > > > > > _______________________________________________ > > yt-users mailing list -- yt-users@python.org > > To unsubscribe send an email to yt-users-leave@python.org > > > > -- > John Wise > Associate Professor of Physics > Center for Relativistic Astrophysics, Georgia Tech > http://cosmo.gatech.edu > _______________________________________________ > yt-users mailing list -- yt-users@python.org > To unsubscribe send an email to yt-users-leave@python.org > _______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org

_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org

_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org

_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org

_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org

_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org

Nathan Goldbaum

10:19 p.m.

This looks like a bug to me, it would be great if you could file a bug on github along with a script we can use to reproduce the issue. On Thu, Oct 4, 2018 at 11:45 AM Stephanie Tonnesen wrote:

...

Okay, so I was trying to figure out why I am getting strange results and went back to the IsolatedGalaxies cookbook case. I am attaching the cut_region verion here. This may be a cut_region problem, not a clump problem, though!

First, I ran the plain old cookbook case and got the same very lovely clumps.

[image: clumps_reloaded_Projection_z_density_cookbook.png]

Then I changed c_max to 2e-24 based on some of the density values I was getting, and got this:

[image: clumps_reloaded_Projection_z_density_max2en24.png] Okay, fine, I just didn't set my limit high enough. So when I run with max = 7e-24:

[image: clumps_reloaded_Projection_z_density_max7en24.png] But, when I select the same (or slightly larger) region with cut_region and the same c_min and c_max (or just using the standard max and min definitions) I cannot find any clumps!

[image: clumps_reloaded_Projection_z_density.png]

Do you think this is a cut_region or a clump_finder problem?

Thanks! Stephanie

-- Dr. Stephanie Tonnesen Associate Research Scientist CCA, Flatiron Institute New York, NY

stonnes@gmail.com

On Wed, Oct 3, 2018 at 3:29 PM Britton Smith wrote:

...
It's my fault. I forgot to remove the imports from ./yt/analysis_modules/level_sets/api.py. If you change the import to "from yt.data_objects.level_sets.api import ..." then it will work. I'll update the PR now.

On Wed, Oct 3, 2018 at 12:10 PM Stephanie Tonnesen wrote:

...
Okay, my git skills are not so good, so definitely not *sure*. Just tried to check by doing this:

cd yt-conda/yt (root) -bash-4.2$ git pull https://github.com/brittonsmith/yt.git clurmps From https://github.com/brittonsmith/yt * branch clurmps -> FETCH_HEAD Already up-to-date.

so...I think yes? I also just did the same thing in my script and got the same output. -- Dr. Stephanie Tonnesen Associate Research Scientist CCA, Flatiron Institute New York, NY

stonnes@gmail.com

On Wed, Oct 3, 2018 at 2:56 PM Nathan Goldbaum wrote:

...
Hi Stephanie,

Are you *sure* you pulled britton's pull request? He deleted that entry in yt/analysis_modules/level_sets/api.py, so it looks like you're using a version of that file from before his pull request.

-Nathan

On Wed, Oct 3, 2018 at 1:44 PM Stephanie Tonnesen wrote:

...
Go it, thanks!

I think I have pulled your update, Britton, and now my code crashes almost immediately with this error.

yt_clump_fingert.py:4: VisibleDeprecationWarning: The level_sets module has been moved to yt.data_objects.level_sets.This import is deprecated and will be removed in a future release.Please, change the import in your scripts from 'from yt.analysis_modules.level_sets' to 'from yt.data_objects.level_sets.'. from yt.analysis_modules.level_sets.api import * Traceback (most recent call last): File "yt_clump_fingert.py", line 4, in <module> from yt.analysis_modules.level_sets.api import * File "/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/api.py", line 28, in <module> from yt.data_objects.level_sets.clump_handling import \ ImportError: cannot import name 'write_clump_index'

I am running: yt version

yt module located at: /mnt/home/stonnesen/yt-conda/yt

The current version of yt is:

--- Version = 3.5.dev0 Changeset = ff956e8708c5 --- This installation CAN be automatically updated.

I am guessing that maybe I didn't include some other necessary update?

Thanks!! Stephanie -- Dr. Stephanie Tonnesen Associate Research Scientist CCA, Flatiron Institute New York, NY

stonnes@gmail.com

On Tue, Oct 2, 2018 at 6:54 PM Britton Smith wrote:

...
Hi Stephanie,

While looking into this issue, I found some low hanging refactoring fruit, so I've issued a pull request that cleans things up and takes care of this as well. You can have a look here: https://github.com/yt-project/yt/pull/2045

In running the example you attached, I found that the calculation resulted in a single clump comprised of the entire domain, so the clump annotation may be effectively invisible as it will draw a box around the entire figure. If I used a slightly smaller smaller data container, I was able to see the annotation, so I think things are ok. Let me know if it still doesn't seem right.

Britton

On Tue, Oct 2, 2018 at 1:05 PM Stephanie Tonnesen wrote:

> Hi John, > > Thanks! I *think* it works--here is my only confusion. If I set > c_max to some value less than the max density (say, 1e-24 in the code I > attached previously), it doesn't crash anymore but it doesn't actually plot > a clump contour either. If I set c_max to the max density setting (or I > guess a bit above, 10**np.floor(np.log10(data_source[field]).max()+1) ), > then I get a contour plotted. Does this seem right to you? > > Also, they both give me this message > "/mnt/xfs1/home/stonnesen/yt-conda/lib/python3.6/site-packages/matplotlib/contour.py:1180: > UserWarning: No contour levels w > ere found within the data range. > warnings.warn("No contour levels were found" > " > Even though I seem to have one central contour in the c_max = > 10**np.floor(np.log10(data_source[field]).max()+1) case. > > Thanks! > Stephanie > > -- > Dr. Stephanie Tonnesen > Associate Research Scientist > CCA, Flatiron Institute > New York, NY > > stonnes@gmail.com > > > On Tue, Oct 2, 2018 at 7:38 AM Wise, John H < > jwise@physics.gatech.edu> wrote: > >> Hi Stephanie, >> >> I actually ran into this bug in the past week. I fixed in for my >> case >> by making this change in clump_handling.py >> >> - elif (child._validate()): >> + elif ((child.children is not None) and >> child._validate()): >> >> I haven't had the time to submit a PR. If this works for you, let >> me >> know, and I can I create one this week. >> >> Thanks, >> John >> >> >> On 10/01/2018 09:48 PM, Stephanie Tonnesen wrote: >> > Hi, >> > >> > Sorry about the crazy delay, here is a script that reproduces the >> error >> > with the IsolatedGalaxy dataset! >> > >> > The script is attached. I am happy to help someone look into it, >> or >> > look more myself with a bit of extra guidance. >> > >> > Thanks!! >> > >> > Stephanie >> > >> > -- >> > Dr. Stephanie Tonnesen >> > Associate Research Scientist >> > CCA, Flatiron Institute >> > New York, NY >> > >> > stonnes@gmail.com mailto:stonnes@gmail.com >> > >> > >> > On Fri, Sep 14, 2018 at 6:51 AM Stephanie Tonnesen < >> stonnes@gmail.com >> > mailto:stonnes@gmail.com> wrote: >> > >> > Hi Britton! >> > >> > No, I have not resolved the issue, unfortunately. I will try >> and >> > get a script that reproduces the problem up soon. >> > >> > Best, >> > Stephanie >> > >> > On Thu, Sep 13, 2018 at 7:45 PM Britton Smith >> > mailto:brittonsmith@gmail.com> >> wrote: >> > >> > HI Stephanie, >> > >> > Have you managed to resolve this issue? If so, would you >> be >> > willing to submit a pull request to get this fix in the >> main >> > repo? If not, could you post a script that could be used >> to >> > reproduce this error with some of the public data? >> > >> > Britton >> > >> > On Fri, Sep 7, 2018 at 1:05 PM Nathan Goldbaum >> > mailto:nathan12343@gmail.com> >> wrote: >> > >> > Generally an error like this is due to a bug in yt. I >> don't >> > know enough about the code you're using to provide >> useful >> > advice, but perhaps we should be checking if >> `children` is >> > `None` and doing something else. The "workaround" is >> likely >> > to fix whatever bug you've run into. >> > >> > On Fri, Sep 7, 2018 at 2:56 PM, Stephanie Tonnesen >> > mailto:stonnes@gmail.com> wrote: >> > >> > Hi yt-users! >> > >> > I have been fiddling some more with yt's clump >> finder >> > and have another question. I want to use density >> to >> > find clumps, but want my maximum threshold value >> to be >> > below the fiducial setting--instead of above the >> maximum >> > possible density in the code I just want the >> finder to >> > stop searching at, say 3.2e-24. >> > >> > The code was working but running out of memory >> until I >> > set the max threshold, and I now get this error: >> > >> > ....lots of pleasant looking output messages.... >> > Updating joins ... : 0%| | 0/7 >> [00:00> > Updating joins ... : 100%|██████████| 7/7 >> [00:00<00:00, >> > 48933.55it/s] >> > yt : [INFO ] 2018-09-07 14:08:55,007 >> Investigating 2 >> > children. >> > yt : [INFO ] 2018-09-07 14:08:55,007 Finding >> clumps: >> > min: 3.200000e-24, max: 3.200000e-24, step: >> 2.000000 >> > yt : [INFO ] 2018-09-07 14:08:55,041 Finding >> clumps: >> > min: 3.200000e-24, max: 3.200000e-24, step: >> 2.000000 >> > yt : [INFO ] 2018-09-07 14:08:55,042 >> Eliminating >> > invalid, childless clump with 2 cells. >> > yt : [INFO ] 2018-09-07 14:08:55,042 1 of 2 >> children >> > survived, linking its children to parent. >> > Traceback (most recent call last): >> > File "yt_clump_fingert.py", line 59, in >> <module> >> > find_clumps(master_clump, c_min, c_max, step) >> > File >> > >> "/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/clump_handling.py", >> > line 444, in find_clumps >> > find_clumps(child, min_val*d_clump, max_val, >> d_clump) >> > File >> > >> "/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/clump_handling.py", >> > line 438, in find_clumps >> > find_clumps(clump, min_val*d_clump, max_val, >> d_clump) >> > File >> > >> "/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/clump_handling.py", >> > line 438, in find_clumps >> > find_clumps(clump, min_val*d_clump, max_val, >> d_clump) >> > File >> > >> "/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/clump_handling.py", >> > line 438, in find_clumps >> > find_clumps(clump, min_val*d_clump, max_val, >> d_clump) >> > File >> > >> "/mnt/home/stonnesen/yt-conda/yt/yt/analysis_modules/level_sets/clump_handling.py", >> > line 461, in find_clumps >> > for child in clump.children: >> > TypeError: 'NoneType' object is not iterable >> > >> > I tried to circumvent this error by changing my >> max to >> > 2e-24 (not a multiple of 2 of 1e-25), and got the >> same >> > error when the min was 3.2e-24 and the max was >> 2e-24 (!). >> > >> > Is there any way I can get around this error and >> have >> > the clumpfinder stop looking once it has clump >> edges at >> > my maximum density? >> > >> > Thanks! >> > Stephanie >> > -- >> > Dr. Stephanie Tonnesen >> > Associate Research Scientist >> > CCA, Flatiron Institute >> > New York, NY >> > >> > stonnes@gmail.com mailto:stonnes@gmail.com >> > >> > _______________________________________________ >> > yt-users mailing list -- yt-users@python.org >> > mailto:yt-users@python.org >> > To unsubscribe send an email to >> > yt-users-leave@python.org > yt-users-leave@python.org> >> > >> > >> > _______________________________________________ >> > yt-users mailing list -- yt-users@python.org >> > mailto:yt-users@python.org >> > To unsubscribe send an email to >> yt-users-leave@python.org >> > mailto:yt-users-leave@python.org >> > >> > _______________________________________________ >> > yt-users mailing list -- yt-users@python.org >> > mailto:yt-users@python.org >> > To unsubscribe send an email to yt-users-leave@python.org >> > mailto:yt-users-leave@python.org >> > >> > -- >> > -- >> > Dr. Stephanie Tonnesen >> > Associate Research Scientist >> > CCA, Flatiron Institute >> > New York, NY >> > >> > stonnes@gmail.com mailto:stonnes@gmail.com >> > >> > >> > >> > _______________________________________________ >> > yt-users mailing list -- yt-users@python.org >> > To unsubscribe send an email to yt-users-leave@python.org >> > >> >> -- >> John Wise >> Associate Professor of Physics >> Center for Relativistic Astrophysics, Georgia Tech >> http://cosmo.gatech.edu >> _______________________________________________ >> yt-users mailing list -- yt-users@python.org >> To unsubscribe send an email to yt-users-leave@python.org >> > _______________________________________________ > yt-users mailing list -- yt-users@python.org > To unsubscribe send an email to yt-users-leave@python.org > _______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org

_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org

_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org

_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org

_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org

_______________________________________________ yt-users mailing list -- yt-users@python.org To unsubscribe send an email to yt-users-leave@python.org

2024

Age (days ago)

2051

Last active (days ago)

List overview

Download

13 comments

4 participants

participants (4)

Britton Smith
Nathan Goldbaum
Stephanie Tonnesen
Wise, John H

clump finder error with maximum density set

tags

participants (4)