Mailman 3 annotate_contour takes much longer time to run than expected on some machines - yt-users

newer
problems installing rockstar with...

annotate_contour takes much longer time to run than expected on some machines

older
custom axes tickmarks projection...

Bili Dong

10 Aug 2015 10 Aug '15

10:10 p.m.

Hi yt-users, I’m using yt-3.2 (Changeset = 329b5039b031 stable). The problem is that the same script that could finish in seconds on my local machine will take much (order of magnitude) longer time to run on the server I’m using. In some cases, it get stuck for hours that I don’t know if it will ever finish. I’ve tracked down the bottleneck to be the `annotate_contour` method. The server should be faster than my local machine. (I’m not exactly sure about how to compare it. But the server is generally much faster in other situations.) For example, for this script (http://paste.yt-project.org/show/5808/ http://paste.yt-project.org/show/5808/), on my local machine the output shows: ``` yt : [INFO ] 2015-08-10 14:48:15,401 Making a fixed resolution buffer of (('gas', 'density')) 800 by 800 yt : [INFO ] 2015-08-10 14:48:15,751 Saving plot snapshot_200_Projection_z_density.png yt : [INFO ] 2015-08-10 14:48:16,716 Saving plot snapshot_200_Projection_z_density.png ``` However, on the server the output shows: ``` yt : [INFO ] 2015-08-10 14:51:25,551 Making a fixed resolution buffer of (('gas', 'density')) 800 by 800 yt : [INFO ] 2015-08-10 14:51:25,917 Saving plot snapshot_200_Projection_z_density.png yt : [INFO ] 2015-08-10 14:56:21,922 Saving plot snapshot_200_Projection_z_density.png ``` That’s 296 times slower!!! This is likely to be a very specific problem with the server. But I’d like to know if anyone else has encountered similar problems before so I can learn from your experience. For now, I have no idea what could have gone wrong. BTW, I’m using TSCC (http://idi.ucsd.edu/computing/ http://idi.ucsd.edu/computing/) at UC San Diego, if you happen to know it. Thanks, Bili

Attachments:

attachment.html (text/html — 2.8 KB)

Show replies by date

Nathan Goldbaum

10 Aug 10 Aug

10:15 p.m.

Hey Bili, Just a shot in the dark, but do you have different matplotlib versions installed on your machine and server at UCSD? The contour callback is using some internal machinery in matplotlib that changed relatively recently. It might also be worthwhile to run your script under cProfile so we get an idea of which function exactly is taking so long. Even better if you can run *that* function under the line_profiler. -Nathan On Mon, Aug 10, 2015 at 5:10 PM, Bili Dong wrote:

...

Hi yt-users,

I’m using yt-3.2 (Changeset = 329b5039b031 stable).

The problem is that the same script that could finish in seconds on my local machine will take much (order of magnitude) longer time to run on the server I’m using. In some cases, it get stuck for hours that I don’t know if it will ever finish. I’ve tracked down the bottleneck to be the `annotate_contour` method. The server should be faster than my local machine. (I’m not exactly sure about how to compare it. But the server is generally much faster in other situations.)

For example, for this script (http://paste.yt-project.org/show/5808/), on my local machine the output shows:

``` yt : [INFO ] 2015-08-10 *14:48:15*,401 Making a fixed resolution buffer of (('gas', 'density')) 800 by 800 yt : [INFO ] 2015-08-10 *14:48:15*,751 Saving plot snapshot_200_Projection_z_density.png yt : [INFO ] 2015-08-10 *14:48:16*,716 Saving plot snapshot_200_Projection_z_density.png ```

However, on the server the output shows: ``` yt : [INFO ] 2015-08-10 *14:51:25*,551 Making a fixed resolution buffer of (('gas', 'density')) 800 by 800 yt : [INFO ] 2015-08-10 *14:51:25*,917 Saving plot snapshot_200_Projection_z_density.png yt : [INFO ] 2015-08-10 *14:56:21*,922 Saving plot snapshot_200_Projection_z_density.png ```

That’s *296* times slower!!!

This is likely to be a very specific problem with the server. But I’d like to know if anyone else has encountered similar problems before so I can learn from your experience. For now, I have no idea what could have gone wrong.

BTW, I’m using TSCC (http://idi.ucsd.edu/computing/) at UC San Diego, if you happen to know it.

Thanks, Bili

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

Cameron Hummels

10:33 p.m.

I recently ran into some major slowdowns in going from yt version 3.1 to 3.2 on certain machines, although it wasn't related to my calling the annotate_contour callback. When I tracked it down with pdb, the slowdown was in indexing certain arrays, so maybe there is a similar issue you're encountering. I'm going to track down the changeset that this is tied to this week, and I'll let you know what I find. Cameron On Mon, Aug 10, 2015 at 3:15 PM, Nathan Goldbaum wrote:

...

Hey Bili,

Just a shot in the dark, but do you have different matplotlib versions installed on your machine and server at UCSD? The contour callback is using some internal machinery in matplotlib that changed relatively recently.

It might also be worthwhile to run your script under cProfile so we get an idea of which function exactly is taking so long. Even better if you can run *that* function under the line_profiler.

-Nathan

On Mon, Aug 10, 2015 at 5:10 PM, Bili Dong wrote:

...
Hi yt-users,

I’m using yt-3.2 (Changeset = 329b5039b031 stable).

The problem is that the same script that could finish in seconds on my local machine will take much (order of magnitude) longer time to run on the server I’m using. In some cases, it get stuck for hours that I don’t know if it will ever finish. I’ve tracked down the bottleneck to be the `annotate_contour` method. The server should be faster than my local machine. (I’m not exactly sure about how to compare it. But the server is generally much faster in other situations.)

For example, for this script (http://paste.yt-project.org/show/5808/), on my local machine the output shows:

``` yt : [INFO ] 2015-08-10 *14:48:15*,401 Making a fixed resolution buffer of (('gas', 'density')) 800 by 800 yt : [INFO ] 2015-08-10 *14:48:15*,751 Saving plot snapshot_200_Projection_z_density.png yt : [INFO ] 2015-08-10 *14:48:16*,716 Saving plot snapshot_200_Projection_z_density.png ```

However, on the server the output shows: ``` yt : [INFO ] 2015-08-10 *14:51:25*,551 Making a fixed resolution buffer of (('gas', 'density')) 800 by 800 yt : [INFO ] 2015-08-10 *14:51:25*,917 Saving plot snapshot_200_Projection_z_density.png yt : [INFO ] 2015-08-10 *14:56:21*,922 Saving plot snapshot_200_Projection_z_density.png ```

That’s *296* times slower!!!

This is likely to be a very specific problem with the server. But I’d like to know if anyone else has encountered similar problems before so I can learn from your experience. For now, I have no idea what could have gone wrong.

BTW, I’m using TSCC (http://idi.ucsd.edu/computing/) at UC San Diego, if you happen to know it.

Thanks, Bili

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

-- Cameron Hummels NSF Postdoctoral Fellow Department of Astronomy California Institute of Technology http://chummels.org

Bili Dong

10:41 p.m.

Hi Nathan, I checked that the matplotlib versions are the same. I changed my script to http://paste.yt-project.org/show/5809/. The results of cProfile on the server are http://paste.yt-project.org/show/5810/. Judging by eye, it seems to be related to the triangulation functionality. -Bili

...

On Aug 10, 2015, at 3:15 PM, Nathan Goldbaum wrote:

Hey Bili,

Just a shot in the dark, but do you have different matplotlib versions installed on your machine and server at UCSD? The contour callback is using some internal machinery in matplotlib that changed relatively recently.

It might also be worthwhile to run your script under cProfile so we get an idea of which function exactly is taking so long. Even better if you can run *that* function under the line_profiler.

-Nathan

On Mon, Aug 10, 2015 at 5:10 PM, Bili Dong mailto:qobilidop@gmail.com> wrote: Hi yt-users,

I’m using yt-3.2 (Changeset = 329b5039b031 stable).

The problem is that the same script that could finish in seconds on my local machine will take much (order of magnitude) longer time to run on the server I’m using. In some cases, it get stuck for hours that I don’t know if it will ever finish. I’ve tracked down the bottleneck to be the `annotate_contour` method. The server should be faster than my local machine. (I’m not exactly sure about how to compare it. But the server is generally much faster in other situations.)

For example, for this script (http://paste.yt-project.org/show/5808/ http://paste.yt-project.org/show/5808/), on my local machine the output shows:

``` yt : [INFO ] 2015-08-10 14:48:15,401 Making a fixed resolution buffer of (('gas', 'density')) 800 by 800 yt : [INFO ] 2015-08-10 14:48:15,751 Saving plot snapshot_200_Projection_z_density.png yt : [INFO ] 2015-08-10 14:48:16,716 Saving plot snapshot_200_Projection_z_density.png ```

However, on the server the output shows: ``` yt : [INFO ] 2015-08-10 14:51:25,551 Making a fixed resolution buffer of (('gas', 'density')) 800 by 800 yt : [INFO ] 2015-08-10 14:51:25,917 Saving plot snapshot_200_Projection_z_density.png yt : [INFO ] 2015-08-10 14:56:21,922 Saving plot snapshot_200_Projection_z_density.png ```

That’s 296 times slower!!!

This is likely to be a very specific problem with the server. But I’d like to know if anyone else has encountered similar problems before so I can learn from your experience. For now, I have no idea what could have gone wrong.

BTW, I’m using TSCC (http://idi.ucsd.edu/computing/ http://idi.ucsd.edu/computing/) at UC San Diego, if you happen to know it.

Thanks, Bili

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org mailto:yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

Bili Dong

10:53 p.m.

The previous output in pastebin is truncated. Here’s the full version http://use.yt/upload/410b970d http://use.yt/upload/410b970d. -Bili

...

On Aug 10, 2015, at 3:41 PM, Bili Dong wrote:

Hi Nathan,

I checked that the matplotlib versions are the same.

I changed my script to http://paste.yt-project.org/show/5809/ http://paste.yt-project.org/show/5809/. The results of cProfile on the server are http://paste.yt-project.org/show/5810/ http://paste.yt-project.org/show/5810/. Judging by eye, it seems to be related to the triangulation functionality.

-Bili

...
On Aug 10, 2015, at 3:15 PM, Nathan Goldbaum mailto:nathan12343@gmail.com> wrote:

Hey Bili,

Just a shot in the dark, but do you have different matplotlib versions installed on your machine and server at UCSD? The contour callback is using some internal machinery in matplotlib that changed relatively recently.

It might also be worthwhile to run your script under cProfile so we get an idea of which function exactly is taking so long. Even better if you can run *that* function under the line_profiler.

-Nathan

On Mon, Aug 10, 2015 at 5:10 PM, Bili Dong mailto:qobilidop@gmail.com> wrote: Hi yt-users,

I’m using yt-3.2 (Changeset = 329b5039b031 stable).

The problem is that the same script that could finish in seconds on my local machine will take much (order of magnitude) longer time to run on the server I’m using. In some cases, it get stuck for hours that I don’t know if it will ever finish. I’ve tracked down the bottleneck to be the `annotate_contour` method. The server should be faster than my local machine. (I’m not exactly sure about how to compare it. But the server is generally much faster in other situations.)

For example, for this script (http://paste.yt-project.org/show/5808/ http://paste.yt-project.org/show/5808/), on my local machine the output shows:

``` yt : [INFO ] 2015-08-10 14:48:15,401 Making a fixed resolution buffer of (('gas', 'density')) 800 by 800 yt : [INFO ] 2015-08-10 14:48:15,751 Saving plot snapshot_200_Projection_z_density.png yt : [INFO ] 2015-08-10 14:48:16,716 Saving plot snapshot_200_Projection_z_density.png ```

However, on the server the output shows: ``` yt : [INFO ] 2015-08-10 14:51:25,551 Making a fixed resolution buffer of (('gas', 'density')) 800 by 800 yt : [INFO ] 2015-08-10 14:51:25,917 Saving plot snapshot_200_Projection_z_density.png yt : [INFO ] 2015-08-10 14:56:21,922 Saving plot snapshot_200_Projection_z_density.png ```

That’s 296 times slower!!!

This is likely to be a very specific problem with the server. But I’d like to know if anyone else has encountered similar problems before so I can learn from your experience. For now, I have no idea what could have gone wrong.

BTW, I’m using TSCC (http://idi.ucsd.edu/computing/ http://idi.ucsd.edu/computing/) at UC San Diego, if you happen to know it.

Thanks, Bili

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org mailto:yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org mailto:yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

Kacper Kowalik

11:30 p.m.

On 08/10/2015 05:10 PM, Bili Dong wrote:

...

Hi yt-users,

I’m using yt-3.2 (Changeset = 329b5039b031 stable).

The problem is that the same script that could finish in seconds on my local machine will take much (order of magnitude) longer time to run on the server I’m using. In some cases, it get stuck for hours that I don’t know if it will ever finish. I’ve tracked down the bottleneck to be the `annotate_contour` method. The server should be faster than my local machine. (I’m not exactly sure about how to compare it. But the server is generally much faster in other situations.)

For example, for this script (http://paste.yt-project.org/show/5808/ http://paste.yt-project.org/show/5808/), on my local machine the output shows:

``` yt : [INFO ] 2015-08-10 14:48:15,401 Making a fixed resolution buffer of (('gas', 'density')) 800 by 800 yt : [INFO ] 2015-08-10 14:48:15,751 Saving plot snapshot_200_Projection_z_density.png yt : [INFO ] 2015-08-10 14:48:16,716 Saving plot snapshot_200_Projection_z_density.png ```

However, on the server the output shows: ``` yt : [INFO ] 2015-08-10 14:51:25,551 Making a fixed resolution buffer of (('gas', 'density')) 800 by 800 yt : [INFO ] 2015-08-10 14:51:25,917 Saving plot snapshot_200_Projection_z_density.png yt : [INFO ] 2015-08-10 14:56:21,922 Saving plot snapshot_200_Projection_z_density.png ```

That’s 296 times slower!!!

This is likely to be a very specific problem with the server. But I’d like to know if anyone else has encountered similar problems before so I can learn from your experience. For now, I have no idea what could have gone wrong.

BTW, I’m using TSCC (http://idi.ucsd.edu/computing/ http://idi.ucsd.edu/computing/) at UC San Diego, if you happen to know it.

Thanks, Bili

Hi Bili, make sure you pass "-DNDEBUG" to CPPFLAGS when you compile matplotlib. Without it mpl is a good cpu cycle burner. There's a debug routine in triangulation module that's enabled by default. Cheers, Kacper

...

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

Bili Dong

11 Aug 11 Aug

1:27 a.m.

I reinstalled matplotlib and it works. Thanks Kacper! You saved my life! -Bili

...

On Aug 10, 2015, at 4:30 PM, Kacper Kowalik wrote:

On 08/10/2015 05:10 PM, Bili Dong wrote:

...
Hi yt-users,

I’m using yt-3.2 (Changeset = 329b5039b031 stable).

The problem is that the same script that could finish in seconds on my local machine will take much (order of magnitude) longer time to run on the server I’m using. In some cases, it get stuck for hours that I don’t know if it will ever finish. I’ve tracked down the bottleneck to be the `annotate_contour` method. The server should be faster than my local machine. (I’m not exactly sure about how to compare it. But the server is generally much faster in other situations.)

For example, for this script (http://paste.yt-project.org/show/5808/ http://paste.yt-project.org/show/5808/), on my local machine the output shows:

``` yt : [INFO ] 2015-08-10 14:48:15,401 Making a fixed resolution buffer of (('gas', 'density')) 800 by 800 yt : [INFO ] 2015-08-10 14:48:15,751 Saving plot snapshot_200_Projection_z_density.png yt : [INFO ] 2015-08-10 14:48:16,716 Saving plot snapshot_200_Projection_z_density.png ```

However, on the server the output shows: ``` yt : [INFO ] 2015-08-10 14:51:25,551 Making a fixed resolution buffer of (('gas', 'density')) 800 by 800 yt : [INFO ] 2015-08-10 14:51:25,917 Saving plot snapshot_200_Projection_z_density.png yt : [INFO ] 2015-08-10 14:56:21,922 Saving plot snapshot_200_Projection_z_density.png ```

That’s 296 times slower!!!

This is likely to be a very specific problem with the server. But I’d like to know if anyone else has encountered similar problems before so I can learn from your experience. For now, I have no idea what could have gone wrong.

BTW, I’m using TSCC (http://idi.ucsd.edu/computing/ http://idi.ucsd.edu/computing/) at UC San Diego, if you happen to know it.

Thanks, Bili

Hi Bili, make sure you pass "-DNDEBUG" to CPPFLAGS when you compile matplotlib. Without it mpl is a good cpu cycle burner. There's a debug routine in triangulation module that's enabled by default. Cheers, Kacper

...
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

John Wise

1:54 a.m.

Hi, I just wanted to follow-up with a question since -DNDEBUG should be specified by default from the Python installation. Does anyone know why CPPFLAGS wasn't passed into mpl's setup.py? Bili, did you install yt with the install_script or on top of the system's or module's Python stack? Thanks, John On 08/10/2015 09:27 PM, Bili Dong wrote:

...

I reinstalled matplotlib and it works. Thanks Kacper! You saved my life!

-Bili

...
On Aug 10, 2015, at 4:30 PM, Kacper Kowalik wrote:

On 08/10/2015 05:10 PM, Bili Dong wrote:

...
Hi yt-users,

I’m using yt-3.2 (Changeset = 329b5039b031 stable).

The problem is that the same script that could finish in seconds on my local machine will take much (order of magnitude) longer time to run on the server I’m using. In some cases, it get stuck for hours that I don’t know if it will ever finish. I’ve tracked down the bottleneck to be the `annotate_contour` method. The server should be faster than my local machine. (I’m not exactly sure about how to compare it. But the server is generally much faster in other situations.)

For example, for this script (http://paste.yt-project.org/show/5808/ http://paste.yt-project.org/show/5808/), on my local machine the output shows:

``` yt : [INFO ] 2015-08-10 14:48:15,401 Making a fixed resolution buffer of (('gas', 'density')) 800 by 800 yt : [INFO ] 2015-08-10 14:48:15,751 Saving plot snapshot_200_Projection_z_density.png yt : [INFO ] 2015-08-10 14:48:16,716 Saving plot snapshot_200_Projection_z_density.png ```

However, on the server the output shows: ``` yt : [INFO ] 2015-08-10 14:51:25,551 Making a fixed resolution buffer of (('gas', 'density')) 800 by 800 yt : [INFO ] 2015-08-10 14:51:25,917 Saving plot snapshot_200_Projection_z_density.png yt : [INFO ] 2015-08-10 14:56:21,922 Saving plot snapshot_200_Projection_z_density.png ```

That’s 296 times slower!!!

This is likely to be a very specific problem with the server. But I’d like to know if anyone else has encountered similar problems before so I can learn from your experience. For now, I have no idea what could have gone wrong.

BTW, I’m using TSCC (http://idi.ucsd.edu/computing/ http://idi.ucsd.edu/computing/) at UC San Diego, if you happen to know it.

Thanks, Bili

Hi Bili, make sure you pass "-DNDEBUG" to CPPFLAGS when you compile matplotlib. Without it mpl is a good cpu cycle burner. There's a debug routine in triangulation module that's enabled by default. Cheers, Kacper

...
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

-- John Wise Assistant Professor of Physics Center for Relativistic Astrophysics, Georgia Tech http://cosmo.gatech.edu

Bili Dong

2:03 a.m.

Hi John, Previously I was using the default mpl provided by the system. Then I installed a local version using `pip install --user matplotlib` without specifying -DNDEBUG explicitly, after which the slowdown was gone. I guess somehow the system default version missed the -DNDEBUG flag while compiling. But I don’t know the details. Regards, Bili

...

On Aug 10, 2015, at 6:54 PM, John Wise wrote:

Hi,

I just wanted to follow-up with a question since -DNDEBUG should be specified by default from the Python installation. Does anyone know why CPPFLAGS wasn't passed into mpl's setup.py?

Bili, did you install yt with the install_script or on top of the system's or module's Python stack?

Thanks, John

On 08/10/2015 09:27 PM, Bili Dong wrote:

...
I reinstalled matplotlib and it works. Thanks Kacper! You saved my life!

-Bili

...
On Aug 10, 2015, at 4:30 PM, Kacper Kowalik wrote:

On 08/10/2015 05:10 PM, Bili Dong wrote:

...
Hi yt-users,

I’m using yt-3.2 (Changeset = 329b5039b031 stable).

The problem is that the same script that could finish in seconds on my local machine will take much (order of magnitude) longer time to run on the server I’m using. In some cases, it get stuck for hours that I don’t know if it will ever finish. I’ve tracked down the bottleneck to be the `annotate_contour` method. The server should be faster than my local machine. (I’m not exactly sure about how to compare it. But the server is generally much faster in other situations.)

For example, for this script (http://paste.yt-project.org/show/5808/ http://paste.yt-project.org/show/5808/), on my local machine the output shows:

``` yt : [INFO ] 2015-08-10 14:48:15,401 Making a fixed resolution buffer of (('gas', 'density')) 800 by 800 yt : [INFO ] 2015-08-10 14:48:15,751 Saving plot snapshot_200_Projection_z_density.png yt : [INFO ] 2015-08-10 14:48:16,716 Saving plot snapshot_200_Projection_z_density.png ```

However, on the server the output shows: ``` yt : [INFO ] 2015-08-10 14:51:25,551 Making a fixed resolution buffer of (('gas', 'density')) 800 by 800 yt : [INFO ] 2015-08-10 14:51:25,917 Saving plot snapshot_200_Projection_z_density.png yt : [INFO ] 2015-08-10 14:56:21,922 Saving plot snapshot_200_Projection_z_density.png ```

That’s 296 times slower!!!

This is likely to be a very specific problem with the server. But I’d like to know if anyone else has encountered similar problems before so I can learn from your experience. For now, I have no idea what could have gone wrong.

BTW, I’m using TSCC (http://idi.ucsd.edu/computing/ http://idi.ucsd.edu/computing/) at UC San Diego, if you happen to know it.

Thanks, Bili

Hi Bili, make sure you pass "-DNDEBUG" to CPPFLAGS when you compile matplotlib. Without it mpl is a good cpu cycle burner. There's a debug routine in triangulation module that's enabled by default. Cheers, Kacper

...
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

-- John Wise Assistant Professor of Physics Center for Relativistic Astrophysics, Georgia Tech http://cosmo.gatech.edu _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

John Wise

2:16 a.m.

Thanks, Bili. There could have been some loose ends in the system version. I was confused because this shouldn't have happened with the install script, but this sort of clears things up. Thanks again! John On 08/10/2015 10:03 PM, Bili Dong wrote:

...

Hi John,

Previously I was using the default mpl provided by the system. Then I installed a local version using `pip install --user matplotlib` without specifying -DNDEBUG explicitly, after which the slowdown was gone. I guess somehow the system default version missed the -DNDEBUG flag while compiling. But I don’t know the details.

Regards, Bili

...
On Aug 10, 2015, at 6:54 PM, John Wise wrote:

Hi,

I just wanted to follow-up with a question since -DNDEBUG should be specified by default from the Python installation. Does anyone know why CPPFLAGS wasn't passed into mpl's setup.py?

Bili, did you install yt with the install_script or on top of the system's or module's Python stack?

Thanks, John

On 08/10/2015 09:27 PM, Bili Dong wrote:

...
I reinstalled matplotlib and it works. Thanks Kacper! You saved my life!

-Bili

...
On Aug 10, 2015, at 4:30 PM, Kacper Kowalik wrote:

On 08/10/2015 05:10 PM, Bili Dong wrote:

...
Hi yt-users,

I’m using yt-3.2 (Changeset = 329b5039b031 stable).

The problem is that the same script that could finish in seconds on my local machine will take much (order of magnitude) longer time to run on the server I’m using. In some cases, it get stuck for hours that I don’t know if it will ever finish. I’ve tracked down the bottleneck to be the `annotate_contour` method. The server should be faster than my local machine. (I’m not exactly sure about how to compare it. But the server is generally much faster in other situations.)

For example, for this script (http://paste.yt-project.org/show/5808/ http://paste.yt-project.org/show/5808/), on my local machine the output shows:

``` yt : [INFO ] 2015-08-10 14:48:15,401 Making a fixed resolution buffer of (('gas', 'density')) 800 by 800 yt : [INFO ] 2015-08-10 14:48:15,751 Saving plot snapshot_200_Projection_z_density.png yt : [INFO ] 2015-08-10 14:48:16,716 Saving plot snapshot_200_Projection_z_density.png ```

However, on the server the output shows: ``` yt : [INFO ] 2015-08-10 14:51:25,551 Making a fixed resolution buffer of (('gas', 'density')) 800 by 800 yt : [INFO ] 2015-08-10 14:51:25,917 Saving plot snapshot_200_Projection_z_density.png yt : [INFO ] 2015-08-10 14:56:21,922 Saving plot snapshot_200_Projection_z_density.png ```

That’s 296 times slower!!!

This is likely to be a very specific problem with the server. But I’d like to know if anyone else has encountered similar problems before so I can learn from your experience. For now, I have no idea what could have gone wrong.

BTW, I’m using TSCC (http://idi.ucsd.edu/computing/ http://idi.ucsd.edu/computing/) at UC San Diego, if you happen to know it.

Thanks, Bili

Hi Bili, make sure you pass "-DNDEBUG" to CPPFLAGS when you compile matplotlib. Without it mpl is a good cpu cycle burner. There's a debug routine in triangulation module that's enabled by default. Cheers, Kacper

...
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

-- John Wise Assistant Professor of Physics Center for Relativistic Astrophysics, Georgia Tech http://cosmo.gatech.edu _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

-- John Wise Assistant Professor of Physics Center for Relativistic Astrophysics, Georgia Tech http://cosmo.gatech.edu

Nathan Goldbaum

2:17 a.m.

Might be worth sending a message to the triton sysadmins too. On Monday, August 10, 2015, John Wise wrote:

...

Thanks, Bili. There could have been some loose ends in the system version. I was confused because this shouldn't have happened with the install script, but this sort of clears things up.

Thanks again! John

On 08/10/2015 10:03 PM, Bili Dong wrote:

...
Hi John,

Previously I was using the default mpl provided by the system. Then I installed a local version using `pip install --user matplotlib` without specifying -DNDEBUG explicitly, after which the slowdown was gone. I guess somehow the system default version missed the -DNDEBUG flag while compiling. But I don’t know the details.

Regards, Bili

On Aug 10, 2015, at 6:54 PM, John Wise wrote:

...
Hi,

I just wanted to follow-up with a question since -DNDEBUG should be specified by default from the Python installation. Does anyone know why CPPFLAGS wasn't passed into mpl's setup.py?

Bili, did you install yt with the install_script or on top of the system's or module's Python stack?

Thanks, John

On 08/10/2015 09:27 PM, Bili Dong wrote:

...
I reinstalled matplotlib and it works. Thanks Kacper! You saved my life!

-Bili

On Aug 10, 2015, at 4:30 PM, Kacper Kowalik

...
wrote:

On 08/10/2015 05:10 PM, Bili Dong wrote:

...
Hi yt-users,

I’m using yt-3.2 (Changeset = 329b5039b031 stable).

The problem is that the same script that could finish in seconds on my local machine will take much (order of magnitude) longer time to run on the server I’m using. In some cases, it get stuck for hours that I don’t know if it will ever finish. I’ve tracked down the bottleneck to be the `annotate_contour` method. The server should be faster than my local machine. (I’m not exactly sure about how to compare it. But the server is generally much faster in other situations.)

For example, for this script (http://paste.yt-project.org/show/5808/ http://paste.yt-project.org/show/5808/), on my local machine the output shows:

``` yt : [INFO ] 2015-08-10 14:48:15,401 Making a fixed resolution buffer of (('gas', 'density')) 800 by 800 yt : [INFO ] 2015-08-10 14:48:15,751 Saving plot snapshot_200_Projection_z_density.png yt : [INFO ] 2015-08-10 14:48:16,716 Saving plot snapshot_200_Projection_z_density.png ```

However, on the server the output shows: ``` yt : [INFO ] 2015-08-10 14:51:25,551 Making a fixed resolution buffer of (('gas', 'density')) 800 by 800 yt : [INFO ] 2015-08-10 14:51:25,917 Saving plot snapshot_200_Projection_z_density.png yt : [INFO ] 2015-08-10 14:56:21,922 Saving plot snapshot_200_Projection_z_density.png ```

That’s 296 times slower!!!

This is likely to be a very specific problem with the server. But I’d like to know if anyone else has encountered similar problems before so I can learn from your experience. For now, I have no idea what could have gone wrong.

BTW, I’m using TSCC (http://idi.ucsd.edu/computing/ < http://idi.ucsd.edu/computing/>) at UC San Diego, if you happen to know it.

Thanks, Bili

Hi Bili, make sure you pass "-DNDEBUG" to CPPFLAGS when you compile matplotlib. Without it mpl is a good cpu cycle burner. There's a debug routine in triangulation module that's enabled by default. Cheers, Kacper

...
_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

-- John Wise Assistant Professor of Physics Center for Relativistic Astrophysics, Georgia Tech http://cosmo.gatech.edu _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

_______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

-- John Wise Assistant Professor of Physics Center for Relativistic Astrophysics, Georgia Tech http://cosmo.gatech.edu _______________________________________________ yt-users mailing list yt-users@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

3180

Age (days ago)

3181

Last active (days ago)

List overview

Download

10 comments

5 participants

participants (5)

Bili Dong
Cameron Hummels
John Wise
Kacper Kowalik
Nathan Goldbaum

annotate_contour takes much longer time to run than expected on some machines

tags

participants (5)