2010/9/10 Fred Drake :

On Fri, Sep 10, 2010 at 4:32 PM, Georg Brandl wrote:

...

IMO this runs contrary to the decision we made when DeprecationWarnings were made silent by default: it spews messages not only at developers, but also at users, who don't need it and probably are going to be quite confused by it,

Agreed; this should be silent by default.

+1. I suggest to enable it only when Py_DEBUG (or Py_TRACE_REFS or Py_REF_DEBUG?) is defined. -- Amaury Forgeot d'Arc

On Sat, Sep 11, 2010 at 9:42 AM, Glyph Lefkowitz wrote:

On Sep 10, 2010, at 5:10 PM, Amaury Forgeot d'Arc wrote:

...

2010/9/10 Fred Drake :

...

On Fri, Sep 10, 2010 at 4:32 PM, Georg Brandl wrote:

...

IMO this runs contrary to the decision we made when DeprecationWarnings were made silent by default: it spews messages not only at developers, but also at users, who don't need it and probably are going to be quite confused by it,

Agreed; this should be silent by default.

+1. I suggest to enable it only when Py_DEBUG (or Py_TRACE_REFS or Py_REF_DEBUG?) is defined.

Would it be possible to treat it the same way as a deprecation warning, and show it under the same conditions?  It would be nice to know if my Python program is leaking uncollectable objects without rebuilding the interpreter.

My suggestion: 1. Add a gc.WARN_UNCOLLECTABLE flag on gc.set_debug that enables the warning message. 2. Have regrtest explicitly set this for our own test suite As far as automatically turning it on for third party test suites goes, we could either: - require them to turn it on explicitly via gc.set_debug - have gc.WARN_UNCOLLECTABLE default to true for non-optimised runs (__debug__ == True) and false for runs with -O or -OO (__debug__ == False) - set it by looking at the -W arguments passed in at interpreter startup (e.g. enable it when all warnings are enabled, leave it disabled by default otherwise) Cheers, Nick. -- Nick Coghlan   |   ncoghlan@gmail.com   |   Brisbane, Australia

On Fri, Sep 10, 2010 at 14:55, Daniel Stutzbach wrote:

On Fri, Sep 10, 2010 at 3:32 PM, Georg Brandl wrote:

...

IMO this runs contrary to the decision we made when DeprecationWarnings were made silent by default: it spews messages not only at developers, but also at users, who don't need it and probably are going to be quite confused by it, assuming it came from their console application (imagine Mercurial printing this).

A non-empty gc.garbage often indicates that there is a bug in the program and that it is probably leaking memory [1].  That's a little different from a DeprecationWarning which doesn't indicate a bug; it just indicates that the program might not run correctly using a future version of Python.  I think a better comparison would be with exceptions throw from a __del__, which (as far as I know) are still printed to the console.

Sure, but exceptions printed out by a __del__ method during interpreter shutdown are not explicitly done as part of the shutdown process, they just happen to typically be triggered by a shutdown. This gc info, OTOH, is explicitly debugging information that is only printed out (typically) at shutdown and thus at a point where it will no longer effect semantics or the performance of the application. So I view this as entirely debugging information and thus in no way critical for a user to know about. -Brett

+1 on adding a way to enable/disable the feature. -1 on removing the feature -0 on making it disabled by default

[1] I know that some large, long-running programs periodically check gc.garbage and carefully choose where to break cycles, but those are the exception and not the rule. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC

_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org

2010/9/29 Antoine Pitrou :

Hello,

...

I'd like to ask your opinion on this change; I think it should be reverted or at least made silent by default.  Basically, it prints a warning like

      gc: 2 uncollectable objects at shutdown:           Use gc.set_debug(gc.DEBUG_UNCOLLECTABLE) to list them.

at interpreter shutdown if gc.garbage is nonempty.

I would like to piggy-back on this discussion to suggest further warnings (either by default, or switchable).

One feature I've often considered would be to add a warning in FileIO and socket dealloc if these objects haven't been closed explicitly. In most situations, relying on garbage collection to shutdown OS resources (here, file descriptors) is something we like to discourage. Furthermore, it can produce real bugs, especially under Windows when coupled with refererence cycles created by traceback objects (the random test_tarfile failures on the Windows buildbots were a symptom of that; their cause would have been obvious with such warnings).

What do you think?

It seems like a slippery slope. Sometimes you really don't care like when you're just hacking together a quick script. -- Regards, Benjamin

Le mercredi 29 septembre 2010 à 07:27 -0500, Benjamin Peterson a écrit :

...

I would like to piggy-back on this discussion to suggest further warnings (either by default, or switchable).

One feature I've often considered would be to add a warning in FileIO and socket dealloc if these objects haven't been closed explicitly. In most situations, relying on garbage collection to shutdown OS resources (here, file descriptors) is something we like to discourage. Furthermore, it can produce real bugs, especially under Windows when coupled with refererence cycles created by traceback objects (the random test_tarfile failures on the Windows buildbots were a symptom of that; their cause would have been obvious with such warnings).

What do you think?

It seems like a slippery slope. Sometimes you really don't care like when you're just hacking together a quick script.

Isn't the "with" statement appropriate in these cases? My assumption is/was that the benefit of warning against leaks in real applications (or even - sigh - the standard library) would outweigh the inconvenience when hacking together a quick script. But if it doesn't, what about enabling it with a command-line switch?

On Sep 29, 2010, at 11:11 PM, Steven D'Aprano wrote:

On Wed, 29 Sep 2010 10:42:27 pm Antoine Pitrou wrote:

...

My assumption is/was that the benefit of warning against leaks in real applications (or even - sigh - the standard library) would outweigh the inconvenience when hacking together a quick script.

But if it doesn't, what about enabling it with a command-line switch?

I think the ability to detect such file descriptor leaks would be valuable, but I'm not sure that it should be running all the time. At the risk of bike-shedding, is it something which could be controlled at runtime, like garbage collection? E.g. something like:

gc.enable_file_warnings() run_my_tests_for_leakage() gc.disable_file_warnings()

or similar. (I'm not wedded to it being in the gc module.)

I don't think it should be in the gc module, but I would prefer it be enabled and controlled through a separate module, rather than something Python does automatically for your convenience. -Barry

On 29 September 2010 22:25, Nick Coghlan wrote:

On Wed, Sep 29, 2010 at 11:40 PM, Barry Warsaw wrote:

...

I don't think it should be in the gc module, but I would prefer it be enabled and controlled through a separate module, rather than something Python does automatically for your convenience.

The os module would seem to be the place to enable/disable tracking of OS level resource leaks (i.e. file descriptors and possible HANDLES on Windows).

Heh, I was expecting the sys module to be the natural choice because this would be changing interpreter behaviour. It's just random bikeshedding at this point however. Regards Floris -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org

On Wed, Sep 29, 2010 at 05:42, Antoine Pitrou wrote:

Le mercredi 29 septembre 2010 à 07:27 -0500, Benjamin Peterson a écrit :

...

...

I would like to piggy-back on this discussion to suggest further warnings (either by default, or switchable).

One feature I've often considered would be to add a warning in FileIO and socket dealloc if these objects haven't been closed explicitly. In most situations, relying on garbage collection to shutdown OS resources (here, file descriptors) is something we like to discourage. Furthermore, it can produce real bugs, especially under Windows when coupled with refererence cycles created by traceback objects (the random test_tarfile failures on the Windows buildbots were a symptom of that; their cause would have been obvious with such warnings).

What do you think?

It seems like a slippery slope. Sometimes you really don't care like when you're just hacking together a quick script.

Isn't the "with" statement appropriate in these cases?

Yes, which is why I suspect people saying they don't bother have been programming Python for a while and are not in the habit yet of using a 'with' statement. The amount of extra typing compared to inlining a call is minimal.

My assumption is/was that the benefit of warning against leaks in real applications (or even - sigh - the standard library) would outweigh the inconvenience when hacking together a quick script.

Does everyone here run all their code under py-debug? If not then I say switch it on when py-debug is on so that we at least detect the leaks in the stdlib without having to think about it.

But if it doesn't, what about enabling it with a command-line switch?

Sure, but I say always turn it on under py-debug.

Sorry for late post. On 2010/09/29 20:01, Antoine Pitrou wrote:

Furthermore, it can produce real bugs, especially under Windows when coupled with refererence cycles created by traceback objects

I think this can be relaxed with the patch in #9815. ;-)