Mailman 3 November 2010 - Python-Dev

Garbage announcement printed on interpreter shutdown
by Georg Brandl 02 Nov '10

02 Nov '10

Hey #python-dev, I'd like to ask your opinion on this change; I think it should be reverted or at least made silent by default. Basically, it prints a warning like gc: 2 uncollectable objects at shutdown: Use gc.set_debug(gc.DEBUG_UNCOLLECTABLE) to list them. at interpreter shutdown if gc.garbage is nonempty. IMO this runs contrary to the decision we made when DeprecationWarnings were made silent by default: it spews messages not only at developers, but also at users, who don't need it and probably are going to be quite confused by it, assuming it came from their console application (imagine Mercurial printing this). Opinions? Georg Am 09.08.2010 00:18, schrieb antoine.pitrou: > Author: antoine.pitrou > Date: Mon Aug 9 00:18:46 2010 > New Revision: 83861 > > Log: > Issue #477863: Print a warning at shutdown if gc.garbage is not empty. > > > > Modified: > python/branches/py3k/Doc/library/gc.rst > python/branches/py3k/Doc/whatsnew/3.2.rst > python/branches/py3k/Include/pythonrun.h > python/branches/py3k/Lib/test/test_gc.py > python/branches/py3k/Misc/NEWS > python/branches/py3k/Modules/gcmodule.c > python/branches/py3k/Python/pythonrun.c > > Modified: python/branches/py3k/Doc/library/gc.rst > ============================================================================== > --- python/branches/py3k/Doc/library/gc.rst (original) > +++ python/branches/py3k/Doc/library/gc.rst Mon Aug 9 00:18:46 2010 > @@ -177,6 +177,15 @@ > If :const:`DEBUG_SAVEALL` is set, then all unreachable objects will be added to > this list rather than freed. > > + .. versionchanged:: 3.2 > + If this list is non-empty at interpreter shutdown, a warning message > + gets printed: > + > + :: > + > + gc: 2 uncollectable objects at shutdown: > + Use gc.set_debug(gc.DEBUG_UNCOLLECTABLE) to list them. > + > The following constants are provided for use with :func:`set_debug`: > > > @@ -197,6 +206,9 @@ > reachable but cannot be freed by the collector). These objects will be added to > the ``garbage`` list. > > + .. versionchanged:: 3.2 > + Also print the contents of the :data:`garbage` list at interpreter > + shutdown (rather than just its length), if it isn't empty. > > .. data:: DEBUG_SAVEALL > > > Modified: python/branches/py3k/Doc/whatsnew/3.2.rst > ============================================================================== > --- python/branches/py3k/Doc/whatsnew/3.2.rst (original) > +++ python/branches/py3k/Doc/whatsnew/3.2.rst Mon Aug 9 00:18:46 2010 > @@ -119,6 +119,11 @@ > * The :class:`ftplib.FTP` class now supports the context manager protocol > (Contributed by Tarek Ziadé and Giampaolo Rodolà; :issue:`4972`.) > > +* A warning message will now get printed at interpreter shutdown if > + the :data:`gc.garbage` list isn't empty. This is meant to make the > + programmer aware that his code contains object finalization issues. > + (Added by Antoine Pitrou; :issue:`477863`.) > + > * The :func:`shutil.copytree` function has two new options: > > * *ignore_dangling_symlinks*: when ``symlinks=False`` (meaning that the > > Modified: python/branches/py3k/Include/pythonrun.h > ============================================================================== > --- python/branches/py3k/Include/pythonrun.h (original) > +++ python/branches/py3k/Include/pythonrun.h Mon Aug 9 00:18:46 2010 > @@ -148,6 +148,7 @@ > PyAPI_FUNC(void) PyByteArray_Fini(void); > PyAPI_FUNC(void) PyFloat_Fini(void); > PyAPI_FUNC(void) PyOS_FiniInterrupts(void); > +PyAPI_FUNC(void) _PyGC_Fini(void); > > /* Stuff with no proper home (yet) */ > PyAPI_FUNC(char *) PyOS_Readline(FILE *, FILE *, char *); > > Modified: python/branches/py3k/Lib/test/test_gc.py > ============================================================================== > --- python/branches/py3k/Lib/test/test_gc.py (original) > +++ python/branches/py3k/Lib/test/test_gc.py Mon Aug 9 00:18:46 2010 > @@ -1,5 +1,5 @@ > import unittest > -from test.support import verbose, run_unittest > +from test.support import verbose, run_unittest, strip_python_stderr > import sys > import gc > import weakref > @@ -466,6 +466,42 @@ > # would be damaged, with an empty __dict__. > self.assertEqual(x, None) > > + def test_garbage_at_shutdown(self): > + import subprocess > + code = """if 1: > + import gc > + class X: > + def __init__(self, name): > + self.name = name > + def __repr__(self): > + return "<X %%r>" %% self.name > + def __del__(self): > + pass > + > + x = X('first') > + x.x = x > + x.y = X('second') > + del x > + if %d: > + gc.set_debug(gc.DEBUG_UNCOLLECTABLE) > + """ > + def run_command(code): > + p = subprocess.Popen([sys.executable, "-c", code], > + stdout=subprocess.PIPE, > + stderr=subprocess.PIPE) > + stdout, stderr = p.communicate() > + self.assertEqual(p.returncode, 0) > + self.assertEqual(stdout.strip(), b"") > + return strip_python_stderr(stderr) > + > + stderr = run_command(code % 0) > + self.assertIn(b"gc: 2 uncollectable objects at shutdown", stderr) > + self.assertNotIn(b"[<X 'first'>, <X 'second'>]", stderr) > + # With DEBUG_UNCOLLECTABLE, the garbage list gets printed > + stderr = run_command(code % 1) > + self.assertIn(b"gc: 2 uncollectable objects at shutdown", stderr) > + self.assertIn(b"[<X 'first'>, <X 'second'>]", stderr) > + > class GCTogglingTests(unittest.TestCase): > def setUp(self): > gc.enable() > > Modified: python/branches/py3k/Misc/NEWS > ============================================================================== > --- python/branches/py3k/Misc/NEWS (original) > +++ python/branches/py3k/Misc/NEWS Mon Aug 9 00:18:46 2010 > @@ -30,6 +30,8 @@ > Extensions > ---------- > > +- Issue #477863: Print a warning at shutdown if gc.garbage is not empty. > + > - Issue #6869: Fix a refcount problem in the _ctypes extension. > > - Issue #5504: ctypes should now work with systems where mmap can't > > Modified: python/branches/py3k/Modules/gcmodule.c > ============================================================================== > --- python/branches/py3k/Modules/gcmodule.c (original) > +++ python/branches/py3k/Modules/gcmodule.c Mon Aug 9 00:18:46 2010 > @@ -1295,17 +1295,16 @@ > > static struct PyModuleDef gcmodule = { > PyModuleDef_HEAD_INIT, > - "gc", > - gc__doc__, > - -1, > - GcMethods, > - NULL, > - NULL, > - NULL, > - NULL > + "gc", /* m_name */ > + gc__doc__, /* m_doc */ > + -1, /* m_size */ > + GcMethods, /* m_methods */ > + NULL, /* m_reload */ > + NULL, /* m_traverse */ > + NULL, /* m_clear */ > + NULL /* m_free */ > }; > > - > PyMODINIT_FUNC > PyInit_gc(void) > { > @@ -1364,6 +1363,37 @@ > return n; > } > > +void > +_PyGC_Fini(void) > +{ > + if (garbage != NULL && PyList_GET_SIZE(garbage) > 0) { > + PySys_WriteStderr( > + "gc: " > + "%" PY_FORMAT_SIZE_T "d uncollectable objects at shutdown:\n", > + PyList_GET_SIZE(garbage) > + ); > + if (debug & DEBUG_UNCOLLECTABLE) { > + PyObject *repr = NULL, *bytes = NULL; > + repr = PyObject_Repr(garbage); > + if (!repr || !(bytes = PyUnicode_EncodeFSDefault(repr))) > + PyErr_WriteUnraisable(garbage); > + else { > + PySys_WriteStderr( > + " %s\n", > + PyBytes_AS_STRING(bytes) > + ); > + } > + Py_XDECREF(repr); > + Py_XDECREF(bytes); > + } > + else { > + PySys_WriteStderr( > + " Use gc.set_debug(gc.DEBUG_UNCOLLECTABLE) to list them.\n" > + ); > + } > + } > +} > + > /* for debugging */ > void > _PyGC_Dump(PyGC_Head *g) > > Modified: python/branches/py3k/Python/pythonrun.c > ============================================================================== > --- python/branches/py3k/Python/pythonrun.c (original) > +++ python/branches/py3k/Python/pythonrun.c Mon Aug 9 00:18:46 2010 > @@ -404,6 +404,9 @@ > while (PyGC_Collect() > 0) > /* nothing */; > #endif > + /* We run this while most interpreter state is still alive, so that > + debug information can be printed out */ > + _PyGC_Fini(); > > /* Destroy all modules */ > PyImport_Cleanup(); > _______________________________________________ > Python-checkins mailing list > Python-checkins(a)python.org > http://mail.python.org/mailman/listinfo/python-checkins -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out.

17 19

Change to logging Formatters: support for alternative format styles
by Vinay Sajip 01 Nov '10

01 Nov '10

I've just checked in a change to logging into the py3k branch (r85835), including doc changes and tests, for providing slightly more flexibility in alternative format styles for logging. Basically, Formatter.__init__ gets an extra optional keyword arg style=<one of '%' (default), '{' or '$'>. This is then used to merge the format string with the LogRecord: either fmt % record.__dict__, or fmt.format(**record.dict), or string.Template(fmt).substitute(**record.dict). Backward compatibility is maintained (unless I've missed something). This does not cater for how you combine the logging message + its args when you make a logging call, but should work with any existing library which uses %-formatting in its logging calls. We discussed this here around a year ago in the context of generally encouraging a move from %-formatting to {}-formatting, but there seemed to be no obvious easy path forward. As of now, and even without this change I've just checked in, developers can use {}- or $-formatting in their logging messages using e.g. logger.debug(__('Message with {} {}', 2, 'place-holders')) with a suitable class standing in for __. A bit ugly, I know, with extra parentheses and what not, but hopefully for those who must have {} now it shouldn't be too much of a price to pay and there's no major performance implication, as actual rendering to string is done as late as possible, just as it is now. Comments welcome. Assuming there are no strong objections asking for reversion of this change, I'll publicise to the wider community in a few days. Regards, Vinay Sajip

7 13

PyMem_MALLOC vs PyMem_Malloc
by Hirokazu Yamamoto 01 Nov '10

01 Nov '10

Hello. I found several codes using PyMem_Free to free allocated memory with PyMem_MALLOC (ie: PyUnicode_AsWideCharString) Is it safe?

2 3

time.wallclock()
by Kristján Valur Jónsson 01 Nov '10

01 Nov '10

Working on Condition variables and semaphores (see http://bugs.python.org/issue10260) I noticed that time.time() was being used to correctly time blocking system calls. On windows, I would have used time.clock() but reading the documentation made me realize that on Unix that would return CPU seconds which are useless when blocking. However, on Windows, time.clock() has a much higher resolution, apart from being a "wallclock" time, and is thus better suited to timing that time.time(). In addition, time.time() has the potential of giving unexpected results if someone messes with the system clock. I was wondering if it were helpful to have a function such as time.wallclock() which is specified to give relative wallclock time between invocations or an approximation thereof, to the system's best ability? We could then choose this to be an alias of time.clock() on windows and time.time() on any other machine, or even have custom implementations on machines that support such a notion. Kristján

2 2

okay to remove argparse.__all__?
by Steven Bethard 01 Nov '10

01 Nov '10

I think the easiest and most sensible way to address http://bugs.python.org/issue9353 is to simply remove the __all__ definition from argparse - everything that doesn't start with an underscore in the module is already meant to be exposed. But then I wonder - is __all__ considered part of the public API of a module? Or is it okay to just remove it and assume that no one should have been accessing it directly anyway? Steve -- Where did you get that preposterous hypothesis? Did Steve tell you that? --- The Hiphopopotamus

5 7