Mailman 3 October 2010 - Python-Dev

Re: [Python-Dev] [issue1633863] AIX: configure ignores $CC
by Sébastien Sablé Nov. 8, 2010

Nov. 8, 2010

Hi Martin, I have started to correct quite a lot of issues I have with Python on AIX, and since I had to test quite a lot of patchs, I though it would be more convenient to setup a buildbot for that platform. So I now have a buildbot environment with 2 slaves (AIX 5.3 and 6.1) that builds and tests Python (branch py3k) with both gcc and xlc (the native AIX compiler) (I have 4 builders ("py3k-aix6-xlc", "py3k-aix5-xlc", "py3k-aix6-gcc", "py3k-aix5-gcc"). I expect to add 4 more builders … [View More]

7 16

new buffer in python2.7
by Kristján Valur Jónsson Nov. 8, 2010

Nov. 8, 2010

Although 2.7 has the new buffer interface and memoryview objects, these are widely not accepted in the built in modules. Examples are the structmodule, some of the socketmodule apis, structmodule, etc. IMHO this is unfortunate. For example when doign network io, you would want code like this: Buffer = bytearray(10) Socket.recv_into(Buffer) Header = struct.unpack("i", memoryview(Buffer)[:4])[0] In other words, you want to minimize coping the data around. Currently in 2.7 you have to cast the … [View More]

7 20

Re: [Python-Dev] [Python-checkins] r85902 - in python/branches/py3k/Lib: os.py test/test_os.py
by Benjamin Peterson Nov. 4, 2010

Nov. 4, 2010

2010/10/28 victor.stinner <python-checkins(a)python.org>: > Author: victor.stinner > Date: Fri Oct 29 02:38:58 2010 > New Revision: 85902 > > Log: > Issue #10210: os.get_exec_path() ignores BytesWarning warnings > > > Modified: > python/branches/py3k/Lib/os.py > python/branches/py3k/Lib/test/test_os.py > > Modified: python/branches/py3k/Lib/os.py > ============================================================================== > ---… [View More]

3 9

str.format_from_mapping
by Eric Smith Nov. 4, 2010

Nov. 4, 2010

What are your thoughts on adding a str.format_from_mapping (or similar name, maybe the suggested "format_map") to 3.2? See http://bugs.python.org/issue6081 . This method would be similar to "%(foo)s %(bar)s" % d, where d is a dict (or rather any mapping object), but of course would use str.format syntax: "{foo} {bar}".format_from_mapping(d). I like the idea. It's particularly handy when converting from %-formatting. Eric.

6 11

On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/
by Raymond Hettinger Nov. 3, 2010

Nov. 3, 2010

On Oct 26, 2010, at 9:31 AM, Guido van Rossum wrote: > I would like Gregor Lingl's approval of turning turtle.py into a package. It might make some things harder for novices, e.g. trackebacks and just browsing the source code. > > Also many people don't expect to find any code in a file named __init__.py (and most of the time I agree with this). But the alternative isn't so great either, assuming we'll want strict backwards compatibility (I wouldn't want the instructions in Gregor's … [View More]or anyone's book to start failing because of this). You can't rename turtle to turtle/turtle.py either, because then there'd be horrible confusion between turtle/ and turtle.py. > > IOW, yes, flat still seems better than nested here! Thanks for saying this. It might be a good time to also have a discussion about what may be an emerging trend of wanting to split standard library modules into packages. I understand the urge to split longer modules into multiple source files but would like to mention some of the advantages of keeping modules in a single source file. Taking the decimal module as an example: * Right now, I can pull up the source and entire history from subversion by knowing just the module name: http://svn.python.org/view/python/branches/py3k/Lib/decimal.py?view=markup , and I can search the entire module with nothing more sophisticated than "find" in the web browser. That no longer works with unittest. * I can easily email the file to someone or copy it back to a previous version of Python because only one file is involved. To look at the code, I don't have to guess how someone would have partitioned it into decimal.context.utils.threadlocal.py or somesuch. I don't need a packaging tool to distribute a module (pyparsing and BeautifulSoup are two examples of tools that are easily managed by just being in one file). * Some editors are set up to easily dance across multiple files and directories as if they were a single unit; however, many editors are not. Most file viewers and editors work better with single files. Take two operations in IDLE for example. If I press Alt-M for open module, I can retrieve all of the source for decimal (but this won't work with the logging or unittest packages). Once the source is visible, I press Alt-C to bring up a class browser for quickly reviewing the source (but this also won't work with packages). Several Python editors have this capability (provided by the pyclbr module) but they are defeated by packaging. * The Lib directory is more easily grepped when it is flat. The more we nest the Lib directory, the harder it will be to search the sources (while excluding the test directory). Packaging is not always wrong. Maybe it was the right thing to do for unittest, maybe not. I just wanted to bring up some of the advantages of having a single file for a library module. It would not be a good thing if some of the newer committers embarked on taking modules over 2000 lines and split them into packages. Right now, we have hundreds of source files, but it wouldn't take long for someone on a packaging mission to turn that into thousands. If someone wants to reorganize code for clarity, I would prefer keeping it within one file, bringing related functions together and using comment lines to mark the major sections. ISTM, this is cleaner than introducing a new directory and having multiple files that cross-import one another. Raymond [View Less]

22 79

Cleaning-up the new unittest API
by Raymond Hettinger Nov. 2, 2010

Nov. 2, 2010

The API for the unittest module has grown fat (the documentation is approaching 2,000 lines and 10,000 words like a small book). I think we can improve learnability by focusing on the most important parts of the API. I would like to simplify and clean-up the API for the unittest module by de-documenting assertSetEqual(), assertDictEqual(), assertListEqual, and assertTupleEqual(). All of those methods are already called automatically by assertEqual(), so those methods never need to be … [View More]called directly. Or, if you need to be more explicit about the type checking for sequences, the assertSequenceEqual() method will suffice. Either way, there's no need to expose the four type specific methods. Besides de-documenting those four redundant methods, I propose that assertItemsEqual() be deprecated just like its brother assertSameElements(). I haven't found anyone who accurately guesses what those methods entail based on their method names ("items" usually implies key/value pairs elsewhere in the language; nor is it clear whether order is important, whether the elements need to be hashable or orderable or just define equality tests, nor is is clear whether duplicates cause the test to fail). Given the purpose of the unittest module, it's important that the reader have a crystal clear understanding of what a test is doing. Their attention needs to be focused on the subject of the test, not on questioning the semantics of the test method. IMO, users are far better-off sticking with assertEqual() so they can be specific about the nature of the test: # hashable elements; ignore dups assertEqual(set(a), set(b)) # orderable elements; dups matter, order doesn't assertEqual(sorted(a), sorted(b)) # eq tested elements, dups matter, order matters assertEqual(list(a), list(b)) # hashable keys, eq tested values # ignore dups, ignore order assertEqual(dict(a), dict(b)) These take just a few more characters than assertSameElements() and assertItemsEqual(), but they are far more clear about their meaning. You won't have to second guess what semantics are hidden behind the abstraction. There are a couple other problems with the new API but it is probably too late to do anything about it. * elsewhere in Python we spell comparison names with abbreviations like eq, ne, lt, le, gt, ge. In unittest, those are spelled in an awkward, not easily remembered manner: assertLessEqual(a, b), etc. Fortunately, it's clear what the mean; however, it's not easy to guess their spelling. * the names for assertRegexpMatches() and assertNotRegexpMatches are deeply misleading since they are implemented in terms of re.search(), not re.match(). Raymond P.S. I also looked ar assertDictContainsSubset(a,b). It is a bit over-specialized, but at least it is crystal clear what is does and it beats the awkward alternative using dict views: assertLessEqual(a.items(), b.items()) [View Less]

16 38

Garbage announcement printed on interpreter shutdown
by Georg Brandl Nov. 2, 2010

Nov. 2, 2010

Hey #python-dev, I'd like to ask your opinion on this change; I think it should be reverted or at least made silent by default. Basically, it prints a warning like gc: 2 uncollectable objects at shutdown: Use gc.set_debug(gc.DEBUG_UNCOLLECTABLE) to list them. at interpreter shutdown if gc.garbage is nonempty. IMO this runs contrary to the decision we made when DeprecationWarnings were made silent by default: it spews messages not only at developers, but also at users, who … [View More]don't need it and probably are going to be quite confused by it, assuming it came from their console application (imagine Mercurial printing this). Opinions? Georg Am 09.08.2010 00:18, schrieb antoine.pitrou: > Author: antoine.pitrou > Date: Mon Aug 9 00:18:46 2010 > New Revision: 83861 > > Log: > Issue #477863: Print a warning at shutdown if gc.garbage is not empty. > > > > Modified: > python/branches/py3k/Doc/library/gc.rst > python/branches/py3k/Doc/whatsnew/3.2.rst > python/branches/py3k/Include/pythonrun.h > python/branches/py3k/Lib/test/test_gc.py > python/branches/py3k/Misc/NEWS > python/branches/py3k/Modules/gcmodule.c > python/branches/py3k/Python/pythonrun.c > > Modified: python/branches/py3k/Doc/library/gc.rst > ============================================================================== > --- python/branches/py3k/Doc/library/gc.rst (original) > +++ python/branches/py3k/Doc/library/gc.rst Mon Aug 9 00:18:46 2010 > @@ -177,6 +177,15 @@ > If :const:`DEBUG_SAVEALL` is set, then all unreachable objects will be added to > this list rather than freed. > > + .. versionchanged:: 3.2 > + If this list is non-empty at interpreter shutdown, a warning message > + gets printed: > + > + :: > + > + gc: 2 uncollectable objects at shutdown: > + Use gc.set_debug(gc.DEBUG_UNCOLLECTABLE) to list them. > + > The following constants are provided for use with :func:`set_debug`: > > > @@ -197,6 +206,9 @@ > reachable but cannot be freed by the collector). These objects will be added to > the ``garbage`` list. > > + .. versionchanged:: 3.2 > + Also print the contents of the :data:`garbage` list at interpreter > + shutdown (rather than just its length), if it isn't empty. > > .. data:: DEBUG_SAVEALL > > > Modified: python/branches/py3k/Doc/whatsnew/3.2.rst > ============================================================================== > --- python/branches/py3k/Doc/whatsnew/3.2.rst (original) > +++ python/branches/py3k/Doc/whatsnew/3.2.rst Mon Aug 9 00:18:46 2010 > @@ -119,6 +119,11 @@ > * The :class:`ftplib.FTP` class now supports the context manager protocol > (Contributed by Tarek Ziadé and Giampaolo Rodolà; :issue:`4972`.) > > +* A warning message will now get printed at interpreter shutdown if > + the :data:`gc.garbage` list isn't empty. This is meant to make the > + programmer aware that his code contains object finalization issues. > + (Added by Antoine Pitrou; :issue:`477863`.) > + > * The :func:`shutil.copytree` function has two new options: > > * *ignore_dangling_symlinks*: when ``symlinks=False`` (meaning that the > > Modified: python/branches/py3k/Include/pythonrun.h > ============================================================================== > --- python/branches/py3k/Include/pythonrun.h (original) > +++ python/branches/py3k/Include/pythonrun.h Mon Aug 9 00:18:46 2010 > @@ -148,6 +148,7 @@ > PyAPI_FUNC(void) PyByteArray_Fini(void); > PyAPI_FUNC(void) PyFloat_Fini(void); > PyAPI_FUNC(void) PyOS_FiniInterrupts(void); > +PyAPI_FUNC(void) _PyGC_Fini(void); > > /* Stuff with no proper home (yet) */ > PyAPI_FUNC(char *) PyOS_Readline(FILE *, FILE *, char *); > > Modified: python/branches/py3k/Lib/test/test_gc.py > ============================================================================== > --- python/branches/py3k/Lib/test/test_gc.py (original) > +++ python/branches/py3k/Lib/test/test_gc.py Mon Aug 9 00:18:46 2010 > @@ -1,5 +1,5 @@ > import unittest > -from test.support import verbose, run_unittest > +from test.support import verbose, run_unittest, strip_python_stderr > import sys > import gc > import weakref > @@ -466,6 +466,42 @@ > # would be damaged, with an empty __dict__. > self.assertEqual(x, None) > > + def test_garbage_at_shutdown(self): > + import subprocess > + code = """if 1: > + import gc > + class X: > + def __init__(self, name): > + self.name = name > + def __repr__(self): > + return "<X %%r>" %% self.name > + def __del__(self): > + pass > + > + x = X('first') > + x.x = x > + x.y = X('second') > + del x > + if %d: > + gc.set_debug(gc.DEBUG_UNCOLLECTABLE) > + """ > + def run_command(code): > + p = subprocess.Popen([sys.executable, "-c", code], > + stdout=subprocess.PIPE, > + stderr=subprocess.PIPE) > + stdout, stderr = p.communicate() > + self.assertEqual(p.returncode, 0) > + self.assertEqual(stdout.strip(), b"") > + return strip_python_stderr(stderr) > + > + stderr = run_command(code % 0) > + self.assertIn(b"gc: 2 uncollectable objects at shutdown", stderr) > + self.assertNotIn(b"[<X 'first'>, <X 'second'>]", stderr) > + # With DEBUG_UNCOLLECTABLE, the garbage list gets printed > + stderr = run_command(code % 1) > + self.assertIn(b"gc: 2 uncollectable objects at shutdown", stderr) > + self.assertIn(b"[<X 'first'>, <X 'second'>]", stderr) > + > class GCTogglingTests(unittest.TestCase): > def setUp(self): > gc.enable() > > Modified: python/branches/py3k/Misc/NEWS > ============================================================================== > --- python/branches/py3k/Misc/NEWS (original) > +++ python/branches/py3k/Misc/NEWS Mon Aug 9 00:18:46 2010 > @@ -30,6 +30,8 @@ > Extensions > ---------- > > +- Issue #477863: Print a warning at shutdown if gc.garbage is not empty. > + > - Issue #6869: Fix a refcount problem in the _ctypes extension. > > - Issue #5504: ctypes should now work with systems where mmap can't > > Modified: python/branches/py3k/Modules/gcmodule.c > ============================================================================== > --- python/branches/py3k/Modules/gcmodule.c (original) > +++ python/branches/py3k/Modules/gcmodule.c Mon Aug 9 00:18:46 2010 > @@ -1295,17 +1295,16 @@ > > static struct PyModuleDef gcmodule = { > PyModuleDef_HEAD_INIT, > - "gc", > - gc__doc__, > - -1, > - GcMethods, > - NULL, > - NULL, > - NULL, > - NULL > + "gc", /* m_name */ > + gc__doc__, /* m_doc */ > + -1, /* m_size */ > + GcMethods, /* m_methods */ > + NULL, /* m_reload */ > + NULL, /* m_traverse */ > + NULL, /* m_clear */ > + NULL /* m_free */ > }; > > - > PyMODINIT_FUNC > PyInit_gc(void) > { > @@ -1364,6 +1363,37 @@ > return n; > } > > +void > +_PyGC_Fini(void) > +{ > + if (garbage != NULL && PyList_GET_SIZE(garbage) > 0) { > + PySys_WriteStderr( > + "gc: " > + "%" PY_FORMAT_SIZE_T "d uncollectable objects at shutdown:\n", > + PyList_GET_SIZE(garbage) > + ); > + if (debug & DEBUG_UNCOLLECTABLE) { > + PyObject *repr = NULL, *bytes = NULL; > + repr = PyObject_Repr(garbage); > + if (!repr || !(bytes = PyUnicode_EncodeFSDefault(repr))) > + PyErr_WriteUnraisable(garbage); > + else { > + PySys_WriteStderr( > + " %s\n", > + PyBytes_AS_STRING(bytes) > + ); > + } > + Py_XDECREF(repr); > + Py_XDECREF(bytes); > + } > + else { > + PySys_WriteStderr( > + " Use gc.set_debug(gc.DEBUG_UNCOLLECTABLE) to list them.\n" > + ); > + } > + } > +} > + > /* for debugging */ > void > _PyGC_Dump(PyGC_Head *g) > > Modified: python/branches/py3k/Python/pythonrun.c > ============================================================================== > --- python/branches/py3k/Python/pythonrun.c (original) > +++ python/branches/py3k/Python/pythonrun.c Mon Aug 9 00:18:46 2010 > @@ -404,6 +404,9 @@ > while (PyGC_Collect() > 0) > /* nothing */; > #endif > + /* We run this while most interpreter state is still alive, so that > + debug information can be printed out */ > + _PyGC_Fini(); > > /* Destroy all modules */ > PyImport_Cleanup(); > _______________________________________________ > Python-checkins mailing list > Python-checkins(a)python.org > http://mail.python.org/mailman/listinfo/python-checkins -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. [View Less]

17 19

Change to logging Formatters: support for alternative format styles
by Vinay Sajip Nov. 1, 2010

Nov. 1, 2010

I've just checked in a change to logging into the py3k branch (r85835), including doc changes and tests, for providing slightly more flexibility in alternative format styles for logging. Basically, Formatter.__init__ gets an extra optional keyword arg style=<one of '%' (default), '{' or '$'>. This is then used to merge the format string with the LogRecord: either fmt % record.__dict__, or fmt.format(**record.dict), or string.Template(fmt).substitute(**record.dict). Backward compatibility … [View More]

7 13

PyMem_MALLOC vs PyMem_Malloc
by Hirokazu Yamamoto Nov. 1, 2010

Nov. 1, 2010

Hello. I found several codes using PyMem_Free to free allocated memory with PyMem_MALLOC (ie: PyUnicode_AsWideCharString) Is it safe?

2 3

Re: [Python-Dev] [Python-checkins] r85934 - in python/branches/py3k: Misc/NEWS Modules/socketmodule.c
by Hirokazu Yamamoto Oct. 31, 2010

Oct. 31, 2010

On 2010/10/30 3:20, martin.v.loewis wrote: > Modified: python/branches/py3k/Modules/socketmodule.c > ============================================================================== > --- python/branches/py3k/Modules/socketmodule.c (original) > +++ python/branches/py3k/Modules/socketmodule.c Fri Oct 29 20:20:08 2010 > @@ -3093,6 +3093,27 @@ > static PyObject * > socket_gethostname(PyObject *self, PyObject *unused) > { > +#ifdef MS_WINDOWS > + /* Don't use … [View More]

3 2