Mailman 3 April 2005 - Python-Dev

python-dev Summary for 2005-04-01 through 2005-04-15 [draft]
by Tim Lesher April 18, 2005

April 18, 2005

Here's the first draft of the python-dev summary for the first half of April. Please send any corrections or suggestions to the summarizers. ====================== Summary Announcements ====================== --------------------------- New python-dev summary team --------------------------- This summary marks the first by the team of Steve Bethard, Tim Lesher, and Tony Meyer. We're trying a collaborative approach to the summaries: each fortnight, we'll be getting together in a virtual smoke-filled back room to divide up the interesting threads. Then we'll stitch together the summaries in roughly the same form as you've seen in the past. We'll mark each editor's entries with his initials. Thanks to Brett Cannon for sixty-one excellent python-dev summaries. Also, thanks for providing scripts to help get the new summaries off the ground! We're looking forward to the contributions you'll make to the Python core, now that the summaries aren't taking up all your time. [TDL] ========= Summaries ========= ---------------------- Right Operator Methods ---------------------- Greg Ewing explored an issue with new-style classes that define only right operator methods (__radd__, __rmul__, etc.) Instances of such a class cannot be added/multiplied/etc. together as Python raises a TypeError. Armin Rigo explained the rule: if the instances on both sides of an operator are of the same class, only the non-reversed method is ever called. Armin also explained that an __add__ or __mul__ method that returns NotImplemented may be called twice when Python attempts to differentiate between numeric and sequence operations. Contributing threads: - `New style classes and operator methods <http://mail.python.org/pipermail/python-dev/2005-April/052577.html>`__ [SJB] ------------------------------------------ Hierarchical groups in regular expressions ------------------------------------------ Chris Ottrey demoed his `pyre2 project`_ that can extract a hierarchy of strings when nested groups match in a regular expression. The current re module (in the stdlib) only matches the last occurrence of a group in the string, throwing away any preceding matches. People discussed some of pyre2's proposed API, with the main suggestion being to extend the API to support unnamed (positional) groups in addition to named groups. Though a number of people expressed interest in the idea, it was not clear whether the functionality should be included in the standard library. However, most agreed that if it was included, it should be integrated with the existing re module. Gustavo Niemeyer offered to perform this integration if an API could be agreed upon. Further discussion was moved to the pyre2 `development wiki`_ and `mailing list`_. Contributing threads: - `hierarchicial named groups extension to the re library <http://mail.python.org/pipermail/python-dev/2005-April/052508.html>`__ .. _pyre2 project: http://pyre2.sourceforge.net/ .. _development wiki: http://py.redsoft.be/pyre2/wiki/ .. _mailing list: http://lists.sourceforge.net/lists/listinfo/pyre2-devel [SJB] ------------------------------- Security capabilities in Python ------------------------------- The issue of security came up again, and Ka-Ping Yee suggested that in Python's restricted execution mode secure proxies can be created by using lexical scoping. He posted `some code`_ for revealing only certain "facets" of an object by using a function to declare a proxy class that used function local variables to build the proxy. Thus to access the attributes used in the proxy class, you need to access things like im_func or func_closure, which are not accessible in restricted execution mode. James Y Knight illustrated how strategic overriding of __eq__ in a subclass of str could allow access to the hidden "facets". Eyal Lotem suggested that such an attack could be countered by implementing "facets" in C, but having to turn to C every time you needed a particular security construct seemed unappealing. Contributing threads: - `Security capabilities in Python <http://mail.python.org/pipermail/python-dev/2005-April/052580.html>`__ .. _some code: http://zesty.ca/python/facet.py [SJB] --------------------------------- Improving GilState API Robustness --------------------------------- Michael Hudson noted that his changes to thread handling in the readline module appeared to trigger `bug 1176893`_ ("Readline segfault"). However, he believed the problem lay in the GilState API, rather than in his changes: PyGilState_Release crashes if PyEval_InitThreads wasn't called, even if the code you're writing doesn't use multiple threads. He proposed several solutions, none of which met with resounding approbation, and Tim Peters noted that `PEP 311`_, Simplified Global Interpreter Lock Acquisition for Extensions, "specifically disowns responsibility for worrying about whether Py_Initialize and PyEval_InitThreads have been called." Bob Ippolito wondered whether just calling PyEval_InitThreads directly in Py_Initialize might be a better idea. No objections were raised, so long as the underlying OS locking mechanisms weren't overly expensive; some initial benchmarks indicated that this approach was viable, at least on Linux and OS X. Contributing threads: - `threading (GilState) question <http://mail.python.org/pipermail/python-dev/2005-April/052562.html>`__ .. _bug 1176893: http://sourceforge.net/tracker/index.php?func=detail&aid=1176893&group_id=5… .. _PEP 311: http://www.python.org/peps/pep-0311.html [TDL] ---------------------------------------- Unicode byte order mark decoding ---------------------------------------- Evan Jones saw that the UTF-16 decoder discards the byte-order mark (BOM) from Unicode files, while the UTF-8 decoder doesn't. Although the BOM isn't really required in UTF-8 files, many Unicode-generating applications, especially on Microsoft platforms, add it. Walter Dörwald created a patch_ to add a UTF-8-Sig codec that generates a BOM on writing and skips it on reading, but after a long discussion on the history of the Unicode, Microsoft's influence over its evolution, the consensus was that BOM and signature handling belong at a higher level (for example, a stream API) than the codec. Contributing threads: - `Unicode byte order mark decoding <http://mail.python.org/pipermail/python-dev/2005-April/052501.html>`__ .. _patch: http://sourceforge.net/tracker/index.php?func=detail&aid=1177307&group_id=5… [TDL] --------------- Developers List --------------- Raymond Hettinger has started a `project to track developers`_ and the (tracker and commit) privileges they have, and who gave them the privileges, and why (for example, was it for a one-shot project). Removing inactive developers should improve clarity, institutional memory, security, and makes everything tidier. Raymond has begun contacting recently inactive developers to check whether they still require the privileges they have. Contributing threads: - `Developer list update <http://mail.python.org/pipermail/python-dev/2005-April/052540.html>`__ .. _project to track developers: http://cvs.sourceforge.net/viewcvs.py/*checkout*/python/python/dist/src/Mis… [TAM] -------------------- Marshalling Infinity -------------------- Scott David Daniels kicked off a very long thread by asking what (un)marshal should do with floating point NaNs. The current behaviour (as with any NaN, infinity, or signed zero) is undefined: a platform-dependant accident, because Python is written to C89, which has no such concepts. Tim Peters pointed out all code for (de)serialing C doubles should go through _PyFloat_Pack8()/_PyFloat_Unpack8(), and that the current implementation suggests that the routines could simply copy bytes on platforms that use the standard IEEE-754 single and double formats natively. Michael Hudson obliged by creating a `patch to implement this`_. The consensus was that the correct behaviour is that packing a NaN or infinity shouldn't cause an exception. When unpacking, an IEEE-754 platform shouldn't cause an exception, but a non-754 platform should, since there's no sensible value that it can be unpacked to, and errors should never pass silently. Contributing threads: - `marshal / unmarshal <http://mail.python.org/pipermail/python-dev/2005-April/052593.html>`__ .. _patch to implement this: http://python.org/sf/1181301 [TAM] --------------------------------- Location of the sign bit in longs --------------------------------- Michael Hudson asked about the possibility of longs storing the sign bit somewhere other than the current location, suggesting the top bit of ob_digit[0]. Tim Peters suggested that it would be better to give struct _longobject a distinct sign member. This simplifies code, costs no extra bytes for some longs, and 8 extra bytes for others, and shouldn't hurt binary compatibility. Michael coughed up a `longobject patch`_, which seems likely to be checked in. Contributing threads: - `marshal / unmarshal <http://mail.python.org/pipermail/python-dev/2005-April/052593.html>`__ .. _longobject patch: http://python.org/sf/1177779 [TAM] ----------------------- Acceptable diff formats ----------------------- Nick Coghlan asked if context diffs are still favoured for patches. Historically, context diffs were preferred, but it appears that unified diffs are the today's choice. Raymond Hettinger made the sensible suggestion that whichever is most informative for the particular patch should be used, and Bob Ippolito pointed out that if CVS is replaced with subversion, unified diffs will have better support. The `patch submission guidelines`_ will be updated at some point to reflect the preference for unified diffs, although if your diff program doesn't support '-u', then context diffs are ok - plain patches are, of course, not. Contributing threads: - `Unified or context diffs? <http://mail.python.org/pipermail/python-dev/2005-April/052657.html>`__ .. _patch submission guidelines: http://www.python.org/patches/ [TAM] =============== Skipped Threads =============== - python-dev Summary for 2005-03-16 through 2005-03-31 [draft] - [Python-checkins] python/dist/src/Lib/logging handlers.py, 1.19, 1.19.2.1 - [Python-checkins] python/dist/src/Modules mathmodule.c, 2.74, 2.75 - Weekly Python Patch/Bug Summary - Mail.python.org - New bug, directly assigned, okay? - inconsistency when swapping obj.__dict__ with a dict-like object... - Pickling instances of nested classes - args attribute of Exception objects -- Tim Lesher <tlesher(a)gmail.com>

6 5

Security capabilities in Python
by Eyal Lotem April 18, 2005

April 18, 2005

I would like to experiment with security based on Python references as security capabilities. Unfortunatly, there are several problems that make Python references invalid as capabilities: * There is no way to create secure proxies because there are no private attributes. * Lots of Python objects are reachable unnecessarily breaking the principle of least privelege (i.e: object.__subclasses__() etc.) I was wondering if any such effort has already begun or if there are other considerations making Python unusable as a capability platform? (Please cc the reply to my email)

10 17

Fwd: CFP: DLS05: ACM Dynamic Languages Symposium
by Guido van Rossum April 18, 2005

April 18, 2005

See you all at OOPSLA! ---------- Forwarded message ---------- From: Roel Wuyts <Roel.Wuyts(a)ulb.ac.be> Date: Apr 17, 2005 10:59 PM Subject: CFP: DLS05: ACM Dynamic Languages Symposium To: python-announce-list(a)python.org CALL FOR PAPERS FOR THE ACM Dynamic Languages Symposium 2005 October 18, 2005 San Diego, California (co-located with OOPSLA'05) URL: http://decomp.ulb.ac.be:8082/events/dls05/ ----------- Abstract ----------- In industry, static languages (such as Java, C++ and C#) are much more widely used than their dynamic counterparts (like CLOS, Python, Self, Perl, php or Smalltalk). So it appears as though dynamic language concepts were forgotten and lost the race. But this is not the case. Java and C#, the latest mainstream static languages, popularized to a certain extent dynamic language features such as garbage collection, portability and (limited forms of) reflection. In the near future, we expect this dynamicity to increase even further. E.g., it is getting clearer year after year that pervasive computing is becoming the rule and that concepts such as meta programming, reflection, mobility, dynamic reconfigurability and distribution are becoming increasingly popular. All of these features are the domain of dynamic languages, and hence it is only logical that more dynamic language concepts have to be taken up by static languages, or that dynamic languages can make a breakthrough. Currently, the dynamic language community is fragmented, split over a multitude of paradigms (from functional over logic to object-oriented), languages and syntaxes. This fragmentation severely hinders research as well as acceptance, and results in either language wars or, even worse, language ignorance. The goal of this symposium is to provide a highly visible, international forum for researchers working on dynamic features and languages. We explicitly invite submissions from all kinds of paradigms (object-oriented, functional, logic, ...), as can be seen from the structure of the program committee. Areas of interests include, but are not limited to: - closures - delegation - actors, active objects - constraint systems - mixins and traits - reflection and meta-programming - language symbiosis and multi-paradigm languages - experience reports on successful application of dynamic languages Accepted Papers will be published in the ACM Digital Library. ------------------------------- Submission Guidelines ------------------------------- Papers will need to be submitted using an online tracking system, of which the URL will be given later. All papers must be submitted electronically in PDF format (or PostScript, if you do not have access to PDF-producing programs, but this is not recommended). Submissions, as well as final versions, must be formatted to conform to ACM Proceedings requirements: Nine point font on ten point baseline, two columns per page, each column 3.33 inches wide by 9 inches tall, with a column gutter of 0.33 inches, etc. See the ACM Proceedings Guidelines. You can save preparation time by using one of the templates from that page. Note that MS Word documents must be converted to PDF before being submitted. ---------------------- Important Dates ---------------------- - Deadline for receipt of submissions: June 24th 2005 - Notification of acceptance or rejection: August 5th 2005 - Final version for the proceedings: To be announced later --------------------------- Program Committee --------------------------- - Gilad Bracha - Wolfgang De Meuter - Stephane Ducasse - Gopal Gupta - Robert Hirschfeld - Dan Ingalls - Yukihiro Matsumoto - Mark Miller - Eliot Miranda - Philippe Mougin - Oscar Nierstrasz - Dave Thomas - David Ungar - Guido Van Rossum - Peter Van Roy - Jon L White (G) - Roel Wuyts (Chair) -- Roel Wuyts DeComp roel.wuyts(a)ulb.ac.be Université Libre de Bruxelles http://homepages.ulb.ac.be/~rowuyts/ Belgique Vice-President of the European Smalltalk Users Group: www.esug.org -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations.html -- --Guido van Rossum (home page: http://www.python.org/~guido/)

1 0

Inconsistent exception for read-only properties?
by Barry Warsaw April 18, 2005

April 18, 2005

I've noticed an apparent inconsistency in the exception thrown for read-only properties for C extension types vs. Python new-style classes. I'm wondering if this is intentional, a bug, a bug worth fixing, or whether I'm just missing something. class other(object): def __init__(self, value): self._value = value def _get_value(self): return self._value value = property(_get_value) With this class, if you attempt "other(1).value = 7" you will get an AttributeError. However, if you define something similar in C using a tp_getset, where the structure has NULL for the setter, you will get a TypeError (code available upon request). At best, this is inconsistent. What's the "right" exception to raise? I think the documentation I've seen (e.g. Raymond's How To for Descriptors) describes AttributeError as the thing to raise when trying to set read-only properties. Thoughts? Should this be fixed (in 2.4?). -Barry

6 12

shadow password module (spwd) is never built due to error in setup.py
by Irmen de Jong April 15, 2005

April 15, 2005

Hello, A modification was made in setup.py, cvs rel 1.213 (see diff here: http://cvs.sourceforge.net/viewcvs.py/python/python/dist/src/setup.py?r1=1.… ) which appears to be wrong. At least, on my system, the spwd module is never built anymore, because the if statement is never true. Actually, the sysconfig doesn't contain *any* of the HAVE_XXXX vars that occur in pyconfig.h (I checked by printing all vars). I don't really understand the distutils magic that is done in setup.py, but it appears to me that either the if statement is wrong (because the vars never exist) or distutils does something wrong by leaving out all HAVE_XXX vars from pyconfig.h. Please advise? I want my spwd module back ;-) --Irmen de Jong PS I checked that pyconfig.h correctly #defines both HAVE_GETSPNAM and HAVE_GETSPENT to 1 on my system (Mandrake linux 10.1), so the rest of the configure script runs fine (it should, I created the original patches for it... see SF patch # 579435)

3 3

PyCon 2005 keynote on-line
by Guido van Rossum April 15, 2005

April 15, 2005

http://python.org/doc/essays/ppt/ -- scroll to the end. -- --Guido van Rossum (home page: http://www.python.org/~guido/)

1 0

Check out a new way to read threaded conversations.
by Ka-Ping Yee April 15, 2005

April 15, 2005

I hope you will not mind too much if I ask a small favor. Sorry for this off-topic post. I am working on a new design for displaying online conversations. (Some of you saw this at PyCon.) I'm conducting a short survey to gather some opinions on the current design. If you have just a few minutes to spare, would you please visit: http://zesty.ca/threadmap/pydev.cgi You'll see a new way of looking at this discussion list that you may find pretty interesting. I look forward to learning what you think of it. I am very grateful for your time and assistance. (If you reply to this message, please reply to me only -- I don't want to clutter up python-dev with lots of off-topic messages.) -- Ping

2 1

Weekly Python Patch/Bug Summary
by Kurt B. Kaiser April 14, 2005

April 14, 2005

Patch / Bug Summary ___________________ Patches : 314 open ( +6) / 2824 closed ( +5) / 3138 total (+11) Bugs : 898 open (+16) / 4921 closed ( +8) / 5819 total (+24) RFE : 177 open ( +1) / 151 closed ( +0) / 328 total ( +1) New / Reopened Patches ______________________ typos in rpc.py (2005-04-09) CLOSED http://python.org/sf/1179503 opened by engelbert gruber [AST] Fix for core in test_grammar.py (2005-04-08) http://python.org/sf/1179513 opened by logistix no html file for modulefinder (2005-04-10) http://python.org/sf/1180012 opened by George Yoshida fix typos in Library Reference (2005-04-10) http://python.org/sf/1180062 opened by George Yoshida great improvement for locale.py formatting functions (2005-04-10) http://python.org/sf/1180296 opened by Georg Brandl clarify behavior of StringIO objects when preinitialized (2005-04-10) CLOSED http://python.org/sf/1180305 opened by Georg Brandl st_gen and st_birthtime support for FreeBSD (2005-04-11) http://python.org/sf/1180695 opened by Antti Louko binary formats for marshalling floats (2005-04-11) http://python.org/sf/1180995 opened by Michael Hudson make float packing copy bytes when they can (2005-04-12) http://python.org/sf/1181301 opened by Michael Hudson range() in for loops, again (2005-04-12) http://python.org/sf/1181334 opened by Armin Rigo HMAC hexdigest and general review (2005-04-13) http://python.org/sf/1182394 opened by Shane Holloway Patches Closed ______________ Complex commented (2005-04-06) http://python.org/sf/1177597 closed by loewis typos in rpc.py (2005-04-08) http://python.org/sf/1179503 closed by rhettinger clarify behavior of StringIO objects when preinitialized (2005-04-10) http://python.org/sf/1180305 closed by rhettinger Improved output for unittest failUnlessEqual (2003-04-22) http://python.org/sf/725569 closed by purcell [AST] Generator expressions (2005-03-21) http://python.org/sf/1167628 closed by bcannon New / Reopened Bugs ___________________ 256 should read 255 in operator module docs (2005-04-06) CLOSED http://python.org/sf/1178255 opened by Dan Everhart operator.isMappingType and isSequenceType on instances (2005-04-06) CLOSED http://python.org/sf/1178269 opened by Dan Everhart Erroneous line number error in Py2.4.1 (2005-04-07) http://python.org/sf/1178484 opened by Timo Linna configure: refuses setgroups (2005-04-07) http://python.org/sf/1178510 opened by zosh 2.4.1 breaks pyTTS (2005-04-07) http://python.org/sf/1178624 opened by Dieter Deyke Variable.__init__ uses self.set(), blocking specialization (2005-04-07) http://python.org/sf/1178863 opened by Emil Variable.__init__ uses self.set(), blocking specialization (2005-04-07) http://python.org/sf/1178872 opened by Emil IDLE bug - changing shortcuts (2005-04-08) http://python.org/sf/1179168 opened by Przemysław Gocyła can't import thru cygwin symlink (2005-04-08) http://python.org/sf/1179412 opened by steveward Missing def'n of equality for set elements (2005-04-09) CLOSED http://python.org/sf/1179957 opened by Skip Montanaro codecs.readline sometimes removes newline chars (2005-04-02) http://python.org/sf/1175396 reopened by doerwalter locale.format question (2005-04-10) CLOSED http://python.org/sf/1180002 opened by Andrew Ma test_posix fails on cygwin (2005-04-10) http://python.org/sf/1180147 opened by Henrik Wist subprocess.Popen fails with closed stdout (2005-04-10) http://python.org/sf/1180160 opened by neuhauser broken pyc files (2005-04-10) http://python.org/sf/1180193 opened by Armin Rigo Python keeps file references after calling close methode (2005-04-10) http://python.org/sf/1180237 opened by Eelco expanding platform module and making it work as it should (2005-04-10) http://python.org/sf/1180267 opened by Nikos Kouremenos StringIO's docs should mention overwriting of initial value (2005-04-10) CLOSED http://python.org/sf/1180392 opened by Leif K-Brooks BaseHTTPServer uses deprecated mimetools.Message (2005-04-11) http://python.org/sf/1180470 opened by Paul Jimenez lax error-checking in new-in-2.4 marshal stuff (2005-04-11) http://python.org/sf/1180997 opened by Michael Hudson Bad sys.executable value for bdist_wininst install script (2005-04-12) http://python.org/sf/1181619 opened by follower asyncore.loop() documentation (2005-04-13) http://python.org/sf/1181939 opened by Graham re.escape(s) prints wrong for chr(0) (2005-04-13) http://python.org/sf/1182603 opened by Nick Jacobson dir() does not include _ (2005-04-13) http://python.org/sf/1182614 opened by Nick Jacobson ZipFile __del__/close problem with longint/long files (2005-04-14) http://python.org/sf/1182788 opened by Robert Kiendl Bugs Closed ___________ 256 should read 255 in operator module docs (2005-04-06) http://python.org/sf/1178255 closed by rhettinger operator.isMappingType and isSequenceType on instances (2005-04-06) http://python.org/sf/1178269 closed by rhettinger GNU readline 4.2 prompt issue (2002-12-30) http://python.org/sf/660083 closed by mwh non-ascii readline input crashes python (2004-08-14) http://python.org/sf/1009263 closed by mwh readline+no threads (2003-09-24) http://python.org/sf/811844 closed by mwh compiler module didn't get updated for "class foo():pass" (2005-04-03) http://python.org/sf/1176012 closed by bcannon Missing def'n of equality for set elements (2005-04-09) http://python.org/sf/1179957 closed by rhettinger locale.format question (2005-04-10) http://python.org/sf/1180002 closed by andrewma StringIO's docs should mention overwriting of initial value/ (2005-04-10) http://python.org/sf/1180392 closed by rhettinger New / Reopened RFE __________________ making builtin exceptions more informative (2005-04-13) http://python.org/sf/1182143 opened by Sebastien de Menten

1 0

PyCallable_Check redeclaration
by Nicholas Bastin April 14, 2005

April 14, 2005

Why is PyCallable_Check declared in both object.h and abstract.h? It appears that it's been this way for quite some time (exists in both 2.3.4 and 2.4.1). -- Nick

1 0

Unified or context diffs?
by Nick Coghlan April 14, 2005

April 14, 2005

Are context diffs still favoured for patches? The patch submission guidelines [1] still say that, but is it actually true these days? I personally prefer unified diffs, but have been generating context diffs because of what the guidelines say. Brett can probably guess why I'm asking :) Cheers, Nick. [1] http://www.python.org/patches/ -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net

11 11