Mailman 3 September 2008 - Python-Dev

PEP 1, PEP Purpose and Guidelines
by barry＠zope.com 18 May '21

18 May '21

It has been a while since I posted a copy of PEP 1 to the mailing lists and newsgroups. I've recently done some updating of a few sections, so in the interest of gaining wider community participation in the Python development process, I'm posting the latest revision of PEP 1 here. A version of the PEP is always available on-line at http://www.python.org/peps/pep-0001.html Enjoy, -Barry -------------------- snip snip -------------------- PEP: 1 Title: PEP Purpose and Guidelines Version: $Revision: 1.36 $ Last-Modified: $Date: 2002/07/29 18:34:59 $ Author: Barry A. Warsaw, Jeremy Hylton Status: Active Type: Informational Created: 13-Jun-2000 Post-History: 21-Mar-2001, 29-Jul-2002 What is a PEP? PEP stands for Python Enhancement Proposal. A PEP is a design document providing information to the Python community, or describing a new feature for Python. The PEP should provide a concise technical specification of the feature and a rationale for the feature. We intend PEPs to be the primary mechanisms for proposing new features, for collecting community input on an issue, and for documenting the design decisions that have gone into Python. The PEP author is responsible for building consensus within the community and documenting dissenting opinions. Because the PEPs are maintained as plain text files under CVS control, their revision history is the historical record of the feature proposal[1]. Kinds of PEPs There are two kinds of PEPs. A standards track PEP describes a new feature or implementation for Python. An informational PEP describes a Python design issue, or provides general guidelines or information to the Python community, but does not propose a new feature. Informational PEPs do not necessarily represent a Python community consensus or recommendation, so users and implementors are free to ignore informational PEPs or follow their advice. PEP Work Flow The PEP editor, Barry Warsaw <peps(a)python.org>, assigns numbers for each PEP and changes its status. The PEP process begins with a new idea for Python. It is highly recommended that a single PEP contain a single key proposal or new idea. The more focussed the PEP, the more successfully it tends to be. The PEP editor reserves the right to reject PEP proposals if they appear too unfocussed or too broad. If in doubt, split your PEP into several well-focussed ones. Each PEP must have a champion -- someone who writes the PEP using the style and format described below, shepherds the discussions in the appropriate forums, and attempts to build community consensus around the idea. The PEP champion (a.k.a. Author) should first attempt to ascertain whether the idea is PEP-able. Small enhancements or patches often don't need a PEP and can be injected into the Python development work flow with a patch submission to the SourceForge patch manager[2] or feature request tracker[3]. The PEP champion then emails the PEP editor <peps(a)python.org> with a proposed title and a rough, but fleshed out, draft of the PEP. This draft must be written in PEP style as described below. If the PEP editor approves, he will assign the PEP a number, label it as standards track or informational, give it status 'draft', and create and check-in the initial draft of the PEP. The PEP editor will not unreasonably deny a PEP. Reasons for denying PEP status include duplication of effort, being technically unsound, not providing proper motivation or addressing backwards compatibility, or not in keeping with the Python philosophy. The BDFL (Benevolent Dictator for Life, Guido van Rossum) can be consulted during the approval phase, and is the final arbitrator of the draft's PEP-ability. If a pre-PEP is rejected, the author may elect to take the pre-PEP to the comp.lang.python newsgroup (a.k.a. python-list(a)python.org mailing list) to help flesh it out, gain feedback and consensus from the community at large, and improve the PEP for re-submission. The author of the PEP is then responsible for posting the PEP to the community forums, and marshaling community support for it. As updates are necessary, the PEP author can check in new versions if they have CVS commit permissions, or can email new PEP versions to the PEP editor for committing. Standards track PEPs consists of two parts, a design document and a reference implementation. The PEP should be reviewed and accepted before a reference implementation is begun, unless a reference implementation will aid people in studying the PEP. Standards Track PEPs must include an implementation - in the form of code, patch, or URL to same - before it can be considered Final. PEP authors are responsible for collecting community feedback on a PEP before submitting it for review. A PEP that has not been discussed on python-list(a)python.org and/or python-dev(a)python.org will not be accepted. However, wherever possible, long open-ended discussions on public mailing lists should be avoided. Strategies to keep the discussions efficient include, setting up a separate SIG mailing list for the topic, having the PEP author accept private comments in the early design phases, etc. PEP authors should use their discretion here. Once the authors have completed a PEP, they must inform the PEP editor that it is ready for review. PEPs are reviewed by the BDFL and his chosen consultants, who may accept or reject a PEP or send it back to the author(s) for revision. Once a PEP has been accepted, the reference implementation must be completed. When the reference implementation is complete and accepted by the BDFL, the status will be changed to `Final.' A PEP can also be assigned status `Deferred.' The PEP author or editor can assign the PEP this status when no progress is being made on the PEP. Once a PEP is deferred, the PEP editor can re-assign it to draft status. A PEP can also be `Rejected'. Perhaps after all is said and done it was not a good idea. It is still important to have a record of this fact. PEPs can also be replaced by a different PEP, rendering the original obsolete. This is intended for Informational PEPs, where version 2 of an API can replace version 1. PEP work flow is as follows: Draft -> Accepted -> Final -> Replaced ^ +----> Rejected v Deferred Some informational PEPs may also have a status of `Active' if they are never meant to be completed. E.g. PEP 1. What belongs in a successful PEP? Each PEP should have the following parts: 1. Preamble -- RFC822 style headers containing meta-data about the PEP, including the PEP number, a short descriptive title (limited to a maximum of 44 characters), the names, and optionally the contact info for each author, etc. 2. Abstract -- a short (~200 word) description of the technical issue being addressed. 3. Copyright/public domain -- Each PEP must either be explicitly labelled as placed in the public domain (see this PEP as an example) or licensed under the Open Publication License[4]. 4. Specification -- The technical specification should describe the syntax and semantics of any new language feature. The specification should be detailed enough to allow competing, interoperable implementations for any of the current Python platforms (CPython, JPython, Python .NET). 5. Motivation -- The motivation is critical for PEPs that want to change the Python language. It should clearly explain why the existing language specification is inadequate to address the problem that the PEP solves. PEP submissions without sufficient motivation may be rejected outright. 6. Rationale -- The rationale fleshes out the specification by describing what motivated the design and why particular design decisions were made. It should describe alternate designs that were considered and related work, e.g. how the feature is supported in other languages. The rationale should provide evidence of consensus within the community and discuss important objections or concerns raised during discussion. 7. Backwards Compatibility -- All PEPs that introduce backwards incompatibilities must include a section describing these incompatibilities and their severity. The PEP must explain how the author proposes to deal with these incompatibilities. PEP submissions without a sufficient backwards compatibility treatise may be rejected outright. 8. Reference Implementation -- The reference implementation must be completed before any PEP is given status 'Final,' but it need not be completed before the PEP is accepted. It is better to finish the specification and rationale first and reach consensus on it before writing code. The final implementation must include test code and documentation appropriate for either the Python language reference or the standard library reference. PEP Template PEPs are written in plain ASCII text, and should adhere to a rigid style. There is a Python script that parses this style and converts the plain text PEP to HTML for viewing on the web[5]. PEP 9 contains a boilerplate[7] template you can use to get started writing your PEP. Each PEP must begin with an RFC822 style header preamble. The headers must appear in the following order. Headers marked with `*' are optional and are described below. All other headers are required. PEP: <pep number> Title: <pep title> Version: <cvs version string> Last-Modified: <cvs date string> Author: <list of authors' real names and optionally, email addrs> * Discussions-To: <email address> Status: <Draft | Active | Accepted | Deferred | Final | Replaced> Type: <Informational | Standards Track> * Requires: <pep numbers> Created: <date created on, in dd-mmm-yyyy format> * Python-Version: <version number> Post-History: <dates of postings to python-list and python-dev> * Replaces: <pep number> * Replaced-By: <pep number> The Author: header lists the names and optionally, the email addresses of all the authors/owners of the PEP. The format of the author entry should be address(a)dom.ain (Random J. User) if the email address is included, and just Random J. User if the address is not given. If there are multiple authors, each should be on a separate line following RFC 822 continuation line conventions. Note that personal email addresses in PEPs will be obscured as a defense against spam harvesters. Standards track PEPs must have a Python-Version: header which indicates the version of Python that the feature will be released with. Informational PEPs do not need a Python-Version: header. While a PEP is in private discussions (usually during the initial Draft phase), a Discussions-To: header will indicate the mailing list or URL where the PEP is being discussed. No Discussions-To: header is necessary if the PEP is being discussed privately with the author, or on the python-list or python-dev email mailing lists. Note that email addresses in the Discussions-To: header will not be obscured. Created: records the date that the PEP was assigned a number, while Post-History: is used to record the dates of when new versions of the PEP are posted to python-list and/or python-dev. Both headers should be in dd-mmm-yyyy format, e.g. 14-Aug-2001. PEPs may have a Requires: header, indicating the PEP numbers that this PEP depends on. PEPs may also have a Replaced-By: header indicating that a PEP has been rendered obsolete by a later document; the value is the number of the PEP that replaces the current document. The newer PEP must have a Replaces: header containing the number of the PEP that it rendered obsolete. PEP Formatting Requirements PEP headings must begin in column zero and the initial letter of each word must be capitalized as in book titles. Acronyms should be in all capitals. The body of each section must be indented 4 spaces. Code samples inside body sections should be indented a further 4 spaces, and other indentation can be used as required to make the text readable. You must use two blank lines between the last line of a section's body and the next section heading. You must adhere to the Emacs convention of adding two spaces at the end of every sentence. You should fill your paragraphs to column 70, but under no circumstances should your lines extend past column 79. If your code samples spill over column 79, you should rewrite them. Tab characters must never appear in the document at all. A PEP should include the standard Emacs stanza included by example at the bottom of this PEP. A PEP must contain a Copyright section, and it is strongly recommended to put the PEP in the public domain. When referencing an external web page in the body of a PEP, you should include the title of the page in the text, with a footnote reference to the URL. Do not include the URL in the body text of the PEP. E.g. Refer to the Python Language web site [1] for more details. ... [1] http://www.python.org When referring to another PEP, include the PEP number in the body text, such as "PEP 1". The title may optionally appear. Add a footnote reference that includes the PEP's title and author. It may optionally include the explicit URL on a separate line, but only in the References section. Note that the pep2html.py script will calculate URLs automatically, e.g.: ... Refer to PEP 1 [7] for more information about PEP style ... References [7] PEP 1, PEP Purpose and Guidelines, Warsaw, Hylton http://www.python.org/peps/pep-0001.html If you decide to provide an explicit URL for a PEP, please use this as the URL template: http://www.python.org/peps/pep-xxxx.html PEP numbers in URLs must be padded with zeros from the left, so as to be exactly 4 characters wide, however PEP numbers in text are never padded. Reporting PEP Bugs, or Submitting PEP Updates How you report a bug, or submit a PEP update depends on several factors, such as the maturity of the PEP, the preferences of the PEP author, and the nature of your comments. For the early draft stages of the PEP, it's probably best to send your comments and changes directly to the PEP author. For more mature, or finished PEPs you may want to submit corrections to the SourceForge bug manager[6] or better yet, the SourceForge patch manager[2] so that your changes don't get lost. If the PEP author is a SF developer, assign the bug/patch to him, otherwise assign it to the PEP editor. When in doubt about where to send your changes, please check first with the PEP author and/or PEP editor. PEP authors who are also SF committers, can update the PEPs themselves by using "cvs commit" to commit their changes. Remember to also push the formatted PEP text out to the web by doing the following: % python pep2html.py -i NUM where NUM is the number of the PEP you want to push out. See % python pep2html.py --help for details. Transferring PEP Ownership It occasionally becomes necessary to transfer ownership of PEPs to a new champion. In general, we'd like to retain the original author as a co-author of the transferred PEP, but that's really up to the original author. A good reason to transfer ownership is because the original author no longer has the time or interest in updating it or following through with the PEP process, or has fallen off the face of the 'net (i.e. is unreachable or not responding to email). A bad reason to transfer ownership is because you don't agree with the direction of the PEP. We try to build consensus around a PEP, but if that's not possible, you can always submit a competing PEP. If you are interested assuming ownership of a PEP, send a message asking to take over, addressed to both the original author and the PEP editor <peps(a)python.org>. If the original author doesn't respond to email in a timely manner, the PEP editor will make a unilateral decision (it's not like such decisions can be reversed. :). References and Footnotes [1] This historical record is available by the normal CVS commands for retrieving older revisions. For those without direct access to the CVS tree, you can browse the current and past PEP revisions via the SourceForge web site at http://cvs.sourceforge.net/cgi-bin/cvsweb.cgi/python/nondist/peps/?cvsroot=… [2] http://sourceforge.net/tracker/?group_id=5470&atid=305470 [3] http://sourceforge.net/tracker/?atid=355470&group_id=5470&func=browse [4] http://www.opencontent.org/openpub/ [5] The script referred to here is pep2html.py, which lives in the same directory in the CVS tree as the PEPs themselves. Try "pep2html.py --help" for details. The URL for viewing PEPs on the web is http://www.python.org/peps/ [6] http://sourceforge.net/tracker/?group_id=5470&atid=305470 [7] PEP 9, Sample PEP Template http://www.python.org/peps/pep-0009.html Copyright This document has been placed in the public domain. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 End:

8 14

pthreads, fork, import, and execvp
by Rotem Yaari 16 Sep '09

16 Sep '09

Hello everyone! We have been encountering several deadlocks in a threaded Python application which calls subprocess.Popen (i.e. fork()) in some of its threads. This has occurred on Python 2.4.1 on a 2.4.27 Linux kernel. Preliminary analysis of the hang shows that the child process blocks upon entering the execvp function, in which the import_lock is acquired due to the following line: def _ execvpe(file, args, env=None): from errno import ENOENT, ENOTDIR ... It is known that when forking from a pthreaded application, acquisition attempts on locks which were already locked by other threads while fork() was called will deadlock. Due to these oddities we were wondering if it would be better to extract the above import line from the execvpe call, to prevent lock acquisition attempts in such cases. Another workaround could be re-assigning a new lock to import_lock (such a thing is done with the global interpreter lock) at PyOS_AfterFork or pthread_atfork. We'd appreciate any opinions you might have on the subject. Thanks in advance, Yair and Rotem

12 29

Re: subprocess and EINTR errnos
by Peter Astrand 06 Jul '09

06 Jul '09

On Wed, 10 Nov 2004, John P Speno wrote: Hi, sorry for the delayed response. > While using subprocess (aka popen5), I came across one potential gotcha. I've had > exceptions ending like this: > > File "test.py", line 5, in test > cmd = popen5.Popen(args, stdout=PIPE) > File "popen5.py", line 577, in __init__ > data = os.read(errpipe_read, 1048576) # Exceptions limited to 1 MB > OSError: [Errno 4] Interrupted system call > > (on Solaris 9) > > Would it make sense for subprocess to use a more robust read() function > which can handle these cases, i.e. when the parent's read on the pipe > to the child's stderr is interrupted by a system call, and returns EINTR? > I imagine it could catch EINTR and EAGAIN and retry the failed read(). I assume you are using signals in your application? The os.read above is not the only system call that can fail with EINTR. subprocess.py is full of other system calls that can fail, and I suspect that many other Python modules are as well. I've made a patch (attached) to subprocess.py (and test_subprocess.py) that should guard against EINTR, but I haven't committed it yet. It's quite large. Are Python modules supposed to handle EINTR? Why not let the C code handle this? Or, perhaps the signal module should provide a sigaction function, so that users can use SA_RESTART. Index: subprocess.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/subprocess.py,v retrieving revision 1.8 diff -u -r1.8 subprocess.py --- subprocess.py 7 Nov 2004 14:30:34 -0000 1.8 +++ subprocess.py 17 Nov 2004 19:42:30 -0000 @@ -888,6 +888,50 @@ pass + def _read_no_intr(self, fd, buffersize): + """Like os.read, but retries on EINTR""" + while True: + try: + return os.read(fd, buffersize) + except OSError, e: + if e.errno == errno.EINTR: + continue + else: + raise + + + def _read_all(self, fd, buffersize): + """Like os.read, but retries on EINTR, and reads until EOF""" + all = "" + while True: + data = self._read_no_intr(fd, buffersize) + all += data + if data == "": + return all + + + def _write_no_intr(self, fd, s): + """Like os.write, but retries on EINTR""" + while True: + try: + return os.write(fd, s) + except OSError, e: + if e.errno == errno.EINTR: + continue + else: + raise + + def _waitpid_no_intr(self, pid, options): + """Like os.waitpid, but retries on EINTR""" + while True: + try: + return os.waitpid(pid, options) + except OSError, e: + if e.errno == errno.EINTR: + continue + else: + raise + def _execute_child(self, args, executable, preexec_fn, close_fds, cwd, env, universal_newlines, startupinfo, creationflags, shell, @@ -963,7 +1007,7 @@ exc_value, tb) exc_value.child_traceback = ''.join(exc_lines) - os.write(errpipe_write, pickle.dumps(exc_value)) + self._write_no_intr(errpipe_write, pickle.dumps(exc_value)) # This exitcode won't be reported to applications, so it # really doesn't matter what we return. @@ -979,7 +1023,7 @@ os.close(errwrite) # Wait for exec to fail or succeed; possibly raising exception - data = os.read(errpipe_read, 1048576) # Exceptions limited to 1 MB + data = self._read_all(errpipe_read, 1048576) # Exceptions limited to 1 MB os.close(errpipe_read) if data != "": child_exception = pickle.loads(data) @@ -1003,7 +1047,7 @@ attribute.""" if self.returncode == None: try: - pid, sts = os.waitpid(self.pid, os.WNOHANG) + pid, sts = self._waitpid_no_intr(self.pid, os.WNOHANG) if pid == self.pid: self._handle_exitstatus(sts) except os.error: @@ -1015,7 +1059,7 @@ """Wait for child process to terminate. Returns returncode attribute.""" if self.returncode == None: - pid, sts = os.waitpid(self.pid, 0) + pid, sts = self._waitpid_no_intr(self.pid, 0) self._handle_exitstatus(sts) return self.returncode @@ -1049,27 +1093,33 @@ stderr = [] while read_set or write_set: - rlist, wlist, xlist = select.select(read_set, write_set, []) + try: + rlist, wlist, xlist = select.select(read_set, write_set, []) + except select.error, e: + if e[0] == errno.EINTR: + continue + else: + raise if self.stdin in wlist: # When select has indicated that the file is writable, # we can write up to PIPE_BUF bytes without risk # blocking. POSIX defines PIPE_BUF >= 512 - bytes_written = os.write(self.stdin.fileno(), input[:512]) + bytes_written = self._write_no_intr(self.stdin.fileno(), input[:512]) input = input[bytes_written:] if not input: self.stdin.close() write_set.remove(self.stdin) if self.stdout in rlist: - data = os.read(self.stdout.fileno(), 1024) + data = self._read_no_intr(self.stdout.fileno(), 1024) if data == "": self.stdout.close() read_set.remove(self.stdout) stdout.append(data) if self.stderr in rlist: - data = os.read(self.stderr.fileno(), 1024) + data = self._read_no_intr(self.stderr.fileno(), 1024) if data == "": self.stderr.close() read_set.remove(self.stderr) Index: test/test_subprocess.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_subprocess.py,v retrieving revision 1.14 diff -u -r1.14 test_subprocess.py --- test/test_subprocess.py 12 Nov 2004 15:51:48 -0000 1.14 +++ test/test_subprocess.py 17 Nov 2004 19:42:30 -0000 @@ -7,6 +7,7 @@ import tempfile import time import re +import errno mswindows = (sys.platform == "win32") @@ -35,6 +36,16 @@ fname = tempfile.mktemp() return os.open(fname, os.O_RDWR|os.O_CREAT), fname + def read_no_intr(self, obj): + while True: + try: + return obj.read() + except IOError, e: + if e.errno == errno.EINTR: + continue + else: + raise + # # Generic tests # @@ -123,7 +134,7 @@ p = subprocess.Popen([sys.executable, "-c", 'import sys; sys.stdout.write("orange")'], stdout=subprocess.PIPE) - self.assertEqual(p.stdout.read(), "orange") + self.assertEqual(self.read_no_intr(p.stdout), "orange") def test_stdout_filedes(self): # stdout is set to open file descriptor @@ -151,7 +162,7 @@ p = subprocess.Popen([sys.executable, "-c", 'import sys; sys.stderr.write("strawberry")'], stderr=subprocess.PIPE) - self.assertEqual(remove_stderr_debug_decorations(p.stderr.read()), + self.assertEqual(remove_stderr_debug_decorations(self.read_no_intr(p.stderr)), "strawberry") def test_stderr_filedes(self): @@ -186,7 +197,7 @@ 'sys.stderr.write("orange")'], stdout=subprocess.PIPE, stderr=subprocess.STDOUT) - output = p.stdout.read() + output = self.read_no_intr(p.stdout) stripped = remove_stderr_debug_decorations(output) self.assertEqual(stripped, "appleorange") @@ -220,7 +231,7 @@ stdout=subprocess.PIPE, cwd=tmpdir) normcase = os.path.normcase - self.assertEqual(normcase(p.stdout.read()), normcase(tmpdir)) + self.assertEqual(normcase(self.read_no_intr(p.stdout)), normcase(tmpdir)) def test_env(self): newenv = os.environ.copy() @@ -230,7 +241,7 @@ 'sys.stdout.write(os.getenv("FRUIT"))'], stdout=subprocess.PIPE, env=newenv) - self.assertEqual(p.stdout.read(), "orange") + self.assertEqual(self.read_no_intr(p.stdout), "orange") def test_communicate(self): p = subprocess.Popen([sys.executable, "-c", @@ -305,7 +316,8 @@ 'sys.stdout.write("\\nline6");'], stdout=subprocess.PIPE, universal_newlines=1) - stdout = p.stdout.read() + + stdout = self.read_no_intr(p.stdout) if hasattr(open, 'newlines'): # Interpreter with universal newline support self.assertEqual(stdout, @@ -343,7 +355,7 @@ def test_no_leaking(self): # Make sure we leak no resources - max_handles = 1026 # too much for most UNIX systems + max_handles = 10 # too much for most UNIX systems if mswindows: max_handles = 65 # a full test is too slow on Windows for i in range(max_handles): @@ -424,7 +436,7 @@ 'sys.stdout.write(os.getenv("FRUIT"))'], stdout=subprocess.PIPE, preexec_fn=lambda: os.putenv("FRUIT", "apple")) - self.assertEqual(p.stdout.read(), "apple") + self.assertEqual(self.read_no_intr(p.stdout), "apple") def test_args_string(self): # args is a string @@ -457,7 +469,7 @@ p = subprocess.Popen(["echo $FRUIT"], shell=1, stdout=subprocess.PIPE, env=newenv) - self.assertEqual(p.stdout.read().strip(), "apple") + self.assertEqual(self.read_no_intr(p.stdout).strip(), "apple") def test_shell_string(self): # Run command through the shell (string) @@ -466,7 +478,7 @@ p = subprocess.Popen("echo $FRUIT", shell=1, stdout=subprocess.PIPE, env=newenv) - self.assertEqual(p.stdout.read().strip(), "apple") + self.assertEqual(self.read_no_intr(p.stdout).strip(), "apple") def test_call_string(self): # call() function with string argument on UNIX @@ -525,7 +537,7 @@ p = subprocess.Popen(["set"], shell=1, stdout=subprocess.PIPE, env=newenv) - self.assertNotEqual(p.stdout.read().find("physalis"), -1) + self.assertNotEqual(self.read_no_intr(p.stdout).find("physalis"), -1) def test_shell_string(self): # Run command through the shell (string) @@ -534,7 +546,7 @@ p = subprocess.Popen("set", shell=1, stdout=subprocess.PIPE, env=newenv) - self.assertNotEqual(p.stdout.read().find("physalis"), -1) + self.assertNotEqual(self.read_no_intr(p.stdout).find("physalis"), -1) def test_call_string(self): # call() function with string argument on Windows /Peter Åstrand <astrand(a)lysator.liu.se>

5 5

deprecating BaseException.message
by Brett Cannon 07 Apr '09

07 Apr '09

During the PyCon sprint I tried to make BaseException accept only a single argument and bind it to BaseException.message . I was successful (see the p3yk_no_args_on_exc branch), but it was very painful to pull off as anyone who sat around me the last three days of the sprint will tell you as they had to listen to me curse incessantly. Because of the pain that I went through in the transition and thus the lessons learned, Guido and I discussed it and we think it would be best to give up on forcing BaseException to accept only a single argument. I think it is still doable, but requires a multi-release transition period and not the one that 2.6 -> 3.0 is offering. And so Guido and I plan on deprecating BaseException.message as its entire point in existence was to help transition to what we are not going to have happen. =) Now that means BaseException.message might hold the record for shortest lived feature as it was only introduced in 2.5 and is now to be deprecated in 2.6 and removed in 2.7/3.0. =) Below is PEP 352, revised to reflect the removal of BaseException.messageand for letting the 'args' attribute stay (along with suggesting one should only pass a single argument to BaseException). Basically the interface for exceptions doesn't really change in 3.0 except for the removal of __getitem__. -------------------------------------------------------------------------- PEP: 352 Title: Required Superclass for Exceptions Version: $Revision: 53592 $ Last-Modified: $Date: 2007-01-28 21:54:11 -0800 (Sun, 28 Jan 2007) $ Author: Brett Cannon <brett(a)python.org> Guido van Rossum <guido(a)python.org> Status: Final Type: Standards Track Content-Type: text/x-rst Created: 27-Oct-2005 Post-History: Abstract ======== In Python 2.4 and before, any (classic) class can be raised as an exception. The plan for 2.5 was to allow new-style classes, but this makes the problem worse -- it would mean *any* class (or instance) can be raised! This is a problem as it prevents any guarantees from being made about the interface of exceptions. This PEP proposes introducing a new superclass that all raised objects must inherit from. Imposing the restriction will allow a standard interface for exceptions to exist that can be relied upon. It also leads to a known hierarchy for all exceptions to adhere to. One might counter that requiring a specific base class for a particular interface is unPythonic. However, in the specific case of exceptions there's a good reason (which has generally been agreed to on python-dev): requiring hierarchy helps code that wants to *catch* exceptions by making it possible to catch *all* exceptions explicitly by writing ``except BaseException:`` instead of ``except *:``. [#hierarchy-good]_ Introducing a new superclass for exceptions also gives us the chance to rearrange the exception hierarchy slightly for the better. As it currently stands, all exceptions in the built-in namespace inherit from Exception. This is a problem since this includes two exceptions (KeyboardInterrupt and SystemExit) that often need to be excepted from the application's exception handling: the default behavior of shutting the interpreter down without a traceback is usually more desirable than whatever the application might do (with the possible exception of applications that emulate Python's interactive command loop with ``>>>`` prompt). Changing it so that these two exceptions inherit from the common superclass instead of Exception will make it easy for people to write ``except`` clauses that are not overreaching and not catch exceptions that should propagate up. This PEP is based on previous work done for PEP 348 [#pep348]_. Requiring a Common Superclass ============================= This PEP proposes introducing a new exception named BaseException that is a new-style class and has a single attribute, ``args``. Below is the code as the exception will work in Python 3.0 (how it will work in Python 2.x is covered in the `Transition Plan`_ section):: class BaseException(object): """Superclass representing the base of the exception hierarchy. Provides a 'message' attribute that contains either the single argument to the constructor or the empty string. This attribute is used in the string representation for the exception. This is so that it provides the extra details in the traceback. """ def __init__(self, *args): """Set the 'message' attribute'""" self.args = args def __str__(self): """Return the str of 'message'""" if len(self.args) == 1: return str(self.args[0]) else: return str(self.args) def __repr__(self): return "%s(*%s)" % (self.__class__.__name__, repr(self.args)) No restriction is placed upon what may be passed in for ``args`` for backwards-compatibility reasons. In practice, though, only a single string argument should be used. This keeps the string representation of the exception to be a useful message about the exception that is human-readable; this is why the ``__str__`` method special-cases on length-1 ``args`` value. Including programmatic information (e.g., an error code number) should be stored as a separate attribute in a subclass. The ``raise`` statement will be changed to require that any object passed to it must inherit from BaseException. This will make sure that all exceptions fall within a single hierarchy that is anchored at BaseException [#hierarchy-good]_. This also guarantees a basic interface that is inherited from BaseException. The change to ``raise`` will be enforced starting in Python 3.0 (see the `Transition Plan`_ below). With BaseException being the root of the exception hierarchy, Exception will now inherit from it. Exception Hierarchy Changes =========================== With the exception hierarchy now even more important since it has a basic root, a change to the existing hierarchy is called for. As it stands now, if one wants to catch all exceptions that signal an error *and* do not mean the interpreter should be allowed to exit, you must specify all but two exceptions specifically in an ``except`` clause or catch the two exceptions separately and then re-raise them and have all other exceptions fall through to a bare ``except`` clause:: except (KeyboardInterrupt, SystemExit): raise except: ... That is needlessly explicit. This PEP proposes moving KeyboardInterrupt and SystemExit to inherit directly from BaseException. :: - BaseException |- KeyboardInterrupt |- SystemExit |- Exception |- (all other current built-in exceptions) Doing this makes catching Exception more reasonable. It would catch only exceptions that signify errors. Exceptions that signal that the interpreter should exit will not be caught and thus be allowed to propagate up and allow the interpreter to terminate. KeyboardInterrupt has been moved since users typically expect an application to exit when the press the interrupt key (usually Ctrl-C). If people have overly broad ``except`` clauses the expected behaviour does not occur. SystemExit has been moved for similar reasons. Since the exception is raised when ``sys.exit()`` is called the interpreter should normally be allowed to terminate. Unfortunately overly broad ``except`` clauses can prevent the explicitly requested exit from occurring. To make sure that people catch Exception most of the time, various parts of the documentation and tutorials will need to be updated to strongly suggest that Exception be what programmers want to use. Bare ``except`` clauses or catching BaseException directly should be discouraged based on the fact that KeyboardInterrupt and SystemExit almost always should be allowed to propagate up. Transition Plan =============== Since semantic changes to Python are being proposed, a transition plan is needed. The goal is to end up with the new semantics being used in Python 3.0 while providing a smooth transition for 2.x code. All deprecations mentioned in the plan will lead to the removal of the semantics starting in the version following the initial deprecation. Here is BaseException as implemented in the 2.x series:: class BaseException(object): """Superclass representing the base of the exception hierarchy. The __getitem__ method is provided for backwards-compatibility and will be deprecated at some point. """ def __init__(self, *args): """Set the 'args' attribute.""" self.args = args def __str__(self): """Return the str of args[0] or args, depending on length.""" return str(self.args[0] if len(self.args) <= 1 else self.args) def __repr__(self): func_args = repr(self.args) if self.args else "()" return self.__class__.__name__ + func_args def __getitem__(self, index): """Index into arguments passed in during instantiation. Provided for backwards-compatibility and will be deprecated. """ return self.args[index] Deprecation of features in Python 2.9 is optional. This is because it is not known at this time if Python 2.9 (which is slated to be the last version in the 2.x series) will actively deprecate features that will not be in 3.0 . It is conceivable that no deprecation warnings will be used in 2.9 since there could be such a difference between 2.9 and 3.0 that it would make 2.9 too "noisy" in terms of warnings. Thus the proposed deprecation warnings for Python 2.9 will be revisited when development of that version begins to determine if they are still desired. * Python 2.5 [done] - all standard exceptions become new-style classes - introduce BaseException - Exception, KeyboardInterrupt, and SystemExit inherit from BaseException - deprecate raising string exceptions * Python 2.6 - deprecate catching string exceptions - deprecate ``message`` attribute (see `Retracted Ideas`_) * Python 2.7 - deprecate raising exceptions that do not inherit from BaseException - remove ``message`` attribute * Python 2.8 - deprecate catching exceptions that do not inherit from BaseException * Python 2.9 - deprecate ``__getitem__`` (optional) * Python 3.0 [done] - drop everything that was deprecated above: + string exceptions (both raising and catching) + all exceptions must inherit from BaseException + drop ``__getitem__`` Retracted Ideas =============== A previous version of this PEP that was implemented in Python 2.5 included a 'message' attribute on BaseException. Its purpose was to begin a transition to BaseException accepting only a single argument. This was to tighten the interface and to force people to use attributes in subclasses to carry arbitrary information with an exception instead of cramming it all into ``args``. Unfortunately, while implementing the removal of the ``args`` attribute in Python 3.0 at the PyCon 2007 sprint [#pycon2007-sprint-email]_, it was discovered that the transition was very painful, especially for C extension modules. It was decided that it would be better to deprecate the ``message`` attribute in Python 2.6 (and remove in Python 2.7 and Python 3.0) and consider a more long-term transition strategy in Python 3.0 to remove multiple-argument support in BaseException in preference of accepting only a single argument. Thus the introduction of ``message`` and the original deprecation of ``args`` has been retracted. References ========== .. [#pep348] PEP 348 (Exception Reorganization for Python 3.0) http://www.python.org/peps/pep-0348.html .. [#hierarchy-good] python-dev Summary for 2004-08-01 through 2004-08-15 http://www.python.org/dev/summary/2004-08-01_2004-08-15.html#an-exception-i… .. [#SF_1104669] SF patch #1104669 (new-style exceptions) http://www.python.org/sf/1104669 .. [#pycon2007-sprint-email] python-3000 email ("How far to go with cleaning up exceptions") http://mail.python.org/pipermail/python-3000/2007-March/005911.html Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 End:

3 3

Re: [Python-Dev] [Distutils] unicode bug in distutils
by Phillip J. Eby 04 Apr '09

04 Apr '09

At 02:47 PM 2/24/2007 -0600, Tarek Ziadé wrote: >I have created a setup.py file for distirbution and I bumped into >a small bug when i tried to set my name in the contact field (Tarek Ziadé) > >Using string (utf8 file): > >setup( > maintainer="Tarek Ziadé" >) > >leads to: > > File ".../lib/python2.5/distutils/command/register.py", line 162, in > send_metadata > auth) > File ".../lib/python2.5/distutils/command/register.py", line 257, in > post_to_server > value = unicode(value).encode("utf-8") >UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 10: >ordinal not in range(128) > > >Using unicode: > >setup( > maintainer=u"Tarek Ziadé" >) > >leads to: > > File ".../lib/python2.5/distutils/dist.py", line 1094, in write_pkg_file > file.write('Author: %s\n' % self.get_contact() ) >UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in >position 18: ordinal not in range(128) > >I would propose a patch for this problem but i don't know what would be >the best input (i guess unicode > for names) At 05:45 PM 2/24/2007 -0500, Tres Seaver wrote: >Don't you still need to tell Python about the encoding of your string >literals [1] [2] ? E.g.:: That's not the problem, it's that the code that writes the PKG-INFO file doesn't handle Unicode. See distutils.dist.DistributionMetadata.write_pkg_info(). It needs to use a file with encoding support, if it's doing unicode However, there's currently no standard, as far as I know, for what encoding the PKG-INFO file should use. Meanwhile, the 'register' command accepts Unicode, but is broken in handling it. Essentially, the problem is that Python 2.5 broke this by adding a unicode *requirement* to the "register" command. Previously, register simply sent whatever you gave it, and the PKG-INFO writing code still does. Unfortunately, this means that there is no longer any one value that you can use for your name that will be accepted by both "register" and anything that writes a PKG-INFO file. Both register and write_pkg_info() are arguably broken here, and should be able to work with either strings or unicode, and degrade gracefully in the event of non-ASCII characters in a string. (Because even though "register" is only run by the package's author, users may run other commands that require a PKG-INFO, so a package prepared using Python <2.5 must still be usable with Python 2.5 distutils, and Python <2.5 allows 8-bit maintainer names.) Unfortunately, this isn't fixable until there's a new 2.5.x release. For previous Python versions, both register and write_pkg_info() accepted 8-bit strings and passed them on as-is, so the only workaround for this issue at the moment is to revert to Python 2.4 or less.

4 5

Python + Java Integration
by Chas Emerick 05 Dec '08

05 Dec '08

This may seem like it's coming out of left field for a minute, but bear with me. There is no doubt that Ruby's success is a concern for anyone who sees it as diminishing Python's status. One of the reasons for Ruby's success is certainly the notion (originally advocated by Bruce Tate, if I'm not mistaken) that it is the "next Java" -- the language and environment that mainstream Java developers are, or will, look to as a natural next step. One thing that would help Python in this "debate" (or, perhaps simply put it in the running, at least as a "next Java" candidate) would be if Python had an easier migration path for Java developers that currently rely upon various third-party libraries. The wealth of third-party libraries available for Java has always been one of its great strengths. Ergo, if Python had an easy-to-use, recommended way to use those libraries within the Python environment, that would be a significant advantage to present to Java developers and those who would choose Ruby over Java. Platform compatibility is always a huge motivator for those looking to migrate or upgrade. In that vein, I would point to JPype (http://jpype.sourceforge.net). JPype is a module that gives "python programs full access to java class libraries". My suggestion would be to either: (a) include JPype in the standard library, or barring that, (b) make a very strong push to support JPype (a) might be difficult or cumbersome technically, as JPype does need to build against Java headers, which may or may not be possible given the way that Python is distributed, etc. However, (b) is very feasible. I can't really say what "supporting JPype" means exactly -- maybe GvR and/or other heavyweights in the Python community make public statements regarding its existence and functionality, maybe JPype gets a strong mention or placement on python.org....all those details are obviously not up to me, and I don't know the workings of the "official" Python organizations enough to make serious suggestions. Regardless of the form of support, I think raising people's awareness of JPype and what it adds to the Python environment would be a Good Thing (tm). For our part, we've used JPype to make PDFTextStream (our previously Java-only PDF text extraction library) available and supported for Python. You can read some about it here: http://snowtide.com/PDFTextStream.Python And I've blogged about how PDFTextStream.Python came about, and how we worked with Steve Ménard, the maintainer of JPype, to make it all happen (watch out for this URL wrapping): http://blog.snowtide.com/2006/08/21/working-together-pythonjava-open- sourcecommercial Cheers, Chas Emerick Founder, Snowtide Informatics Systems Enterprise-class PDF content extraction cemerick(a)snowtide.com http://snowtide.com | +1 413.519.6365

3 2

I would like an Python account
by Victor Stinner 06 Oct '08

06 Oct '08

Hi, Would it possible to get more permissions on Python bugtracker, especially to add keywords, close a duplicate bug, etc.? I also would like an SVN account for Python trunk and Python 3000 branch. I know that both branches are near "frozen" and commit are only allowed after patch review and/or discussion (on the mailing list or the bug tracker. So I will after the final versions (2.6 and 3.0) to commit. Or would it be possible to have a "mentor"? Someone to ask questions about Python SVN/bug tracker, or someone who reviews my patchs/commits. -- Victor Stinner aka haypo http://www.haypocalc.com/blog/

3 4

Re: [Python-Dev] [Python-3000] New proposition for Python3 bytes filename issue
by Victor Stinner 01 Oct '08

01 Oct '08

Le Monday 29 September 2008 18:45:28 Georg Brandl, vous avez écrit : > If I had to choose, I'd still argue for the modified UTF-8 as filesystem > encoding (if it were UTF-8 otherwise), despite possible surprises when a > such-encoded filename escapes from Python. If I understand correctly this solution. The idea is to change the default file system encoding, right? Eg. if your filesystem is UTF-8, use ISO-8859-1 to make sure that UTF-8 conversion will never fail. Let's try with an ugly directory on my UTF-8 file system: $ find . ./têste ./ô ./a?b ./dossié ./dossié/abc ./dir?name ./dir?name/xyz Python3 using encoding=ISO-8859-1: >>> import os; os.listdir(b'.') [b't\xc3\xaaste', b'\xc3\xb4', b'a\xffb', b'dossi\xc3\xa9', b'dir\xffname'] >>> files=os.listdir('.'); files ['tÃªste', 'Ã´', 'aÿb', 'dossiÃ©', 'dirÿname'] >>> open(files[0]).close() >>> os.listdir(files[-1]) ['xyz'] Ok, I have unicode filenames and I'm able to open a file and list a directory. The problem is now to display correctly the filenames. For me "unicode" sounds like "text (characters) encoded in the correct charset". In this case, unicode is just a storage for *bytes* in a custom charset. How can we mix <custom unicode (bytes encoded in ISO-8859-1)> with <real unicode>? Eg. os.path.join('dossiÃ©', "fichié") : first argument is encoded in ISO-8859-1 whereas the second argument is encoding in Unicode. It's something like that: str(b'dossi\xc3\xa9', 'ISO-8859-1') + '/' + 'fichi\xe9' Whereas the correct (unicode) result should be: 'dossié/fichié' as bytes in ISO-8859-1: b'dossi\xc3\xa9/fichi\xc3\xa9' as bytes in UTF-8: b'dossi\xe9/fichi\xe9' Change the default file system encoding to store bytes in Unicode is like introducing a new Python type: <fake Unicode for filename hacks>. -- Victor Stinner aka haypo http://www.haypocalc.com/blog/

11 26

Python security team
by Victor Stinner 01 Oct '08

01 Oct '08

Hi, I would like to know if a Python security team does exist. I sent an email about an imageop issue, and I didn't get any answer. Later I learned that a security ticket was created, I don't have access to it. First, I would like to access to these informations. Not only this issue, but all security related issues. I have some knowledges about security and I can help to resolve issues and/or estimate the criticity of an issue. Second, I would like to help to fix all Python security issues. It looks like Python community isn't very reactive (proactive?) about security. Eg. a DoS was reported in smtpd server (integrated to Python)... 15 months ago. A patch is available but it's not applied in Python trunk. Third, I'm also looking for a document explaining "how Python is secure" (!). If an user can run arbitrary Python code, we know that it can do anything (read/remove any file, create/kill any process, read/write anywhere in memory, etc.). Brett wrote a paper about CPython sandboxing. PyPy is also working on sandboxing using two interpreters: one has high priviledge and execute instructions from the second interpreter (after checking the permissions and arguments). So is there somewhere a document to explain to current status of Python security? -- Victor Stinner aka haypo http://www.haypocalc.com/blog/

10 14

Re: [Python-Dev] Patch for an initial support of bytes filename in Python3
by glyph＠divmod.com 01 Oct '08

01 Oct '08

On 12:47 am, victor.stinner(a)haypocalc.com wrote: This is the most sane contribution I've seen so far :). >See attached patch: python3_bytes_filename.patch > >Using the patch, you will get: >- open() support bytes >- listdir(unicode) -> only unicode, *skip* invalid filenames > (as asked by Guido) Forgive me for being a bit dense, but I couldn't find this hunk in the patch. Do I understand properly that (listdir(bytes) -> bytes)? If so, this seems basically sane to me, since it provides text behavior where possible and allows more sophisticated filesystem wrappers (i.e. Twisted's FilePath, Will McGugan's "FS") to do more tricky things, separating filenames for display to the user and filenames for exchange with the FS. >- remove os.getcwdu() >- create os.getcwdb() -> bytes >- glob.glob() support bytes >- fnmatch.filter() support bytes >- posixpath.join() and posixpath.split() support bytes It sounds like maybe there should be some 2to3 fixers in here somewhere, too? Not necessarily as part of this patch, but somewhere related? I don't know what they would do, but it does seem quite likely that code which was previously correct under 2.6 (using bytes) would suddenly be mixing bytes and unicode with these APIs.

5 14

Re: [Python-Dev] [Python-3000] New proposition for Python3 bytes filename issue
by "Martin v. Löwis" 01 Oct '08

01 Oct '08

>> On Windows, we might reject bytes filenames for all file operations: open(), >> unlink(), os.path.join(), etc. (raise a TypeError or UnicodeError) > > Since I've seen no objections to this yet: please no. If we offer a > "lower-level" bytes filename API, it should work for all platforms. Unfortunately, it can't. You cannot represent all possible file names in a byte string in Windows (just as you can't do so in a Unicode string on Unix). So using byte strings on Windows would work for some files, but fail for others. In particular, listdir might give you a list of file names which you then can't open/stat/recurse into. (of course, you could use UTF-8 as the file system encoding on Windows, but then you will have to rewrite a lot of C code first) Regards, Martin

6 8

Re: [Python-Dev] [Python-3000] New proposition for Python3 bytes filename issue
by Guido van Rossum 30 Sep '08

30 Sep '08

On Tue, Sep 30, 2008 at 12:42 PM, Terry Reedy <tjreedy(a)udel.edu> wrote: > Guido van Rossum wrote: >> >> On Tue, Sep 30, 2008 at 11:13 AM, Georg Brandl <g.brandl(a)gmx.net> wrote: >>> >>> Victor Stinner schrieb: >>>> >>>> On Windows, we might reject bytes filenames for all file operations: >>>> open(), >>>> unlink(), os.path.join(), etc. (raise a TypeError or UnicodeError) >>> >>> Since I've seen no objections to this yet: please no. If we offer a >>> "lower-level" bytes filename API, it should work for all platforms. >> >> I'm not sure either way. I've heard it claim that Windows filesystem >> APIs use Unicode natively. Does Python 3.0 on Windows currently >> support filenames expressed as bytes? Are they encoded first before >> passing to the Unicode APIs? Using what encoding? > > In 3.0rc1, the listdir doc needs updating: > "os.listdir(path) > Return a list containing the names of the entries in the directory. The list > is in arbitrary order. It does not include the special entries '.' and '..' > even if they are present in the directory. Availability: Unix, Windows. > > On Windows NT/2k/XP and Unix, if path is a Unicode object, the result will > be a list of Unicode objects." > > s/Unicode/bytes/ at least for Windows. > >>>> os.listdir(b'.') > [b'countries.txt', b'multeetest.py', b't1.py', b't1.pyc', b't2.py', b'tem', > b'temp.py', b'temp.pyc', b'temp2.py', b'temp3.py', b'temp4.py', b'test.py', > b'z', b'z.txt'] > > The bytes names do not work however: > >>>> t=open(b'tem') > Traceback (most recent call last): > File "<pyshell#23>", line 1, in <module> > t=open(b'tem') > File "C:\Programs\Python30\lib\io.py", line 284, in __new__ > return open(*args, **kwargs) > File "C:\Programs\Python30\lib\io.py", line 184, in open > raise TypeError("invalid file: %r" % file) > TypeError: invalid file: b'tem' > > Is this what you were asking? No, that's because bytes is missing from the explicit list of allowable types in io.open. Victor has a one-line trivial patch for this. Could you try this though? >>> import _fileio >>> _fileio._FileIO(b'tem') -- --Guido van Rossum (home page: http://www.python.org/~guido/)

2 1

Filename as byte string in python 2.6 or 3.0?
by Victor Stinner 30 Sep '08

30 Sep '08

Hi, I read that Python 2.6 is planned to Wednesday. One bug is is still open and important for me: Python 2.6/3.0 are unable to use filename as byte strings. http://bugs.python.org/issue3187 The problem =========== On Windows, all filenames are unicode strings (I guess UTF-16-LE), but on UNIX for historical reasons, filenames are byte strings. On Linux, you can expect UTF-8 valid filenames but sometimes (eg. copy from a FAT32 USB key to an ext3 filesystem) you get invalid filename (byte string in a different charset than your default filesystem encoding (utf8)). Python functions using filenames ================================ In Python, you have (incomplete list): - filename producer: os.listdir(), os.walk(), glob.glob() - filename manipulation: os.path.*() - access file: open(), os.unlink(), shutil.rmtree() If you give unicode to producer, they return unicode _or_ byte strings (type may change for each filename :-/). Guido proposed to break this behaviour: raise an exception if unicode conversion fails. We may consider an option like "skip invalid". If you give bytes to producer, they only return byte strings. Great. Filename manipulation: in python 2.6/3.0, os.path.*() is not compatible with the type "bytes". So you can not use os.path.join(<your unicode path>, <bytes filename>) *nor* os.path.join(<your bytes path>, <bytes filename>) because os.path.join() (eg. with the posix version) uses path.endswith('/'). Access file: open() rejects the type bytes (it's just a test, open() supports bytes if you remove the test). As I remember, unlink() is compatible with bytes. But rmtree() fails because it uses os.path.join() (even if you give bytes directory, join() fails). Solutions ========= - producer: unicode => *only* unicode // bytes => bytes - manipulation: support both unicode and bytes but avoid (when it's possible) to mix bytes and characters - open(): allow bytes I implemented these solutions as a patch set attached to the issue #3187: * posix_path_bytes.patch: fix posixpath.join() to support bytes * io_byte_filename.patch: open() allows bytes filename * fnmatch_bytes.patch: patch fnmatch.filter() to accept bytes filenames * glob1_bytes.patch: fix glob.glob() to accept invalid directory name Mmmh, there is no patch for stop os.listdir() on invalid filename. Priority ======== I think that the problem is important because it's a regression from 2.5 to 2.6/3.0. Python 2.5 uses bytes filename, so it was possible to open/unlink "invalid" unicode strings (since it's not unicode but bytes). Well, if it's too late for the final versions, this problem should be at least fixed quickly. Test the problem ================ Example to create invalid filenames on Linux: $ mkdir /tmp/test $ cd /tmp/test $ touch $(echo -e "a\xffb") $ mkdir $(echo -e "dir\xffname") $ touch $(echo -e "dir\xffname/file") $ find . ./a?b ./dir?name ./dir?name/file Python 2.5: >>> import os >>> os.listdir('.') ['a\xffb', 'dir\xffname'] >>> open(os.listdir('.')[0]).close() # open file: ok >>> os.unlink(os.listdir('.')[0]) # remove file: ok >>> os.listdir('.') ['dir\xffname'] >>> shutil.rmtree(os.listdir('.')[0]) # remove dir: ok Wrong solutions =============== New type -------- I proposed an ugly type "InvalidFilename" mixing bytes and characters. As everybody using unicode knows, it's a bad idea :-) (and it introduces a new type). Convert bytes to unicode (replace) ---------------------------------- unicode_filename = unicode(bytes_filename, charset, "replace") Ok, you will get valid unicode strings which can be used in os.path.join() & friends, but open() or unlink() will fails because this filename doesn't exist! -- Victor Stinner aka haypo http://www.haypocalc.com/blog/

17 38

Re: [Python-Dev] [Python-3000] New proposition for Python3 bytes filename issue
by "Martin v. Löwis" 30 Sep '08

30 Sep '08

> Oh, ok. I had assumed Windows just uses a fixed encoding without the problem > of misencoded filenames. It's the other way 'round: On Windows, Unicode file names are the natural choice, and byte strings have limitations. In a sense, Windows got it right - but then, they started later. Unix missed the opportunity of declaring that all file APIs are UTF-8 (except for Plan-9 and OS X, neither being "true" Unix). Regards, Martin

2 3

Re: [Python-Dev] [Python-3000] New proposition for Python3 bytes filename issue
by Guido van Rossum 30 Sep '08

30 Sep '08

On Mon, Sep 29, 2008 at 11:22 PM, Georg Brandl <g.brandl(a)gmx.net> wrote: > No, that was not what I meant (although it is another possibility). As I wrote, > Martin's proposal that I support here is using the modified UTF-8 codec that > successfully roundtrips otherwise invalid UTF-8 data. I thought that the "successful rountripping" pretty much stopped as soon as the unicode data is exported to somewhere else -- doesn't it contain invalid surrogate sequences? In general, I'm very reluctant to use utf-8b given that it doesn't seem to be well documented as a standard anywhere. Providing some minimal APIs that can process raw-bytes filenames still makes more sense -- it is mostly analogous of our treatment of text files, where the underlying binary data is also accessible. > You seem to forget that (disregarding OSX here, since it already enforces > UTF-8) the majority of file names on Posix systems will be encoded correctly. Apparently under certain circumstances (external FS mounted) OSX can also have non-UTF-8 filenames. [...] > With the filenames decoded by UTF-8, your files named têste, ô, dossié will > be displayed and handled correctly. The others are *invalid* in the filesystem > encoding UTF-8 and therefore would be represented by something like > > u'dir\uXXffname' where XX is some private use Unicode namespace. It won't look > pretty when printed, but then, what do other applications do? They e.g. display > a question mark as you show above, which is not better in terms of readability. > > But it will work when given to a filename-handling function. Valid filenames > can be compared to Unicode strings. > > A real-world example: OpenOffice can't open files with invalid bytes in their > name. They are displayed in the "Open file" dialog, but trying to open fails. > This regularly drives me crazy. Let's not make Python not work this way too, > or, even worse, not even display those filenames. How can it *regularly* drive you crazy when "the majority of fie names [...] encoded correctly" (as you assert above)? -- --Guido van Rossum (home page: http://www.python.org/~guido/)

4 8

Re: [Python-Dev] [Python-3000] New proposition for Python3 bytes filename issue
by Marcin 'Qrczak' Kowalczyk 30 Sep '08

30 Sep '08

2008/9/30 Glenn Linderman <v+python(a)g.nevcal.com>: > So the problem is that a Unicode file system interface can't deal with > non-UTF-8 byte streams as file names. > > So it seems there are four suggested approaches, all of which have aspects > that are inconvenient. Let's not forget what happens when a non-UTF-8 file name is read from a file or written to a file, under the assumption that the filename is written to the file directly (which probably breaks for filenames containing newlines or such). > 4) Use of bytes APIs on FS interfaces. This seems to be the "solution" > adopted by Posix that creates the "problem" encountered by Unicode-native > applications. It is cumbersome to deal with within applications that > attempt to display the names. What do Posix-style "open file" dialog boxes > do in this case? http://library.gnome.org/devel/glib/stable/glib-Character-Set-Conversion.ht… I used to observe three different ways to display such filenames within gedit (including %xx and \xx escapes), but now it is consistent, probably because it switched to using the above function everywhere: $ touch $'abc\xffz' $ gedit The Open dialog shows: abc�z (invalid encoding) When the file is open, the window title and the tab title show: abc�z and the same is in recent file list. It has a bug: it appends " (invalid encoding)" even if the filename contains a correctly encoded U+FFFD character. Nautilus has the same behavior and the same bug because this is a design bug of that function which does not allow to tell whether the conversion was successful. A filename containing a newline is sometimes displayed in two lines, and sometimes with a U+000A character from a fallback font (hex character number in a box). -- Marcin Kowalczyk qrczak(a)knm.org.pl http://qrnik.knm.org.pl/~qrczak/

1 0

Re: [Python-Dev] [Python-3000] New proposition for Python3 bytes filename issue
by Guido van Rossum 30 Sep '08

30 Sep '08

On Tue, Sep 30, 2008 at 1:12 PM, Terry Reedy <tjreedy(a)udel.edu> wrote: > Terry Reedy wrote: >> >> Guido van Rossum wrote: > >>> I'm not sure either way. I've heard it claim that Windows filesystem >>> APIs use Unicode natively. Does Python 3.0 on Windows currently >>> support filenames expressed as bytes? Are they encoded first before >>> passing to the Unicode APIs? Using what encoding? > >> [os.listdir(bytes) returns list of bytes, open(bytes) fails] > > More: > > The path functions seem also do not work: > >>>> op.abspath(b'tem') > ... > path = path.replace("/", "\\") > TypeError: expected an object with the buffer interface > > The error message is a bit cryptic given that the problem is that the > arguments to replace should be bytes instead of strings for a bytes path. > > .basename fails with > ... > while i and p[i-1] not in '/\\': > TypeError: 'in <string>' requires string as left operand, not int > > os.rename, os.stat, os.mkdir, os.rmdir work. I presume same is true for > others that normally work on windows. It looks roughly like the system calls do support bytes (using what encoding?) but the Python code in os.path doesn't. This is the same as the status quo on Linux. -- --Guido van Rossum (home page: http://www.python.org/~guido/)

1 0

Re: [Python-Dev] [Python-3000] New proposition for Python3 bytes filename issue
by Guido van Rossum 30 Sep '08

30 Sep '08

On Tue, Sep 30, 2008 at 11:13 AM, Georg Brandl <g.brandl(a)gmx.net> wrote: > Victor Stinner schrieb: >> On Windows, we might reject bytes filenames for all file operations: open(), >> unlink(), os.path.join(), etc. (raise a TypeError or UnicodeError) > > Since I've seen no objections to this yet: please no. If we offer a > "lower-level" bytes filename API, it should work for all platforms. I'm not sure either way. I've heard it claim that Windows filesystem APIs use Unicode natively. Does Python 3.0 on Windows currently support filenames expressed as bytes? Are they encoded first before passing to the Unicode APIs? Using what encoding? -- --Guido van Rossum (home page: http://www.python.org/~guido/)

2 1

Re: [Python-Dev] Patch for an initial support of bytes filename in Python3
by Bill Janssen 30 Sep '08

30 Sep '08

Victor Stinner <victor.stinner(a)haypocalc.com> wrote: > - listdir(unicode) -> only unicode, *skip* invalid filenames > (as asked by Guido) Is there an option listdir(bytes) which will return *all* filenames (as byte sequences)? Otherwise, this seems troubling to me; *something* should be returned for filenames which can't be represented, even if it's only None. Bill

2 3

Re: [Python-Dev] [Python-3000] New proposition for Python3 bytes filename issue
by Guido van Rossum 30 Sep '08

30 Sep '08

> Victor Stinner schrieb: (Thanks Victor for moving this to the list. Having a discussion in the tracker is really painful, I find.) >> POSIX OS >> -------- >> >> The default behaviour should be to use unicode and raise an error if >> conversion to unicode fails. It should also be possible to use bytes using >> bytes arguments and optional arguments (for getcwd). >> >> - listdir(unicode) -> unicode and raise an error on invalid filename I know I keep flipflopping on this one, but the more I think about it the more I believe it is better to drop those names than to raise an exception. Otherwise a "naive" program that happens to use os.listdir() can be rendered completely useless by a single non-UTF-8 filename. Consider the use of os.listdir() by the glob module. If I am globbing for *.py, why should the presence of a file named b'\xff' cause it to fail? Robust programs using os.listdir() should use the bytes->bytes version. >> - listdir(bytes) -> bytes >> - getcwd() -> unicode >> - getcwd(bytes=True) -> bytes >> - open(): accept bytes or unicode >> >> os.path.*() should accept operations on bytes filenames, but maybe not on >> bytes+unicode arguments. os.path.join('directory', b'filename'): raise an >> error (or use *implicit* conversion to bytes)? (Yeah, it should be all bytes or all strings.) On Mon, Sep 29, 2008 at 9:45 AM, Georg Brandl <g.brandl(a)gmx.net> wrote: > This approach (changing all path-handling functions to accept either bytes > or string, but not both) is doomed in my eyes. First, there are lots of them, > second, they are not only in os.path but in many modules and also in user > code, and third, I see no clean way of implementing them in the specified way. > (Just try to do it with os.path.join as an example; I couldn't find the > good way to write it, only the bad and the ugly...) It doesn't have to be supported for all operations -- just enough to be able to access all the system calls. and do the most basic pathname manipulations (split and join -- almost everything else can be built out of those). > If I had to choose, I'd still argue for the modified UTF-8 as filesystem > encoding (if it were UTF-8 otherwise), despite possible surprises when a > such-encoded filename escapes from Python. I'm having a hard time finding info about UTF-8b. Does anyone have a decent link? I noticed that OSX has a different approach yet. I believe it insists on valid UTF-8 filenames. It may even require some normalization but I don't know if the kernel enforces this. I tried to create a file named b'\xff' and it came out as %ff. Then "rm %ff" worked. So I think it may be replacing all bad UTF8 sequences with their % encoding. The "set filesystem encoding to be Latin-1" approach has a certain charm as well, but clearly would be a mistake on OSX, and probably on other systems too (whenever the user doesn't think in Latin-1). -- --Guido van Rossum (home page: http://www.python.org/~guido/)

9 12

when is path==NULL?
by Ulrich Eckhardt 30 Sep '08

30 Sep '08

Hi! I'm looking at trunk/Python/sysmodule.c, function PySys_SetArgv(). In that function, there is code like this: PyObject* path = PySys_GetObject("path"); ... if (path != NULL) { ... } My intuition says that if path==NULL, something is very wrong. At least I would expect to get 'None', but never NULL, except when out of memory. So, for the case that path==NULL', I would simply invoke Py_FatalError("no mem for sys.path"), similarly to the other call there. Sounds reasonable? Uli -- Sator Laser GmbH Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932 ************************************************************************************** Visit our website at <http://www.satorlaser.de/> ************************************************************************************** Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, weitergeleitet, veröffentlicht oder anderweitig benutzt werden. E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht verantwortlich. **************************************************************************************

4 4

Re: [Python-Dev] [Python-3000] New proposition for Python3 bytes filename issue
by Adam Olsen 30 Sep '08

30 Sep '08

On Tue, Sep 30, 2008 at 12:22 AM, Georg Brandl <g.brandl(a)gmx.net> wrote: > Victor Stinner schrieb: >> Le Monday 29 September 2008 18:45:28 Georg Brandl, vous avez écrit : >>> If I had to choose, I'd still argue for the modified UTF-8 as filesystem >>> encoding (if it were UTF-8 otherwise), despite possible surprises when a >>> such-encoded filename escapes from Python. >> >> If I understand correctly this solution. The idea is to change the default >> file system encoding, right? Eg. if your filesystem is UTF-8, use ISO-8859-1 >> to make sure that UTF-8 conversion will never fail. > > No, that was not what I meant (although it is another possibility). As I wrote, > Martin's proposal that I support here is using the modified UTF-8 codec that > successfully roundtrips otherwise invalid UTF-8 data. > > You seem to forget that (disregarding OSX here, since it already enforces > UTF-8) the majority of file names on Posix systems will be encoded correctly. > >> Let's try with an ugly directory on my UTF-8 file system: >> $ find >> .. >> ../têste >> ../ô >> ../a?b >> ../dossié >> ../dossié/abc >> ../dir?name >> ../dir?name/xyz >> >> Python3 using encoding=ISO-8859-1: >>>>> import os; os.listdir(b'.') >> [b't\xc3\xaaste', b'\xc3\xb4', b'a\xffb', b'dossi\xc3\xa9', b'dir\xffname'] >>>>> files=os.listdir('.'); files >> ['tÃªste', 'Ã´', 'aÿb', 'dossiÃ(c)', 'dirÿname'] >>>>> open(files[0]).close() >>>>> os.listdir(files[-1]) >> ['xyz'] >> >> Ok, I have unicode filenames and I'm able to open a file and list a directory. >> The problem is now to display correctly the filenames. >> >> For me "unicode" sounds like "text (characters) encoded in the correct >> charset". In this case, unicode is just a storage for *bytes* in a custom >> charset. > >> How can we mix <custom unicode (bytes encoded in ISO-8859-1)> with <real >> unicode>? Eg. os.path.join('dossiÃ(c)', "fichié") : first argument is encoded >> in ISO-8859-1 whereas the second argument is encoding in Unicode. It's >> something like that: >> str(b'dossi\xc3\xa9', 'ISO-8859-1') + '/' + 'fichi\xe9' >> >> Whereas the correct (unicode) result should be: >> 'dossié/fichié' >> as bytes in ISO-8859-1: >> b'dossi\xc3\xa9/fichi\xc3\xa9' >> as bytes in UTF-8: >> b'dossi\xe9/fichi\xe9' > > With the filenames decoded by UTF-8, your files named têste, ô, dossié will > be displayed and handled correctly. The others are *invalid* in the filesystem > encoding UTF-8 and therefore would be represented by something like > > u'dir\uXXffname' where XX is some private use Unicode namespace. It won't look > pretty when printed, but then, what do other applications do? They e.g. display > a question mark as you show above, which is not better in terms of readability. > > But it will work when given to a filename-handling function. Valid filenames > can be compared to Unicode strings. > > A real-world example: OpenOffice can't open files with invalid bytes in their > name. They are displayed in the "Open file" dialog, but trying to open fails. > This regularly drives me crazy. Let's not make Python not work this way too, > or, even worse, not even display those filenames. The only way to display that file would be to transform it into some other valid unicode string. However, as that string is already valid, you've just made any files named after it impossible to open. If you extend unicode then you're unable to display that extended name[1]. I think Guido's right on this one. If I have to choose between openoffice crashing or skipping my file, I'd vastly prefer it skip it. A warning would be a nice bonus (from python or from openoffice), telling me there's a buggered file I should go fix. Renaming the file is the end solution. [1] You could argue that Unicode should add new scalars to handle all currently invalid UTF-8 sequences. They could then output to their original forms if in UTF-8, or a mundane form in UTF-16 and UTF-32. However, I suspect "we don't want to add validation to linux" will not be a very persuasive argument. -- Adam Olsen, aka Rhamphoryncus

2 2

Status of MS Windows CE port
by Ulrich Eckhardt 30 Sep '08

30 Sep '08

Greetings! I'm currently trying to assess the effort required for a CE port. I'm already aware of the project at http://pythonce.sourceforge.net, but I find it a waste of time to have two separate projects which then require syncing changes back and forth. Questions: - What are the feelings towards a port in the mainline? I'm aware that you are trying to get 2.6 and 3.0 out of the door and are probably not willing to risk such changes to the release code right now, but in general? - Is anyone already working on this? I know that Brad Clements made an attempt some years ago but didn't get it integrated, are there any others? Maybe even any ongoing work? - I see that you are using Subversion and Bazaar, would it be possible to get a feature branch for the CE port? I'm thinking about the effort going on at pythonce.sf.net and some further porting that I need for my specific platform. IMHO both would be better off in the same repository as your upstream sources, since merging changes back and forth is much easier that way. Thanks! Uli -- Sator Laser GmbH Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932 ************************************************************************************** Visit our website at <http://www.satorlaser.de/> ************************************************************************************** Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, weitergeleitet, veröffentlicht oder anderweitig benutzt werden. E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht verantwortlich. **************************************************************************************

4 11

Re: [Python-Dev] Patch for an initial support of bytes filename in Python3
by Brett Cannon 29 Sep '08

29 Sep '08

On Mon, Sep 29, 2008 at 5:47 PM, Victor Stinner <victor.stinner(a)haypocalc.com> wrote: > Hi, > > See attached patch: python3_bytes_filename.patch > Patches should go on the tracker, not the mailing list. Otherwise it will just get lost. -Brett

1 0

Re: [Python-Dev] Real segmentation fault handler
by Ralf W. Grosse-Kunstleve 29 Sep '08

29 Sep '08

FWIW: I didn't have much luck translating segfaults into exceptions. It (seemed) to work on some platforms, but not others; this was in the context of C++. In my experience, it is more useful to generate Python and C stack traces and bail out. I also do this for floating-point exceptions. The handlers are installed at runtime from a low-level extension module: http://cctbx.svn.sourceforge.net/viewvc/cctbx/trunk/boost_adaptbx/meta_ext.… Example output is below. It works under Linux and partially under Mac OS X. Ralf % boost_adaptbx.segmentation_fault Now dereferencing null-pointer ... show_stack(1): /net/chevy/raid1/rwgk/dist/boost_adaptbx/command_line/segmentation_fault.py(10) run show_stack(2): /net/chevy/raid1/rwgk/dist/boost_adaptbx/command_line/segmentation_fault.py(14) <module> libc backtrace (18 frames, most recent call last): /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python [0x4118e9] /lib64/libc.so.6(__libc_start_main+0xf4) [0x363241e074] /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python(Py_Main+0x935) [0x4123c5] /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python(PyRun_SimpleFileExFlags+0x1a0) [0x4a8860] /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python(PyRun_FileExFlags+0x10e) [0x4a85ce] /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python(PyEval_EvalCode+0x32) [0x487402] /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python(PyEval_EvalCodeEx+0x81f) [0x4873bf] /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python(PyEval_EvalFrameEx+0x6bc1) [0x486541] /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python(PyEval_EvalFrameEx+0x2bb9) [0x482539] /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python(PyObject_Call+0x13) [0x415ae3] /net/chevy/raid1/rwgk/bintbx_py252/lib/libboost_python.so [0x2aaaaba7c6f7] /net/chevy/raid1/rwgk/bintbx_py252/lib/libboost_python.so(boost::python::handle_exception_impl(boost::function0<void>)+0x28) [0x2aaaaba87148] /net/chevy/raid1/rwgk/bintbx_py252/lib/libboost_python.so(boost::function0<void>::operator()() const+0x19e) [0x2aaaaba8816e] /net/chevy/raid1/rwgk/bintbx_py252/lib/libboost_python.so [0x2aaaaba7fef8] /net/chevy/raid1/rwgk/bintbx_py252/lib/libboost_python.so(boost::python::objects::function::call(_object*, _object*) const+0x7d) [0x2aaaaba7fb5d] /net/chevy/raid1/rwgk/bintbx_py252/lib/boost_python_meta_ext.so(boost::python::objects::caller_py_function_impl<boost::python::detail::caller<char (*)(char const*), boost::python::default_call_policies, boost::mpl::vector2<char, char const*> > >::operator()(_object*, _object*)+0x29) [0x2aaaab8470a9] /net/chevy/raid1/rwgk/bintbx_py252/lib/boost_python_meta_ext.so [0x2aaaab843790] /lib64/libc.so.6 [0x3632430f30] Segmentation fault (Python and libc call stacks above) % boost_adaptbx.divide_by_zero Now dividing by zero (in C++) ... show_stack(1): /net/chevy/raid1/rwgk/dist/boost_adaptbx/command_line/divide_by_zero.py(10) run show_stack(2): /net/chevy/raid1/rwgk/dist/boost_adaptbx/command_line/divide_by_zero.py(14) <module> libc backtrace (18 frames, most recent call last): /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python [0x4118e9] /lib64/libc.so.6(__libc_start_main+0xf4) [0x363241e074] /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python(Py_Main+0x935) [0x4123c5] /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python(PyRun_SimpleFileExFlags+0x1a0) [0x4a8860] /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python(PyRun_FileExFlags+0x10e) [0x4a85ce] /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python(PyEval_EvalCode+0x32) [0x487402] /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python(PyEval_EvalCodeEx+0x81f) [0x4873bf] /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python(PyEval_EvalFrameEx+0x6bc1) [0x486541] /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python(PyEval_EvalFrameEx+0x2bb9) [0x482539] /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python(PyObject_Call+0x13) [0x415ae3] /net/chevy/raid1/rwgk/bintbx_py252/lib/libboost_python.so [0x2aaaaba7c6f7] /net/chevy/raid1/rwgk/bintbx_py252/lib/libboost_python.so(boost::python::handle_exception_impl(boost::function0<void>)+0x28) [0x2aaaaba87148] /net/chevy/raid1/rwgk/bintbx_py252/lib/libboost_python.so(boost::function0<void>::operator()() const+0x19e) [0x2aaaaba8816e] /net/chevy/raid1/rwgk/bintbx_py252/lib/libboost_python.so [0x2aaaaba7fef8] /net/chevy/raid1/rwgk/bintbx_py252/lib/libboost_python.so(boost::python::objects::function::call(_object*, _object*) const+0x7d) [0x2aaaaba7fb5d] /net/chevy/raid1/rwgk/bintbx_py252/lib/boost_python_meta_ext.so(boost::python::objects::caller_py_function_impl<boost::python::detail::caller<double (*)(double const&, double const&), boost::python::default_call_policies, boost::mpl::vector3<double, double const&, double const&> > >::operator()(_object*, _object*)+0x12a) [0x2aaaab84759a] /net/chevy/raid1/rwgk/bintbx_py252/lib/boost_python_meta_ext.so [0x2aaaab8437a4] /lib64/libc.so.6 [0x3632430f30] Floating-point error (Python and libc call stacks above) ----- Original Message ---- From: Victor Stinner <victor.stinner(a)haypocalc.com> To: python-dev(a)python.org Sent: Monday, September 29, 2008 4:05:53 PM Subject: [Python-Dev] Real segmentation fault handler Hi, I would like to be able to catch SIGSEGV in my Python code! So I started to hack Python trunk to support this feature. The idea is to use a signal handler which call longjmp(), and add setjmp() at Py_EvalFrameEx() enter. See attached ("small") patch: segfault.patch Example read.py with the *evil* ctypes module of invalid memory read: ------------------- 8< -------------- from ctypes import string_at def fault(): text = string_at(1, 10) print("text = {0!r}".format(text)) def test(): print("test: 1") try: fault() except MemoryError, err: print "ooops!" print err print("test: 2") try: fault() except MemoryError, err: print "ooops!" print err print("test: end") def main(): test() if __name__ == "__main__": main() ------------------- 8< -------------- Result: ------------------- 8< -------------- $ python read.py test: 1 sizeof()=160 ooops! segmentation fault test: 2 sizeof()=160 ooops! segmentation fault test: end ------------------- 8< -------------- Example bug1.py of a stack overflow: ---------- loop = None, for i in xrange(10**5): loop = loop, None ---------- Result: ---------- $ python -i bug1.py >>> print loop (((((((((...Traceback (most recent call last): File "<stdin>", line 1, in <module> MemoryError: segmentation fault >>> ---------- Python is able to restore a valid state (stack/heap) after a segmentation fault and raise a classical Python exception (I choosed MemoryError, but it could be a specific exception). On my computer (Ubuntu Gutsy/i386), each segfault_frame takes sizeof(sigjmpbuf) + sizeof(void*) = 160 bytes, allocated on the stack. I don't know if it's huge or not, but that will limit the number of recursive calls. The feature can be optional if we add a configure option and some #ifdef/#endif. A dedicated stack is needed to be call the signal handler on stack overflow error. I choosed 4 KB, but since I only call longjmp(), smaller stack might also works. Does other VM support such feature? JVM, Mono, .NET, etc. ? I had the idea of catching SIGSEGV after reading the issue 1069092 (stack overflow because of too many recursive calls). -- Victor Stinner aka haypo http://www.haypocalc.com/blog/

1 0

New proposition for Python3 bytes filename issue
by Victor Stinner 29 Sep '08

29 Sep '08

Hi, After reading the previous discussion, here is new proposition. Python 2.x and Windows are not affected by this issue. Only Python3 on POSIX (eg. Linux or *BSD) is affected. Some system are broken, but Python have to be able to open/copy/move/remove files with an "invalid filename". The issue can wait for Python 3.0.1 / 3.1. Windows ------- On Windows, we might reject bytes filenames for all file operations: open(), unlink(), os.path.join(), etc. (raise a TypeError or UnicodeError) POSIX OS -------- The default behaviour should be to use unicode and raise an error if conversion to unicode fails. It should also be possible to use bytes using bytes arguments and optional arguments (for getcwd). - listdir(unicode) -> unicode and raise an error on invalid filename - listdir(bytes) -> bytes - getcwd() -> unicode - getcwd(bytes=True) -> bytes - open(): accept bytes or unicode os.path.*() should accept operations on bytes filenames, but maybe not on bytes+unicode arguments. os.path.join('directory', b'filename'): raise an error (or use *implicit* conversion to bytes)? When the user wants to display a filename to the screen, he can uses: text = str(filename, fs_encoding, "replace") -- Victor Stinner aka haypo http://www.haypocalc.com/blog/

4 5

',' precedence in documentation
by C. Titus Brown 29 Sep '08

29 Sep '08

Hi folks, over on the pygr list, we were recently discussing a mildly confusing edit I made: assert 'seq1' in self.db, self.db.keys() This was interpreted by someone as being assert 'seq1' in (self.db, self.db.keys()) which is very different from the actual meaning, assert ('seq1' in self.db), self.db.keys() where 'self.db.keys()' serves as the message to be printed out when the assertion fails. Apart from questions of why I was apparently out to confuse people with this edit, the question of ',' precedence came up. Looking at http://docs.python.org/ref/summary.html there's no mention of ',' binding and tuple creation vs the other operators. I'm not really sure how to think of it but it seems like this is something that could be cleared up in the docs with a brief note somewhere. Thoughts? cheers, --titus -- C. Titus Brown, ctb(a)msu.edu

16 28

Broken link to NASM download location
by Ulrich Eckhardt 29 Sep '08

29 Sep '08

Hi! In trunk/PCbuild/readme.txt it says NASM is available from kernel.org. The project has moved to Sourceforge though, please replace the link with http://nasm.sf.net Thanks! Uli -- Sator Laser GmbH Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932 ************************************************************************************** Visit our website at <http://www.satorlaser.de/> ************************************************************************************** Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, weitergeleitet, veröffentlicht oder anderweitig benutzt werden. E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht verantwortlich. **************************************************************************************

2 1

Re: [Python-Dev] [Python-3000] Filename as byte string in python 2.6 or 3.0?
by Victor Stinner 29 Sep '08

29 Sep '08

Le Monday 29 September 2008 06:43:55, vous avez écrit : > It will make users happy, and it's simple enough to implement for > python 3.0. I dislike your argument. A "quick and dirty hack" is always faster to implement than a real solution, but we may hits later new issues if we don't choose the right solution. -- Victor Stinner aka haypo http://www.haypocalc.com/blog/

1 0

Reminder: Python 2.6 final next Wednesday
by Barry Warsaw 28 Sep '08

28 Sep '08

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 This is a reminder that Python 2.6 final is schedule for release next Wednesday, October 1st. Once again, I've gone through the release blocker issues and knocked anything that doesn't specifically affect 2.6 to deferred blocker. This leaves us with 7 open blocking issues. Please spend some time over the next several days reviewing patches, making comments and working toward closing these issues. Email me directly if you have any questions, or ping me on irc. Unfortunately, my 'net connection may be a little bit flakey until Wednesday, but I will do my best to get online and follow up as needed. Please pitch in to help get Python 2.6 released on time! Thanks, - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSN1hJnEjvBPtnXfVAQIQfQQAgYr0tuzJhm3LZX/1SaCwRJcX09UvNH1I CjZHs2TKS22MjF9d3mmBgcEJPl9AwGE+6EF6OiSgrsNRoRtnN0MMT3nQo+deRkan P3jUgMFJMFkA7Uq5MmuNnEnKZXa/bsu/8Om/4wqHqvtDXbUQkZPfyE8BFwBzJJSM Aa6Wp3wieFs= =r4iD -----END PGP SIGNATURE-----

4 3

call for help: C-API 2 to 3 porting guide
by Benjamin Peterson 27 Sep '08

27 Sep '08

Hi guys, I remember a while back we were bemoaning the fact that little documentation exists for the extension module transition between 2.x and 3.x. In the name of this effort, I've created a c porting howto (Doc/howto/cporting.rst). I have started a few sections, but I can't do it all alone. Please consider writing a section. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1."

3 2

PyCon 2009 Call for Proposals
by Aahz 27 Sep '08

27 Sep '08

Call for proposals -- PyCon 2009 -- <http://us.pycon.org/2009/> =============================================================== Want to share your experience and expertise? PyCon 2009 is looking for proposals to fill the formal presentation tracks. The PyCon conference days will be March 27-29, 2009 in Chicago, Illinois, preceded by the tutorial days (March 25-26), and followed by four days of development sprints (March 30-April 2). Previous PyCon conferences have had a broad range of presentations, from reports on academic and commercial projects to tutorials and case studies. We hope to continue that tradition this year. Online proposal submission will open on September 29, 2008. Proposals will be accepted through November 03, with acceptance notifications coming out on December 15. For the detailed call for proposals, please see: <http://us.pycon.org/2009/conference/proposals/> We look forward to seeing you in Chicago! -- Aahz (aahz(a)pythoncraft.com) <*> http://www.pythoncraft.com/ "Argue for your limitations, and sure enough they're yours." --Richard Bach

6 10

json decoder speedups, any time left for 2.6?
by Bob Ippolito 27 Sep '08

27 Sep '08

I'm out of town this week for a conference (ICFP/CUFP in Victoria) and my hotel's connection has been bad enough such that I can't get any Real Work done so I've managed to hammer on the json library's decoding quite a bit instead. I just released simplejson 1.9.3 which improves decoding performance by about 2x and I've got some more changes along the way in trunk for 1.9.4 that will increase it even further (over 3x my original 1.9.2 benchmark perf). How much time do I have left to get this into Python 2.6? FWIW the changes are all on the Python side, no C code has been harmed (yet). The test suite still passes of course. -bob

4 6

2to3 and reST text format
by Victor Stinner 27 Sep '08

27 Sep '08

Hi, As I remember, last month 2to3 was able to fix my reST documentation using "-d" option. But trunk version fails to parse my reST documents. Should I run fsck on my brain or is it really a regression? Short example: ----------------- Title ===== Some text before the example: >>> print "Hello world!" Hello world! Verbatim block: :: >>> print "Hello world!" Hello world! ----------------- The document can be checked with: python2.5 -c "import doctest; doctest.testfile('test.rst')" 2to3 output: ----------------- $ 2to3 -d test.rst (...) RefactoringTool: Can't parse test.rst: ParseError: bad input: type=28, value='==', context=('', (2, 0)) (...) ----------------- -- Victor Stinner aka haypo http://www.haypocalc.com/blog/

2 1

Summary of Python tracker Issues
by Python tracker 26 Sep '08

26 Sep '08

ACTIVITY SUMMARY (09/19/08 - 09/26/08) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 2052 open (+35) / 13746 closed (+32) / 15798 total (+67) Open issues with patches: 660 Average duration of open issues: 713 days. Median duration of open issues: 1818 days. Open Issues Breakdown open 2037 (+34) pending 15 ( +1) Issues Created Or Reopened (69) _______________________________ turtle in the tkinter package? 09/20/08 CLOSED http://bugs.python.org/issue3884 reopened loewis patch subprocess failing in GUI applications on Windows 09/21/08 http://bugs.python.org/issue3905 reopened georg.brandl Building PDF documentation from tex files 09/19/08 http://bugs.python.org/issue3909 created wplappert 2.6 regression in socket.ssl method 09/19/08 http://bugs.python.org/issue3910 created matejcik patch ftplib.FTP.makeport() bug 09/19/08 http://bugs.python.org/issue3911 created giampaolo.rodola patch unittest. assertAlmostEqual() documentation incomplete 09/20/08 CLOSED http://bugs.python.org/issue3912 created roysmith patch compound_stmt syntax includes 'decorated' 09/20/08 CLOSED http://bugs.python.org/issue3913 created dangyogi augop definition does not include "//=" 09/20/08 CLOSED http://bugs.python.org/issue3914 created dangyogi Broken link in 14.7 curses, Python Library Reference 09/20/08 CLOSED http://bugs.python.org/issue3915 created jasoneth layout of build directories for Windows not current 09/20/08 CLOSED http://bugs.python.org/issue3916 created gagenellina patch set_display definition allows empty '{' '}' in Language Definiti 09/21/08 CLOSED http://bugs.python.org/issue3917 created dangyogi random.uniform suprisingly (good kind) does not work as document 09/21/08 CLOSED http://bugs.python.org/issue3918 created xuinkrbin PySys_SetObject crashes after Py_NewInterpreter(). 09/21/08 CLOSED http://bugs.python.org/issue3919 created grahamd OpenBSD 4.4 still doesn't support _XOPEN_SOURCE 09/21/08 http://bugs.python.org/issue3920 created loewis patch, needs review smtplib cannot sendmail over TLS 09/21/08 http://bugs.python.org/issue3921 created shidot 3.0rc1 missing tk lib in sys.path? 09/21/08 CLOSED http://bugs.python.org/issue3922 created jmfauth 'genexpr_for' in definition of 'call' in Language Reference. 09/21/08 CLOSED http://bugs.python.org/issue3923 created dangyogi cookielib chokes on non-integer cookie version, should ignore it 09/21/08 http://bugs.python.org/issue3924 created DenNukem test_distutils fails on cygwin 09/21/08 CLOSED http://bugs.python.org/issue3925 created ocean-city patch, easy Idle doesn't obey the new improved warnings arguements 09/21/08 http://bugs.python.org/issue3926 created scott_daniels dummy multiprocessing needs to use properties 09/22/08 CLOSED http://bugs.python.org/issue3927 created benjamin.peterson patch os.mknod missing on Solaris 09/22/08 http://bugs.python.org/issue3928 created sandberg Incorrect exception raising in dbm.open on non-existing DB 09/22/08 CLOSED http://bugs.python.org/issue3929 created hagen patch, needs review urllib.request.urlopen() different on Windows than Linux 09/22/08 CLOSED http://bugs.python.org/issue3930 created mark codecs.charmap_build is untested and undocumented 09/22/08 http://bugs.python.org/issue3931 created fijal HTMLParser cannot handle '&' and non-ascii characters in attribu 09/22/08 http://bugs.python.org/issue3932 created yanne presence of .pythonstartup.py causes __main__.__file__ to be set 09/22/08 CLOSED http://bugs.python.org/issue3933 created exarkun sphinx - building muppy docs fails on Linux 09/22/08 CLOSED http://bugs.python.org/issue3934 created robwolfe patch bisect insort C implementation ignores methods on list subclasse 09/22/08 http://bugs.python.org/issue3935 created jek Faulty suppression of 'as' keyword warning 09/22/08 CLOSED http://bugs.python.org/issue3936 created tjreedy patch platform.dist(): detect Linux distribution version in a robust, 09/22/08 CLOSED http://bugs.python.org/issue3937 created zooko patch Clearing globals; interpreter -- IDLE difference 09/22/08 http://bugs.python.org/issue3938 created tjreedy Patch to implement a real ftplib test suite 09/22/08 http://bugs.python.org/issue3939 created giampaolo.rodola patch turtle.py - minor bugfixes 09/23/08 CLOSED http://bugs.python.org/issue3940 created gregorlingl patch Help in IDLE should open the chm file 09/23/08 http://bugs.python.org/issue3941 created gregorlingl Usability issue from not being able to use defined start and end 09/23/08 CLOSED http://bugs.python.org/issue3942 created Mez IDLE won't start in 3.0rc1 "Subprocess didn't make connection... 09/23/08 CLOSED http://bugs.python.org/issue3943 created rbtyod faster long multiplication 09/23/08 http://bugs.python.org/issue3944 created pernici patch compile error in _fileio.c (cygwin) 09/23/08 CLOSED http://bugs.python.org/issue3945 created ocean-city patch, easy PyObject_CheckReadBuffer crashes on memoryview object 09/23/08 http://bugs.python.org/issue3946 created rupole patch, needs review configure --with-threads on cygwin => crash on thread related te 09/23/08 http://bugs.python.org/issue3947 created ocean-city readline steals sigwinch 09/23/08 http://bugs.python.org/issue3948 created shish curses' sigwinch handler isn't visible from python 09/23/08 http://bugs.python.org/issue3949 created shish turtle.py: bug in TurtleScreenBase._drawimage 09/23/08 CLOSED http://bugs.python.org/issue3950 created gregorlingl patch Disable Py_USING_MEMORY_DEBUGGER! 09/24/08 http://bugs.python.org/issue3951 created haypo patch _lsprof: clear() should call flush_unmatched() 09/24/08 http://bugs.python.org/issue3952 created haypo patch __cmp__ still documented in 3.0rc1? 09/24/08 CLOSED http://bugs.python.org/issue3953 created hniksic _hotshot: invalid error control in logreader() 09/24/08 http://bugs.python.org/issue3954 created haypo patch maybe doctest doesn't understand unicode_literals? 09/24/08 http://bugs.python.org/issue3955 created mark turtle.py - bug in Screen.__init__() 09/24/08 http://bugs.python.org/issue3956 created gregorlingl patch [contextlib] Reverse order of locking 09/24/08 http://bugs.python.org/issue3957 created DragonSA strage default value behaviour 09/24/08 CLOSED http://bugs.python.org/issue3958 created dominik.holler Add Google's ipaddr.py to the stdlib 09/24/08 http://bugs.python.org/issue3959 created gvanrossum Section permalink html anchors are wrong 09/25/08 CLOSED http://bugs.python.org/issue3960 created ronny Arrows key do not browse in the IDLE 09/25/08 http://bugs.python.org/issue3961 created italian-boy single architecture framework build fails on OS X 10.5 09/25/08 http://bugs.python.org/issue3962 created cekees Problems when calling exec from a function 09/25/08 CLOSED http://bugs.python.org/issue3963 created sandberg quiet the freeze makefile 09/25/08 http://bugs.python.org/issue3964 created docwhat patch 2.6rc2 crashes when trying to open unicode filename with unprint 09/25/08 CLOSED http://bugs.python.org/issue3965 created giltay patch Win32ErrorTests from test_os.py 09/25/08 http://bugs.python.org/issue3966 created rpetrov bytearray().count() 09/25/08 http://bugs.python.org/issue3967 created haypo patch, needs review turtle.py: fill() and end_fill() do not work correctly 09/25/08 CLOSED http://bugs.python.org/issue3968 created gregorlingl patch turtle.py - setup() doesn't work correctly 09/25/08 CLOSED http://bugs.python.org/issue3969 created gregorlingl patch Tix Tree widget no longer instantiable. 09/26/08 CLOSED http://bugs.python.org/issue3970 created ronLongo s_push: parser stack overflow MemoryError 09/26/08 http://bugs.python.org/issue3971 created netcaf Add Option to Bind to a Local IP Address in httplib.py 09/26/08 http://bugs.python.org/issue3972 created eldonz(a)atlanticdb.com patch Invalid line number in Exception traceback with header # -*- cod 09/26/08 CLOSED http://bugs.python.org/issue3973 created haypo collections.namedtuple uses exec to create new classes 09/26/08 http://bugs.python.org/issue3974 created audax patch PyTraceBack_Print() doesn't respect # coding: xxx header 09/26/08 http://bugs.python.org/issue3975 created haypo Issues Now Closed (55) ______________________ Incorrectly displayed non ascii characters in prompt using "inpu 274 days http://bugs.python.org/issue1688 amaury.forgeotdarc patch ctypes.wstring_at and string_at call Python API without the GIL 40 days http://bugs.python.org/issue3554 haypo subprocess + stdout redirection + wait + svn= hang 34 days http://bugs.python.org/issue3593 gregory.p.smith IDLE does not run with Py30b3 30 days http://bugs.python.org/issue3628 benjamin.peterson Duplicated test name in regex test script 29 days http://bugs.python.org/issue3654 timehorse sqlite: enumeration value 'TYPE_STRING' not handled in switch 29 days http://bugs.python.org/issue3659 ghaering patch, easy atexit.register with bad input segfaults on exit 29 days http://bugs.python.org/issue3666 skip.montanaro patch Ignored LDFLAGS during linking libpython$(VERSION).so 27 days http://bugs.python.org/issue3678 gregory.p.smith patch, easy test_tarfile error on cygwin (Directory not empty) 10 days http://bugs.python.org/issue3838 ocean-city patch, easy kqueue.control requires 2 params while docs say max_events (the 8 days http://bugs.python.org/issue3852 georg.brandl patch provide a "cmem" role 9 days http://bugs.python.org/issue3875 georg.brandl 2.6 regression in urllib.getproxies_environment 5 days http://bugs.python.org/issue3879 benjamin.peterson patch, needs review turtle in the tkinter package? 2 days http://bugs.python.org/issue3884 kirill_simonov patch errors on _bsddb creation and dealloc 7 days http://bugs.python.org/issue3885 haypo patch, needs review Python 2.6 doesn't run after installation on amd64 3 days http://bugs.python.org/issue3887 jpe Demo/parser/unparse.py 4 days http://bugs.python.org/issue3889 georg.brandl patch collections.deque should have empty() method 2 days http://bugs.python.org/issue3891 rhettinger collections 3 days http://bugs.python.org/issue3897 georg.brandl Slight readme.txt fix (VC9) 3 days http://bugs.python.org/issue3901 georg.brandl patch Strange heapq behavior on Python 3.0 when overriding __le__ 0 days http://bugs.python.org/issue3908 giampaolo.rodola unittest. assertAlmostEqual() documentation incomplete 1 days http://bugs.python.org/issue3912 georg.brandl patch compound_stmt syntax includes 'decorated' 1 days http://bugs.python.org/issue3913 georg.brandl augop definition does not include "//=" 1 days http://bugs.python.org/issue3914 georg.brandl Broken link in 14.7 curses, Python Library Reference 1 days http://bugs.python.org/issue3915 georg.brandl layout of build directories for Windows not current 0 days http://bugs.python.org/issue3916 georg.brandl patch set_display definition allows empty '{' '}' in Language Definiti 0 days http://bugs.python.org/issue3917 georg.brandl random.uniform suprisingly (good kind) does not work as document 0 days http://bugs.python.org/issue3918 georg.brandl PySys_SetObject crashes after Py_NewInterpreter(). 1 days http://bugs.python.org/issue3919 benjamin.peterson 3.0rc1 missing tk lib in sys.path? 0 days http://bugs.python.org/issue3922 loewis 'genexpr_for' in definition of 'call' in Language Reference. 0 days http://bugs.python.org/issue3923 georg.brandl test_distutils fails on cygwin 0 days http://bugs.python.org/issue3925 ocean-city patch, easy dummy multiprocessing needs to use properties 1 days http://bugs.python.org/issue3927 benjamin.peterson patch Incorrect exception raising in dbm.open on non-existing DB 4 days http://bugs.python.org/issue3929 hagen patch, needs review urllib.request.urlopen() different on Windows than Linux 0 days http://bugs.python.org/issue3930 amaury.forgeotdarc presence of .pythonstartup.py causes __main__.__file__ to be set 0 days http://bugs.python.org/issue3933 loewis sphinx - building muppy docs fails on Linux 1 days http://bugs.python.org/issue3934 schuppenies patch Faulty suppression of 'as' keyword warning 3 days http://bugs.python.org/issue3936 benjamin.peterson patch platform.dist(): detect Linux distribution version in a robust, 1 days http://bugs.python.org/issue3937 lemburg patch turtle.py - minor bugfixes 0 days http://bugs.python.org/issue3940 loewis patch Usability issue from not being able to use defined start and end 0 days http://bugs.python.org/issue3942 benjamin.peterson IDLE won't start in 3.0rc1 "Subprocess didn't make connection... 3 days http://bugs.python.org/issue3943 amaury.forgeotdarc compile error in _fileio.c (cygwin) 0 days http://bugs.python.org/issue3945 ocean-city patch, easy turtle.py: bug in TurtleScreenBase._drawimage 1 days http://bugs.python.org/issue3950 georg.brandl patch __cmp__ still documented in 3.0rc1? 0 days http://bugs.python.org/issue3953 georg.brandl strage default value behaviour 0 days http://bugs.python.org/issue3958 dominik.holler Section permalink html anchors are wrong 1 days http://bugs.python.org/issue3960 ronny Problems when calling exec from a function 1 days http://bugs.python.org/issue3963 sandberg 2.6rc2 crashes when trying to open unicode filename with unprint 0 days http://bugs.python.org/issue3965 giltay patch turtle.py: fill() and end_fill() do not work correctly 0 days http://bugs.python.org/issue3968 georg.brandl patch turtle.py - setup() doesn't work correctly 0 days http://bugs.python.org/issue3969 georg.brandl patch Tix Tree widget no longer instantiable. 0 days http://bugs.python.org/issue3970 amaury.forgeotdarc Invalid line number in Exception traceback with header # -*- cod 0 days http://bugs.python.org/issue3973 pitrou SRE: x++ isn't supported 2658 days http://bugs.python.org/issue433031 georg.brandl have a way to search backwards for re 2413 days http://bugs.python.org/issue516762 benjamin.peterson Add current dir when running try_run test program 1282 days http://bugs.python.org/issue1168055 lianos patch Top Issues Most Discussed (10) ______________________________ 20 Regexp 2.7 (modifications to current re 2.2.2) 164 days open http://bugs.python.org/issue2636 11 Major reworking of Python 2.5.2 re module 17 days open http://bugs.python.org/issue3825 11 Failed to build Python 2.5.1 with sqlite3 521 days open http://bugs.python.org/issue1706863 10 zero-length match confuses re.finditer() 606 days open http://bugs.python.org/issue1647489 9 26.rc1: test_signal issue on FreeBSD 6.3 12 days open http://bugs.python.org/issue3864 9 test_tarfile fails on cygwin (unicode decode error) 17 days open http://bugs.python.org/issue3824 9 Ctypes is confused by bitfields of varying integer types 45 days open http://bugs.python.org/issue3547 8 subprocess failing in GUI applications on Windows 5 days open http://bugs.python.org/issue3905 8 platform.dist() has unpredictable result under Linux 338 days open http://bugs.python.org/issue1322 7 Faulty suppression of 'as' keyword warning 3 days closed http://bugs.python.org/issue3936

1 0

preliminary findings of compiling python to javascript and running it under google "V8"
by Luke Kenneth Casson Leighton 26 Sep '08

26 Sep '08

http://groups.google.com/group/pyjamas-dev/browse_thread/thread/5e14ac70508… may be of interest to some people.

1 0

Update to httplib.py
by Eldon Ziegler 25 Sep '08

25 Sep '08

I updated httplib.py, python 2.4, to be able to bind to a specific IP address when connecting to a remote site. Would there be any interest in making this available to others? If so, are there instructions on how to post an update? Eldon Ziegler

4 3

okay to add new test cases to 2.6?
by skip＠pobox.com 23 Sep '08

23 Sep '08

Benjamin Peterson wound up writing a test case for the new C atexit module on the py3k branch. A similar test, though different in detail, makes sense for the Python atexit module on trunk. Is this something I can check in or should I just wait until after 2.6 is released? def test_badargs(self): s = StringIO.StringIO() sys.stdout = sys.stderr = s save_handlers = atexit._exithandlers atexit._exithandlers = [] try: atexit.register(lambda: 1, 0, 0, (x for x in (1,2)), 0, 0) self.assertRaises(TypeError, atexit._run_exitfuncs) finally: sys.stdout = sys.__stdout__ sys.stderr = sys.__stderr__ atexit._exithandlers = save_handlers Thx, Skip

2 1

Status of dictnotes.txt?
by Brendan O'Connor 23 Sep '08

23 Sep '08

Hi everyone, I was enjoying reading Raymond Hettinger's Objects/dictnotes.txt, and was wondering, which (if any) of its suggestions implemented? I see most of it was written back in 2003. Thanks, Brendan

2 2

Release blocker with no assignee (now me)
by skip＠pobox.com 22 Sep '08

22 Sep '08

Can someone comment on this ticket? http://bugs.python.org/issue3666 It's marked as a release blocker for 3.0. I added a patch then refined it based on a comment by Christian Heimes. It was unassigned. I just took it and will apply the patch if it gets a thumbs up from another developer. Skip

1 0

Patch review needed
by Jesus Cea 22 Sep '08

22 Sep '08

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Sorry to bother you, but I would like a patch review, since this code must be merged for 2.6. http://bugs.python.org/issue3885 Thanks in advance. - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea(a)jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBSNfaK5lgi5GaxT1NAQJCOAP+IVtGPcjjzWT/AefvEzxPajRmqf0H+MVn 5TO0STwkWuJHyYdv/z4OtA9UD6NTQ3Z4gLPOc5ZwlvhTW1gd856IBhVEFy1JARoJ ZERR7cp/WcdUSksHlC5qgTd2ZD1Y1VYcgCwTkL59D8R7I2wq752om6ZzhKs3SKC7 SG8kLYMlo9I= =P9mB -----END PGP SIGNATURE-----

1 0

Python documentation
by Barry Warsaw 22 Sep '08

22 Sep '08

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Martin points out that in the past, as part of the release process, we've built separate downloadable documentation. Do we still want to do that for Python 2.6 and 3.0, and if so, how do we go about doing that? I have this feeling that building the documentation is much different now than in the past, and I don't really have a good feel for how it's done now. If you think we should release separate downloadable documentation and can help integrate that into the release project, you just might be a Documentation Expert <wink>. Please let me know if you can help. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSNOLBnEjvBPtnXfVAQIQnAQAm6thEThGufep6hzHxBwAN8MTsLb9jxsu Z8GAtX1bdMNOrJczYpU6by0oXPLR2pupnGV1YrAyQyoqpk+K7W8by5Qtg8+ZZcYH GerkqMVtNYn2zY1HhKigivp2JvlqIidRc5D36XS2EJixhZEPcOQDVm34THNQyRJT QasCQwdSAHI= =MbMY -----END PGP SIGNATURE-----

8 10

turtle.Screen.__init__ issue
by Gregor Lingl 22 Sep '08

22 Sep '08

Hello there, its high time to resolve an issue, which I have already addressed twice some weeks ago. (You can find a more elaborate description in my former posting cited below) There is a tiny difference (also in behaviour!) in turtle.Screen.__init__() between the versions for 2.6 and 3.0. The difference results from the fact, that I submitted the 3.0 version approx. a week later, after having it ported to 3.0. In this process I had found what I now consider to be a bug in 2.6 and changed it accordingly. Shortly: If you have already a Screen object containing some turtles and some graphics, in 2.6: s = Screen() returns an object with identical state and behaviour, but clears (re-initializes) the screen and thus destroys the content in 3.0 s = Screen() returns an object with identical state and behaviour, but leaves the content untouched The difference in code consist only in indenting the call of the __init__ method of the parent class, so it will be executed only conditionally. Anyway, as this difference between the two versions is highly undesirable there are (imho) three options to proceed: (1) correct 2.6 in order that it will work like 3.0 (2) undo the change in 3.0 in order that it will work like 2.6 (3) find a different solution for both I would (like Vern, see below) decisevely prefer option (1), and I suppose that there is not enough time left to chose option (3) as this would probably need some discussions. What is your opinion, and who should decide? For your convenience I've attached a diff-file which also contains the description of three other small bugs, which I've found in the meantime and which shouldn't cause any controversies. Regards, Gregor %%%%%%%%%% Here follows the answer of Vern Ceder - a long term turtle graphics user and author of several patches for the old turtle module - to my former posting: >> Gregor, >> >> I don't feel authoritative on the correctness/appropriateness of the implementation, >> but I do agree completely that behavior b, or what you have in the 3.0 version, >> is vastly preferable. >> >> Cheers, >> Vern -------- Original-Nachricht -------- Betreff: [Python-Dev] turtle.Screen- how to implement best a Singleton Datum: Mon, 18 Aug 2008 10:15:45 +0200 Von: Gregor Lingl <gregor.lingl(a)aon.at> An: python-dev(a)python.org CC: Toby Donaldson <tjd(a)sfu.ca>, python-3000(a)python.org, jjposner(a)snet.net, Brad Miller <bonelake(a)gmail.com>, Vern Ceder <vceder(a)canterburyschool.org> Hi, this posting - concerning the new turtle module - goes to the Python-Dev and Python-3000 lists and to a couple of 'power users' of turtle graphics, hoping to recieve feedback from the developer's point of view as well as from the user's point of view. Currently the implementations of the turtle.Screen class for Python 2.6 and Python 3.0 differ by a 'tiny' detail with an important difference in behaviour. So clearly this has to be resolved before the final release.(The origin of this difference is, that when I ported turtle.py to Python 3.0 I discovered (and 'fixed') what I now consider to be a bug in the 2.6 version.) I'd like to ask you kindly for your advice to achieve an optimal solution. The posting consists of three parts: 1. Exposition of design goals 2. Problem with the implementation 3. How to solve it? Preliminary remark: I've had some discussions on this topic before but I still do not see a clear solution. Moreover I'm well aware of the fact that using the Singleton pattern is controversial. So ... 1. Exposition of design goals ... why use the Singleton design pattern? The turtle module contains a TurtleScreen class, which implements methods to control the drawing area the turtle is (turtles are) drawing on. It's constructor needs a Tkinter Canvas as argument. In order to avoid the need for users to tinker around with Tkinter stuff there is the Screen(TurtleScreen) class, designed to be used by beginners(students, kids,...), particularly in interactive sessions. A (THE (!)) Screen object is essentially a window containing a scrolled canvas, the TurtleScreen. So it's a ressource which should exist only once. It can be constructed in several ways: - implicitely by calling an arbitrary function derived from a Turtle-method, such as forward(100) or by constructing a Turtle such as bob = Turtle() - implicitely by calling an arbitrary function derived from a Screen method, such as bgcolor("red") - explicitely by calling it's constructor such as s = Screen() Anyway this construction should only happen if a Screen object doesn't exist yet. Now for the pending question: What should happen, when s = Screen() is called explicitely and there exists already 'the' Screen object. (i) Clearly s should get a reference to the existing Screen object, but ... (ii) (a)... should s be reinitialized (this is the case now in Python 2.6), or (b)... should s be left untouched (this is the case now in Python 3.0) I, for my part, prefer the latter solution (b). Example: a student, having (interactively) produced some design using some turtle t = Turtle() decides spontaneously to change backgroundcolor. s = Screen(); s.bgcolor("pink") should do this for her - instead of deleting her design and moreover her turtle. To reinitialize the screen she still can use s.clear(). Of course, there are workarounds to achieve the same effect also with solution (a), for instance by assigning s = Screen() *before* drawing anything or by assigning s = t.getscreen(). But imho (which derives itself from my experience as a teacher) solution (b) supports better the oop-view as well as experimenting spontaneously in interactive sessions. 2. Problem with the implementation The task is to derive a Singleton class from a Nonsingleton class (Screen from TurtleScreen). The current implementations of the Screen 'Singleton' both use the Borg idiom. Just for *explaining* the difference between the two versions of class Screen here concisely, I'll use a 'standard' Singleton pattern (roughly equivalent to the Borg idiom): class Spam(object): def __init__(self, s): self.s = s class SingleSpam(Spam): _inst = None def __new__(cls, *args, **kwargs): if cls != type(cls._inst): cls._inst = Spam.__new__(cls, *args, **kwargs) return cls._inst def __init__(self, s): if vars(self): return ###### should this be here??? Spam.__init__(self, s) Shortly, this means that SingleSpam.__init__() acts like an empty method whenever a (the!) SingleSpam object already exists. 3.0 version of Screen acts like this. By contrast 2.6 version of Screen acts as if the butlast line were not there and thus reinitializes the Screen object. 3. How to solve it? Main question: which *behaviour* of the Screen class should be preferred. If 3.0, is it feasible and correct not to call the constructor of the parent class if the object already exists? Additional question: Do you consider the Borg idiom a good solution for this task or should the standard singleton pattern as shown above be preferred. Or would you suggest a solution/an approach different from both? Thanks for your patience, and - in advance - for your assistance Regard, Gregor _______________________________________________ Python-Dev mailing list Python-Dev(a)python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/gregor.lingl%40aon.at --- turtle26rc2.py Thu Jun 12 20:39:36 2008 +++ turtle.py Mon Sep 22 00:02:25 2008 @@ -359,7 +359,9 @@ def __init__(self, master, width=500, height=350, canvwidth=600, canvheight=500): TK.Frame.__init__(self, master, width=width, height=height) - self._root = self.winfo_toplevel() + # renamed: self._root ==> self._rootwindow + # to avoid name conflict with TK.Frame method self._root + self._rootwindow = self.winfo_toplevel() self.width, self.height = width, height self.canvwidth, self.canvheight = canvwidth, canvheight self.bg = "white" @@ -379,7 +381,7 @@ self.hscroll.grid(padx=1, in_ = self, pady=1, row=1, column=0, rowspan=1, columnspan=1, sticky='news') self.reset() - self._root.bind('<Configure>', self.onResize) + self._rootwindow.bind('<Configure>', self.onResize) def reset(self, canvwidth=None, canvheight=None, bg = None): """Ajust canvas and scrollbars according to given canvas size.""" @@ -732,7 +734,7 @@ """Configure image item as to draw image object at position (x,y) on canvas) """ - self.cv.coords(item, (x, -y)) + self.cv.coords(item, (x * self.xscale, -y * self.yscale)) self.cv.itemconfig(item, image=image) def _setbgpic(self, item, image): @@ -3571,7 +3573,7 @@ self._root.setupcanvas(width, height, canvwidth, canvheight) Screen._canvas = self._root._getcanvas() self.setup(width, height, leftright, topbottom) - TurtleScreen.__init__(self, Screen._canvas) + TurtleScreen.__init__(self, Screen._canvas) Turtle._screen = self def setup(self, width=_CFG["width"], height=_CFG["height"], @@ -3612,6 +3614,7 @@ if starty is None: starty = (sh - height) / 2 self._root.set_geometry(width, height, startx, starty) + self.update() def title(self, titlestring): """Set title of turtle-window

2 2

Possible issue in warnings.py
by Scott David Daniels 21 Sep '08

21 Sep '08

Does someone who knows the design of warnings.py a bit better than I know whether it would be an improvement to switch from: > try: > file.write(formatwarning(message, category, filename, > lineno, line)) > except IOError: > pass # the file (probably stderr) is invalid > # - this warning gets lost. to: > complaint = formatwarning(message, category, filename, > lineno, line) > try: > file.write(complaint) > except IOError: > pass # the file (probably stderr) is invalid > # - this warning gets lost. on the grounds that you might not want failures in the linecache code to behave the same as failures in writing the complaint to the target area? I'm working on a patch where the Idle warnings code seems to accidentally escalating warnings into errors, and it looked to me like this would accidentally swallow errors getting warning context and make them fail silently. The Idle issue that I'm fiddling with is that it doesn't take the new showwarning call format, and it looked like this should possibly be fixed at the same time. --Scott David Daniels Scott.Daniels(a)Acm.Org

2 1

Deploying a Python Service on Apache Axis2
by Heshan Suriyaarachchi 21 Sep '08

21 Sep '08

Hi guys, Apache Axis2/Java, is a popular open source Web service engine. It currently supports exposing services written in Java, Javascript as Web services. This article [1] discusses the Python data Binding that enable exposing Web services written in Python. [1] - http://wso2.org/library/articles/deploying-python-service-axis2 [2] - http://heshans.blogspot.com/2008/09/wso2-wsfjython-10-alpha.html [3] - http://wso2.org/library/invoking-enterprise-web-services-using-jython -- Regards, Heshan Suriyaarachchi http://heshans.blogspot.com/

2 1

how about updating PEP 290?
by Fredrik Lundh 21 Sep '08

21 Sep '08

the "Code Migration and Modernization" PEP hasn't been updated for 2.5 and 2.6. http://www.python.org/dev/peps/pep-0290/ surely there's something new in 2.5 and 2.6 that might be worth mentioning? </F>

1 0

bsddb tests disabled by default
by Jesus Cea 20 Sep '08

20 Sep '08

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Just installing 2.6rc2, I see that bsddb3 testsuite is disabled by default. Current testsuite is far more fast and stable that the old one (entire test: 17 seconds in my machine). I was wondering if it is time to enable bsddb3 testsuite by default. BTW: How is a "resource" enabled by an user, without touching sourcecode?. - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea(a)jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBSNJktZlgi5GaxT1NAQI7YQP+PwnNpfnJCsd3u/bAgjFQfHaRXMYlS1PN dZPb8lkzMbyanNituTC9VLxI97BXsSPSM7VNnbyO3lBVSvJxvsDaRDmoUynno+VX rw7+dD/mqKdyPujBjLzqYhbvQAoUOxLNac44/pTjvqoGiDa5CeR0AunUDnqnVVJa 4by7SBBxYrs= =sllZ -----END PGP SIGNATURE-----

5 7

Code coverage
by Walter Dörwald 19 Sep '08

19 Sep '08

Hello all! The code coverage site at http://coverage.livinglogic.de/ was broken for the last few months. It's fixed again now and runs the test suite once per day with regrtest.py -T -N -uurlfetch,largefile,network,decimal Servus, Walter

2 1

What this code should do?
by Maciej Fijalkowski 19 Sep '08

19 Sep '08

Hello, I'm a little clueless about exact semantics of following snippets: http://paste.pocoo.org/show/85698/ is this fine? or shall I fill the bug? (the reason to ask is because a) django is relying on this b) pypy implements it differently) cheers, fijal

5 5

ANNOUNCE: CapPython, an object-capability subset of Python
by Mark Seaborn 19 Sep '08

19 Sep '08

During the past couple of months I have been working on an object-capability subset of Python - in other words, a restricted execution scheme for sandboxing Python code. It has been influenced by other object-capability subset languages, such as Joe-E (a subset of Java [1]), Caja/Cajita (subsets of Javascript [2]) and Caperl (based on Perl [3]). I'm calling it CapPython because the name doesn't seem to have been taken yet. :-) I believe it is now secure, so it seems like a good time to announce it here! The basic idea behind CapPython is to enforce encapsulation by restricting access to private attributes of objects. This is achieved through a combination of static checking and limiting access to unsafe builtins and modules. Private attributes may only be accessed through "self" variables. "Self" variables are defined as being the first arguments of functions defined inside class definitions, with a few restrictions intended to prevent these functions from escaping without being safely wrapped. Private attribute names are those starting with "_". Additionally, "im_self", "im_func" and some other special cases are treated as private attributes. Assignments to attributes are only allowed via "self" variables. For example, the following code is accepted by the static verifier: class Counter(object): def __init__(self): self._count = 0 def get_next(self): self._count += 1 return self._count But the following code reads a private attribute and so it is rejected as violating encapsulation: counter._count -= 1 CapPython consists of three parts: - a static verifier; - a "safe exec" function, which will check code before executing it and can run code in a safe scope; - a module loader which implements a safe __import__ function. Eventually this will be runnable as untrusted CapPython code. I am documenting CapPython via my blog at the moment, with the following posts so far: http://lackingrhoticity.blogspot.com/2008/08/introducing-cappython.html http://lackingrhoticity.blogspot.com/2008/09/dealing-with-modules-and-built… http://lackingrhoticity.blogspot.com/2008/09/cappython-unbound-methods-and-… The code is available from a Bazaar repository on Launchpad: https://code.launchpad.net/cappython I am currently working on creating a simple example program, which will be a wsgiref-based web server with a form for executing CapPython code. This involves taming some of the standard libraries to pass the verifier. There are some design notes here - http://plash.beasts.org/wiki/CapPython - although these notes are more a list of references and problems CapPython needs to address than an explanation of the current design. There was also a thread about CapPython on the e-lang mailing list: http://www.eros-os.org/pipermail/e-lang/2008-August/012828.html Mark [1] http://code.google.com/p/joe-e/ [2] http://code.google.com/p/google-caja/ [3] http://caperl.links.org/

6 8

Summary of Python tracker Issues
by Python tracker 19 Sep '08

19 Sep '08

ACTIVITY SUMMARY (09/12/08 - 09/19/08) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 2035 open (+36) / 13696 closed (+24) / 15731 total (+60) Open issues with patches: 652 Average duration of open issues: 715 days. Median duration of open issues: 1793 days. Open Issues Breakdown open 2022 (+36) pending 13 ( +0) Issues Created Or Reopened (60) _______________________________ FUD in documentation for urllib.urlopen() 09/12/08 http://bugs.python.org/issue3849 created raz find_recursion_limit.py is broken 09/12/08 CLOSED http://bugs.python.org/issue3850 created pitrou patch, easy, needs review IDLE: Pressing "Home" on Windows places cursor before ">>>" inst 09/12/08 http://bugs.python.org/issue3851 created serwy patch kqueue.control requires 2 params while docs say max_events (the 09/12/08 http://bugs.python.org/issue3852 created ionel.mc Windows SQLite DLL should be built with multithreading enabled 09/12/08 CLOSED http://bugs.python.org/issue3853 created ghaering easy Document sqlite3 vs. threads 09/12/08 http://bugs.python.org/issue3854 created ghaering Windows installation did not work; tried on two machines 09/13/08 CLOSED http://bugs.python.org/issue3855 created MLModel IDLE fails on startup on Mac 10.5 for 2.6b3 and 3.0b3 09/13/08 CLOSED http://bugs.python.org/issue3856 created MLModel ImportError: No module named test.test_support 09/13/08 CLOSED http://bugs.python.org/issue3857 created wplappert make install tries to install files outside of --prefix 09/13/08 CLOSED http://bugs.python.org/issue3858 created jjlee test_sys.Sizeof fails on win64 09/13/08 CLOSED http://bugs.python.org/issue3859 created amaury.forgeotdarc patch GzipFile and BZ2File should support context manager protocol 09/13/08 http://bugs.python.org/issue3860 created hagen distutils CCompiler._compile doesn't require lang keyword argume 09/14/08 CLOSED http://bugs.python.org/issue3861 created ikelos test_array fails on FreeBSD7 amd64 09/14/08 http://bugs.python.org/issue3862 created robrien patch 2.6rc1: test_threading hangs on FreeBSD 6.3 i386 09/14/08 http://bugs.python.org/issue3863 created aimacintyre patch 26.rc1: test_signal issue on FreeBSD 6.3 09/14/08 http://bugs.python.org/issue3864 created aimacintyre explain that profilers should be used for profiling, not benchma 09/14/08 http://bugs.python.org/issue3865 created effbot int() doesn't 'guess' 09/14/08 CLOSED http://bugs.python.org/issue3866 created kjohnson What's New in 2.6 doesn't mention stricter object.__init__ 09/14/08 CLOSED http://bugs.python.org/issue3867 created spiv patch for review: OS/2 EMX port fixes for 2.6 09/14/08 http://bugs.python.org/issue3868 created aimacintyre patch, patch Arrow keys not working with Python 2.6rc1 09/14/08 CLOSED http://bugs.python.org/issue3869 created Chewie Parser/asdl_c.py requires python in order to build python 09/14/08 CLOSED http://bugs.python.org/issue3870 created gregory.p.smith cross building python for mingw32 with distutils 09/14/08 http://bugs.python.org/issue3871 created rpetrov Tix ComboBox error 09/15/08 http://bugs.python.org/issue3872 created dwahli Unpickling is really slow 09/15/08 http://bugs.python.org/issue3873 created hagen documentation bug: HTMLParser needs to document unknown_decl 09/15/08 http://bugs.python.org/issue3874 created freyley provide a "cmem" role 09/15/08 http://bugs.python.org/issue3875 created benjamin.peterson multiprocessing does not compile on *BSD and potentialy other sy 09/16/08 http://bugs.python.org/issue3876 created henry.precheur test_fileio fails on OpenBSD 4.4 09/16/08 http://bugs.python.org/issue3877 created henry.precheur patch urllib2 is not working with proxy for HTTPS 09/16/08 CLOSED http://bugs.python.org/issue3878 created anupam 2.6 regression in urllib.getproxies_environment 09/16/08 http://bugs.python.org/issue3879 created vila patch, needs review _tkinter._flatten() doesn't check PySequence_Size() error code 09/16/08 http://bugs.python.org/issue3880 created haypo patch IDLE won't start in custom directory. 09/16/08 http://bugs.python.org/issue3881 created kimsey0 Bottom buttons of IDLE Preferences Pane on Mac not wide enough f 09/16/08 CLOSED http://bugs.python.org/issue3882 created MLModel Bottom buttons of IDLE Preferences Pane on Mac not wide enough f 09/16/08 http://bugs.python.org/issue3883 created MLModel turtle in the tkinter package? 09/16/08 CLOSED http://bugs.python.org/issue3884 created kirill_simonov errors on _bsddb creation and dealloc 09/16/08 http://bugs.python.org/issue3885 created haypo patch, needs review Integer overflow in _hashopenssl.c (CVE-2008-2316) 09/18/08 http://bugs.python.org/issue3886 reopened benjamin.peterson patch, patch, 64bit Python 2.6 doesn't run after installation on amd64 09/17/08 CLOSED http://bugs.python.org/issue3887 created jpe [PATCH] Document more deprecated modules in What's New in Python 09/17/08 CLOSED http://bugs.python.org/issue3888 created pjenvey patch Demo/parser/unparse.py 09/17/08 http://bugs.python.org/issue3889 created gregdarke patch ssl.SSLSocket.recv() implementation may not work with non-blocki 09/17/08 http://bugs.python.org/issue3890 created giampaolo.rodola collections.deque should have empty() method 09/17/08 http://bugs.python.org/issue3891 created roysmith bsddb: test01_basic_replication fails on Windows sometimes 09/18/08 http://bugs.python.org/issue3892 created benjamin.peterson timedelta overflows in a surprising way 09/18/08 CLOSED http://bugs.python.org/issue3893 created jjinux imageop issue 09/18/08 http://bugs.python.org/issue3894 created brett.cannon _lsprof issue 09/18/08 http://bugs.python.org/issue3895 created brett.cannon idle should be installed as idle3.0 09/18/08 http://bugs.python.org/issue3896 created HWJ collections 09/18/08 http://bugs.python.org/issue3897 created kadelka 3.0 documentation fails to build 09/18/08 CLOSED http://bugs.python.org/issue3898 created loewis test_ssl.py doesn't properly test ssl integration with asyncore 09/18/08 http://bugs.python.org/issue3899 created giampaolo.rodola patch ctypes: wrong calling convention for _string_at 09/18/08 CLOSED http://bugs.python.org/issue3900 created haypo patch Slight readme.txt fix (VC9) 09/18/08 http://bugs.python.org/issue3901 created ocean-city patch distutils does not correctly create packages for compiled extens 09/18/08 http://bugs.python.org/issue3902 created Peaker pickle error in python3.0rc1 09/18/08 CLOSED http://bugs.python.org/issue3903 created Georg asynchat async_chat __init__() arguments changed in python 2.6 09/18/08 CLOSED http://bugs.python.org/issue3904 created forest subprocess failing in GUI applications on Windows 09/18/08 http://bugs.python.org/issue3905 created twhitema lib2to3\main.py will not run 09/19/08 CLOSED http://bugs.python.org/issue3906 created rupole "for line in file" doesn't work for pipes 09/19/08 CLOSED http://bugs.python.org/issue3907 created endolith Strange heapq behavior on Python 3.0 when overriding __le__ 09/19/08 http://bugs.python.org/issue3908 created giampaolo.rodola Issues Now Closed (51) ______________________ Add a factorial function 182 days http://bugs.python.org/issue2138 maix Nested module import clutters package namespace 202 days http://bugs.python.org/issue2210 brett.cannon Fixer for new metaclass syntax is needed 179 days http://bugs.python.org/issue2366 benjamin.peterson patch, 26backport Fixer for new integer literals are needed 180 days http://bugs.python.org/issue2369 benjamin.peterson 26backport Multiple buffer overflows in unicode processing 158 days http://bugs.python.org/issue2620 brett.cannon patch bsddb3 needs to be ported to Python 3.0 124 days http://bugs.python.org/issue2887 jcea 2to3 doesn't handle print(whatever); print nor string.* function 106 days http://bugs.python.org/issue3000 benjamin.peterson integer overflow in hashlib causes wrong results for cryptograph 108 days http://bugs.python.org/issue3026 loewis patch sqlite defines a few global symbols. 92 days http://bugs.python.org/issue3103 ghaering Documentation: timeit: "lower bound" should read "upper bound" 68 days http://bugs.python.org/issue3318 unutbu file read preallocs 'size' bytes which can cause memory problems 36 days http://bugs.python.org/issue3531 pitrou Add MS EULA to the list of third-party licenses in the Windows i 25 days http://bugs.python.org/issue3617 loewis patch test_cpickle crash on AMD64 Windows build 22 days http://bugs.python.org/issue3640 pitrou patch, 64bit latexwriter: \footnotemark can only take numbers as arguments 20 days http://bugs.python.org/issue3655 georg.brandl incompatible arguments in warning formatting for idle 17 days http://bugs.python.org/issue3698 gpolo os.urandom(1.1): infinite loop 22 days http://bugs.python.org/issue3708 rpetrov patch, easy socket.socket.recv broken (unbearably slow) 10 days http://bugs.python.org/issue3766 pitrou os.write accepts unicode strings 11 days http://bugs.python.org/issue3782 pitrou patch python-2.6b3.msi and python-2.6b3.amd64.msi can't both be instal 2 days http://bugs.python.org/issue3833 jretz can't run close through itertools.chain on inner generator 1 days http://bugs.python.org/issue3842 rhettinger hexadecimal, not decimal 2 days http://bugs.python.org/issue3843 benjamin.peterson sqlite3 module: Improved concurrency 0 days http://bugs.python.org/issue3846 ghaering needs review Begging for review 0 days http://bugs.python.org/issue3847 loewis find_recursion_limit.py is broken 1 days http://bugs.python.org/issue3850 pitrou patch, easy, needs review Windows SQLite DLL should be built with multithreading enabled 6 days http://bugs.python.org/issue3853 loewis easy Windows installation did not work; tried on two machines 0 days http://bugs.python.org/issue3855 loewis IDLE fails on startup on Mac 10.5 for 2.6b3 and 3.0b3 2 days http://bugs.python.org/issue3856 gpolo ImportError: No module named test.test_support 0 days http://bugs.python.org/issue3857 benjamin.peterson make install tries to install files outside of --prefix 1 days http://bugs.python.org/issue3858 pitrou test_sys.Sizeof fails on win64 3 days http://bugs.python.org/issue3859 schuppenies patch distutils CCompiler._compile doesn't require lang keyword argume 0 days http://bugs.python.org/issue3861 ikelos int() doesn't 'guess' 0 days http://bugs.python.org/issue3866 benjamin.peterson What's New in 2.6 doesn't mention stricter object.__init__ 1 days http://bugs.python.org/issue3867 benjamin.peterson Arrow keys not working with Python 2.6rc1 0 days http://bugs.python.org/issue3869 benjamin.peterson Parser/asdl_c.py requires python in order to build python 3 days http://bugs.python.org/issue3870 theller urllib2 is not working with proxy for HTTPS 0 days http://bugs.python.org/issue3878 amaury.forgeotdarc Bottom buttons of IDLE Preferences Pane on Mac not wide enough f 0 days http://bugs.python.org/issue3882 benjamin.peterson turtle in the tkinter package? 0 days http://bugs.python.org/issue3884 brett.cannon Python 2.6 doesn't run after installation on amd64 3 days http://bugs.python.org/issue3887 loewis [PATCH] Document more deprecated modules in What's New in Python 0 days http://bugs.python.org/issue3888 georg.brandl patch timedelta overflows in a surprising way 1 days http://bugs.python.org/issue3893 benjamin.peterson 3.0 documentation fails to build 0 days http://bugs.python.org/issue3898 georg.brandl ctypes: wrong calling convention for _string_at 0 days http://bugs.python.org/issue3900 haypo patch pickle error in python3.0rc1 0 days http://bugs.python.org/issue3903 benjamin.peterson asynchat async_chat __init__() arguments changed in python 2.6 0 days http://bugs.python.org/issue3904 josiahcarlson lib2to3\main.py will not run 0 days http://bugs.python.org/issue3906 benjamin.peterson "for line in file" doesn't work for pipes 0 days http://bugs.python.org/issue3907 amaury.forgeotdarc -S hides standard dynamic modules 2247 days http://bugs.python.org/issue586680 brett.cannon classic division in demos/ directory 2035 days http://bugs.python.org/issue687648 akuchling patch, easy digit-only tag values are mishandled in Tkinter 1286 days http://bugs.python.org/issue1160383 gpolo tkapp read-only attributes 1130 days http://bugs.python.org/issue1257772 gpolo Top Issues Most Discussed (10) ______________________________ 11 errors on _bsddb creation and dealloc 3 days open http://bugs.python.org/issue3885 11 Major reworking of Python 2.5.2 re module 10 days open http://bugs.python.org/issue3825 9 2.6rc1: test_threading hangs on FreeBSD 6.3 i386 5 days open http://bugs.python.org/issue3863 9 test_array fails on FreeBSD7 amd64 6 days open http://bugs.python.org/issue3862 8 bsddb: test01_basic_replication fails on Windows sometimes 2 days open http://bugs.python.org/issue3892 8 Integer overflow in _hashopenssl.c (CVE-2008-2316) 2 days open http://bugs.python.org/issue3886 8 tkinter goes into an infinite loop (pydoc.gui) 8 days open http://bugs.python.org/issue3835 7 Add MS EULA to the list of third-party licenses in the Windows 25 days closed http://bugs.python.org/issue3617 6 find_recursion_limit.py is broken 1 days closed http://bugs.python.org/issue3850 6 sqlite3 module: Improved concurrency 0 days closed http://bugs.python.org/issue3846

1 0

What's New in 2.6 link wasn't what I expected
by skip＠pobox.com 19 Sep '08

19 Sep '08

>From this page: http://www.python.org/download/releases/2.6/ I clicked the link labelled "What's new in Python 2.6rc2". This didn't take me to the What's New page. Instead it took me to: http://www.python.org/download/releases/2.6/NEWS.txt Seems a bit mislabelled if nothing else. Poking around I see that the 2.5 What's New document is at: http://docs.python.org/dev/whatsnew/whatsnew25.html but http://docs.python.org/dev/whatsnew/whatsnew26.html doesn't exist. Am I missing something? Skip

5 6

ssl module, non-blocking sockets and asyncore integration
by Giampaolo Rodola' 18 Sep '08

18 Sep '08

Hi, I try to revamp a discussion I raised some time ago and which seems to be not solved yet. I'm interested in using the ssl module with asyncore but since there's no real documentation about how using ssl module with non-blocking sockets I've not been able to write something useful with it. I took a look at Test/test_ssl.py which seems to include a test consisting in an asyncore-based secure server but I noticed something strange: def __init__(self, conn, certfile): asyncore.dispatcher_with_send.__init__(self, conn) self.socket = ssl.wrap_socket(conn, server_side=True, certfile=certfile, do_handshake_on_connect=True) I'm not sure about the real meaning of the do_handshake_on_connect parameter but if it does what I think it should block the application as long as the handshake isn't finished, which is not acceptable in an asynchronous environment. Am I misunderstanding something? Does the ssl module can actually be used with non-blocking sockets? If so which is the proper/recommended way to do that? Thanks in advance --- Giampaolo http://code.google.com/p/pyftpdlib/

4 12

FreeBSD 7 amd64 and large memory tests
by Andrew MacIntyre 18 Sep '08

18 Sep '08

I noticed issue 3862 (http://bugs.python.org/issue3862), which appears to be exposed only on FreeBSD 7.0 amd64 (perhaps 64 bit FreeBSD 7, but amd64 is the only 64bit hardware I can check). As far as I can make out from a couple of quick checks, this appears to be happening because the process is killed before the malloc() that exhausts the available swap space is allowed to report an out of memory condition. FreeBSD's default process virtual memory limit (ulimit -v) on both i386 and amd64 is "unlimited" for 6.3 and 7.0. Setting ulimit -v to a finite value on 7.0 amd64 seems to give rise to "normal" malloc() behaviour so that test_array passes. FreeBSD 7 uses a different malloc() implementation than FreeBSD 6 and earlier, and the new implementation uses mmap() in preference to sbrk(). So I suspect that behaviour was "normal" on FreeBSD 6 amd64 (can't test at the moment as I don't have that installed anywhere). Behaviour appears "normal" on FreeBSD 7 i386 as well. I haven't yet tried posting a query to a FreeBSD list, as it could simply be a bug on amd64, but I was wondering whether there was anything (other than deactivating tests and documenting use of ulimit -v on this platform) that could be done to work around this behaviour. Andrew. -- ------------------------------------------------------------------------- Andrew I MacIntyre "These thoughts are mine alone..." E-mail: andymac(a)bullseye.apana.org.au (pref) | Snail: PO Box 370 andymac(a)pcug.org.au (alt) | Belconnen ACT 2616 Web: http://www.andymac.org/ | Australia

6 10

RELEASED Python 2.6rc2 and 3.0rc1
by Barry Warsaw 18 Sep '08

18 Sep '08

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On behalf of the Python development team and the Python community, I am happy to announce the second and final planned release candidate for Python 2.6, as well as the first release candidate for Python 3.0. These are release candidates, so while they are not suitable for production environments, we strongly encourage you to download and test them on your software. We expect only critical bugs to be fixed between now and the final releases. Currently Python 2.6 is scheduled for October 1st, 2008. Python 3.0 release candidate 2 is planned for October 1st, with the final release planned for October 15, 2008. If you find things broken or incorrect, please submit bug reports at http://bugs.python.org For more information and downloadable distributions, see the Python 2.6 website: http://www.python.org/download/releases/2.6/ and the Python 3.0 web site: http://www.python.org/download/releases/3.0/ See PEP 361 for release schedule details: http://www.python.org/dev/peps/pep-0361/ Enjoy, - -Barry Barry Warsaw barry(a)python.org Python 2.6/3.0 Release Manager (on behalf of the entire python-dev team) -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSNHpw3EjvBPtnXfVAQLW9wP/RBCaUvhuheIh+BjLLIHQFBQi7D3uVgqi l0+4fhhoKGJvtWklLfSM9I1prcjH/d6tzUu4fIOjX7aM+wZSG++vkfmBoehnhyZW AvU9Lax4mqDwhOJA2QA0WMx0obpYYVHeUl7D1g9kWzbRUkZDX9NZGMWThhEOC1qA UA3bBYbvWiQ= =BFNH -----END PGP SIGNATURE-----

1 0

switching on -3 from within a program?
by Anthon van der Neut 17 Sep '08

17 Sep '08

With a minor modification to the Makefile I was able to get mod_wsgi v2.3 to work with python2.6rc1. I promptly got a warning in my apache log files on the depcrecated use of 'sha' in the paste's cookie module, good! And easily fixed. After that I was looking for a way to switch on the -3 warnings from within my code to have extra warnings show up in my apache log file. After reading some documentation I tried from the python2.6 prompt: import sys sys.py3kwarning = True x = { 'abc': 1 }; x.has_key('abc') which does not give a warning (starting python2.6 with the -3 option of course does). Is there anyway to switch this on from within a program with a Python statement? If not, would I need to make a small C-extension module (to be imported as the first module) that sets Py_Py3WarningFlag or something else at the C level, or would that better be done by mod_wsgi's C code. Regards Anthon

6 5

RELEASED Python 2.6rc1
by Barry Warsaw 17 Sep '08

17 Sep '08

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On behalf of the Python development team and the Python community, I am happy to announce the first release candidate for Python 2.6. This is a release candidate, so while it is not suitable for production environments, we strongly encourage you to download the release and test it on your software. We expect only critical bugs to be fixed between now and the final 2.6 release, still scheduled for October 1st, 2008. There is one more release candidate planned for September 17th. You might notice that unlike earlier releases, we are /not/ releasing Python 3.0rc1 at this time. It was decided that 3.0 still needs time to resolve open issues and that we would not hold up the 2.6 release for this. We feel that Python 2.6 is nearly ready for its final release. If you find things broken or incorrect, please submit bug reports at http://bugs.python.org For more information and downloadable distributions, see the Python 2.6 website: http://www.python.org/download/releases/2.6/ (Note that the Windows installers will be uploaded shortly.) See PEP 361 for release schedule details: http://www.python.org/dev/peps/pep-0361/ Enjoy, - -Barry Barry Warsaw barry(a)python.org Python 2.6/3.0 Release Manager (on behalf of the entire python-dev team) -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSMsXV3EjvBPtnXfVAQJFsgP9GxZYQocbDTd0Z/0yEjpHfZ/FTd8y83jV 5JouO07lB8XtLawnWB9hF8sUrCuBVog5He3mLVUPDmlyn30qvjYWMG2J6zW0yYMX yZdjUyUmta0IMCsXe7YXj369xebh4nWuwG4tDygly4donA7GYPXAlxI48MmyDJxw 1v07LM4Dttw= =Nd3s -----END PGP SIGNATURE-----

3 3

Re: [Python-Dev] ',' precedence in documentation]
by Christopher Lee 16 Sep '08

16 Sep '08

> ----- Forwarded message from Guido van Rossum <guido(a)python.org> ----- > > Note that in any case, the 'in' operator binds more tightly than the > comma: e.g. f(x in y, z) means f((x in y), z). Exactly the point I was trying to make. As far as the user's concerned, an expression list x,y can itself be part of an expression; x in y is also an expression. Such expressions can be combined to form larger expressions, so it follows logically that you need to know which has precedence when *both* occur in an expression, e.g. whether x in y,z evaluates as (x in y),z or x in (y,z) The page that Fredrik sent you to (http://docs.python.org/ref/exprlists.html ) doesn't address that question. I still think the precedence table (http://docs.python.org/ref/summary.html ) should show that "in" has higher precedence than comma in an expression. Can anyone show us "where it is written" in the Python docs that "in" has higher precedence than comma, *and* why there is a good reason that this information should NOT be included in the precedence table? -- Chris Adding comma to the precedence table would also disambiguate expressions like: x and y,z x,y or z not x,y

8 18

Re: [Python-Dev] [Python-checkins] Python Regression Test Failures basics (1)
by Nick Coghlan 15 Sep '08

15 Sep '08

Neal Norwitz wrote: > test_epoll skipped -- kernel doesn't support epoll() ... > test_ioctl skipped -- Unable to open /dev/tty ... > test_multiprocessing skipped -- OSError raises on RLock creation, see issue 3111! ... > test test_normalization failed -- Traceback (most recent call last): > File "/tmp/python-test/local/lib/python2.6/test/test_normalization.py", line 90, in test_main > self.failUnless(X == NFC(X) == NFD(X) == NFKC(X) == NFKD(X), c) > AssertionError: 6918 ... > 326 tests OK. > 1 test failed: > test_normalization ... > 3 skips unexpected on linux2: > test_epoll test_multiprocessing test_ioctl Neal, What environment are you using to run the debug-mode regression tests? The above four tests run without any problems for me, but I'm just running them in a normal Kubuntu desktop shell. Cheers, Nick. P.S. I'm in the process of creating a --with-pydebug build to see if that makes any difference. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org

3 4

Weak Dictionary Iteration Behavior in Python 3
by Armin Ronacher 15 Sep '08

15 Sep '08

Hi everybody, In Python 2.x when iterating over a weak key dictionary for example, the common idom for doing that was calling dictionary.keys() to ensure that a list of all objects is returned it was safe to iterate over as a weak reference could stop existing during dict iteration which of course raises a runtime error by the dict iterator. This was documented behavior and worked pretty well, with the small problem that suddenly all references in the dict wouldn't die until iteration is over because the list holds references to the object. This no longer works in Python 3 because .keys() on the weak key dictionary returns a generator over the key view of the internal dict which of course has the same problem as iterkeys in Python 2.x. The following code shows the problem:: from weakref import WeakKeyDictionary f1 = Foo() f2 = Foo() d = WeakKeyDictionary() d[f1] = 42 d[f2] = 23 i = iter(d.keys()) # or use d.keyrefs() here which has the same problem print(next(i)) del f2 print(next(i)) This example essentially dies with "RuntimeError: dictionary changed size during iteration" as soon as f2 is deleted. Iterating over weak key dictionaries might not be the most common task but I know some situations where this is necessary. Unfortunately I can't see a way to achieve that in Python 3. Regards, Armin

9 15

Do we still support MacOS < X?
by Georg Brandl 14 Sep '08

14 Sep '08

If not, I'll remove the traces from the docs, where they only serve to confuse where MacOS X actually belongs under "Unix", not "Mac". Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out.

4 3

Possible bug in array?
by Reed O'Brien 13 Sep '08

13 Sep '08

or in FreeBSD? 2.6rc1 and 3.0b3 both fail test_array on FreeBSD7 amd64 test_array passes in 2.5.2 on the same machine but fails test_list the same as test_array... *** Signal 9 Not sure if I should file this as a bug or not. I didn't see anything on bugs.pyhton.org... ~ro

2 1

2.6 rc1 performance results
by A.M. Kuchling 13 Sep '08

13 Sep '08

Three weeks ago, Antoine Pitrou posted the pybench results for 2.6 trunk: http://mail.python.org/pipermail/python-dev/2008-August/081951.html The big discovery in those results were TryExcept being 48% slower, but there was a patch in the bug tracker to improve things. I've re-run the tests to check the results. Disclaimer: these results are probably not directly comparable. Antoine was using a "32-bit Linux installation on an Athlon 3600+ X2"; I'm on a Macbook. Good news: TryExcept is now only 10% slower than 2.5, not 48%. Bad news: the big slowdowns are: CompareFloats: 117ms 98ms +19.2% 118ms 99ms +19.0% CompareIntegers: 110ms 104ms +5.6% 110ms 105ms +4.9% DictWithStringKeys: 118ms 105ms +12.8% 133ms 108ms +22.7% NestedForLoops: 125ms 116ms +7.7% 127ms 118ms +8.0% Recursion: 193ms 159ms +21.5% 197ms 163ms +20.8% SecondImport: 139ms 129ms +8.4% 143ms 130ms +9.9% SecondPackageImport: 150ms 139ms +8.6% 152ms 140ms +8.1% SecondSubmoduleImport: 211ms 191ms +10.5% 214ms 195ms +9.4% SimpleComplexArithmetic: 130ms 119ms +9.4% 131ms 120ms +9.2% Antoine, your Recursion results were actually about the same (+2.2%) from 2.5 to 2.6, so this big slowdown is novel. I wonder if these tests are simply slower on MacOS for some reason (compiler, CPU cache size, etc.). Does anyone see similar results? Any idea what might have made DictForStringKeys and Recursion slow down? Complete results: Test minimum run-time average run-time this other diff this other diff ------------------------------------------------------------------------------- BuiltinFunctionCalls: 140ms 148ms -5.4% 142ms 153ms -7.5% BuiltinMethodLookup: 120ms 135ms -11.2% 122ms 137ms -11.0% CompareFloats: 117ms 98ms +19.2% 118ms 99ms +19.0% CompareFloatsIntegers: 109ms 119ms -8.9% 109ms 121ms -9.3% CompareIntegers: 110ms 104ms +5.6% 110ms 105ms +4.9% CompareInternedStrings: 128ms 153ms -16.3% 131ms 158ms -16.8% CompareLongs: 102ms 99ms +3.5% 105ms 101ms +3.9% CompareStrings: 164ms 161ms +2.0% 166ms 165ms +0.7% CompareUnicode: 141ms 158ms -10.5% 143ms 164ms -12.6% ComplexPythonFunctionCalls: 159ms 272ms -41.3% 164ms 277ms -40.6% ConcatStrings: 173ms 168ms +3.2% 177ms 172ms +3.1% ConcatUnicode: 108ms 121ms -10.8% 111ms 124ms -10.4% CreateInstances: 168ms 180ms -6.4% 176ms 182ms -3.7% CreateNewInstances: 129ms 153ms -15.6% 132ms 158ms -16.0% CreateStringsWithConcat: 156ms 157ms -0.7% 158ms 161ms -1.9% CreateUnicodeWithConcat: 112ms 114ms -1.8% 114ms 117ms -2.2% DictCreation: 104ms 112ms -7.1% 106ms 114ms -7.2% DictWithFloatKeys: 149ms 162ms -7.7% 153ms 168ms -8.7% DictWithIntegerKeys: 123ms 148ms -16.8% 127ms 151ms -15.9% DictWithStringKeys: 118ms 105ms +12.8% 133ms 108ms +22.7% ForLoops: 91ms 88ms +3.6% 91ms 88ms +3.0% IfThenElse: 108ms 102ms +5.2% 109ms 103ms +5.5% ListSlicing: 155ms 239ms -35.0% 157ms 241ms -34.6% NestedForLoops: 125ms 116ms +7.7% 127ms 118ms +8.0% NormalClassAttribute: 135ms 140ms -3.8% 139ms 146ms -4.7% NormalInstanceAttribute: 123ms 126ms -2.4% 125ms 130ms -4.4% PythonFunctionCalls: 126ms 126ms +0.0% 129ms 128ms +0.9% PythonMethodCalls: 165ms 165ms -0.1% 168ms 170ms -1.1% Recursion: 193ms 159ms +21.5% 197ms 163ms +20.8% SecondImport: 139ms 129ms +8.4% 143ms 130ms +9.9% SecondPackageImport: 150ms 139ms +8.6% 152ms 140ms +8.1% SecondSubmoduleImport: 211ms 191ms +10.5% 214ms 195ms +9.4% SimpleComplexArithmetic: 130ms 119ms +9.4% 131ms 120ms +9.2% SimpleDictManipulation: 124ms 146ms -14.6% 128ms 150ms -14.8% SimpleFloatArithmetic: 127ms 132ms -3.6% 131ms 144ms -9.3% SimpleIntFloatArithmetic: 93ms 100ms -6.5% 94ms 100ms -5.6% SimpleIntegerArithmetic: 94ms 91ms +2.8% 95ms 92ms +3.1% SimpleListManipulation: 108ms 110ms -1.1% 110ms 111ms -1.2% SimpleLongArithmetic: 141ms 136ms +3.8% 143ms 139ms +2.8% SmallLists: 157ms 151ms +4.3% 160ms 156ms +2.6% SmallTuples: 123ms 123ms -0.3% 125ms 127ms -1.5% SpecialClassAttribute: 136ms 152ms -10.5% 140ms 155ms -10.2% SpecialInstanceAttribute: 225ms 256ms -11.9% 227ms 258ms -11.8% StringMappings: 197ms 206ms -4.4% 201ms 209ms -3.8% StringPredicates: 154ms 194ms -20.5% 157ms 196ms -20.2% StringSlicing: 144ms 146ms -1.1% 152ms 155ms -1.9% TryExcept: 111ms 101ms +9.5% 114ms 102ms +11.2% TryFinally: 120ms 129ms -6.9% 122ms 131ms -6.8% TryRaiseExcept: 145ms 150ms -4.0% 148ms 153ms -3.2% TupleSlicing: 141ms 140ms +1.3% 152ms 159ms -4.5% UnicodeMappings: 140ms 133ms +5.9% 143ms 136ms +5.5% UnicodePredicates: 123ms 137ms -9.9% 125ms 138ms -9.9% UnicodeProperties: 136ms 146ms -6.7% 142ms 151ms -6.3% UnicodeSlicing: 132ms 129ms +2.0% 134ms 131ms +2.2% WithFinally: 167ms 168ms -1.0% 172ms 173ms -0.3% WithRaiseExcept: 134ms 150ms -11.1% 140ms 156ms -9.8% ------------------------------------------------------------------------------- Totals: 7659ms 8031ms -4.6% 7834ms 8229ms -4.8% (this=p26.pybench, other=p25.pybench) --amk

7 7

Re: [Python-Dev] Add python.exe to PATH environment variable
by Henning von Bargen 13 Sep '08

13 Sep '08

One of the things I like about Python is that * it doesn't actually need an installation. * It is sufficient to set up PATH (and, if you like, PYTHONPATH) accordingly to start python. However, you need the DLLs somewhere in the Python directory, too. That way it is possible to install Python applications in a minimal-invasive fashion: Just copy one directory to the hard-disk (or a network drive) and create a shortcut to some start-app.cmd script For example, this is a great advantage if you need to install a Python application on a client's application server, since the client's IT departments in reponsibility for the server absolutely don't like it when programs modify PATH or install DLLs in the %WINDIR%\System32 directory - and they are right here!!! Think of what happens if a program modifies PATH: The modification does not affect running services, of course. But *after a reboot* (possibly a few months later), the old services will find another version of some program than before, and then someting is going to crash! I have seen too much trouble of this kind after the installation of a different Perl version or a another Oracle product. The reason is then quite hard to find: if the installation of another software was 3 months ago, will anyone remember? And will anyone even consider it could be the reason for the fault after the reboot? Well - I will, but most others probably won't. I don't want this kind of trouble too happen with Python! So, please try keeping the Python installation as minimal invasive as possible. Note: This could also be an option for the installation. Henning

1 0

Summary of Python tracker Issues
by Python tracker 12 Sep '08

12 Sep '08

ACTIVITY SUMMARY (09/05/08 - 09/12/08) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 2019 open (+32) / 13652 closed (+30) / 15671 total (+62) Open issues with patches: 637 Average duration of open issues: 716 days. Median duration of open issues: 1772 days. Open Issues Breakdown open 2005 (+31) pending 14 ( +1) Issues Created Or Reopened (67) _______________________________ readline configuration for shared libs w/o curses dependencies 09/07/08 CLOSED http://bugs.python.org/issue1204 reopened gregory.p.smith patch, 64bit py3k fails under Windows if "-c" or "-m" is given a non-ascii va 09/09/08 http://bugs.python.org/issue3705 reopened amaury.forgeotdarc patch asyncore differences between 2.x and 3.x 09/07/08 CLOSED http://bugs.python.org/issue3764 reopened gregory.p.smith easy tkinter Menu.delete bug 09/12/08 http://bugs.python.org/issue3774 reopened gpolo patch, easy, needs review warnings.catch_warnings fails gracelessly when recording warning 09/09/08 http://bugs.python.org/issue3781 reopened ncoghlan patch Make PyInstanceMethod_Type available or remove it 09/05/08 http://bugs.python.org/issue3787 created christian.heimes patch, patch, needs review test_cookie isn't comprehensive 09/05/08 http://bugs.python.org/issue3788 created benjamin.peterson easy multiprocessing deadlocks when sending large data through Queue 09/05/08 CLOSED http://bugs.python.org/issue3789 created DavidDecotigny in zlib decompressor objects, unused_data and unconsumed_tail mu 09/06/08 CLOSED http://bugs.python.org/issue3790 created pitrou patch, patch, needs review bsddb not completely removed 09/06/08 http://bugs.python.org/issue3791 created amaury.forgeotdarc patch, patch Module variable overridden in child processes with multiprocessi 09/06/08 CLOSED http://bugs.python.org/issue3792 created TarkaSteve Small RST fix in datamodel.rst 09/06/08 CLOSED http://bugs.python.org/issue3793 created mgiuca __div__ still documented in Python 3 09/06/08 CLOSED http://bugs.python.org/issue3794 created mgiuca patch wsgiref.simple_server fails to run demo_app 09/06/08 CLOSED http://bugs.python.org/issue3795 created Walling some tests are not run in test_float 09/06/08 CLOSED http://bugs.python.org/issue3796 created amaury.forgeotdarc patch, patch, needs review mmap, dbm, ossaudiodev, marshal & winreg return bytearray instea 09/06/08 CLOSED http://bugs.python.org/issue3797 created gregory.p.smith patch, patch, easy, needs review SystemExit incorrectly displays unicode message 09/06/08 http://bugs.python.org/issue3798 created amaury.forgeotdarc Byte/string inconsistencies between different dbm modules 09/06/08 http://bugs.python.org/issue3799 created skip.montanaro patch Fix for formatter.py 09/07/08 CLOSED http://bugs.python.org/issue3800 created skomoroh patch cgi.parse_qsl does not return list 09/07/08 CLOSED http://bugs.python.org/issue3801 created acapnotic patch smtpd.py __getaddr insufficient handling 09/08/08 http://bugs.python.org/issue3802 created marcus(a)internetnowasp.net Comparison operators - New rules undocumented in Python 3.0 09/08/08 CLOSED http://bugs.python.org/issue3803 created mgiuca patch Test for issue2222 (r65745) 09/08/08 CLOSED http://bugs.python.org/issue3804 created ocean-city patch sslobj.read py3k takes odd arguments 09/08/08 http://bugs.python.org/issue3805 created gregory.p.smith LockTests in test_imp should be skipped when thread is not avail 09/08/08 CLOSED http://bugs.python.org/issue3806 created ocean-city patch _multiprocessing build fails when configure --without-threads 09/08/08 http://bugs.python.org/issue3807 created ocean-city patch, needs review test_cgi is giving deprecation warnings 09/09/08 CLOSED http://bugs.python.org/issue3808 created benjamin.peterson test_logging leaving a 'test.blah' file behind 09/09/08 CLOSED http://bugs.python.org/issue3809 created brett.cannon os.chdir() et al: is the path str or bytes? 09/09/08 http://bugs.python.org/issue3810 created ceder Update Unicode database to 5.1.0 09/09/08 CLOSED http://bugs.python.org/issue3811 created loewis patch, needs review py3k build fails if configure --without-threads 09/09/08 CLOSED http://bugs.python.org/issue3812 created ocean-city patch cannot lanch python.exe via symbolic link if HAVE_BROKEN_MBSTOWC 09/09/08 CLOSED http://bugs.python.org/issue3813 created ocean-city patch Add VCS support 09/09/08 CLOSED http://bugs.python.org/issue3814 created sdouche patch Python 3.0b3 - Idle doesn't start on win XPh 09/09/08 CLOSED http://bugs.python.org/issue3815 created vbr __newobj__ pickle feature is not documented 09/09/08 http://bugs.python.org/issue3816 created christian.heimes easy ftplib: ABOR does not consider 225 response code 09/09/08 http://bugs.python.org/issue3817 created giampaolo.rodola patch ftplib.FTP.abort() should consider "225" a positive response cod 09/09/08 CLOSED http://bugs.python.org/issue3818 created giampaolo.rodola patch urllib2 sends Basic auth across redirects 09/09/08 http://bugs.python.org/issue3819 created TFKyle Python 3.0b3 doesn't start on German Win XP SP3/SP2 09/09/08 http://bugs.python.org/issue3820 created pietzcker trace module bug when using --missing 09/09/08 http://bugs.python.org/issue3821 created jjdominguezm patch, needs review zfill doc string uses inconsistent variable names 09/09/08 CLOSED http://bugs.python.org/issue3822 created tjd ssl.wrap_socket() is incompatible with servers that drop privile 09/09/08 http://bugs.python.org/issue3823 created forest test_tarfile fails on cygwin (unicode decode error) 09/09/08 http://bugs.python.org/issue3824 created ocean-city patch Major reworking of Python 2.5.2 re module 09/09/08 http://bugs.python.org/issue3825 created mrabarnett patch Self-reference in BaseHTTPRequestHandler descendants causes stuc 09/09/08 http://bugs.python.org/issue3826 created romkyns memoryview.size is redundant 09/09/08 CLOSED http://bugs.python.org/issue3827 created benjamin.peterson patch, needs review Bound methods compare 'successfully' with ints 09/10/08 CLOSED http://bugs.python.org/issue3828 created inducer Tuple comparison masking exception 09/10/08 http://bugs.python.org/issue3829 created fdrake Tarfile has incorrect USTAR "VERSION" field (should be 00; is 0 09/10/08 CLOSED http://bugs.python.org/issue3830 created SunriseProgrammer Multiprocessing: Expose underlying pipe in queues 09/11/08 http://bugs.python.org/issue3831 created TarkaSteve add force_shared Library option to create shared lib even with u 09/11/08 CLOSED http://bugs.python.org/issue3832 created vajda python-2.6b3.msi and python-2.6b3.amd64.msi can't both be instal 09/11/08 http://bugs.python.org/issue3833 created jretz wsgiref.validator.InputWrapper readline method has wrong signatu 09/11/08 CLOSED http://bugs.python.org/issue3834 created strogon14 tkinter goes into an infinite loop (pydoc.gui) 09/11/08 http://bugs.python.org/issue3835 created HWJ 2to3 broken due to mixed 2.5 and 3.0 syntax 09/11/08 CLOSED http://bugs.python.org/issue3836 created bhy patch Comment for html_logo setting is out of date 09/11/08 CLOSED http://bugs.python.org/issue3837 created jmswisher test_tarfile error on cygwin (Directory not empty) 09/11/08 http://bugs.python.org/issue3838 created ocean-city patch, easy wsgi.simple_server resets 'Content-Length' header on empty conte 09/11/08 http://bugs.python.org/issue3839 created nosklo patch if TESTFN == "/tmp/@test", some tests fail 09/11/08 http://bugs.python.org/issue3840 created ocean-city patch IDLE: quirky behavior when displaying strings longer than 4093 c 09/11/08 http://bugs.python.org/issue3841 created spmcinerney can't run close through itertools.chain on inner generator 09/11/08 http://bugs.python.org/issue3842 created dangyogi hexadecimal, not decimal 09/11/08 CLOSED http://bugs.python.org/issue3843 created maix Script: find untested C functions 09/12/08 http://bugs.python.org/issue3844 created ajaksu2 memory access before short string when checking suffix 09/12/08 http://bugs.python.org/issue3845 created doko sqlite3 module: Improved concurrency 09/12/08 http://bugs.python.org/issue3846 created ghaering needs review Begging for review 09/12/08 http://bugs.python.org/issue3847 created ghaering select.epoll calling register with the same fd fails 09/12/08 http://bugs.python.org/issue3848 created ionel.mc Issues Now Closed (73) ______________________ readline configuration for shared libs w/o curses dependencies 0 days http://bugs.python.org/issue1204 gregory.p.smith patch, 64bit smtplib.SMTP docs 319 days http://bugs.python.org/issue1317 akuchling New SSL module doesn't seem to verify hostname against commonNam 275 days http://bugs.python.org/issue1589 janssen Tools/i18n/msgfmt.py fixes for Python 3.0 237 days http://bugs.python.org/issue1840 loewis patch ctypes.util.find_library(): posix .so without SONAME 202 days http://bugs.python.org/issue2145 theller Faq 4.28 -- Trailing comas 172 days http://bugs.python.org/issue2420 akuchling Cannot use non-ascii letters in disutils if setuptools is used. 156 days http://bugs.python.org/issue2562 lemburg patch Documentation of new gobject types fails 132 days http://bugs.python.org/issue2747 georg.brandl sphinx and virtualenv 118 days http://bugs.python.org/issue2859 georg.brandl optparse documentation: variable arguments example doesn't work 93 days http://bugs.python.org/issue3040 akuchling ssl.SSLSocket implements methods that are overriden by socket.so 79 days http://bugs.python.org/issue3162 hodgestar patch import of site.py fails on startup 64 days http://bugs.python.org/issue3279 benjamin.peterson patch, needs review Documentation: timeit: "lower bound" should read "upper bound" 64 days http://bugs.python.org/issue3318 georg.brandl Use Python 3 lexer for 3.0 docs 55 days http://bugs.python.org/issue3376 georg.brandl patch Updates to "Macintosh Library Modules" Section 1.1 41 days http://bugs.python.org/issue3472 georg.brandl Zlib compress/decompress functions returning bytearray 37 days http://bugs.python.org/issue3492 pythonhacker patch, easy, needs review zipfile has problem reading zip files over 2GB 27 days http://bugs.python.org/issue3535 pitrou patch, needs review test_unicode.test_raiseMemError fails in UCS4 17 days http://bugs.python.org/issue3601 pitrou patch, needs review broken link in curses module docs 18 days http://bugs.python.org/issue3604 akuchling 2to3: commands varible replaced by subprocess 20 days http://bugs.python.org/issue3606 benjamin.peterson Python won't compile a regex that compiles with 2.5.2 and 30b2 20 days http://bugs.python.org/issue3629 gvanrossum patch, needs review invalid result value of _weakref.__init__() 19 days http://bugs.python.org/issue3634 benjamin.peterson patch test_cpickle crash on AMD64 Windows build 21 days http://bugs.python.org/issue3640 amaury.forgeotdarc patch, 64bit Objects/obmalloc.c:529: warning: comparison is always false due 17 days http://bugs.python.org/issue3642 loewis patch, 64bit readline module Crashs on OpenBSD/amd64 16 days http://bugs.python.org/issue3645 gregory.p.smith patch, 64bit, easy fix for pychecker property complaints 17 days http://bugs.python.org/issue3658 benjamin.peterson patch, easy, needs review sqlite3.Connection.iterdump docs pythonicity 13 days http://bugs.python.org/issue3669 akuchling patch os.urandom(1.1): infinite loop 12 days http://bugs.python.org/issue3708 benjamin.peterson patch, easy PY_FORMAT_SIZE_T is not for PyString_FromFormat 10 days http://bugs.python.org/issue3743 benjamin.peterson patch Sphinx producing duplicate id attributes, HTML fails validation. 9 days http://bugs.python.org/issue3746 georg.brandl re.escape() does not work with bytes() 9 days http://bugs.python.org/issue3756 andrewmcnamara "make check" suggest a testing target under GNU coding standards 3 days http://bugs.python.org/issue3758 brett.cannon patch asyncore differences between 2.x and 3.x 0 days http://bugs.python.org/issue3764 giampaolo.rodola easy Deprecate bsddb for removal in 3.0 2 days http://bugs.python.org/issue3769 brett.cannon patch, patch, needs review deprecate bsddb/dbhash in 2.6 for removal in 3.0 1 days http://bugs.python.org/issue3776 brett.cannon patch, patch, needs review long(4.2) now returns an int 5 days http://bugs.python.org/issue3777 skip.montanaro patch Incorrect compiler options used for cc of Sun Studio 12 3 days http://bugs.python.org/issue3784 loewis Can't build ctypes of Python 2.5.2 with Sun Studio 12 0 days http://bugs.python.org/issue3785 theller multiprocessing deadlocks when sending large data through Queue 0 days http://bugs.python.org/issue3789 jnoller in zlib decompressor objects, unused_data and unconsumed_tail mu 1 days http://bugs.python.org/issue3790 gregory.p.smith patch, patch, needs review Module variable overridden in child processes with multiprocessi 0 days http://bugs.python.org/issue3792 TarkaSteve Small RST fix in datamodel.rst 0 days http://bugs.python.org/issue3793 georg.brandl __div__ still documented in Python 3 0 days http://bugs.python.org/issue3794 georg.brandl patch wsgiref.simple_server fails to run demo_app 0 days http://bugs.python.org/issue3795 pitrou some tests are not run in test_float 0 days http://bugs.python.org/issue3796 amaury.forgeotdarc patch, patch, needs review mmap, dbm, ossaudiodev, marshal & winreg return bytearray instea 0 days http://bugs.python.org/issue3797 gregory.p.smith patch, patch, easy, needs review Fix for formatter.py 2 days http://bugs.python.org/issue3800 georg.brandl patch cgi.parse_qsl does not return list 0 days http://bugs.python.org/issue3801 facundobatista patch Comparison operators - New rules undocumented in Python 3.0 2 days http://bugs.python.org/issue3803 georg.brandl patch Test for issue2222 (r65745) 0 days http://bugs.python.org/issue3804 ocean-city patch LockTests in test_imp should be skipped when thread is not avail 0 days http://bugs.python.org/issue3806 ocean-city patch test_cgi is giving deprecation warnings 0 days http://bugs.python.org/issue3808 facundobatista test_logging leaving a 'test.blah' file behind 1 days http://bugs.python.org/issue3809 brett.cannon Update Unicode database to 5.1.0 2 days http://bugs.python.org/issue3811 loewis patch, needs review py3k build fails if configure --without-threads 0 days http://bugs.python.org/issue3812 ocean-city patch cannot lanch python.exe via symbolic link if HAVE_BROKEN_MBSTOWC 0 days http://bugs.python.org/issue3813 ocean-city patch Add VCS support 0 days http://bugs.python.org/issue3814 pitrou patch Python 3.0b3 - Idle doesn't start on win XPh 0 days http://bugs.python.org/issue3815 vbr ftplib.FTP.abort() should consider "225" a positive response cod 0 days http://bugs.python.org/issue3818 georg.brandl patch zfill doc string uses inconsistent variable names 0 days http://bugs.python.org/issue3822 georg.brandl memoryview.size is redundant 1 days http://bugs.python.org/issue3827 amaury.forgeotdarc patch, needs review Bound methods compare 'successfully' with ints 0 days http://bugs.python.org/issue3828 amaury.forgeotdarc Tarfile has incorrect USTAR "VERSION" field (should be 00; is 0 0 days http://bugs.python.org/issue3830 lars.gustaebel add force_shared Library option to create shared lib even with u 0 days http://bugs.python.org/issue3832 loewis wsgiref.validator.InputWrapper readline method has wrong signatu 0 days http://bugs.python.org/issue3834 pje 2to3 broken due to mixed 2.5 and 3.0 syntax 0 days http://bugs.python.org/issue3836 benjamin.peterson patch Comment for html_logo setting is out of date 0 days http://bugs.python.org/issue3837 georg.brandl hexadecimal, not decimal 0 days http://bugs.python.org/issue3843 benjamin.peterson -S hides standard dynamic modules 2240 days http://bugs.python.org/issue586680 benjamin.peterson threading module can deadlock after fork 20 days http://bugs.python.org/issue874900 gregory.p.smith patch, needs review SSLObject breaks read semantics 1087 days http://bugs.python.org/issue1291446 ellisj Search is to long with regex like ^(.+|dontmatch)*$ 1084 days http://bugs.python.org/issue1297193 pitrou Add httponly to Cookie module 598 days http://bugs.python.org/issue1638033 benjamin.peterson patch Top Issues Most Discussed (10) ______________________________ 31 warnings.catch_warnings fails gracelessly when recording warnin 3 days open http://bugs.python.org/issue3781 25 dbm.sqlite proof of concept 8 days open http://bugs.python.org/issue3783 11 long(4.2) now returns an int 5 days closed http://bugs.python.org/issue3777 10 Update Unicode database to 5.1.0 2 days closed http://bugs.python.org/issue3811 10 ssl.SSLSocket implements methods that are overriden by socket.s 79 days closed http://bugs.python.org/issue3162 9 tkinter Menu.delete bug 0 days open http://bugs.python.org/issue3774 9 socket.socket.recv broken (unbearably slow) 9 days open http://bugs.python.org/issue3766 9 py3k fails under Windows if "-c" or "-m" is given a non-ascii v 3 days open http://bugs.python.org/issue3705 9 readline configuration for shared libs w/o curses dependencies 0 days closed http://bugs.python.org/issue1204 8 ssl.wrap_socket() is incompatible with servers that drop privil 3 days open http://bugs.python.org/issue3823

1 0

Updated release schedule for 2.6 and 3.0
by Barry Warsaw 12 Sep '08

12 Sep '08

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 We had a lot of discussion recently about changing the release schedule and splitting Python 2.6 and 3.0. There was general consensus that this was a good idea, in order to hit our October 1 deadline for Python 2.6 final at least. There is only one open blocker for 2.6, issue 3657. Andrew, Fred, Tim and I (via IRC) will be getting together tonight to do some Python hacking, so we should resolve this issue and release 2.6rc1 tonight. We'll have an abbreviated 2.6rc1, and I will release 2.6rc2 next Wednesday the 17th as planned. The final planned release of 2.6 will be Wednesday October 1st. If 3.0 is looking better, I will release 3.0rc1 on Wednesday, otherwise we'll re-evaluate the release schedule for 3.0 as necessary. This means currently the schedule looks like this: Fri 12-Sep 2.6rc1 Wed 17-Sep 2.6rc2, 3.0rc1 Wed 01-Oct 2.6 final, 3.0rc2 Wed 15-Oct 3.0 final I've updated the Python Release Schedule gcal and will update the PEP momentarily. We'll close the tree later tonight (UTC-4). - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSMpmfHEjvBPtnXfVAQJUiQP/eTXStyp9M0+Ja7iAFfYcpzfM19j9ddBr ocMC+KTDSgci5/rmFw4KdMhqO9TBk2sXIdCd9GInnuMOKtndCKZ/PXaexnVvSVGb P3CpkkMs/vG1itQIc/EXq6CUhzwuxEv9h8Wo8+zcmL05Cc1IrE5d2OYiO0+KQ8ei lW+j/aNKMWY= =w2oI -----END PGP SIGNATURE-----

2 2

Python 3.0: can we make dbm's .keys() return an iterator?
by Gerhard Häring 12 Sep '08

12 Sep '08

As far as I can see, the specification of the dbm interface is the module docstring in dbm/__init__.py, which reads: """ [...] It has the following interface (key and data are strings): d[key] = data # store data at key (may override data at # existing key) data = d[key] # retrieve data at key (raise KeyError if no # such key) del d[key] # delete data stored at key (raises KeyError # if no such key) flag = key in d # true if the key exists list = d.keys() # return a list of all existing keys (slow!) """ Now I thought that in Python 3.0, keys(), values() and friends should return iterators. Can we change at least the specification of the dbm module? We could then later in 3.1 change the implementations to return iterators instead of lists, too. I stumbled upon it cos I'm trying to help Skip with the SQLite-based implementation. -- Gerhard

2 1

HTTPS read-only SVN access is denied?
by techtonik 11 Sep '08

11 Sep '08

SVN checkout over HTTPS protocol requires password. Is it intentional or just temporary server issue? I am behind a proxy that doesn't support PROPFIND requests and I can't test SVN+SSH, because 22 port is closed. Site docs keep silence about that HTTPS is used at all. Shouldn't authentication be asked only on write access in normal SVN setup? BTW, link to "current svn tree" is broken here - http://www.python.org/dev/patches/ -- --anatoly t.

8 13

?spurious? Timeout in BSDDB under MS Windows
by Jesus Cea 10 Sep '08

10 Sep '08

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Trent, are you available to look at the ?spurious? timeout failures in bsddb replication code in the Windows buildbot?. Ten seconds timeout should be plenty enough. I can't debug any MS Windows issue myself; this is a Microsoft-free environment. - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea(a)jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBSMFWXJlgi5GaxT1NAQK3BwQAlG532RIfCKRLRwQWYqUnL87F4ITxgu7W 6QgzIHoCvl/oA5G0MsaSJ6Qh3XdutH4RjovxH7Ky3rQ08lFa7iyg6mHuzkBTDbri Pb7jgWHxQ/FImSD1rxqJHrLpKvRNoR0zjyDUVlEsgLElJ6wKGz4/eclJh0/0Wylv QdTISCI4rR4= =HBAA -----END PGP SIGNATURE-----

3 2

new ssl module is incompatible with servers that drop privileges
by Forest 09 Sep '08

09 Sep '08

I've been trying out the new ssl module, and I love it so far, except for the way it accepts private keys and certificates. It accept them only as paths to their location on the file system, which I believe means that a server can only support SSL if it has read permission to its private key file when client connections arrive. This is a problem for servers that bind to their socket and drop privileges as soon as they start up, a practice that is both common and recommended in the unix world. IMHO, this severely limits the new ssl module's utility, and discourages good security practices. Wouldn't it be better if we could specify keys and certificates as bytes or file-like objects? This would solve the security issue, give applications more flexibility in key management, and might also improve performance slightly (by avoiding file system operations at accept() time). Perhaps there's a workaround that I haven't noticed yet? A quick look at the source code didn't reveal any obvious way to specify keys other than as paths in the file system. http://bugs.python.org/issue3823

2 2

Not releasing rc1 tonight
by Barry Warsaw 09 Sep '08

09 Sep '08

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I'm not going to release rc1 tonight. There are too many open release blockers that I don't want to defer, and I'd like the buildbots to churn through the bsddb removal on all platforms. Let me first thank Benjamin, Brett, Mark and Antoine for their help on IRC tonight. Here are the issues I'm not comfortable with deferring: 3640 test_cpickle crash on AMD64 Windows build 874900 threading module can deadlock after fork 3574 compile() cannot decode Latin-1 source encodings 3657 pickle can pickle the wrong function 3187 os.listdir can return byte strings 3660 reference leaks in 3.0 3594 PyTokenizer_FindEncoding() never succeeds 3629 Py30b3 won't compile a regex that compiles with 2.5.2 and 30b2 In addition, Mark reported in IRC that there are some regressions in the logging module. I appreciate any feedback or fixes you can provide on these issues. You might also want to look at the deferred blockers to see if there's anything that really should be blocking rc1. I'd like to try again on Friday and stick to rc2 on the 17th. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSL9Y53EjvBPtnXfVAQJGXwP+JZUa5EWlQh7yzt7aFdEM3qgiFZnKqWhz TN4Cen0/eK8c4+t8a5WC+OLvc/P3PhMPhLSnE+g6IqQUO+pt+2LANgpAvCUrUahc Nk2pt3gCclcmWlzVvCBspVPZjFPkHsW0uVhgK6x1C/2Re90yjeBqPGgT4LGlmaR3 bz6A3iiUnk0= =Y5aN -----END PGP SIGNATURE-----

17 43

Re: [Python-Dev] [Python-checkins] r66321 - in python/trunk: Doc/library/warnings.rst Lib/asynchat.py Lib/bsddb/test/test_early_close.py Lib/mimetools.py Lib/test/test___all__.py Lib/test/test_exceptions.py Lib/test/test_hmac.py Lib/test/test_import.py Lib/test/test_macostools.py Lib/test/test_pep352.py Lib/test/test_py3kwarn.py Lib/test/test_random.py Lib/test/test_re.py Lib/test/test_struct.py Lib/test/test_structmembers.py Lib/test/test_sundry.py Lib/test/test_support.py Lib/test/test_symtable.py Lib/tes
by M.-A. Lemburg 09 Sep '08

09 Sep '08

On 2008-09-09 02:49, brett.cannon wrote: > Author: brett.cannon > Date: Tue Sep 9 02:49:16 2008 > New Revision: 66321 > > Log: > warnings.catch_warnings() now returns a list or None instead of the custom > WarningsRecorder object. This makes the API simpler to use as no special object > must be learned. > > Closes issue 3781. > Review by Benjamin Peterson. This sounds a lot like a feature. Why was it added just before RC1 ? > Modified: > python/trunk/Doc/library/warnings.rst > python/trunk/Lib/asynchat.py > python/trunk/Lib/bsddb/test/test_early_close.py > python/trunk/Lib/mimetools.py > python/trunk/Lib/test/test___all__.py > python/trunk/Lib/test/test_exceptions.py > python/trunk/Lib/test/test_hmac.py > python/trunk/Lib/test/test_import.py > python/trunk/Lib/test/test_macostools.py > python/trunk/Lib/test/test_pep352.py > python/trunk/Lib/test/test_py3kwarn.py > python/trunk/Lib/test/test_random.py > python/trunk/Lib/test/test_re.py > python/trunk/Lib/test/test_struct.py > python/trunk/Lib/test/test_structmembers.py > python/trunk/Lib/test/test_sundry.py > python/trunk/Lib/test/test_support.py > python/trunk/Lib/test/test_symtable.py > python/trunk/Lib/test/test_urllib.py > python/trunk/Lib/test/test_urllibnet.py > python/trunk/Lib/test/test_userstring.py > python/trunk/Lib/test/test_warnings.py > python/trunk/Lib/warnings.py > python/trunk/Misc/NEWS -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Sep 09 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611

3 3

Add python.exe to PATH environment variable
by Giampaolo Rodola' 09 Sep '08

09 Sep '08

Hi, I've always found it strange that Python Windows installers never managed to add the python executable to the PATH environment variable. Are there plans for adding such a thing? Thanks in advance --- Giampaolo http://code.google.com/p/pyftpdlib/

17 37

Re: [Python-Dev] [issue3769] Deprecate bsddb for removal in 3.0
by Raymond Hettinger 08 Sep '08

08 Sep '08

I think this should be deferred to Py3.1. This decision was not widely discussed and I think it likely that some users will be surprised and dismayed. The release candidate seems to be the wrong time to yank this out (in part because of the surprise factor) and in part because I think the change silently affects shelve performance so that the impact may be significantly negative but not readily apparent. We don't have to take this out. So why do it hastily at the last minute and without some discussion on comp.lang.python at least. If it were any other release, we would have disciplined ourselves to deprecate first and remove a generation or two later. Also, the reason for removal may yet disappear if jcrea steps in an continues to make updates. Also, the referenced note ( http://mail.python.org/pipermail/python-dev/2008-July/081379.html ) say to "start end-of-lifing it" which I took to mean deprecate rather than remove during a release candidate. Raymond ----- Original Message ----- From: "Benjamin Peterson" <report(a)bugs.python.org> To: <python-bugs-list(a)python.org> Sent: Wednesday, September 03, 2008 4:32 PM Subject: [issue3769] Deprecate bsddb for removal in 3.0 > > Benjamin Peterson <musiccomposition(a)gmail.com> added the comment: > > Also see this: > http://mail.python.org/pipermail/python-3000/2008-September/014712.html > > _______________________________________ > Python tracker <report(a)bugs.python.org> > <http://bugs.python.org/issue3769> > _______________________________________ > _______________________________________________ > Python-bugs-list mailing list > Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/python%40rcn.com >

22 50

constness of interpreter data
by Torne Wuff 08 Sep '08

08 Sep '08

libpython2.5.a contains quite a lot of .data that doesn't look like it needs to be writable (my minimal interpreter binary has 105KB of writable allocated data). A lot of these symbols look like they could just be tagged const with no other changes to the interpreter; some of them would require a proliferation of constness. I'm a bit new at this, though, and it's possible that I'm wrong about some/most/all of these, so I'm wondering what people think. Attached is a patch which adds const to the easy ones: * Docstrings for extension functions (PyDoc_VAR in Python.h) * ascii->digit lookup table (_PyLong_DigitValue in longobject.c) * The copyright notice (cprt in getcopyright.c) There are no errors or new warnings introduced in my build by this patch, but I'm only compiling the interpreter itself. If anyone can suggest a reason why any of those shouldn't be const, please do :) Things that take up space that might be const-able with more effort, or might not, I can't tell: * Token names (_PyParser_TokenNames in tokenizer.c) * Module function tables (and ensuing propagation of constness into PyModule_Init etc) * All the type and exception objects Things that take up space but probably aren't const-able unless someone who knows more than me has an idea: * Standard slot definitions (slotdefs in typeobject.c, but the interned string pointers get added at runtime) * The generated tables for the grammar (but the accelerator seems to want to change these at init time) There's probably other things, but I didn't go through the entire symbol table, just started with the big ones :) So, er, is this a remotely sane thing to be doing, and does anyone have suggestions? :) -- Torne Wuff torne(a)wolfpuppy.org.uk

3 2

2.6 doc searching issue?
by skip＠pobox.com 07 Sep '08

07 Sep '08

>From this page: http://docs.python.org/dev/index.html I searched for "csv" and got just one hit: http://docs.python.org/dev/contents.html?highlight=csv Shouldn't it have at least matched the docs for the csv module itself, not just the table of contents? Thx, Skip

2 1

xmlrpc via proxy for bzr and bugzilla in 2.6
by techtonik 05 Sep '08

05 Sep '08

With 2.6rc1 at the doors people are asking if xmlrpclib will be able to communicate through a proxy? It causes bzr and bugzilla tools to fail if used behind firewall, and there is no easy workaround for users. http://bugs.python.org/issue648658 -- --anatoly t.

2 1

site.py and the checkout builddir
by Benjamin Peterson 05 Sep '08

05 Sep '08

In a checkout, site.py is currently responsible for adding the distutils extension module builddir to sys.path. Usually, this is unproblematic. However, when the initialization code needed by site (in this case the standard io streams and the builtin open) relies on C modules added from the builddir in site.py, it causes silent errors. The calls to initstdio and initsite were moved to having the site initialization first in r62778, so that _bytesio and _stringio could be used when io was loaded. Unfortunately, site.py uses open, but this error went undetected because stderr was not set up yet. (See issue #3279) That options as I see it are: 1. Switch the initialization order back to the original (io streams first) and compile _bytesio and _stringio directly into the Python binary. This is probably the easiest option. 2. Switch the initialization order back to the original and have getpath.c put the builddir on sys.path. This might be difficult because finding the distutils buildir seems to involve disutils.utils.get_platform(). (I'm not a disutils expert; can the location of the builddir be simplified?) Anyway, note that this doesn't affect Python which has been installed on a system because the dynlibs are already on sys.path. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1."

3 3

2.6b3 and 3.0b3 Windows installers
by LRN 05 Sep '08

05 Sep '08

I was wondering, when new installers will be published? I made one on my own ( see http://lrn.1986.li/other/python-2.6.14127-x86.msi ), but i am not sure about it's correctness.

1 0

Summary of Python tracker Issues
by Python tracker 05 Sep '08

05 Sep '08

ACTIVITY SUMMARY (08/29/08 - 09/05/08) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 2023 open (+35) / 13586 closed (+21) / 15609 total (+56) Open issues with patches: 642 Average duration of open issues: 710 days. Median duration of open issues: 1765 days. Open Issues Breakdown open 2009 (+35) pending 14 ( +0) Issues Created Or Reopened (57) _______________________________ Misleading names for multiprocessing "convenience" functions 09/01/08 CLOSED http://bugs.python.org/issue3589 reopened jnoller needs review import warning in multiprocessing 08/29/08 CLOSED http://bugs.python.org/issue3731 created pitrou bdist_msi gives a deprecation warning when run with Python 2.6 08/29/08 CLOSED http://bugs.python.org/issue3732 created lemburg patch, easy, needs review Adding bin and Scripts folder into PATH 08/29/08 CLOSED http://bugs.python.org/issue3733 created tarek subclassing complex 08/29/08 http://bugs.python.org/issue3734 created gumtree allow multiple threads to efficiently send the same requests to 08/29/08 http://bugs.python.org/issue3735 created DavidDecotigny super is a built-in type 08/29/08 http://bugs.python.org/issue3736 created tarek An error in Python Library Reference document 08/30/08 CLOSED http://bugs.python.org/issue3737 created zouzhile logging.Handler.close does something 08/30/08 CLOSED http://bugs.python.org/issue3738 created LambertDW unicode-internal encoder reports wrong length 08/30/08 http://bugs.python.org/issue3739 created doerwalter PEP 3121 --- module state is not nul-initalized as claimed in th 08/30/08 http://bugs.python.org/issue3740 created _doublep DISTUTILS_USE_SDK set causes msvc9compiler.py to raise an except 08/31/08 http://bugs.python.org/issue3741 created TBBle patch Parsing XML file with Unicode characters causes problem 08/31/08 http://bugs.python.org/issue3742 created jaylogan PY_FORMAT_SIZE_T is not for PyString_FromFormat 08/31/08 http://bugs.python.org/issue3743 created amaury.forgeotdarc patch, needs review make altinstall installs pydoc instead of pydoc3.0 09/01/08 http://bugs.python.org/issue3744 created kune _sha256 et al. encode to UTF-8 by default 09/01/08 http://bugs.python.org/issue3745 created hagen Sphinx producing duplicate id attributes, HTML fails validation. 09/01/08 http://bugs.python.org/issue3746 created gjhiggins Fix caching in ABCMeta.__subclasscheck__ 09/01/08 CLOSED http://bugs.python.org/issue3747 created ncoghlan patch, patch, needs review platform.architecture() prints bogus message on windows 09/01/08 CLOSED http://bugs.python.org/issue3748 created ocean-city patch incrementalencoder and incrementalencoder 09/01/08 CLOSED http://bugs.python.org/issue3749 created mft test_bsddb3 skipped -- cannot import name test_support 09/01/08 CLOSED http://bugs.python.org/issue3750 created pitrou str.rpartition fails silently with unicode argument 09/01/08 CLOSED http://bugs.python.org/issue3751 created forest_atq patch test_bsddb broken 09/01/08 CLOSED http://bugs.python.org/issue3752 created pitrou bytearray incompatible with y# 09/01/08 CLOSED http://bugs.python.org/issue3753 created Frostburn minimal cross-compilation support for configure 09/01/08 http://bugs.python.org/issue3754 created rpetrov patch Lists propagate through new instances of class if using append 09/02/08 CLOSED http://bugs.python.org/issue3755 created supernova_hq re.escape() does not work with bytes() 09/02/08 http://bugs.python.org/issue3756 created andrewmcnamara threading.local doesn't support cyclic garbage collecting 09/02/08 http://bugs.python.org/issue3757 created pitrou "make check" suggest a testing target under GNU coding standards 09/02/08 http://bugs.python.org/issue3758 created ralph.corderoy test_asyncore.py leaks handle 09/02/08 CLOSED http://bugs.python.org/issue3759 created ocean-city patch, easy PEP 3121 --- PyType_Copy is missing 09/02/08 http://bugs.python.org/issue3760 created _doublep urllib.request and urllib.response cannot handle HTTP1.1 chunked 09/02/08 http://bugs.python.org/issue3761 created chrisleow platform.architecture() fails if python is lanched via its symbo 09/03/08 CLOSED http://bugs.python.org/issue3762 created ocean-city patch, easy Python 3.0 beta 2 : json and urllib not working together? 09/03/08 http://bugs.python.org/issue3763 created swaroopch asyncore differences between 2.x and 3.x 09/03/08 http://bugs.python.org/issue3764 created giampaolo.rodola [patch] allow mmap take file offset as argument 09/03/08 http://bugs.python.org/issue3765 created chrisl socket.socket.recv broken (unbearably slow) 09/03/08 http://bugs.python.org/issue3766 created thorben tkColorChooser may fail if no color is selected 09/03/08 http://bugs.python.org/issue3767 created gpolo patch Move test_py3kwarn over to new catch_warnings API 09/03/08 CLOSED http://bugs.python.org/issue3768 created brett.cannon patch, patch, needs review Deprecate bsddb for removal in 3.0 09/03/08 CLOSED http://bugs.python.org/issue3769 created brett.cannon patch, patch, needs review test_multiprocessing fails on systems with HAVE_SEM_OPEN=0 09/04/08 http://bugs.python.org/issue3770 created djmdjm test_httpservers intermittent failure 09/04/08 http://bugs.python.org/issue3771 created djmdjm logging module fails with non-ascii data 09/04/08 CLOSED http://bugs.python.org/issue3772 created mhammond patch Check for errors when using PyTokenizer_FindEncoding() 09/04/08 CLOSED http://bugs.python.org/issue3773 created brett.cannon patch tkinter Menu.delete bug 09/04/08 http://bugs.python.org/issue3774 created skomoroh patch, easy, needs review Update RELNOTES file 09/04/08 http://bugs.python.org/issue3775 created barry deprecate bsddb/dbhash in 2.6 for removal in 3.0 09/04/08 http://bugs.python.org/issue3776 created brett.cannon patch, patch, needs review PyNumber_Long fails from Float 09/04/08 http://bugs.python.org/issue3777 created barry-scott python uninstaller leave registry entries 09/04/08 http://bugs.python.org/issue3778 created donovaly log into bugs.python.org requires cookies 09/04/08 CLOSED http://bugs.python.org/issue3779 created donovaly No way to write unit tests for warnings 09/04/08 CLOSED http://bugs.python.org/issue3780 created exarkun warnings.catch_warnings fails gracelessly when recording warning 09/04/08 http://bugs.python.org/issue3781 created exarkun patch, needs review os.write accepts unicode strings 09/04/08 http://bugs.python.org/issue3782 created pitrou dbm.sqlite proof of concept 09/04/08 http://bugs.python.org/issue3783 created skip.montanaro patch Incorrect compiler options used for cc of Sun Studio 12 09/05/08 http://bugs.python.org/issue3784 created mschmarck Can't build ctypes of Python 2.5.2 with Sun Studio 12 09/05/08 http://bugs.python.org/issue3785 created mschmarck _curses, _curses_panel & _multiprocessing can't be build in 2.6b 09/05/08 http://bugs.python.org/issue3786 created mschmarck Issues Now Closed (67) ______________________ LookupError: unknown encoding: X-MAC-JAPANESE 322 days http://bugs.python.org/issue1276 josm patch New SSL module doesn't seem to verify hostname against commonNam 269 days http://bugs.python.org/issue1589 heikki "RuntimeError: dictionary changed size during iteration" in Tkin 260 days http://bugs.python.org/issue1658 gpolo patch __eq__ / __hash__ check doesn't take inheritance into account 3 days http://bugs.python.org/issue2235 ncoghlan patch Update What's new in 2.6 172 days http://bugs.python.org/issue2305 facundobatista Clean up getargs.c and its formatting possibilities 170 days http://bugs.python.org/issue2322 barry xml.sax.parser() doesn't terminate when given a filename 160 days http://bugs.python.org/issue2501 benjamin.peterson patch Cannot use non-ascii letters in disutils if setuptools is used. 150 days http://bugs.python.org/issue2562 lemburg patch Update interface of weakref dictionaries 102 days http://bugs.python.org/issue2965 barry patch VS8 include dirs grow without bound 99 days http://bugs.python.org/issue2975 jkleckner patch Building a Win32 binary installer crashes 75 days http://bugs.python.org/issue3160 pitrou patch, needs review Deficiencies in multiprocessing/threading API 50 days http://bugs.python.org/issue3352 ncoghlan patch, needs review multiprocessing module is racy 45 days http://bugs.python.org/issue3419 jnoller patch, needs review In function call, keyword arguments could follow *args 35 days http://bugs.python.org/issue3473 amaury.forgeotdarc patch New Global Module Index glitch on WinXP 25 days http://bugs.python.org/issue3520 georg.brandl Glitch in eval() doc 14 days http://bugs.python.org/issue3569 georg.brandl Interpreter name: python vs. python-3.0 13 days http://bugs.python.org/issue3577 georg.brandl Misleading names for multiprocessing "convenience" functions 1 days http://bugs.python.org/issue3589 ncoghlan needs review sax.parser considers XML as text rather than bytes 16 days http://bugs.python.org/issue3590 benjamin.peterson PyTokenizer_FindEncoding() never succeeds 16 days http://bugs.python.org/issue3594 brett.cannon patch, needs review Move test.test_suport.catch_warning() to the 'warnings' module 13 days http://bugs.python.org/issue3602 brett.cannon patch test_multiprocessing failure (Unserializable message) 15 days http://bugs.python.org/issue3607 jnoller 2to3 refactoring 12 days http://bugs.python.org/issue3637 benjamin.peterson patch, needs review segfaults calling warnings.warn() with non-string message 11 days http://bugs.python.org/issue3639 brett.cannon patch segfault calling sys.excepthook with non-Exception argument 9 days http://bugs.python.org/issue3653 georg.brandl patch, needs review Reloading an extension module always leaks 10 days http://bugs.python.org/issue3667 benjamin.peterson patch, needs review "s*" argument parser marker leaks memory 5 days http://bugs.python.org/issue3668 pitrou patch, needs review bsddb module leaks memory 10 days http://bugs.python.org/issue3673 jcea patch, patch compilation --without-threads fails 7 days http://bugs.python.org/issue3683 benjamin.peterson patch, needs review Source tarball for Sphinx 0.4.3 missing 3 days http://bugs.python.org/issue3695 georg.brandl Error parsing arguments on OpenBSD <= 4.4 8 days http://bugs.python.org/issue3696 pitrou patch, needs review "Fatal Python error: Cannot recover from stack overflow" on Wind 7 days http://bugs.python.org/issue3697 pitrou patch, needs review open() on directory raises IOError with unhelpful message 5 days http://bugs.python.org/issue3703 benjamin.peterson patch, needs review help('finally') behaves bizarrely 3 days http://bugs.python.org/issue3707 georg.brandl patch, easy, needs review os.urandom(1.1): infinite loop 5 days http://bugs.python.org/issue3708 gregory.p.smith patch, easy memoryview leaks references 4 days http://bugs.python.org/issue3712 pitrou patch hashlib's docstring throws exception in pydoc 3 days http://bugs.python.org/issue3715 gregory.p.smith easy mistake in 3.4.2 Customizing attribute access 2 days http://bugs.python.org/issue3716 georg.brandl platform.py: _syscmd_file() can't handle target path with space 5 days http://bugs.python.org/issue3719 jfdp patch invalid literal for int() with base 16: '' 7 days http://bugs.python.org/issue3721 amaury.forgeotdarc Allow ',<whitespace>' delimiters in logging.config.fileConfig() 5 days http://bugs.python.org/issue3726 vsajip patch Update BaseHTTPServer docs 1 days http://bugs.python.org/issue3730 georg.brandl import warning in multiprocessing 3 days http://bugs.python.org/issue3731 pitrou bdist_msi gives a deprecation warning when run with Python 2.6 3 days http://bugs.python.org/issue3732 ocean-city patch, easy, needs review Adding bin and Scripts folder into PATH 4 days http://bugs.python.org/issue3733 amaury.forgeotdarc An error in Python Library Reference document 0 days http://bugs.python.org/issue3737 georg.brandl logging.Handler.close does something 2 days http://bugs.python.org/issue3738 vsajip Fix caching in ABCMeta.__subclasscheck__ 1 days http://bugs.python.org/issue3747 ncoghlan patch, patch, needs review platform.architecture() prints bogus message on windows 1 days http://bugs.python.org/issue3748 ocean-city patch incrementalencoder and incrementalencoder 0 days http://bugs.python.org/issue3749 georg.brandl test_bsddb3 skipped -- cannot import name test_support 0 days http://bugs.python.org/issue3750 jcea str.rpartition fails silently with unicode argument 0 days http://bugs.python.org/issue3751 amaury.forgeotdarc patch test_bsddb broken 2 days http://bugs.python.org/issue3752 jcea bytearray incompatible with y# 0 days http://bugs.python.org/issue3753 georg.brandl Lists propagate through new instances of class if using append 0 days http://bugs.python.org/issue3755 benjamin.peterson test_asyncore.py leaks handle 2 days http://bugs.python.org/issue3759 ocean-city patch, easy platform.architecture() fails if python is lanched via its symbo 1 days http://bugs.python.org/issue3762 ocean-city patch, easy Move test_py3kwarn over to new catch_warnings API 0 days http://bugs.python.org/issue3768 brett.cannon patch, patch, needs review Deprecate bsddb for removal in 3.0 2 days http://bugs.python.org/issue3769 barry patch, patch, needs review logging module fails with non-ascii data 0 days http://bugs.python.org/issue3772 vsajip patch Check for errors when using PyTokenizer_FindEncoding() 1 days http://bugs.python.org/issue3773 amaury.forgeotdarc patch log into bugs.python.org requires cookies 0 days http://bugs.python.org/issue3779 benjamin.peterson No way to write unit tests for warnings 0 days http://bugs.python.org/issue3780 brett.cannon socket-module SSL is broken 2412 days http://bugs.python.org/issue508944 janssen relocate cgi.parse_qs() into urlparse 2200 days http://bugs.python.org/issue600362 facundobatista patch New class special method lookup change 2106 days http://bugs.python.org/issue643841 ncoghlan patch SSLObject breaks read semantics 1086 days http://bugs.python.org/issue1291446 ellisj Top Issues Most Discussed (10) ______________________________ 17 Deprecate bsddb for removal in 3.0 2 days closed http://bugs.python.org/issue3769 17 segfault in for loop with evil iterator 7 days open http://bugs.python.org/issue3720 13 Deficiencies in multiprocessing/threading API 50 days closed http://bugs.python.org/issue3352 12 reference leaks in 3.0 12 days open http://bugs.python.org/issue3660 10 platform.py: _syscmd_file() can't handle target path with space 5 days closed http://bugs.python.org/issue3719 10 Misleading names for multiprocessing "convenience" functions 1 days closed http://bugs.python.org/issue3589 9 Check for errors when using PyTokenizer_FindEncoding() 1 days closed http://bugs.python.org/issue3773 8 dbm.sqlite proof of concept 1 days open http://bugs.python.org/issue3783 8 Precompute range length 133 days open http://bugs.python.org/issue2690 7 No way to write unit tests for warnings 0 days closed http://bugs.python.org/issue3780

1 0

Python performance through times
by Antoine Pitrou 05 Sep '08

05 Sep '08

Here's an interesting blog post comparing Python performance of various versions from 2.2.3 upto the latest 3.0 beta. http://www.ogre.com/node/147 The fact that only Mandelbrot calculation (a hardly representative benchmark for a high-level dynamic language such as Python) has become significantly slower is rather a good sign IMO. cheers Antoine.

2 1

Bug in SimpleHTTPRequestHandler.send_head?
by Kim Gräsman 05 Sep '08

05 Sep '08

Hi all, I'm new to this group and the Python language as such. I stumbled on it when I joined a project to build a rich network library for C++, which in turn uses Python and its CGI HTTP server implementation as part of its unit test suite. We're having a little trouble when serving a text file containing Windows line endings (CRLF) -- the resulting content contains Unix line endings only (LF). This breaks our tests, because we can't verify that the body, as parsed by our HTTP client, is the same as the source file we're serving through the Python HTTP server. I've isolated it to the SimpleHTTPRequestHandler.send_head method in SimpleHTTPServer.py: -- ctype = self.guess_type(path) if ctype.startswith('text/'): mode = 'r' else: mode = 'rb' try: f = open(path, mode) except IOError: self.send_error(404, "File not found") return None -- The f object is returned from this method, and used with shutil.copyfileobj to copy the contents to the output stream. This is easily fixed by omitting the content-type check entirely, and blindly using mode 'rb', and I think that makes sense, because the server should not be concerned with the contents of the body, so treating it as a binary stream seems right. This also fixes another issue, where the actual body size differs from what's specified in the Content-Length header, because CR characters are stripped when the body is served, but Content-Length contains the source file's binary size. I'm not sure which source control system you're using, so I won't try to provide a patch, but I believe the code should read: -- if os.path.isdir(path): if not self.path.endswith('/'): # redirect browser - doing basically what apache does self.send_response(301) self.send_header("Location", self.path + "/") self.end_headers() return None for index in "index.html", "index.htm": index = os.path.join(path, index) if os.path.exists(index): path = index break else: return self.list_directory(path) #patch: removed content-type check try: f = open(path, 'rb') #patch: always open in binary mode except IOError: self.send_error(404, "File not found") return None self.send_response(200) self.send_header("Content-type", self.guess_type(path)) #patch: content-type check here instead fs = os.fstat(f.fileno()) -- My changes marked with "#patch[...]". Grateful for any comments! Best wishes, - Kim

3 2

Can/should built-in functions get __get__?
by Terry Reedy 05 Sep '08

05 Sep '08

One of the nice features of 3.0 is that differences between classes defined in C and Python (other than speed) are mostly erased or hidden from the view of a Python programmer. However, there are still sometimes surprising and quite visible differences between 'functions' written in C and Python. Can these be better unified also? In particular, built-in functions, in spite of of being labeled 'builtin_function_or_method', are not usable as methods because they lack the __get__ method needed to bind function to instance. [Q. Any reason to not shorten that to 'built-in function'?] So, as a c.l.p poster discovered, within a class statement, __hash__ = lambda x: id(x) # works, while the apparently cleaner __hash__ = id # raises TypeError: id() takes exactly one argument (0 given) [Irony: trivial lambda wrapping, lambda x: f(x) has been described here as useless and diseased, but is seems that casting 'built-in function' to 'function' is not so completely useless. ;-] In this case, __hash__ = object.__hash__ is available, as is object.__repr__, but this is not true in general for C functions. The difference between a C function and a C method wrapper is tantalizingly small: >>> i = set(dir(id)) >>> h = set(dir(object.__hash__)) >>> i-h {'__module__', '__self__'} >>> h-i {'__objclass__', '__get__'} Similarly, for >>> def f(): pass >>> ff = set(dir(f)) >>> ff - i {'__defaults__', '__annotations__', '__kwdefaults__', '__globals__', '__closure__', '__dict__', '__code__', '__get__'} the only un-obvious difference is __get__, So I cannot help but wonder: is this is essential? or could it be removed but has not yet? Could the object wrapper for C functions get a __get__ method so they become methods when accessed via a class just like Python functions do? Terry Jan Reedy

3 3

Re: [Python-Dev] 'warnings' module changes still breaking Twisted, still looking for a fix
by glyph＠divmod.com 04 Sep '08

04 Sep '08

On 10:18 pm, musiccomposition(a)gmail.com wrote: >That's why catch_warning keeps track of the warnings filter too, so >you can call warnings.simplefilter("always") within the context >manager and the filter state will be restored. Thanks for the pointer - this is interesting. I misunderstood the way the warnings filter worked. It looks like this combination of features may in fact be all we need. It would be nice if application code could otherwise "officially" keep track of the warnings filter (my understanding is that 'filters', despite the public-looking name, is private, since it is absent from the documentation?) but that doesn't sound like a release blocker.

2 1

'warnings' module changes still breaking Twisted, still looking for a fix
by glyph＠divmod.com 04 Sep '08

04 Sep '08

With the 2.6 final release impending, the Twisted community buildbot is still red, <http://bit.ly/zFymN>, but there only seems to be one real issue: the warn_explicit change. This seems like it could be a pretty minor bit of maintenance to clear up on our end, if Python provided the appropriate hook. The current solution we have in mind for this problem is to build our own warnings module <http://bit.ly/4F3Qr2>, but that is obviously inelegant. While we do want to experiment with our own features for deprecation, users of Twisted will (rightfully) expect our "assertWarns" to continue covering warnings produced by the Python 2.6 warnings system. The proposed solution to this problem seems to have been "warnings.catch_warnings()" <http://bugs.python.org/issue3602>, but it is insufficient for testing, since subsequent calls to "catch_warnings()" will not catch warnings suppressed by duplication suppression; without even getting into multiple tests testing the same code path, this breaks features like our 'trial' tool's "--until- failure" option, which runs a fixed group of tests repeatedly in an attempt to flush out buggy tests which have race conditions that cause them to fail intermittently. We thought we'd filed a ticket for this before, but searching around in the issue tracker hasn't come up with anything. JP Calderone has filed a new ticket for this problem: <http://bugs.python.org/issue3780> . Any chance this could be made into a release blocker? If this one issue were fixed, Python 2.6 would be the smallest disruption to Twisted since Python 2.0 :-).

3 3

patch for Cookie.py to add support for HttpOnly
by Matt Chisholm 04 Sep '08

04 Sep '08

Eighteen months ago, Arvin Schnell contributed a really straightforward three-line patch to Cookie.py adding support for the HttpOnly flag on cookies: http://bugs.python.org/issue1638033 In the last eighteen months, HttpOnly has become a de-facto extension to the cookie standard. It is now supported by IE 7, Firefox 3, and Opera 9.5 (and there's a bug open against WebKit to support it): http://www.owasp.org/index.php/HTTPOnly Ruby, Perl, and PHP all support creating HttpOnly cookies now too. This article explains why HttpOnly is a good way to make cross-site scripting (XSS) attacks significantly more difficult: http://www.codinghorror.com/blog/archives/001167.htmllop Unfortunately this patch appears to have been ignored for the last year. The last thing I want is a delay in the release of 2.6/3.0, but Antoine Pitrou posted on the bug that it will have to wait for Python 2.7/3.1, because it is a feature request. If I'm not mistaken, that means no support for HttpOnly until sometime in 2010. Do we really have to wait two more years to apply a three-line patch which will bring Python in line with the industry state of the art and improve security for Python web applications? Is there a way that this could go in to 2.6.1/3.0.1? -matt

2 1

3rd-party dbms
by Oleg Broytmann 04 Sep '08

04 Sep '08

Brett Cannon <brett(a)python.org> wrote: > for 3.1 there is nothing saying we can't change shelve and the > dbm package to allow 3rd-party code to register with the dbm package > such that bsddb can be used as needed behind the scenes. Many years ago I wrote toy hashes based on ZODB and MetaKit. Registering them with anydbm was easy: import anydbm anydbm._names.insert(len(anydbm._names)-1, ['ZODBhash', 'MKhash']) # Insert before dumbdbm More complex part was to make whichdb to recognize those hashes. I just monkey-patched whichdb. If I were doing this now I'd do something similar to atexit module - every hash module will register its own test function, and whichdb will call them in turn until it finds which db it is. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd(a)phd.pp.ru Programmers don't die, they just GOSUB without RETURN.

2 1

Re: [Python-Dev] Durus memory
by Binger David 04 Sep '08

04 Sep '08

I'm sorry this post is a bit off-topic, but I think I should correct this. On Sep 4, 2008, at 11:54 AM, Oleg wrote: > Durus (and ZODB) has an index of all objects, the index is stored in > memory AFAIK - a real problem if one has millions of objects. Durus now has an option to store the index in the data file in a form that is usable directly from disk. With this option, the in-memory index is only for objects changed since the last pack. Durus works fine with millions of objects, as long as you don't have millions of rapidly-changing objects. I haven't used the Berkeley DB storage for Durus, but I'm pretty sure that the Durus/Python side of that would not use any in-memory index.

1 0

Re: [Python-Dev] C API for gc.enable() and gc.disable()
by Andrey Zhmoginov 04 Sep '08

04 Sep '08

> > Would anyone mind if I did add a public C API for gc.disable() and > gc.enable()? I would like to use it as an optimization for the pickle > module (I found out that I get a good 2x speedup just by disabling the > GC while loading large pickles). Of course, I could simply import the > gc module and call the functions there, but that seems overkill to me. > I included the patch below for review. > I don't know if the following question is relevant, but it seems that many people here are familiar with Python cyclic garbage collector. I see Python [v2.5.2 (r252:60911, Jul 31 2008, 17:28:52)] crashing with Segment fault when I extend Python with a very simple module. This behavior is observed when I create a thousand of lists (it does not crash with 10-100) in the module with the garbage collector turned on. When I turn it off - everything is perfect. I suspect that it is my module, but if it is a Python bug ( _GC_Malloc? memory leak somewhere?), it may be worth reporting. The gdb "where" reply is the following: #0 0x080d8de9 in PyErr_Occurred () #1 0x080f508f in _PyObject_GC_Malloc () #2 0x080f5155 in _PyObject_GC_New () #3 0x08079c98 in PyList_New () #4 0xb7f53519 in draw_d_simple () from ./rt/rt_core.so #5 0xb7cf7833 in ffi_call_SYSV () from /usr/lib/python2.5/lib-dynload/_ctypes.so #6 0xb7cf766a in ffi_call () from /usr/lib/python2.5/lib-dynload/_ctypes.so #7 0xb7cf2534 in _CallProc () from /usr/lib/python2.5/lib-dynload/_ctypes.so #8 0xb7cec02a in ?? () from /usr/lib/python2.5/lib-dynload/_ctypes.so #9 0x0805cb97 in PyObject_Call () #10 0x080c7aa7 in PyEval_EvalFrameEx () #11 0x080c96e5 in PyEval_EvalFrameEx () #12 0x080cb1f7 in PyEval_EvalCodeEx () #13 0x080cb347 in PyEval_EvalCode () #14 0x080ea818 in PyRun_FileExFlags () #15 0x080eaab9 in PyRun_SimpleFileExFlags () #16 0x08059335 in Py_Main () #17 0x080587f2 in main () The crashing python code is: from ctypes import * import gc core = CDLL( "./tst.so" ) core.a.argtypes = [] core.a.restype = py_object #gc.disable() core.a() #gc.enable() And tst.cpp is: #include <Python.h> extern "C" { PyObject *a(); } PyObject *a() { int n = 1000; PyObject *item; for ( int i = 0; i < n; i++ ) item = PyList_New( 0 ); // Crashes here (somewhere in between). return item; } tst.cpp is compiled with: g++ -shared -Wl,-soname,tst.tmp.so -o tst.so tst.o g++ -I /usr/include/python2.5 -Wall -fPIC -o tst.o -c tst.cpp Thanks for your help! - Andrew.

2 1

Python 3.0b3 documentation typo
by Reed O'Brien 03 Sep '08

03 Sep '08

I was reading through the Dictionary views section: http://docs.python.org/dev/3.0/library/stdtypes.html#dictionary-view-objects Unless I am mistaken; the intersection at the end of the example usage should be: >>> keys & {'eggs', 'bacon', 'salad'} {'bacon'} Cheers, ~ro

2 1

Re: [Python-Dev] [Python-3000] bsddb finished for 2.6/3.0 (and "<class 'BytesWarning'>: str() on a bytes instance")
by Barry Warsaw 03 Sep '08

03 Sep '08

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Sep 3, 2008, at 7:01 PM, Jesus Cea wrote: > Barry Warsaw wrote: >> and I know Brett agrees, so that's it. On IRC, I've just asked >> Benjamin >> to do the honors for 3.0 and Brett will add the deprecations for 2.6. > > I just committed the fix for bsddb testsuite in Python 3.0 branch: > http://www.python.org/dev/buildbot/3.0.stable/changes/2687 > > Can I do anything to revert this decision?. If not, what can I do to > be > reconsidered in 3.1?. Start raising some pitchforks. It looks like Raymond will join the march :). Really, this is about what's best for Python and pybsddb. In this article, Guido unambiguously states his opinion: http://mail.python.org/pipermail/python-dev/2008-July/081362.html "+1. In my recollection maintaining bsddb has been nothing but trouble right from the start when we were all sitting together at "Zope Corp North" in a rented office in McLean... We can remove it from 3.0. We can't really remove it from 2.6, but we can certainly start end-of-lifing it in 2.6." Jesus, let me stress that IMO this is not a reflection on your work at all. On the contrary, keeping it alive in 2.x and providing a really solid independent package for 3.0 is critical for its continued relevance to Python programmers. I completely agree with Guido that bsddb (not pybsddb) has been a headache since forever. For example, IIRC Sleepycat was notorious for changing the API in micro releases, though I don't know if that's still the case with the current maintainers. I personally believe that Python and pybsddb are both better off with their own maintenance lifecycles so I stand by my decision that pulling it out of 3.0 is the right thing to do. 3.1 is far enough away that any decision we make in 3.0 can be re-evaluated. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSL85BnEjvBPtnXfVAQLfkwQAtoagOP37uAwL1r2H7w73erTsWBYHf4VH KcTZsjeQ/mEvmaaJIG86ylAtpxmDmMF5x7OClR66bXXxf0oTnWV4KMC9rLdQW8R/ KpMIfuQw/501AQgFmcB0M6SQ6CYyJHU5K+K6X+ScOPHOJoG8usPK1pk8XFGOXBZK UGXCEHVvlrk= =7AOQ -----END PGP SIGNATURE-----

1 0

Further PEP 8 compliance issues in threading and multiprocessing
by Nick Coghlan 03 Sep '08

03 Sep '08

I've been taking a close look at the API for multiprocessing and threading, and have discovered a somewhat strange pattern that occurs multiple times in both interfaces: factory functions with names that start with a capital letter so they look like they're actually a class. At first I thought it was a quirk of the multiprocessing implementation, but then I discovered that the threading module also does it for all of the synchronisation objects as well as threading.Timer, with examples like: def Event(*args, **kwds): return _Event(*args, **kwds) Is this just intended to discourage subclassing? If so, why give the misleading impression that these things can be subclassed by naming them as if they were classes? How should this be handled when it comes to the addition of PEP 8 compliant aliases? Add aliases for the factory functions that start with a lower case letter, but otherwise leave the structure of the modules unchanged? Or should the trivial functions like the one above be dropped and made direct aliases for the classes they are currently wrapping? Option 1 has the virtue of being perfectly backwards compatible in the threading case, while option 2 is a little riskier, so I'm inclined to go with option 1 (keeping the factory functions, but giving them non-class like names in the case of multiprocessing, or aliases in the case of threading). Cheers, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org

11 21

fileobj.read(float): warning or error?
by Victor Stinner 03 Sep '08

03 Sep '08

Hi, Since Python 2.4 (maybe 2.2 or older), fileobj.read(4.2) displays an error and works as fileobj.read(4). >>> i=open('/etc/issue') >>> i.read(4.2) __main__:1: DeprecationWarning: integer argument expected, got float It should raises an error instead of a warning, it has no sense to read a partial byte :-) But that should breaks some applications? Well, the real problem is os.urandom(4.2) which goes to an unlimited loop: while len(bytes) < n: bytes += read(_urandomfd, n - len(bytes)) because read(0.2) works as read(0) :-/ Victor

11 23

Re: [Python-Dev] Add python.exe to PATH environment variable
by Curt Hagenlocher 03 Sep '08

03 Sep '08

One of these days, I'm actually going to remember that I need to click "Reply All" when posting to this list... . Sorry for the duplicate, Greg. On Tue, Sep 2, 2008 at 6:45 PM, Greg Ewing <greg.ewing(a)canterbury.ac.nz> wrote: > > A better way would be to start a command process with > the Python directory added to PATH just for that > process. This is similar to what Visual Studio or the Windows SDK do to give you a command prompt with the PATH and other environmental variables setup "correctly" -- they add a shortcut to a batch file that's set to keep the command prompt open after the batch file runs. > How easy would that be to do on Windows? Do environment > variables get inherited by child processes the way > they do on Unix? Generally, yes. I think there's a catch in that there are ways to start processes that don't make them children of your process, but I don't remember why I think that. One other reason not to mess with the PATH -- at least by default -- is that the user may have multiple copies of Python installed. I know I have at least one machine with 2.4.5, 2.5.2, 2.6b2 and 3.0b2 installed -- and I don't want *any* of them in my path. -- Curt Hagenlocher curt(a)hagenlocher.org

2 1

2.6b3 Windows installers
by M.-A. Lemburg 02 Sep '08

02 Sep '08

The download page doesn't list any Windows installer for 2.6b3: http://www.python.org/download/releases/2.6/ I suppose this is due to Martin building the installers and him not be available at the moment. Since Python on Windows will likely only get very few beta testers without a Windows installer build, I'd suggest to postpone the RC1 release that's planned for tomorrow to get more feedback for the Windows builds. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Sep 02 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611

3 3

Re: [Python-Dev] Things to Know About Super
by average 02 Sep '08

02 Sep '08

It seems that the frustration with super revolves around how Python currently conflates (as well as many users) two very different types of inheritance, both "is-a" and "has-a" (or compositional) inheritance. Unfortunately, Python assists this confusion because the language doesn't provide a distinct enough way to differentiate between them. For the former (i.e. is-a inheritance), users should not have to explicitly make a call to the super class (via super() or otherwise)--this should be automatic for every method held in common so as to guarantee the invariants of the simpler class. For this type of inheritance it makes more sense to be explicit about *bypassing* a call to the parent class--IF necessary (perhaps to take advantage of an optimization available only in the subclass)--rather than explicit about *making* one. This would remove many of the bugs, misuses, and frustrations with super(). No more ambiguity and confusion because the language now guarantees certain behavior. Super() would then be limited to cooperative mixin classes; for all others, explicit naming of the class would be required. But perhaps that there is some language abstraction that has yet to be fully invented that would tie it all together nicely. By the way, regarding your trait implementation, it may be worthwhile revisiting the discussion surrounding the Prothon language discussed a few years ago on comp.lang.python which got rid of classes and metaclasses and replaced them with prototypes. Regards, zipher

2 1

Proposal: add odict to collections
by Armin Ronacher 01 Sep '08

01 Sep '08

Hi, I noticed lately that quite a few projects are implementing their own subclasses of `dict` that retain the order of the key/value pairs. However half of the implementations I came across are not implementing the whole dict interface which leads to weird bugs, also the performance of a Python implementation is not that great. To fight that problem I want to proposed a new class in "collections" called odict which is a dict that keeps the items sorted, similar to a PHP array. The interface would be fully compatible with dict and implemented as dict subclass. Updates to existing keys does not change the order of a key but new keys are inserted at the end. Additionally it would support slicing where a list of key, value tuples is returned and sort/reverse/index methods that work like their list equivalents. Index based lookup could work via odict.byindex(). An implementation of that exists as part of the ordereddict implementation which however goes beyond that and is pretty much a fork of the python dict[1]. Some reasons why ordered dicts are a useful feature: - in XML/HTML processing it's often desired to keep the attributes of an tag ordered during processing. So that input ordering is the same as the output ordering. - Form data transmitted via HTTP is usually ordered by the position of the input/textarea/select field in the HTML document. That information is currently lost in most Python web applications / frameworks. - Eaiser transition of code from Ruby/PHP which have sorted associative arrays / hashmaps. - Having an ordered dict in the standard library would allow other libraries support them. For example a PHP serializer could return odicts rather then dicts which drops the ordering information. XML libraries such as etree could add support for it when creating elements or return attribute dicts. Regards, Armin [1]: http://www.xs4all.nl/~anthon/Python/ordereddict/

28 52