Mailman 3 January 2007 - Python-Dev

PEP 1, PEP Purpose and Guidelines
by barry＠zope.com 18 May '21

18 May '21

It has been a while since I posted a copy of PEP 1 to the mailing lists and newsgroups. I've recently done some updating of a few sections, so in the interest of gaining wider community participation in the Python development process, I'm posting the latest revision of PEP 1 here. A version of the PEP is always available on-line at http://www.python.org/peps/pep-0001.html Enjoy, -Barry -------------------- snip snip -------------------- PEP: 1 Title: PEP Purpose and Guidelines Version: $Revision: 1.36 $ Last-Modified: $Date: 2002/07/29 18:34:59 $ Author: Barry A. Warsaw, Jeremy Hylton Status: Active Type: Informational Created: 13-Jun-2000 Post-History: 21-Mar-2001, 29-Jul-2002 What is a PEP? PEP stands for Python Enhancement Proposal. A PEP is a design document providing information to the Python community, or describing a new feature for Python. The PEP should provide a concise technical specification of the feature and a rationale for the feature. We intend PEPs to be the primary mechanisms for proposing new features, for collecting community input on an issue, and for documenting the design decisions that have gone into Python. The PEP author is responsible for building consensus within the community and documenting dissenting opinions. Because the PEPs are maintained as plain text files under CVS control, their revision history is the historical record of the feature proposal[1]. Kinds of PEPs There are two kinds of PEPs. A standards track PEP describes a new feature or implementation for Python. An informational PEP describes a Python design issue, or provides general guidelines or information to the Python community, but does not propose a new feature. Informational PEPs do not necessarily represent a Python community consensus or recommendation, so users and implementors are free to ignore informational PEPs or follow their advice. PEP Work Flow The PEP editor, Barry Warsaw <peps(a)python.org>, assigns numbers for each PEP and changes its status. The PEP process begins with a new idea for Python. It is highly recommended that a single PEP contain a single key proposal or new idea. The more focussed the PEP, the more successfully it tends to be. The PEP editor reserves the right to reject PEP proposals if they appear too unfocussed or too broad. If in doubt, split your PEP into several well-focussed ones. Each PEP must have a champion -- someone who writes the PEP using the style and format described below, shepherds the discussions in the appropriate forums, and attempts to build community consensus around the idea. The PEP champion (a.k.a. Author) should first attempt to ascertain whether the idea is PEP-able. Small enhancements or patches often don't need a PEP and can be injected into the Python development work flow with a patch submission to the SourceForge patch manager[2] or feature request tracker[3]. The PEP champion then emails the PEP editor <peps(a)python.org> with a proposed title and a rough, but fleshed out, draft of the PEP. This draft must be written in PEP style as described below. If the PEP editor approves, he will assign the PEP a number, label it as standards track or informational, give it status 'draft', and create and check-in the initial draft of the PEP. The PEP editor will not unreasonably deny a PEP. Reasons for denying PEP status include duplication of effort, being technically unsound, not providing proper motivation or addressing backwards compatibility, or not in keeping with the Python philosophy. The BDFL (Benevolent Dictator for Life, Guido van Rossum) can be consulted during the approval phase, and is the final arbitrator of the draft's PEP-ability. If a pre-PEP is rejected, the author may elect to take the pre-PEP to the comp.lang.python newsgroup (a.k.a. python-list(a)python.org mailing list) to help flesh it out, gain feedback and consensus from the community at large, and improve the PEP for re-submission. The author of the PEP is then responsible for posting the PEP to the community forums, and marshaling community support for it. As updates are necessary, the PEP author can check in new versions if they have CVS commit permissions, or can email new PEP versions to the PEP editor for committing. Standards track PEPs consists of two parts, a design document and a reference implementation. The PEP should be reviewed and accepted before a reference implementation is begun, unless a reference implementation will aid people in studying the PEP. Standards Track PEPs must include an implementation - in the form of code, patch, or URL to same - before it can be considered Final. PEP authors are responsible for collecting community feedback on a PEP before submitting it for review. A PEP that has not been discussed on python-list(a)python.org and/or python-dev(a)python.org will not be accepted. However, wherever possible, long open-ended discussions on public mailing lists should be avoided. Strategies to keep the discussions efficient include, setting up a separate SIG mailing list for the topic, having the PEP author accept private comments in the early design phases, etc. PEP authors should use their discretion here. Once the authors have completed a PEP, they must inform the PEP editor that it is ready for review. PEPs are reviewed by the BDFL and his chosen consultants, who may accept or reject a PEP or send it back to the author(s) for revision. Once a PEP has been accepted, the reference implementation must be completed. When the reference implementation is complete and accepted by the BDFL, the status will be changed to `Final.' A PEP can also be assigned status `Deferred.' The PEP author or editor can assign the PEP this status when no progress is being made on the PEP. Once a PEP is deferred, the PEP editor can re-assign it to draft status. A PEP can also be `Rejected'. Perhaps after all is said and done it was not a good idea. It is still important to have a record of this fact. PEPs can also be replaced by a different PEP, rendering the original obsolete. This is intended for Informational PEPs, where version 2 of an API can replace version 1. PEP work flow is as follows: Draft -> Accepted -> Final -> Replaced ^ +----> Rejected v Deferred Some informational PEPs may also have a status of `Active' if they are never meant to be completed. E.g. PEP 1. What belongs in a successful PEP? Each PEP should have the following parts: 1. Preamble -- RFC822 style headers containing meta-data about the PEP, including the PEP number, a short descriptive title (limited to a maximum of 44 characters), the names, and optionally the contact info for each author, etc. 2. Abstract -- a short (~200 word) description of the technical issue being addressed. 3. Copyright/public domain -- Each PEP must either be explicitly labelled as placed in the public domain (see this PEP as an example) or licensed under the Open Publication License[4]. 4. Specification -- The technical specification should describe the syntax and semantics of any new language feature. The specification should be detailed enough to allow competing, interoperable implementations for any of the current Python platforms (CPython, JPython, Python .NET). 5. Motivation -- The motivation is critical for PEPs that want to change the Python language. It should clearly explain why the existing language specification is inadequate to address the problem that the PEP solves. PEP submissions without sufficient motivation may be rejected outright. 6. Rationale -- The rationale fleshes out the specification by describing what motivated the design and why particular design decisions were made. It should describe alternate designs that were considered and related work, e.g. how the feature is supported in other languages. The rationale should provide evidence of consensus within the community and discuss important objections or concerns raised during discussion. 7. Backwards Compatibility -- All PEPs that introduce backwards incompatibilities must include a section describing these incompatibilities and their severity. The PEP must explain how the author proposes to deal with these incompatibilities. PEP submissions without a sufficient backwards compatibility treatise may be rejected outright. 8. Reference Implementation -- The reference implementation must be completed before any PEP is given status 'Final,' but it need not be completed before the PEP is accepted. It is better to finish the specification and rationale first and reach consensus on it before writing code. The final implementation must include test code and documentation appropriate for either the Python language reference or the standard library reference. PEP Template PEPs are written in plain ASCII text, and should adhere to a rigid style. There is a Python script that parses this style and converts the plain text PEP to HTML for viewing on the web[5]. PEP 9 contains a boilerplate[7] template you can use to get started writing your PEP. Each PEP must begin with an RFC822 style header preamble. The headers must appear in the following order. Headers marked with `*' are optional and are described below. All other headers are required. PEP: <pep number> Title: <pep title> Version: <cvs version string> Last-Modified: <cvs date string> Author: <list of authors' real names and optionally, email addrs> * Discussions-To: <email address> Status: <Draft | Active | Accepted | Deferred | Final | Replaced> Type: <Informational | Standards Track> * Requires: <pep numbers> Created: <date created on, in dd-mmm-yyyy format> * Python-Version: <version number> Post-History: <dates of postings to python-list and python-dev> * Replaces: <pep number> * Replaced-By: <pep number> The Author: header lists the names and optionally, the email addresses of all the authors/owners of the PEP. The format of the author entry should be address(a)dom.ain (Random J. User) if the email address is included, and just Random J. User if the address is not given. If there are multiple authors, each should be on a separate line following RFC 822 continuation line conventions. Note that personal email addresses in PEPs will be obscured as a defense against spam harvesters. Standards track PEPs must have a Python-Version: header which indicates the version of Python that the feature will be released with. Informational PEPs do not need a Python-Version: header. While a PEP is in private discussions (usually during the initial Draft phase), a Discussions-To: header will indicate the mailing list or URL where the PEP is being discussed. No Discussions-To: header is necessary if the PEP is being discussed privately with the author, or on the python-list or python-dev email mailing lists. Note that email addresses in the Discussions-To: header will not be obscured. Created: records the date that the PEP was assigned a number, while Post-History: is used to record the dates of when new versions of the PEP are posted to python-list and/or python-dev. Both headers should be in dd-mmm-yyyy format, e.g. 14-Aug-2001. PEPs may have a Requires: header, indicating the PEP numbers that this PEP depends on. PEPs may also have a Replaced-By: header indicating that a PEP has been rendered obsolete by a later document; the value is the number of the PEP that replaces the current document. The newer PEP must have a Replaces: header containing the number of the PEP that it rendered obsolete. PEP Formatting Requirements PEP headings must begin in column zero and the initial letter of each word must be capitalized as in book titles. Acronyms should be in all capitals. The body of each section must be indented 4 spaces. Code samples inside body sections should be indented a further 4 spaces, and other indentation can be used as required to make the text readable. You must use two blank lines between the last line of a section's body and the next section heading. You must adhere to the Emacs convention of adding two spaces at the end of every sentence. You should fill your paragraphs to column 70, but under no circumstances should your lines extend past column 79. If your code samples spill over column 79, you should rewrite them. Tab characters must never appear in the document at all. A PEP should include the standard Emacs stanza included by example at the bottom of this PEP. A PEP must contain a Copyright section, and it is strongly recommended to put the PEP in the public domain. When referencing an external web page in the body of a PEP, you should include the title of the page in the text, with a footnote reference to the URL. Do not include the URL in the body text of the PEP. E.g. Refer to the Python Language web site [1] for more details. ... [1] http://www.python.org When referring to another PEP, include the PEP number in the body text, such as "PEP 1". The title may optionally appear. Add a footnote reference that includes the PEP's title and author. It may optionally include the explicit URL on a separate line, but only in the References section. Note that the pep2html.py script will calculate URLs automatically, e.g.: ... Refer to PEP 1 [7] for more information about PEP style ... References [7] PEP 1, PEP Purpose and Guidelines, Warsaw, Hylton http://www.python.org/peps/pep-0001.html If you decide to provide an explicit URL for a PEP, please use this as the URL template: http://www.python.org/peps/pep-xxxx.html PEP numbers in URLs must be padded with zeros from the left, so as to be exactly 4 characters wide, however PEP numbers in text are never padded. Reporting PEP Bugs, or Submitting PEP Updates How you report a bug, or submit a PEP update depends on several factors, such as the maturity of the PEP, the preferences of the PEP author, and the nature of your comments. For the early draft stages of the PEP, it's probably best to send your comments and changes directly to the PEP author. For more mature, or finished PEPs you may want to submit corrections to the SourceForge bug manager[6] or better yet, the SourceForge patch manager[2] so that your changes don't get lost. If the PEP author is a SF developer, assign the bug/patch to him, otherwise assign it to the PEP editor. When in doubt about where to send your changes, please check first with the PEP author and/or PEP editor. PEP authors who are also SF committers, can update the PEPs themselves by using "cvs commit" to commit their changes. Remember to also push the formatted PEP text out to the web by doing the following: % python pep2html.py -i NUM where NUM is the number of the PEP you want to push out. See % python pep2html.py --help for details. Transferring PEP Ownership It occasionally becomes necessary to transfer ownership of PEPs to a new champion. In general, we'd like to retain the original author as a co-author of the transferred PEP, but that's really up to the original author. A good reason to transfer ownership is because the original author no longer has the time or interest in updating it or following through with the PEP process, or has fallen off the face of the 'net (i.e. is unreachable or not responding to email). A bad reason to transfer ownership is because you don't agree with the direction of the PEP. We try to build consensus around a PEP, but if that's not possible, you can always submit a competing PEP. If you are interested assuming ownership of a PEP, send a message asking to take over, addressed to both the original author and the PEP editor <peps(a)python.org>. If the original author doesn't respond to email in a timely manner, the PEP editor will make a unilateral decision (it's not like such decisions can be reversed. :). References and Footnotes [1] This historical record is available by the normal CVS commands for retrieving older revisions. For those without direct access to the CVS tree, you can browse the current and past PEP revisions via the SourceForge web site at http://cvs.sourceforge.net/cgi-bin/cvsweb.cgi/python/nondist/peps/?cvsroot=… [2] http://sourceforge.net/tracker/?group_id=5470&atid=305470 [3] http://sourceforge.net/tracker/?atid=355470&group_id=5470&func=browse [4] http://www.opencontent.org/openpub/ [5] The script referred to here is pep2html.py, which lives in the same directory in the CVS tree as the PEPs themselves. Try "pep2html.py --help" for details. The URL for viewing PEPs on the web is http://www.python.org/peps/ [6] http://sourceforge.net/tracker/?group_id=5470&atid=305470 [7] PEP 9, Sample PEP Template http://www.python.org/peps/pep-0009.html Copyright This document has been placed in the public domain. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 End:

8 14

pthreads, fork, import, and execvp
by Rotem Yaari 17 Sep '09

17 Sep '09

Hello everyone! We have been encountering several deadlocks in a threaded Python application which calls subprocess.Popen (i.e. fork()) in some of its threads. This has occurred on Python 2.4.1 on a 2.4.27 Linux kernel. Preliminary analysis of the hang shows that the child process blocks upon entering the execvp function, in which the import_lock is acquired due to the following line: def _ execvpe(file, args, env=None): from errno import ENOENT, ENOTDIR ... It is known that when forking from a pthreaded application, acquisition attempts on locks which were already locked by other threads while fork() was called will deadlock. Due to these oddities we were wondering if it would be better to extract the above import line from the execvpe call, to prevent lock acquisition attempts in such cases. Another workaround could be re-assigning a new lock to import_lock (such a thing is done with the global interpreter lock) at PyOS_AfterFork or pthread_atfork. We'd appreciate any opinions you might have on the subject. Thanks in advance, Yair and Rotem

12 29

Re: subprocess and EINTR errnos
by Peter Astrand 07 Jul '09

07 Jul '09

On Wed, 10 Nov 2004, John P Speno wrote: Hi, sorry for the delayed response. > While using subprocess (aka popen5), I came across one potential gotcha. I've had > exceptions ending like this: > > File "test.py", line 5, in test > cmd = popen5.Popen(args, stdout=PIPE) > File "popen5.py", line 577, in __init__ > data = os.read(errpipe_read, 1048576) # Exceptions limited to 1 MB > OSError: [Errno 4] Interrupted system call > > (on Solaris 9) > > Would it make sense for subprocess to use a more robust read() function > which can handle these cases, i.e. when the parent's read on the pipe > to the child's stderr is interrupted by a system call, and returns EINTR? > I imagine it could catch EINTR and EAGAIN and retry the failed read(). I assume you are using signals in your application? The os.read above is not the only system call that can fail with EINTR. subprocess.py is full of other system calls that can fail, and I suspect that many other Python modules are as well. I've made a patch (attached) to subprocess.py (and test_subprocess.py) that should guard against EINTR, but I haven't committed it yet. It's quite large. Are Python modules supposed to handle EINTR? Why not let the C code handle this? Or, perhaps the signal module should provide a sigaction function, so that users can use SA_RESTART. Index: subprocess.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/subprocess.py,v retrieving revision 1.8 diff -u -r1.8 subprocess.py --- subprocess.py 7 Nov 2004 14:30:34 -0000 1.8 +++ subprocess.py 17 Nov 2004 19:42:30 -0000 @@ -888,6 +888,50 @@ pass + def _read_no_intr(self, fd, buffersize): + """Like os.read, but retries on EINTR""" + while True: + try: + return os.read(fd, buffersize) + except OSError, e: + if e.errno == errno.EINTR: + continue + else: + raise + + + def _read_all(self, fd, buffersize): + """Like os.read, but retries on EINTR, and reads until EOF""" + all = "" + while True: + data = self._read_no_intr(fd, buffersize) + all += data + if data == "": + return all + + + def _write_no_intr(self, fd, s): + """Like os.write, but retries on EINTR""" + while True: + try: + return os.write(fd, s) + except OSError, e: + if e.errno == errno.EINTR: + continue + else: + raise + + def _waitpid_no_intr(self, pid, options): + """Like os.waitpid, but retries on EINTR""" + while True: + try: + return os.waitpid(pid, options) + except OSError, e: + if e.errno == errno.EINTR: + continue + else: + raise + def _execute_child(self, args, executable, preexec_fn, close_fds, cwd, env, universal_newlines, startupinfo, creationflags, shell, @@ -963,7 +1007,7 @@ exc_value, tb) exc_value.child_traceback = ''.join(exc_lines) - os.write(errpipe_write, pickle.dumps(exc_value)) + self._write_no_intr(errpipe_write, pickle.dumps(exc_value)) # This exitcode won't be reported to applications, so it # really doesn't matter what we return. @@ -979,7 +1023,7 @@ os.close(errwrite) # Wait for exec to fail or succeed; possibly raising exception - data = os.read(errpipe_read, 1048576) # Exceptions limited to 1 MB + data = self._read_all(errpipe_read, 1048576) # Exceptions limited to 1 MB os.close(errpipe_read) if data != "": child_exception = pickle.loads(data) @@ -1003,7 +1047,7 @@ attribute.""" if self.returncode == None: try: - pid, sts = os.waitpid(self.pid, os.WNOHANG) + pid, sts = self._waitpid_no_intr(self.pid, os.WNOHANG) if pid == self.pid: self._handle_exitstatus(sts) except os.error: @@ -1015,7 +1059,7 @@ """Wait for child process to terminate. Returns returncode attribute.""" if self.returncode == None: - pid, sts = os.waitpid(self.pid, 0) + pid, sts = self._waitpid_no_intr(self.pid, 0) self._handle_exitstatus(sts) return self.returncode @@ -1049,27 +1093,33 @@ stderr = [] while read_set or write_set: - rlist, wlist, xlist = select.select(read_set, write_set, []) + try: + rlist, wlist, xlist = select.select(read_set, write_set, []) + except select.error, e: + if e[0] == errno.EINTR: + continue + else: + raise if self.stdin in wlist: # When select has indicated that the file is writable, # we can write up to PIPE_BUF bytes without risk # blocking. POSIX defines PIPE_BUF >= 512 - bytes_written = os.write(self.stdin.fileno(), input[:512]) + bytes_written = self._write_no_intr(self.stdin.fileno(), input[:512]) input = input[bytes_written:] if not input: self.stdin.close() write_set.remove(self.stdin) if self.stdout in rlist: - data = os.read(self.stdout.fileno(), 1024) + data = self._read_no_intr(self.stdout.fileno(), 1024) if data == "": self.stdout.close() read_set.remove(self.stdout) stdout.append(data) if self.stderr in rlist: - data = os.read(self.stderr.fileno(), 1024) + data = self._read_no_intr(self.stderr.fileno(), 1024) if data == "": self.stderr.close() read_set.remove(self.stderr) Index: test/test_subprocess.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_subprocess.py,v retrieving revision 1.14 diff -u -r1.14 test_subprocess.py --- test/test_subprocess.py 12 Nov 2004 15:51:48 -0000 1.14 +++ test/test_subprocess.py 17 Nov 2004 19:42:30 -0000 @@ -7,6 +7,7 @@ import tempfile import time import re +import errno mswindows = (sys.platform == "win32") @@ -35,6 +36,16 @@ fname = tempfile.mktemp() return os.open(fname, os.O_RDWR|os.O_CREAT), fname + def read_no_intr(self, obj): + while True: + try: + return obj.read() + except IOError, e: + if e.errno == errno.EINTR: + continue + else: + raise + # # Generic tests # @@ -123,7 +134,7 @@ p = subprocess.Popen([sys.executable, "-c", 'import sys; sys.stdout.write("orange")'], stdout=subprocess.PIPE) - self.assertEqual(p.stdout.read(), "orange") + self.assertEqual(self.read_no_intr(p.stdout), "orange") def test_stdout_filedes(self): # stdout is set to open file descriptor @@ -151,7 +162,7 @@ p = subprocess.Popen([sys.executable, "-c", 'import sys; sys.stderr.write("strawberry")'], stderr=subprocess.PIPE) - self.assertEqual(remove_stderr_debug_decorations(p.stderr.read()), + self.assertEqual(remove_stderr_debug_decorations(self.read_no_intr(p.stderr)), "strawberry") def test_stderr_filedes(self): @@ -186,7 +197,7 @@ 'sys.stderr.write("orange")'], stdout=subprocess.PIPE, stderr=subprocess.STDOUT) - output = p.stdout.read() + output = self.read_no_intr(p.stdout) stripped = remove_stderr_debug_decorations(output) self.assertEqual(stripped, "appleorange") @@ -220,7 +231,7 @@ stdout=subprocess.PIPE, cwd=tmpdir) normcase = os.path.normcase - self.assertEqual(normcase(p.stdout.read()), normcase(tmpdir)) + self.assertEqual(normcase(self.read_no_intr(p.stdout)), normcase(tmpdir)) def test_env(self): newenv = os.environ.copy() @@ -230,7 +241,7 @@ 'sys.stdout.write(os.getenv("FRUIT"))'], stdout=subprocess.PIPE, env=newenv) - self.assertEqual(p.stdout.read(), "orange") + self.assertEqual(self.read_no_intr(p.stdout), "orange") def test_communicate(self): p = subprocess.Popen([sys.executable, "-c", @@ -305,7 +316,8 @@ 'sys.stdout.write("\\nline6");'], stdout=subprocess.PIPE, universal_newlines=1) - stdout = p.stdout.read() + + stdout = self.read_no_intr(p.stdout) if hasattr(open, 'newlines'): # Interpreter with universal newline support self.assertEqual(stdout, @@ -343,7 +355,7 @@ def test_no_leaking(self): # Make sure we leak no resources - max_handles = 1026 # too much for most UNIX systems + max_handles = 10 # too much for most UNIX systems if mswindows: max_handles = 65 # a full test is too slow on Windows for i in range(max_handles): @@ -424,7 +436,7 @@ 'sys.stdout.write(os.getenv("FRUIT"))'], stdout=subprocess.PIPE, preexec_fn=lambda: os.putenv("FRUIT", "apple")) - self.assertEqual(p.stdout.read(), "apple") + self.assertEqual(self.read_no_intr(p.stdout), "apple") def test_args_string(self): # args is a string @@ -457,7 +469,7 @@ p = subprocess.Popen(["echo $FRUIT"], shell=1, stdout=subprocess.PIPE, env=newenv) - self.assertEqual(p.stdout.read().strip(), "apple") + self.assertEqual(self.read_no_intr(p.stdout).strip(), "apple") def test_shell_string(self): # Run command through the shell (string) @@ -466,7 +478,7 @@ p = subprocess.Popen("echo $FRUIT", shell=1, stdout=subprocess.PIPE, env=newenv) - self.assertEqual(p.stdout.read().strip(), "apple") + self.assertEqual(self.read_no_intr(p.stdout).strip(), "apple") def test_call_string(self): # call() function with string argument on UNIX @@ -525,7 +537,7 @@ p = subprocess.Popen(["set"], shell=1, stdout=subprocess.PIPE, env=newenv) - self.assertNotEqual(p.stdout.read().find("physalis"), -1) + self.assertNotEqual(self.read_no_intr(p.stdout).find("physalis"), -1) def test_shell_string(self): # Run command through the shell (string) @@ -534,7 +546,7 @@ p = subprocess.Popen("set", shell=1, stdout=subprocess.PIPE, env=newenv) - self.assertNotEqual(p.stdout.read().find("physalis"), -1) + self.assertNotEqual(self.read_no_intr(p.stdout).find("physalis"), -1) def test_call_string(self): # call() function with string argument on Windows /Peter Åstrand <astrand(a)lysator.liu.se>

5 5

Python + Java Integration
by Chas Emerick 05 Dec '08

05 Dec '08

This may seem like it's coming out of left field for a minute, but bear with me. There is no doubt that Ruby's success is a concern for anyone who sees it as diminishing Python's status. One of the reasons for Ruby's success is certainly the notion (originally advocated by Bruce Tate, if I'm not mistaken) that it is the "next Java" -- the language and environment that mainstream Java developers are, or will, look to as a natural next step. One thing that would help Python in this "debate" (or, perhaps simply put it in the running, at least as a "next Java" candidate) would be if Python had an easier migration path for Java developers that currently rely upon various third-party libraries. The wealth of third-party libraries available for Java has always been one of its great strengths. Ergo, if Python had an easy-to-use, recommended way to use those libraries within the Python environment, that would be a significant advantage to present to Java developers and those who would choose Ruby over Java. Platform compatibility is always a huge motivator for those looking to migrate or upgrade. In that vein, I would point to JPype (http://jpype.sourceforge.net). JPype is a module that gives "python programs full access to java class libraries". My suggestion would be to either: (a) include JPype in the standard library, or barring that, (b) make a very strong push to support JPype (a) might be difficult or cumbersome technically, as JPype does need to build against Java headers, which may or may not be possible given the way that Python is distributed, etc. However, (b) is very feasible. I can't really say what "supporting JPype" means exactly -- maybe GvR and/or other heavyweights in the Python community make public statements regarding its existence and functionality, maybe JPype gets a strong mention or placement on python.org....all those details are obviously not up to me, and I don't know the workings of the "official" Python organizations enough to make serious suggestions. Regardless of the form of support, I think raising people's awareness of JPype and what it adds to the Python environment would be a Good Thing (tm). For our part, we've used JPype to make PDFTextStream (our previously Java-only PDF text extraction library) available and supported for Python. You can read some about it here: http://snowtide.com/PDFTextStream.Python And I've blogged about how PDFTextStream.Python came about, and how we worked with Steve Ménard, the maintainer of JPype, to make it all happen (watch out for this URL wrapping): http://blog.snowtide.com/2006/08/21/working-together-pythonjava-open- sourcecommercial Cheers, Chas Emerick Founder, Snowtide Informatics Systems Enterprise-class PDF content extraction cemerick(a)snowtide.com http://snowtide.com | +1 413.519.6365

3 2

Re: [Python-Dev] [Python-checkins] r45510 - python/trunk/Lib/pkgutil.py python/trunk/Lib/pydoc.py
by M.-A. Lemburg 21 Mar '08

21 Mar '08

Phillip.eby wrote: > Author: phillip.eby > Date: Tue Apr 18 02:59:55 2006 > New Revision: 45510 > > Modified: > python/trunk/Lib/pkgutil.py > python/trunk/Lib/pydoc.py > Log: > Second phase of refactoring for runpy, pkgutil, pydoc, and setuptools > to share common PEP 302 support code, as described here: > > http://mail.python.org/pipermail/python-dev/2006-April/063724.html Shouldn't this new module be named "pkglib" to be in line with the naming scheme used for all the other utility modules, e.g. httplib, imaplib, poplib, etc. ? > pydoc now supports PEP 302 importers, by way of utility functions in > pkgutil, such as 'walk_packages()'. It will properly document > modules that are in zip files, and is backward compatible to Python > 2.3 (setuptools installs for Python <2.5 will bundle it so pydoc > doesn't break when used with eggs.) Are you saying that the installation of setuptools in Python 2.3 and 2.4 will then overwrite the standard pydoc included with those versions ? I think that's the wrong way to go if not made an explicit option in the installation process or a separate installation altogether. I bothered by the fact that installing setuptools actually changes the standard Python installation by either overriding stdlib modules or monkey-patching them at setuptools import time. > What has not changed is that pydoc command line options do not support > zip paths or other importer paths, and the webserver index does not > support sys.meta_path. Those are probably okay as limitations. > > Tasks remaining: write docs and Misc/NEWS for pkgutil/pydoc changes, > and update setuptools to use pkgutil wherever possible, then add it > to the stdlib. Add setuptools to the stdlib ? I'm still missing the PEP for this along with the needed discussion touching among other things, the change of the distutils standard "python setup.py install" to install an egg instead of a site package. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 18 2006) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

14 51

Making builtins more efficient
by Steven Elliott 14 Apr '07

14 Apr '07

I'm interested in how builtins could be more efficient. I've read over some of the PEPs having to do with making global variables more efficient (search for "global"): http://www.python.org/doc/essays/pepparade.html But I think the problem can be simplified by focusing strictly on builtins. One of my assumptions is that only a small fractions of modules override the default builtins with something like: import mybuiltins __builtins__ = mybuiltins As you probably know each access of a builtin requires two hash table lookups. First, the builtin is not found in the list of globals. It is then found in the list of builtins. Why not have a means of referencing the default builtins with some sort of index the way the LOAD_FAST op code currently works? In other words, by default each module gets the default set of builtins indexed (where the index indexes into an array) in a certain order. The version stored in the pyc file would be bumped each time the set of default builtins is changed. I don't have very strong feelings whether things like True = (1 == 1) would be a syntax error, but assigning to a builtin could just do the equivalent of STORE_FAST. I also don't have very strong feelings about whether the array of default builtins would be shared between modules. To simulate the current behavior where attempting to assign to builtin actually alters that module's global hashtable a separate array of builtins could be used for each module. As to assigning to __builtins__ (like I mentioned at the beginning of this post) perhaps it could assign to the builtin array for those items that have a name that matches a default builtin (such as "True" or "len"). Those items that don't match a default builtin would just create global variables. Perhaps what I'm suggesting isn't feasible for reasons that have already been discussed. But it seems like it should be possible to make "while True" as efficient as "while 1". -- ----------------------------------------------------------------------- | Steven Elliott | selliott4(a)austin.rr.com | -----------------------------------------------------------------------

13 27

Renaming Include/object.h
by "Martin v. Löwis" 09 Mar '07

09 Mar '07

In #1626545, Anton Tropashko requests that object.h should be renamed, because it causes conflicts with other software. I would like to comply with this requests for 2.6, assuming there shouldn't be many problems with existing software as object.h shouldn't be included directly, anyway. What do you think? Regards, Martin

14 30

cross-compile patches
by Han-Wen Nienhuys 07 Mar '07

07 Mar '07

Hello all, A couple of weeks ago, I posted a collection of patches to Python 2.5 on SF.net (#1597850) that enable python to be cross-compiled using standard tools (ie. configure and environment settings), among others on Linux/Darwin to Mingw32/Linux/Freebsd. In response to Martin v Loewis' initial reply we have sent in contributor agreements, but nothing else has happened so far. I was wondering whether there is any way for me to speed up the review process. For example, I could post each patch separately and/or prepare patches to Python SVN. -- Han-Wen Nienhuys - hanwen(a)xs4all.nl - http://www.xs4all.nl/~hanwen

2 3

GeneratorExit inheriting from Exception
by Nick Coghlan 28 Feb '07

28 Feb '07

Should GeneratorExit inherit from Exception or BaseException? Currently, a generator that catches Exception and continues on to yield another value can't be closed properly (you get a runtime error pointing out that the generator ignored GeneratorExit). The only decent reference I could find to it in the old PEP 348/352 discussions is Guido writing [1]: > when GeneratorExit or StopIteration > reach the outer level of an app, it's a bug like all the others that > bare 'except:' WANTS to catch. (at that point in the conversation, I believe bare except was considered the equivalent of "except Exception:") While I agree with what Guido says about GeneratorExit being a bug if it reaches the outer level of an app, it seems like a bit of a trap that a correctly written generator can't write "except Exception:" without preceding it with an "except GeneratorExit:" that reraises the exception. Isn't that exactly the idiom we're trying to get rid of for SystemExit and KeyboardInterrupt? Regards, Nick. [1] http://mail.python.org/pipermail/python-dev/2005-August/055173.html -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org

16 41

Python's C interface for types
by Jim Jewett 01 Feb '07

01 Feb '07

Nick Maclaren wrote: > Martin v. Löwis wrote: >> It may be a bit problematic to implement, but I think a clean >> specification is possible. If a and b are numbers, and a==b, >> then hash(a)==hash(b). You don't even need that much complication. If a==b, then hash(a) == hash(b) If you have to break this, then at least one (preferably both) of (a,b) must be unhashable, so that it won't get used as a dict key. Hashing is only ever meaningful as a shortcut to quickly show that two objects are *not* equal. >> I'm not sure whether "approximately 5.0" >> equals 5 or not: if it does, it should hash the same as 5, >> if it doesn't, it may or may not hash the same (whatever is >> easier to implement). agreed. >> For 0: hash(+0.0)==hash(-0.0)==hash(0)=hash(0L)=0 > Unfortunately, that assumes that equality is transitive. No, but the (transitively closed set of equivalent objects) must have the same hash. If >>> myfloat(5.0) != 5.0 True then you could just return 47 as the hash. It might not be terribly efficient, but it would work. If >>> myfloat_exact(5.0) == 5.0 == myfloat_approx(5.0) != myfloat_exact(5.0) then at least one of (myfloat_exact, myfloat_approx) needs to be unhashable, so that it can't be used as a dictionary key. > let us say that I am implementing a special function and want to > distinguish -0.0 and +0.0. Why can't I use a dictionary? Because they are equal. They aren't identical, but they are equal. > >>> a = float("+0.0") > >>> b = float("-0.0") > >>> print a, b > 0.0 -0.0 With the standard windows distribution, I get just 0.0 0.0 > No, I don't have an answer. You are damned if you do, and damned > if you don't. It is an insoluble problem, and CURRENTLY doesn't > justify two hashing mechanisms (i.e. ANY difference and EQUALITY > difference). You want something in between "__eq__" and "is". (a.identical(b) ?) Hashing is weaker than either. >>> hash ("JimJ") == hash (2010274390) True -jJ

1 0

Happy Birthday, Guido!
by Aahz 01 Feb '07

01 Feb '07

Thanks again for giving me something fun to do with my life. ;-) -- Aahz (aahz(a)pythoncraft.com) <*> http://www.pythoncraft.com/ "I disrespectfully agree." --SJM

6 6

Scoping [Patience, please]
by lingwitt＠gmail.com 31 Jan '07

31 Jan '07

This email is broken up into multiple sections: (1) Introduction (2) Problem (3) Probably Poor Solution (4) Tangent (Utter Tomfoolery) #### Introduction #### Hello, This is my first post to the list, and I apologize for 4 reasons: (1) I haven't been lurking for more than a few days. (2) I am by no means well versed in python. (3) I know nothing of language design. (4) This isn't a very good introduction. Nonetheless, I have a habit of beginning with institutions that are on the verge of sweeping changes, and python seems no different. To curb the future troubles against which this unfortunate fate of mine often forces me, I have been pouring over PEPs (scanning really, because it's far past my bed time). I came across the discussion of scoping, and while it seems to be a very badly beaten and decaying horse, I ask that you'd be kind enough to include me in this disgruntled discourse. #### Problem #### See http://www.python.org/dev/peps/pep-3104/ Consider the following: >>> x = 10 >>> def a(): ... x = 3 ... def b(): ... global x ... x = 4 ... print x ... ... b() ... print x ... >>> a() 4 3 >>> print x 4 The keyword 'global' is only good for the module scope. #### Probably Poor Solution #### See http://www.python.org/dev/peps/pep-3104/ There seems to be a predilection towards limited and/or general means of outer scope referencing, such as (1) prefixing variables with dots. (2) scope override declarations. However, I don't understand the virtue of this generality (of course, I haven't thought much about it either, so please lend me your expertise). Most of the time, it would seem a person should know exactly which outer variable he is after; he knows exactly which scope. Fortunately, the scopes in question are always named. Can't we just reference what's already there? >>> x = 10 >>> def a(): ... x = 3 ... def b(): ... a.x = 4 ... print a.x ... ... b() ... print x ... >>> a() 4 4 >>> print x 10 Of course, the module itself is unnamed, and it would be completely consistent to use the single prefixed dot to reference it: >>> x = 10 >>> def a(): ... x = 3 ... def b(): ... a.x = 4 ... print a.x ... ... .x = 5 ... b() ... print x ... >>> a() 4 4 >>> print x 5 However, I would think a placeholder is easier to spot: >>> x = 10 >>> def a(): ... x = 3 ... def b(): ... a.x = 4 ... print a.x ... ... @.x = 5 ... b() ... print x ... >>> a() 4 4 >>> print x 5 Of course, what happens if 'a' is a variable name? >>> x = 10 >>> def a(): ... x = 3 ... def b(): ... a = 15 ... a.x = 4 #? ... print a.x ... ... @.x = 5 ... b() ... print x ... This would never happen in practice. Why bother considering it? However, one could get around it with a fuller qualification: >>> x = 10 >>> def a(): ... x = 3 ... def b(): ... a = 15 ... @.a.x = 4 ... print a.x ... ... @.x = 5 ... b() ... print x ... >>> a() 4 4 >>> print x 5 In any case, those are just some thoughts off of the top of my (tired) head. It seems to me that it unifies a lot of scoping ideas. everything can be accessed with a qualified name. Sorry if this is nonsensical. #### Tangent (Utter Tomfoolery) #### More interestingly, one sees (if one squints) the distinction between modules, classes, and functions blur: >>> def new_b(): ... print "tee hee" >>> a.b = new_b() >>>a() tee hee 3 >>> print x 5 or perhaps: >>> # Interestingly, the following code is valid already ... >>> def Person(name, age, location): ... def print_record(): ... print('The last person created had the name ' + Person.name_last) ... print("However, I don't know any of the details") ... ... def instance(): ... def print_record(): ... print('Name: ' + name) ... print('age: ' + str(age)) ... print('location: ' + location) ... ... if name == 'lingwitt': ... name = age = location = 'classified' ... ... Person.name_last = name ... ... return instance ... >>>p1 = Person('spam', 80, 'eggsville') >>>Person.name_last spam >>># Now the code becomes invalid ... >>>Person.print_record() The last person created had the name spam However, I don't know any of the details >>>p1.print_record() name: spam age: 80 location: eggsville It would appear that modules, classes, and functions are scopes with different magic associated with them. #### Warning!!! This is an unlisted section! Sillyness Ensues #### It would appear that a language could be built around scopes; One could define a scope and then use that same scope as a template for other scopes (actually, it sounds kind of like prototyping languages); this seems to encompass the behavior of modules, classes, and functions. Seems like all the "kinds" of scopes could be unified. I'll stop here, because now I'm worried that I've made myself look silly. Thanks for your time. Yours truly, Herr Witten P.S. Be gentle.

2 2

2.5.1 release postponed
by Neal Norwitz 30 Jan '07

30 Jan '07

As you may have noticed we missed the schedule for getting 2.5.1 out. Unfortunately several of us are having scheduling problems with making this release. Since there don't seem to be major problems with the current 2.5 release, our current plan is to defer release until April when everyone should have more time to do it right. n

1 0

Problem with signals in a single threaded application
by Ulisses Furquim 29 Jan '07

29 Jan '07

Hi, I'm aware of the problems with signals in a multithreaded application, but I was using signals in a single-threaded application and noticed something that seemed wrong. Some signals were apparently being lost, but when another signal came in the python handler for that "lost" signal was being called. The problem seems to be inside the signal module. The global variable is_tripped is incremented every time a signal arrives. Then, inside PyErr_CheckSignals() (the pending call that calls all python handlers for signals that arrived) we can return immediately if is_tripped is zero. If is_tripped is different than zero, we loop through all signals calling the registered python handlers and after that we zero is_tripped. This seems to be ok, but what happens if a signal arrives after we've returned from its handler (or even after we've checked if that signal arrived) and before we zero is_tripped? I guess we can have a situation where is_tripped is zero but some Handlers[i].tripped are not. In fact, I've inserted some debugging output and could see that this actually happens and then I've written the following test program to reproduce the problem. #!/usr/bin/env python2.5 import sys import os import time import signal def alarm_handler(*args): sys.stderr.write('alarmmmmmmmmmmm!\n') def sigio_handler(*args): sys.stderr.write('Entering SIGIO handler\n') os.kill(os.getpid(), signal.SIGALRM) sys.stderr.write('Leaving SIGIO handler\n') signal.signal(signal.SIGIO, sigio_handler) signal.signal(signal.SIGALRM, alarm_handler) os.kill(os.getpid(), signal.SIGIO) ini = time.time() while True : if time.time() - ini > 3.0: sys.stderr.write('Loop!\n') ini = time.time() When we run this program, the handler for the SIGALRM isn't called after we return from the SIGIO handler. We return to our main loop and print 'Loop!' every 3 seconds aprox. and the SIGALRM handler is called only when another signal arrives (like when we hit Ctrl-C). I've read some threads about signals in the archives and I was under the impression signals should work reliably on single-threaded applications. Am I right? I've thought about a way to fix this, but I don't know what is the current plan for signals support in python, so what can be done? Best regards, -- Ulisses

5 12

Re: [Python-Dev] Unipath package
by glyph＠divmod.com 29 Jan '07

29 Jan '07

On 28 Jan, 06:30 pm, sluggoster(a)gmail.com wrote: >The discussion has been hampered by the lack of released code. Only >Orendorff's class has been widely used in production systems. The >others have either never been used or only by their authors; they >haven't made it to the Cheeseshop. Unipath is merely to say "Here's >another way to do it; see if it works for you." Certainly the Path >methods need more testing and use in the real world before they'd be >ready for the stdlib. The FSPath methods are more experimental so I'd >say they need a year of use before they can be considered sufficiently >stable. Mike is mistaken here; Twisted has a module, twisted.python.filepath, which has been used extensively in production as well as by other projects. You can see such a project here (just the first hit on google code search): http://www.google.com/codesearch?hl=en&q=+twisted+python+filepath+show:3PfU… and the source to the module itself here: http://twistedmatrix.com/trac/browser/trunk/twisted/python/filepath.py Twisted's filepath implementation also provides a zipfile implementation of the same interface, so that, for example, you can copy a directory in the filesystem and put it into a zipfile with the same function. We plan to eventually also include tarfile and in-memory implementations. Of course, I am FilePath's author, so I have a certain bias; but I think it would be well suited to the standard library, and I would be interested to hear any feedback on it, especially that which would make it unsuitable for inclusion.

1 0

Unipath package
by Mike Orr 29 Jan '07

29 Jan '07

I finally finished my path package (Unipath) and put it in the Cheeseshop. http://sluggo.scrapping.cc/python/unipath/ There's a Path class for pathname calculations, and a FSPath subclass for filesystem calls. I'm hoping Path -- or something resembling it -- will find its way into os.path in Python 2.6 or 3.0. FSPath is full of convenience methods so it may not be everybody's cup of tea, but perhaps something similar can go into Python in the farther future Unipath is an early alpha release so the API may change as it gets more real-world use. There's an extensive unittest suite, which passes on Python 2.5 and 2.4.4 on Linux. Windows and Macintosh testers are needed. Following are highlights from the python-3000 discussion and deviations from it: - Path subclasses unicode, or str if the platform can't handle Unicode pathnames. - FSPath subclasses Path. This allows you to do "from unipath import FSPath as Path" and pretend there's only one class. I find this much more convenient in applications, so you don't have to cast objects back and forth between the classes. Also, it just became infeasable not to let them inherit, because so many methods call other methods. - Nevertheless, you can use Path alone and rest assured it will never touch the filesystem. You can even use Path objects with os.* functions if you are so heretically inclined. If Path is accepted into the stdlib, FSPath will inherit it from there. - I tried splitting FSPath into several mixins according to type of operation, but the need for methods to call other methods in a different category sabotaged that too. So FSPath proudly has about fifty methods. (Path has 10 public methods and 4 properties.) - The dirname property is called .parent. The basename property is .name. The extension property is .ext. The name without extension is .stem. - .components() returns a list of directory components. The first component is "/", a Windows drive root, a UNC share, or "" for a relative path. .split_root() returns the root and the rest. - PosixPath, NTPath, and MacPath are Path subclasses using a specific path library. This allows you to express non-native paths and convert paths. Passing a relative foreign path to a *Path constructor converts it to the destination type. Passing an absolute foreign path is an error, because there's no sane way to interpret "C:\\" on Posix or "/" on Windows. I'm skeptical whether this non-native support is really worth it, because .norm() and .norm_case() already convert slashes to backslashes (which is what Talin really wanted to do -- have Posix paths in a config file and automatically convert them to the native format). And if Python is going to drop Mac OS 9 soon then MacPath is obsolete. So maybe this non-native path code will prove less than useful and will be deleted. On the other hand, if someone is burning to write a zippath or ftppath library, you can use it with this. - Setting Path.auto_norm to true will automatically normalize all paths on construction. I'm not sure if this should be the default, because normalizing may produce the wrong path (e.g., if it contains a symlink). You can always pass norm=True or norm=False to the constructor to enable/disable it. - p.child("subdir", "grandkid") is Glyph's favorite "safe join" method that prevents creating a path pointing above 'p'. You can use it as a .joinpath if you don't mind this restriction. The constructor allows multiple positional arguments, which are joined using os.path.join(). - Listing is p.listdir(pattern=None, filter=None, names_only=False). This returns a non-recursive list of paths. If 'names_only' is true it returns the same as os.listdir(). p.walk(pattern=None, filter=None, top_down=True) is the recursive counterpart, yielding paths. In both cases 'pattern' is a glob pattern, and 'filter' is one of the constants (FILES, DIRS, LINKS, FILES_NO_LINKS, DIRS_NO_LINKS, DEAD_LINKS) or a custom boolean function. I spent a lot of time going back and forth between Orendorff's six listing methods and Raphael's one, and finally decided on this, partly because I'm not satisfied with how either Orendorff's class or Raphael's or os.walk() handle symlinks -- sometimes you want to ignore the links and then iterate them separately. - .read_file(mode) and .write_file(content, mode) are a compromise between Orendorff's seven methods and purists' desire for zero methods. - .mkdir(), .rmdir(), .rmtree(), .copy(), .copy_stat(), .copy_tree(), and .move() are more fancy than their stdlib/Orendorff counterparts. They silently succeed if the operation is already done, and have arguments to smartly create/delete intermediate directories, etc. - Two extra functions are in 'unipath.tools'. 'dict2dir' creates a directory hierarchy modeled after a dict. 'dump_path' displays an ASCII tree of a directory hierarchy, with file sizes and symlink targets showing. Enjoy! and please provide feedback. --Mike Orr <sluggoster(a)gmail.com>

6 11

Re: [Python-Dev] Python's C interface for types
by Nick Maclaren 27 Jan '07

27 Jan '07

=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?= <martin(a)v.loewis.de> wrote: > > [not sure what "And so it goes" means in English] I apologise. I try to restrain myself from using excessive idiom, but sometimes I forget. It means "That is how things are, and there is and will be more of the same." > It may be a bit problematic to implement, but I think a clean > specification is possible. If a and b are numbers, and a==b, > then hash(a)==hash(b). I'm not sure whether "approximately 5.0" > equals 5 or not: if it does, it should hash the same as 5, > if it doesn't, it may or may not hash the same (whatever is > easier to implement). > For 0: hash(+0.0)==hash(-0.0)==hash(0)=hash(0L)=0 Unfortunately, that assumes that equality is transitive. With the advanced floating-point models, it may not be. For example, if you want to avoid the loss of error information, exact infinity and approximate infinity (the result of overflow) have different semantics. Similarly with infinitesimals. Even at present, Python's float (Decimal probably more so) doesn't allow you to do some things that are quite reasonable. For example, let us say that I am implementing a special function and want to distinguish -0.0 and +0.0. Why can't I use a dictionary? >>> a = float("+0.0") >>> b = float("-0.0") >>> print a, b 0.0 -0.0 >>> c = {a: "+0.0", b: "-0.0"} >>> print c[a], c[b] -0.0 -0.0 Well, we all know why. But it is not what some quite reasonable programmers will expect. And Decimal (with its cohorts and variant precisions) has this problem quite badly - as do I. No, I don't have an answer. You are damned if you do, and damned if you don't. It is an insoluble problem, and CURRENTLY doesn't justify two hashing mechanisms (i.e. ANY difference and EQUALITY difference). Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1(a)cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679

1 0

Re: [Python-Dev] Problem with signals in a single threaded application
by Nick Maclaren 27 Jan '07

27 Jan '07

I apologise for going off-topic, but this is an explanation of why I said that signal handling is not reliable. The only relevance to Python is that Python should avoid relying on signals if possible, and try to be a little defensive if not. Signals will USUALLY do what is expected, but not always :-( Anything further by Email, please. Greg Ewing <greg.ewing(a)canterbury.ac.nz> wrote: > > > This one looks like an oversight in Python code, and so is a bug, > > but it is important to note that signals do NOT work reliably under > > any Unix or Microsoft system. > > That's a rather pessimistic way of putting it. In my > experience, signals in Unix mostly do what they're > meant to do quite reliably -- it's just a matter of > understanding what they're meant to do. Yes, it is pessimistic, but I am afraid that my experience is that it is so :-( That doesn't deny your point that they MOSTLY do 'work', but car drivers MOSTLY don't need to wear seat belts, either. I am talking about high-RAS objectives, and ones where very rare failure modes can become common (e.g. HPC and other specialist uses). More commonly, there are plain bugs in the implementations which are sanctioned by the standards (Linux is relatively disdainful of such legalistic games). Because they say that everything is undefined behaviour, many vendors' support mechanisms will refuse to accept bug reports unless you push like hell. And, as some are DIABOLICALLY difficult to explain, let alone demonstrate, they can remain lurking for years or decades. > There may be bugs in certain systems that cause > signals to get lost under obscure circumstances, but > that's no reason for Python to make the situation > worse by introducing bugs of its own. 100% agreed. > > Two related signals received between two 'checkpoints' (i.e. when > > the signal is tested and cleared). You may only get one of them, > > and 'related' does not mean 'the same'. > > I wasn't aware that this could happen between > different signals. If it can, there must be some > rationale as to why the second signal is considered > redundant. Otherwise there's a bug in either the > design or the implementation. Nope. There is often a clash between POSIX and the hardware, or a cause where a 'superior' signal overrides an 'inferior' one. I have seen SIGKILL flush some other signals, for example. And, on some systems, SIGFPE may be divided into the basic hardware exceptions. If you catch SIGFPE as such, all of those may be cleared. I don't think that many (any?) current systems do that. And it is actually specified to occur for the SISSTOP, SIGTSTP, SIGTTIN, SIGTTOU, SIGCONT group. > > A second signal received while the first is being 'handled' by the > > operating system or language run-time system. > > That one sounds odd to me. I would expect a signal > received during the execution of a handler to be > flagged and cause the handler to be called again > after it returns. But then I'm used to the BSD > signal model, which is relatively sane. It's nothing to do with the BSD model, which may be saner but still isn't 100% reliable, but occurs at a lower layer. At the VERY lowest level, when a genuine hardware event causes an interrupt, the FLIH (first-level interrupt handler) runs in God mode (EVERYTHING disabled) until it classifies what is going on. This is a ubiquitous misdesign of modern hardware, but that is off-topic. Hardware 'signals' from other CPUs/devices may well get lost if they occur in that window. And there are other, but less extreme, causes at higher levels in the operating system. Unix and Microsoft do NOT have a reliable signal delivery model, where the sender of a signal checks if the recipient has got it and retries if not. Some operating systems do - but I don't think that BSD does. > > A signal sent while the operating system is doing certain things to > > the application (including, sometimes, when it is swapped out or > > deep in I/O.) > > That sounds like an outright bug. I can't think > of any earthly reason why the handler shouldn't > be called eventually, if it remains installed and > the process lives long enough. See above. It gets lost at a low level. That is why you can cause serious time drift on an "IBM PC" (most modern ones) by hammering the video card or generating streams of floating-point fixups. Most people don't notice, because xntp or equivalent fixes it up. And there are worse problems. I could start on cross-CPU TLB and ECC handling on large shared memory systems. I managed to get an Origin in a state where it wouldn't even power down from the power-off button, and I had to flip breakers, due to THAT one! I have reason to believe that all largish SMP systems have similar problems. Again, it is possible to design an operating system to avoid those issues, but we are talking about mainstream ones, and they don't. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1(a)cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679

1 0

Re: [Python-Dev] Python's C interface for types
by Nick Maclaren 27 Jan '07

27 Jan '07

Having looked into the answers a bit more deeply, I am afraid that I am still a bit puzzled. 1) As I understand it, PyMem_Malloc won't cause trouble, but won't be automatically freed, either, as it doesn't return a new reference. I don't think that immediately following it by PyCObject_FromVoidPtr (which is what I do) helps with that. What I need is some standard type that allows me to allocate an anonymous block of memory; yes, I can define such a type, but that seems excessive. Is there one? 2) _PyLong_FromByteArray and _PyLong_AsByteArray aren't in the API and have no comments. Does that mean that they are unstable, in the sense that they may change behaviour in new versions of Python? And will they be there in 3.0? Thanks for any help, again. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1(a)cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679

3 2

Object creation hook
by Kristján V. Jónsson 27 Jan '07

27 Jan '07

Hello there. I am trying to insert a hook into python enabling a callback for all just-created objects. The intention is to debug and find memory leaks, e.g. by having the hook function insert the object into a WeakKeyDictionary. I have already added a method to "object" to set such a hook, and "object_new" now calls it upon completion, but this is far from covering all places. Initially, I thought object_init were the place, but almost no classes call object.__init__ from their __init__ method. Then there is the separate case of old-style classes. Any suggestions on how to do a global object creation hook in python? Kristján

6 7

Weekly Python Patch/Bug Summary
by Kurt B. Kaiser 27 Jan '07

27 Jan '07

Patch / Bug Summary ___________________ Patches : 421 open ( -2) / 3549 closed (+10) / 3970 total ( +8) Bugs : 943 open (-17) / 6471 closed (+25) / 7414 total ( +8) RFE : 260 open ( +2) / 250 closed ( +1) / 510 total ( +3) New / Reopened Patches ______________________ rlcompleter tab completion in pdb (2007-01-22) http://python.org/sf/1641544 opened by Stephen Emslie logging leaks loggers (2007-01-22) CLOSED http://python.org/sf/1641790 opened by TH Fix error/crash in AST: syntaxerror in complex ifs (2007-01-23) http://python.org/sf/1642547 opened by Thomas Wouters comments to clarify complexobject.c (2007-01-23) http://python.org/sf/1642844 opened by Jim Jewett Fix Bug 1362475 Text.edit_modified() doesn't work (2007-01-24) http://python.org/sf/1643641 opened by Matthias Kievernagel ctypes leaks memory (2007-01-24) CLOSED http://python.org/sf/1643874 opened by Thomas Heller file -> open in stdlib (2007-01-25) http://python.org/sf/1644218 opened by Daniel Nogradi Allow importing built-in submodules (2007-01-25) http://python.org/sf/1644818 opened by Miguel Lobo Patches Closed ______________ Fix for #1601399 (urllib2 does not close sockets properly) (2007-01-03) http://python.org/sf/1627441 closed by gbrandl C99 _Bool support for struct (2006-12-07) http://python.org/sf/1610575 closed by loewis logging leaks loggers (2007-01-22) http://python.org/sf/1641790 closed by vsajip Patch for #1586414 to avoid fragmentation on Windows (2006-10-31) http://python.org/sf/1587674 closed by gustaebel smtplib email renames (2007-01-16) http://python.org/sf/1637162 closed by gbrandl urllib2: email.Utils->email.utils (2007-01-16) http://python.org/sf/1637159 closed by gbrandl urllib: change email.Utils -> email.utils (2007-01-16) http://python.org/sf/1637157 closed by gbrandl tarfile extraction does not honor umask (2006-06-16) http://python.org/sf/1507247 closed by gustaebel Fix crash when replacing sys.stdout in sitecustomize (2007-01-08) http://python.org/sf/1630975 closed by twouters ctypes leaks memory (2007-01-24) http://python.org/sf/1643874 closed by theller New / Reopened Bugs ___________________ 2.3.6.4 Error in append and extend descriptions (2007-01-21) CLOSED http://python.org/sf/1641109 opened by ilalopoulos Python 2.5 gets curses.h warning on HPUX (2007-01-22) http://python.org/sf/1642054 opened by Roy Smith Grammatical Error (2007-01-23) CLOSED http://python.org/sf/1643150 opened by Steve Miller function breakpoints in pdb (2007-01-24) http://python.org/sf/1643369 opened by decitre Emphasize buffering issues when sys.stdin is used (2007-01-24) http://python.org/sf/1643712 opened by Raghuram Devarakonda Problem with signals in a single-threaded application (2007-01-24) http://python.org/sf/1643738 opened by Ulisses Furquim strptime %U broken (2007-01-24) CLOSED http://python.org/sf/1643943 opened by Brian Nahas Error arrow offset wrong (2007-01-25) CLOSED http://python.org/sf/1644239 opened by Cees Timmerman ./configure --prefix=/ breaks, won't build C modules (2007-01-25) http://python.org/sf/1644987 opened by Jim Shankland MIME renderer: wrong header line break with long subject? (2007-01-26) http://python.org/sf/1645148 opened by kxroberto Bugs Closed ___________ Newline skipped in "for line in file" (2007-01-16) http://python.org/sf/1636950 closed by bcannon Over-zealous keyword-arguments check for built-in set class (2006-05-11) http://python.org/sf/1486663 closed by gbrandl urllib2 does not close sockets properly (2006-11-22) http://python.org/sf/1601399 closed by gbrandl subprocess: error redirecting i/o from non-console process (2006-11-27) http://python.org/sf/1603907 closed by astrand Problem running a subprocess (2007-01-13) http://python.org/sf/1634739 closed by astrand subprocess.py: O(N**2) bottleneck (2006-11-17) http://python.org/sf/1598181 closed by astrand 2.3.6.4 Error in append and extend descriptions (2007-01-21) http://python.org/sf/1641109 closed by gbrandl asyncore.py and "handle_expt" (2002-12-16) http://python.org/sf/654766 closed by sf-robot Segfault provoked by generators and exceptions (2006-10-18) http://python.org/sf/1579370 closed by loewis gen_iternext: Assertion `f->f_back != ((void *)0)' failed (2006-05-06) http://python.org/sf/1483133 closed by nnorwitz Python polls unnecessarily every 0.1 second when interactive (2006-09-05) http://python.org/sf/1552726 closed by akuchling subprocess interpreted double quotation wrong on windows (2006-03-09) http://python.org/sf/1446119 closed by astrand tarfile.extract() may cause file fragmentation on Windows XP (2006-10-28) http://python.org/sf/1586414 closed by gustaebel subprocess.py Errors with IDLE (2005-02-17) http://python.org/sf/1126208 closed by astrand subprocess.Popen fails inside a Windows service (2005-07-15) http://python.org/sf/1238747 closed by astrand mailbox.py uses old email names (2007-01-16) http://python.org/sf/1637167 closed by gbrandl subprocess.py fails on Windows when there is no console (2005-11-16) http://python.org/sf/1358527 closed by astrand mailbox.py _fromlinepattern regexp does not support positive (2007-01-11) http://python.org/sf/1633678 closed by akuchling rfc822 module, bug in parsedate_tz (2005-08-01) http://python.org/sf/1249573 closed by gbrandl an extra comma in condition command crashes pdb (2007-01-03) http://python.org/sf/1627316 closed by gbrandl Use of "except:" in logging module (2001-03-28) http://python.org/sf/411881 closed by sf-robot segfaults when using __del__ and weakrefs (2005-12-10) http://python.org/sf/1377858 closed by bcannon Grammatical Error (2007-01-24) http://python.org/sf/1643150 closed by quiver modulefinder is not documented (2004-03-11) http://python.org/sf/914375 closed by theller strptime %U broken (2007-01-24) http://python.org/sf/1643943 closed by bcannon Error arrow offset wrong (2007-01-25) http://python.org/sf/1644239 closed by gbrandl ctypes: Pointer-to-pointer unchanged in callback (2006-10-10) http://python.org/sf/1574588 closed by theller New / Reopened RFE __________________ recursive urlparse (2007-01-24) http://python.org/sf/1643370 opened by anatoly techtonik RFE Closed __________ Add registry functions to windows postinstall (2007-01-14) http://python.org/sf/1635335 closed by loewis

1 0

Re: [Python-Dev] Problem with signals in a single threaded application
by Nick Maclaren 27 Jan '07

27 Jan '07

On Tue, Jan 23, 2007, Ulisses Furquim wrote: > > I've read some threads about signals in the archives and I was under > the impression signals should work reliably on single-threaded > applications. Am I right? I've thought about a way to fix this, but I > don't know what is the current plan for signals support in python, so > what can be done? This one looks like an oversight in Python code, and so is a bug, but it is important to note that signals do NOT work reliably under any Unix or Microsoft system. Inter alia, all of the following are likely to lead to lost signals: Two related signals received between two 'checkpoints' (i.e. when the signal is tested and cleared). You may only get one of them, and 'related' does not mean 'the same'. A second signal received while the first is being 'handled' by the operating system or language run-time system. A signal sent while the operating system is doing certain things to the application (including, sometimes, when it is swapped out or deep in I/O.) And there is more, some of which can cause program misbehaviour or crashes. You are also right that threading makes the situation a lot worse. Obviously, Unix and Microsoft systems depend on signals, so you can't simply regard them as hopelessly broken, but you can't assume that they are RELIABLE. All code should be designed to cope with the case of signals getting lost, if at all possible. Defending yourself against the other failures is an almost hopeless task, but luckily they are extremely rare except on specialist systems. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1(a)cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679

3 2

Re: [Python-Dev] Python's C interface for types
by Nick Maclaren 27 Jan '07

27 Jan '07

Giovanni Bajo <rasky(a)develer.com> wrote: > > I personally consider *very* important that hash(5.0) == hash(5) (and > that 5.0 == 5, of course). It gets a bit problematic with floating-point, when you can have different values "exactly 5.0" and "approximately 5.0". IEEE 754 has signed zeroes. And so it goes. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1(a)cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679

2 1

Python's C interface for types
by Nick Maclaren 27 Jan '07

27 Jan '07

I have a fair amount of my binary floating-point model written, though even of what I have done only some is debugged (and none has been rigorously tested). But I have hit some things that I can't work out, and one query reduced comp.lang.python to a stunned silence :-) Note that I am not intending to do all the following, at least for now, but I have had to restructure half a dozen times to match my implementation requirements to the C interface (as I have learnt more about Python!) and designing to avoid that is always good. Any pointers appreciated. I can't find any detailed description of the methods that I need to provide. Specifically: Does Python use classic division (nb_divide) and inversion (nb_invert) or are they entirely historical? Note that I can very easily provide the latter. Is there any documentation on the coercion function (nb_coerce)? It seems to have unusual properties. How critical is the 'numeric' property of the nb_hash function? I can certainly honour it, but is it worth it? I assume that Python will call nb_richcompare if defined and nb_compare if not. Is that right? Are the inplace methods used and, if so, what is their specification? I assume that I can ignore all of the allocation, deallocation and attribute handling functions, as the default for a VAR object is fine. That seems to work. Except for one thing! My base type is static, but I create some space for every derivation (and it can ONLY be used in derived form). The space creation is donein C but the derivation in Python. I assume that I need a class (not instance) destructor, but what should it do to free the space? Call C to Py_DECREF it? I assume that a class structure will never go away until after all instances have gone away (unless I use Py_DECREF), so a C pointer from an instance to something owned by the class is OK. Is there any documentation on how to support marshalling/pickling and the converse from C types? I would quite like to provide some attributes. They are 'simple' but need code executing to return them. I assume that means that they aren't simple enough, and have to be provided as methods (like conjugate). That's what I have done, anyway. Is there any obvious place for a reduction method to be hooked in? That is a method that takes a sequence, all members of which must be convertible to a single class, and returns a member of that class. Note that it specifically does NOT make sense on a single value of that class. Sorry about the length of this! Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1(a)cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679

4 4

Re: [Python-Dev] Python's C interface for types
by Nick Maclaren 27 Jan '07

27 Jan '07

Josiah Carlson <jcarlson(a)uci.edu> wrote: > > See _PyLong_FromByteArray and _PyLong_AsByteArray . Oops! Thanks very much. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1(a)cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679

1 0

Re: [Python-Dev] Python's C interface for types
by Nick Maclaren 27 Jan '07

27 Jan '07

Oops. Something else fairly major I forgot to ask. Python long. I can't find any clean way of converting to or from this, and would much rather not build a knowledge of long's internals into my code. Going via text is, of course, possible - but is not very efficient, even using hex/octal. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1(a)cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679

2 1

Re: [Python-Dev] Floor division
by Nick Maclaren 26 Jan '07

26 Jan '07

"Guido van Rossum" <guido(a)python.org> wrote: > > "(int)float_or_double" truncates in C (even in K&R C) /provided that/ > the true result is representable as an int. Else behavior is > undefined (may return -1, may cause a HW fault, ...). Actually, I have used Cs that didn't, but haven't seen any in over 10 years. C90 is unclear about its intent, but C99 is specific that truncation is towards zero. This is safe, at least for now. > So Python uses C's modf() for float->int now, which is always defined > for finite floats, and also truncates. Yes. And that is clearly documented and not currently likely to change, as far as I know. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1(a)cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679

3 2

Re: [Python-Dev] Floor division
by Nick Maclaren 26 Jan '07

26 Jan '07

"Tim Peters" <tim.peters(a)gmail.com> wrote: > > OTOH, I am a fan of analyzing FP operations as if the inputs were in > fact exactly what they claim to be, which 754 went a long way toward > popularizing. That largely replaced mountains of idiosyncratic > "probabilistic arguments" (and where it seemed no two debaters ever > agreed on the "proper" approach) with a common approach that > sometimes allows surprisingly sharp analysis. Since I spent a good > part of my early career as a professional apologist for Seymour Cray's > "creative" floating point, I'm probably much more grateful to leave > sloppy arithmetic behind than most. Well, I spent some of it working with code (and writing code) that was expected to work, unchanged, on an ICL 1900, CDC 6600/7600, IBM 370 and others. I have seen the harm caused by the 'exact arithmetic' mindset and so don't like it, but I agree about your objections to the "probabilistic arguments" (which were and are mostly twaddle). But that is seriously off-topic. > [remquo] It's really off-topic for Python-Dev, so > I didn't/don't want to belabor it. Agreed, except in one respect. I stand by my opinion that the C99 specification has no known PRACTICAL use (your example is correct, but I know of no such use in a real application), and so PLEASE don't copy it as a model for Python divmod/remainder. > No, /Python's/ definition of mod is inexact for that example. fmod > (which is not Python's definition) is always exact: fmod(-1, 1e100) = > -1, and -1 is trivially exactly congruent to -1 modulo anything > (including modulo 1e100). The result of fmod(x, y) has the same sign > as x; Python's x.__mod__(y) has the same sign as y; and that makes all > the difference in the world as to whether the exact result is always > exactly representable as a float. Oops. You're right, of course. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1(a)cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679

1 0

Re: [Python-Dev] Floor division
by Nick Maclaren 26 Jan '07

26 Jan '07

"Tim Peters" <tim.peters(a)gmail.com> wrote: > > It could, but who would have a (sane) use for a possibly 2000-bit quotient? Well, the 'exact rounding' camp in IEEE 754 seem to think that there is one :-) As you can gather, I can't think of one. Floating-point is an inherently inaccurate representation for anything other than small integers. > This is a bit peculiar to me, because there are ways to compute > "remainder" using a number of operations proportional to the log of > the exponent difference. It could be that people who spend their life > doing floating point forget how to work with integers ;-) Aargh! That is indeed the key! Given that I claim to know something about integer arithmetic, too, how can I have been so STUPID? Yes, you are right, and that is the only plausible way to calculate the remainder precisely. You don't get the quotient precisely, which is what my (insane) specification would have provided. I would nitpick with your example, because you don't want to reduce modulo 3.14 but modulo pi and therefore the modular arithmetic is rather more expensive (given Decimal). However, it STILL doesn't help to make remquo useful! The reason is that pi is input only to the floating-point precision, and so the result of remquo for very large arguments will depend more on the inaccuracy of pi as input than on the mathematical result. That makes remquo totally useless for the example you quote. Yes, I have implemented 'precise' range reduction, and there is no substitute for using an arbitrary precision pi value :-( > > But it isn't obviously WRONG. > > For floats, fmod(x, y) is exactly congruent to x modulo y -- I don't > think it's possible to get more right than exactly right ;-) But, as a previous example of yours pointed out, it's NOT exactly right. It is also supposed to be in the range [0,y) and it isn't. -1%1e100 is mathematically wrong on two counts. 1a Cc: "Tim Peters" <tim.peters(a)gmail.com> Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1(a)cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679

2 1

Re: [Python-Dev] Floor division
by Nick Maclaren 26 Jan '07

26 Jan '07

"Tim Peters" <tim.peters(a)gmail.com> wrote: > > [Tim (misattributed to Guido)] Apologies to both! > > C90 is unclear about its intent, > > But am skeptical of that. I don't have a copy of C90 here, but before > I wrote that I checked Kernighan & Ritchie's seminal C book, Harbison > & Steele's generally excellent "C: A Reference Manual" (2nd ed), and a > web version of Plauger & Brodie's "Standard C": > > http://www-ccs.ucsd.edu/c/ > > They all agree that the Cs they describe (all of which predate C99) > convert floating to integral types via truncation, when possible. I do. Kernighan & Ritchie's seminal C book describes the Unix style of "K&R" C - one of the reasons that ANSI/ISO had to make incompatible changes was that many important PC and embedded Cs differed. Harbison and Steele is generally reliable, but not always; I haven't looked at the last, but I would regard it suspiciously. What C90 says is: When a value of floating type is converted to integer type, the fractional part is discarded. There is other wording, but none relevant to this issue. Now, given the history of floating-point remainder, that is seriously ambiguous. > > but C99 is specific that truncation is towards zero. > > As opposed to what? Truncation away from zero? I read "truncation" > as implying toward 0, although the Plauger & Brodie source is explicit > about "the integer part of X, truncated toward zero" for the sake of > logic choppers ;-) Towards -infinity, of course. That was as common as truncation towards zero up until the 1980s. It was near-universal on twos complement floating-point systems, and not rare on signed magnitude ones. During the standardisation of C90, the BSI tried to explain to ANSI that this needed spelling out, but were ignored. C99 did not add the normative text "(i.e., the value is truncated toward zero)" because there was no ambiguity, after all! Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1(a)cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679

1 0

Re: [Python-Dev] Floor division
by Tim Peters 26 Jan '07

26 Jan '07

[Tim Peters] >> ... >> Maybe we could introduce "%" as a unary prefix operator, where >> %x means "the fractional part of x" ;-) [Anders J. Munch] > What'ya talking about? Obviously it should be a suffix operator ;-) Na -- that would be confusing ;-) ... >> time.sleep(1.0 - math.fmod(now, 1.0)) >> >> would do the same, except would be easier to reason about because it's >> trivially guaranteed that 0.0 <= math.fmod(x, 1.0) < 1.0 for any >> finite float x >= 0.0. The same may or may not be true of % (I would >> have to think about that, and craft a proof one way or the other -- if >> it is true, it would have to invoke something special about the >> modulus 1.0, as the inequality doesn't hold for % for some other >> modulus values). And as you note later, x%y == fmod(x, y) whenever x and y have the same sign (well, given the way CPython implements float.__mod__ today), so there's actually an easy proof. > Other modulus values are important: On an importance scale of 1 to 10, 9 or 10 ;-) ? > The attraction of Guido's formula is that he could just as easily have > used 60.0 or 0.001 if minute or millisecond intervals were desired, or > even som user-specified arbitrary dt. Then we're comparing dt-now%dt > to (1.0-int(now/dt))*dt or (math.ceil(now/dt)-now/dt)*dt. time.time() is never negative in Python (see other reply), so the trivial respelling dt-fmod(now, dt) does the same. > Fortunately, for all a,b>0, mathematically math.fmod(a,b) is equal to > a%b, so if the former is exactly representable, so is the latter. Yup. Also when `a` and `b` both less than 0. This /follows/ from that when `a` and `b` have the same sign, the mathematical a/b is >= 0, so truncation is the same as the floor. Therefore the mathematical a - floor(a/b)*b # Python __mod__ and a - truncate(a/b)*b # C fmod are exactly the same whenever a and b have the same sign. > Which is borne out in floatobject.c: float_rem and float_divmod just > pass on the C fmod result if (a < 0) == (b < 0). Yes. In fact, I wrote all that code :-)

1 0

Re: [Python-Dev] Floor division
by Nick Maclaren 26 Jan '07

26 Jan '07

Armin Rigo <arigo(a)tunes.org> wrote: > > Thanks for the clarification. Yes, it makes sense that __mod__, > __divmod__ and __floordiv__ on float and decimal would eventually follow > the same path as for complex (where they make even less sense and > already raise a DeprecationWarning). Yes. Though them not doing so would also make sense. The difference is that they make no mathematical sense for complex, but the problems with float are caused by floating-point (and do not occur for the mathematical reals). There is an argument for saying that divmod should return a long quotient and a float remainder, which is what C99 has specified for remquo (except that it requires only the last 3 bits of the quotient for reasons that completely baffle me). Linux misimplemented that the last time I looked. Personally, I think that it is bonkers, as it is fiendishly expensive compared to its usefulness - especially with Decimal! But it isn't obviously WRONG. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1(a)cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679

2 1

Re: [Python-Dev] Floor division
by Tim Peters 26 Jan '07

26 Jan '07

... [Tim] >> fractional part of x == fmod(x, 1.0) == modf(x)[0], so you could use >> either. [Anders J. Munch] > Actually, on the off chance that time.time() is negative, he can use > neither. It has to be math.ceil, float.__mod__ or divmod. If time.time() is negative, I expect this would be the least of his worries :-) Even on most Unixish boxes, Python's time.time() is immune to "the year 2038 problem" anyway, since it uses POSIX's gettimeofday() instead of C's time() when it can.

1 0

Floor division
by Raymond Hettinger 26 Jan '07

26 Jan '07

I bumped into an oddity today: 6.0 // 0.001 != math.floor(6.0 / 0.001) In looking at Objects/floatobject.c, I was surprised to find that float_floor_division() is implemented in terms of float_divmod(). Does anyone know why it takes such a circuitous path? I had expected something simpler and faster: return PyFloat_FromDouble(floor(a/b)); Raymond

9 27

Having trouble committing
by Brett Cannon 26 Jan '07

26 Jan '07

I am trying to commit to the 2.5 branch and I am getting an error: svn: Commit failed (details follow): svn: Can't create directory '/data/repos/projects/db/transactions/53566-1.txn': Permission denied Anyone know what is going on? -Brett

4 4

Re: [Python-Dev] [Python-checkins] buildbot failure in amd64 gentoo 2.5
by "Martin v. Löwis" 24 Jan '07

24 Jan '07

Georg Brandl schrieb: > Do I miss something here, or is the buildbot hit by spammers now? It looks like it is. If that continues, we have to disable the web triggers. Regards, Martin

8 7

Complex constructors [was Re: Floor division]
by Nick Maclaren 24 Jan '07

24 Jan '07

Gareth McCaughan <gareth.mccaughan(a)pobox.com> wrote: > > ... The question is whether > it makes sense to define complex(a,b) = a+ib for all a,b > or whether the two-argument form is always in practice going > to be used with real numbers[1]. If it is, which seems pretty > plausible to me, then changing complex() to complain when > passed two complex numbers would (1) notify users sooner > when they have errors in their programs, (2) simplify the > code, and (3) avoid the arguably broken behaviour Tim was > remarking on, where complex(-0.0).real is +0 instead of -0. > > [1] For the avoidance of ambiguity: "real" is not > synonymous with "double-precision floating-point". Precisely. On this matter, does anyone know of an application where making that change would harm anything? I cannot think of a circumstance under which the current behaviour adds any useful function over the one that raises an exception if there are two arguments and either is complex. Yes, of course, SOME people will find it cool to write complex(a,b) when they really mean a+1j*b, but .... Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1(a)cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679

1 0

compex numbers (was Floor Division)
by Jim Jewett 24 Jan '07

24 Jan '07

Tim Peters wrote: > complex_new() ends with: > cr.real -= ci.imag; > cr.imag += ci.real; > and I have no idea what that thinks it's doing. Surely this isn't intended?!: I think it is. python.org/sf/1642844 adds comments to make it less unclear. > >>> complex(complex(1.0, 2.0), complex(10.0, 20.0)) > (-19+12j) > WTF? >>> help(complex) Help on class complex in module __builtin__: class complex(object) | complex(real[, imag]) -> complex number | | Create a complex number from a real part and an optional imaginary part. | This is equivalent to (real + imag*1j) where imag defaults to 0. If "real" and "imag" are themselves complex numbers, then normalizing the result will move the imaginary portion of the "real" vector into the imaginary part and vice versa. Note that changing this (to discard the imaginary parts) would break passing complex numbers to their own constructor. >>> a=complex(1,2) >>> b=complex(a) >>> a==b True -jJ

2 1

compex numbers (was Floor Division)
by Nick Maclaren 24 Jan '07

24 Jan '07

"Jim Jewett" <jimjjewett(a)gmail.com> wrote: > Tim Peters wrote: > > > complex_new() ends with: > > > cr.real -= ci.imag; > > cr.imag += ci.real; > > > and I have no idea what that thinks it's doing. Surely this isn't intended?! > : > > I think it is. python.org/sf/1642844 adds comments to make it less unclear. Agreed. > > If "real" and "imag" are themselves complex numbers, then normalizing > the result will move the imaginary portion of the "real" vector into > the imaginary part and vice versa. Not really. What it does is to make complex(a,b) exactly equivalent to a+1j*b. For example: >>> a = 1+2j >>> b = 3+4j >>> complex(a) (1+2j) >>> b*1j (-4+3j) >>> complex(a,b) (-3+5j) > Note that changing this (to discard the imaginary parts) would break > passing complex numbers to their own constructor. Eh? Now, I am baffled. There are several ways of changing it, all of which would turn one bizarre behaviour into another - or would raise an exception. Personally, I would do the following: complex(a) would permit a to be complex. complex(a,b) would raise an exception if either a or b were complex. But chacun a son gout (accents omitted). Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1(a)cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679

1 0

Floor division
by Nick Maclaren 23 Jan '07

23 Jan '07

"Jim Jewett" <jimjjewett(a)gmail.com> wrote: > > >... I can work with and teach almost any model, > > and have done so with some pretty weird ones. > > I think python's model is "Whatever your other tools use. Ask them." > And I think that is a reasonable choice. Answer: It's undefined. Just because you have tested your code today doesn't mean it will work tomorrow, or on a different set of values (however similar), or that it will give the same answer every time you do the same operation on the same input, or that the effects will be limited to wrong answers and stray exceptions. Still think that it is reasonable? > > Some people write Python that is intended to be robust and portable; > > it is those people who suffer. > > If your users stick to sensible inputs, then it doesn't matter which > model you used. Sigh. Let's step back a step. Who decides when inputs are sensible? And where is it documented? Answers: God alone knows, and nowhere. One of Python's general principles is that its operations should either do roughly what a reasonable user would expect, or it will raise an exception. It doesn't always get there, but it isn't bad. What you are saying is that is undesirable. The old Fortran and C model of saying that any user error can cause any effect (including nasal demons) is tolerable only if there is agreement on what IS an error, and there is some way for a user to find that out. In the case of C, neither is true. > If not, there is no way to get robust and portable; it is just a > matter of which users you annoy. Well, actually, there is. Though I agree that the techniques have rather been forgotten in the past 30 years. Python implements more of them than most languages. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1(a)cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679

1 0

Floor division
by Jim Jewett 23 Jan '07

23 Jan '07

Nick Maclaren wrote: >... I can work with and teach almost any model, > and have done so with some pretty weird ones. I think python's model is "Whatever your other tools use. Ask them." And I think that is a reasonable choice. For sensible input, the various models all work the same. For dubious input, you get the same answer as you got from some other tool, so you can automate some of your regression testing. >... Python doesn't support any libm >behaviour other than the one that it was compiled with, and that is >often NOT what is wanted. If you care (and know) enough to want a specific behavior on dubious output, then you know more about your specfic problem then python ever could. If you can go on to say "and this platform libm is wrong, but the other one over there is right", then you can probably recompile. > Some people write Python that is intended to be robust and portable; > it is those people who suffer. If your users stick to sensible inputs, then it doesn't matter which model you used. If not, there is no way to get robust and portable; it is just a matter of which users you annoy. I could understand "throw an error, and *make* them choose a model if they want to process bad data", but that isn't really any easier to for a user to fix than the current non-model. -jJ

1 0

Re: [Python-Dev] Floor division
by Anders J. Munch 23 Jan '07

23 Jan '07

[Tim Peters] > [Anders J. Munch] > > I did read the whole thread, and I saw your -1%1e100 example. Mixing > > floating-point numbers of very different magnitude can get you in > > trouble - e.g. -1+1e100==1e100. I don't think -1%1e100 is all that > > worse. > > Except that it's very easy to return an exactly correct result in that > case: -1. Except that the function that computes -1 is a different function. Either it makes a difference which function is computed or it doesn't: If it makes a difference, because the first operand may be negative, then the programmer absolutely has to choose the correct function, or the program will have a bug. If the correct function is one whose result cannot be exactly represented, then there's nothing that can be done about that. Whether the function is called using a % infix operator or using math.untrustworthy_intlike_fmod doesn't change the fact that the result is going to be inexact. If it doesn't make a difference, because the first operand is always positive, then both functions provide exactly representable results. So regardless, the current float.__mod__ doesn't create inexact results where exact results are possible. Suppose for example I'm normalising radians to the [0; 2*math.pi] range using radians %= 2*math.pi Changing that to radians = math.fmod(radians, 2*math.pi) would simply be incorrect. I could of course rewrite that to give the correct result with a conditional and a bit more calculation, still using math.fmod, but then the inexactness would reappear. > It was natural to /want/ to extend it to floats. That didn't work > well, and to the contrary it's surprising precisely /because/ the > behavior people enjoy with integers can fail when it's applied to > floats. What makes you say it didn't work well? What user experiences do you base that on? - Anders

1 0

Fwd: Re: Floor division
by python＠rcn.com 23 Jan '07

23 Jan '07

>> Do note that this discussion is only about Python 3. Under the view >> that it was a (minor, but real) design error to /try/ extending >> Python's integer mod definition to floats, if __mod__, and __divmod__ >> and __floordiv__ go away for binary floats in P3K they should >> certainly go away for decimal floats in P3K too. And that's about >> syntax, not functionality: the IBM spec's "remainder" and >> "remainder-near" still need to be there, it's "just" that a user would >> have to get at "remainder" in a more long-winded way (analogous to [Facundo Batista] >We'll have to deprecate that functionality, with proper warnings (take >not I'm not following the thread that discuss the migration path to 3k). > >And we'll have to add the method "remainder" to decimal objects (right >now we have only "remainder_near" in decimal objects, and both >"remainder" and "remainder_near" in context). Whoa. This needs more discussion with respect to the decimal module. I'm not ready to throw-out operator access to the methods provided by the spec. Also, I do not want the Py2.x decimal module releases to be complexified and slowed-down by Py3.0 deprecation warnings. The module is already slow and has ease-of-use issues. Raymond

2 1

Re: [Python-Dev] Floor division
by Anders J. Munch 23 Jan '07

23 Jan '07

Tim Peters wrote: > Please read the whole thread. Maybe you did, but you said nothing > here that indicated you had. The issues aren't about tiny integers > that happen to be in float format, where the result is exactly > representable as a float too. Those don't create problems for any > definition of mod. But even non-tiny exact integers can. I did read the whole thread, and I saw your -1%1e100 example. Mixing floating-point numbers of very different magnitude can get you in trouble - e.g. -1+1e100==1e100. I don't think -1%1e100 is all that worse. > > It's only Decimal.__mod__ that's inconsistent. float.__mod__ has the > > usual floating-point inaccuracies, but then with float that goes with > > the territory. > > No. Decimal.__mod_ always returns the mathematically exact result. I meant inconsistent with integers. People are familiar with the semantics of % on integers, because they use it all the time. % on float is a natural extension of that and hence unsurprising. % on Decimal is exact and correct, but surprising all the same. regards, Anders

3 2

Re: [Python-Dev] Floor division
by Nick Maclaren 23 Jan '07

23 Jan '07

A generic comment. Many of my postings seem to be being misunderstood. I hold no brief for ANY particular floating-point religion, sect or heresy, except insofar as it affects robustness and portability (i.e. "software engineering"). I can work with and teach almost any model, and have done so with some pretty weird ones. My objections to some proposals is that they are sacrificing those in favour of some ill-defined objectives. "Tim Peters" <tim.peters(a)gmail.com> wrote: > [TIm Peters] > >> That's wrong, if true: it should return something with the opposite > >> sign on the imaginary part, whether or not that equals 0 (+0. and -0. > >> both "equal 0"). > > |[Nick Maclaren] > > Grrk. Why? Seriously. > > Seriously: because there's some reason to do so and no good reason > not to. Hmm. That doesn't fully support the practice, except for IEEE 754(R) numbers. To require a floating-point format to have signed zeroes is a religious matter. But I agree that specifying something different if the numbers are an IEEE 754(R) format makes no sense. > > And, returning to 'floor' - if one is truncating towards -infinity, > > should floor(-0.0) deliver -1.0, 0.0 or -0.0? > > I'd leave a zero argument alone (for ceiling too), and am quite sure > that's "the right" 754-ish behavior. It's not clear, and there was a debate about it! But it is what IEEE 754R ended up specifying. > Couldn't quite parse that, but nearly all of Python's math-module > functions inherit most behavior from the platform libm. This is often > considered to be a feature: the functions called from Python > generally act much like they do when called from C or Fortran on the > same platform, easing cross-language development on a single platform. And making it impossible to write robust, portable code :-( Note that most platforms have several libms and the behaviour even for a single libm can be wildly variable. It can also ENHANCE cross-language problems, where a user needs to use a library that expects a different libm or libm option. > Do note the flip side: to the extent that different platform > religions refuse to standardize libm endcase behavior, Python plays > along with whatever libm gods the platform it's running on worships. > That's of value to some too. Actually, no, it doesn't. Because Python doesn't support any libm behaviour other than the one that it was compiled with, and that is often NOT what is wanted. > So which one would you prefer? As explained, there are 3 plausible > candidates. > > You seem to be having some trouble taking "yes" for an answer here ;-) Actually, there are a lot more candidates, but let that pass. All I am saying is that there should be SOME defined AND SANE behaviour. While I would prefer an exception, I am not dogmatic about it. What I can't stand is completely undefined behaviour, as was introduced into Python by C99. > > What I was trying to point out is that the current behaviour is > > UNDEFINED (and may give total nonsense). That is not > > good. > > Eh -- I can't get excited about it. AFAIK, in 15 years nobody has > complained about passing a 0 modulus to math.fmod (possibly because > most newbies use the Windows distro, and it does raise ValueError > there). Some people write Python that is intended to be robust and portable; it is those people who suffer. > What Guido would rather do, which I agreed with, was to have > x.conjugate() simply return x when x is float/int/long. No change in > value, no change in type, and the obvious implementation would even > make ... Fine. I am happy with that. What I was pointing out that forcible changes of type aren't harmful if you remove the "gotchas" of loss of information with coercions that aren't intended to do so. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1(a)cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679

1 0

Re: [Python-Dev] Floor division
by Nick Maclaren 23 Jan '07

23 Jan '07

"Tim Peters" <tim.peters(a)gmail.com> wrote: > > > I guess the conjugate() function could also just return self (although I see > > that conjugate() for a complex with a zero imaginary part returns > > something whose imaginary part is -0; is that intentional? > > That's wrong, if true: it should return something with the opposite > sign on the imaginary part, whether or not that equals 0 (+0. and -0. > both "equal 0"). Grrk. Why? Seriously. IEEE 754 signed zeroes are deceptive enough for float, but are a gibbering nightmare for complex; Kahan may be able to handle them, but mere mortals can't. Inter alia, the only sane forms of infinity for complex numbers are a SINGLE one (the compactified model) and to may infinity into NaN (which I prefer, as it leads to less nonsense). And, returning to 'floor' - if one is truncating towards -infinity, should floor(-0.0) deliver -1.0, 0.0 or -0.0? > math.fmod is 15 years old -- whether or not someone likes it has > nothing to do with whether Python should stop trying to use the > current integer-derived meaning of % for floats. Eh? No, it isn't. Because of the indirection to the C library, it is changing specification as we speak! THAT is all I am getting at; not that the answer might not be A math.fmod with defined behaviour. > On occasion we've added additional error checking around functions > inherited from C. But adding code to return a NaN has never been > done. If you want special error checking added to the math.fmod > wrapper, it would be easiest to "sell" by far to request that it raise > ZeroDivisionError (as integer mod does) for a modulus of 0, or > ValueError (Python's historic mapping of libm EDOM, and what Python's > fmod(1, 0) already does on some platforms). The `decimal` module > raises InvalidOperation in this case, but that exception is specific > to the `decimal` module for now. I never said that it should; I said that it is reasonable behaviour on systems that support them. I personally much prefer an exception in this case. What I was trying to point out is that the current behaviour is UNDEFINED (and may give total nonsense). That is not good. > >> For ints and floats, real could just return self, and imag could > >> return a 0 of the same type as self. I guess the conjugate() function > >> could also just return self (although I see that conjugate() for a > >> complex with a zero imaginary part returns something whose imaginary > >> part is -0; is that intentional? I'd rather not have to do that when > >> the input is an int or float, what do you think?) > > > I don't see the problem in doing that - WHEN implicit conversion > > to a smaller domain, losing information, raises an exception. > > Take it as a pragmatic fact that it wouldn't. Besides, e.g., the > conjugate of 10**50000 is exactly 10**50000 mathematically. Why raise > an exception just because it can't be represented as a float? The > exact result is easily supplied with a few lines of "obviously > correct" implementation code (incref `self` and return it). Eh? I don't understand. Are you referring to float("1.0e50000"), pow(10,50000), pow(10.0,50000), or a conjugate (and, if so, of what?) float(conjg(1.23)) obviously need not raise an exception, except possibly "Sanity failure" :-) Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1(a)cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679

2 1

Re: [Python-Dev] Floor division
by Anders J. Munch 23 Jan '07

23 Jan '07

Tim Peters wrote: > Do note that this discussion is only about Python 3. Under the view > that it was a (minor, but real) design error to /try/ extending > Python's integer mod definition to floats, if __mod__, and __divmod__ > and __floordiv__ go away for binary floats in P3K they should > certainly go away for decimal floats in P3K too. What design error? float.__mod__ works perfectly. >>> -1 % 50 49 >>> -1.0 % 50.0 49.0 >>> It's only Decimal.__mod__ that's inconsistent. float.__mod__ has the usual floating-point inaccuracies, but then with float that goes with the territory. I've had occasion to use it, and it was a pleasant surprise that it "just worked", so I didn't have to go to the math module and ponder over the difference between modulo or remainder. float modulo is useful, consistent and intuitive; -1 on removing that. - Anders

2 1

Re: [Python-Dev] Eliminating f_tstate
by Delaney, Timothy (Tim) 23 Jan '07

23 Jan '07

"Martin v. Löwis" wrote: > For Python 2.5, for compatibility, it probably has to remain > where it is, and only PyTraceBack_Here should stop using it. > As a consequence, a generator .send() makes exceptions > occur in the current thread, not in the thread where the > generator was created. > > What do you think? That's the behaviour I would expect. There is nothing I can find in either the Python docs nor PEP 342 to suggest that interactions with generators happen on anything except the current thread. Anything else IMO is a bug, and should be a candidate for 2.5.1. Tim Delaney

1 0

Eliminating f_tstate
by "Martin v. Löwis" 22 Jan '07

22 Jan '07

Bug #1579370 reports a crash when accessing the thread state of a terminated thread, when releasing a generator object. In analysing the problem, I found that f_tstate doesn't have much utility: it is used in very few places, and in these places, it might be as good or better to use the *current* thread state (rather than the frame's thread state); in some (most?) of these places, the current thread should be identical to the frame's thread, anyway. So I would like to propose that the f_tstate member is removed from the frame object. For Python 2.5, for compatibility, it probably has to remain where it is, and only PyTraceBack_Here should stop using it. As a consequence, a generator .send() makes exceptions occur in the current thread, not in the thread where the generator was created. What do you think? Regards, Martin

3 2

syntax misfeature (exception)
by Neal Becker 22 Jan '07

22 Jan '07

I accidentally wrote: try: ... except a,b: rather than: try ... except (a,b): It appears that the 1st example syntax is silently accepted, but doesn't seem to work. Is this true? If so, I'd say it's a wart.

5 6

Problem between deallocation of modules and func_globals
by Brett Cannon 20 Jan '07

20 Jan '07

I have discovered an issue relating to func_globals for functions and the deallocation of the module it is contained within. Let's say you store a reference to the function encodings.search_function from the 'encodings' module (this came up in C code, but I don't see why it couldn't happen in Python code). Then you delete the one reference to the module that is stored in sys.modules, leading to its deallocation. That triggers the setting of None to every value in encodings.__dict__. Oops, now the global namespace for that module has everything valued at None. The dict doesn't get deallocated since a reference is held by encodings.search_function.func_globals and there is still a reference to that (technically held in the interpreter's codec_search_path field). So the function can still execute, but throws exceptions like AttributeError because a module variable that once held a dict now has None and thus doesn't have the 'get' method. My question is whether this is at all worth trying to rectify. Since Google didn't turn anything up I am going to guess this is not exactly a common thing. =) That would lead me to believe some (probably most) of you will say, "just leave it alone and work around it". The other option I can think of is to store a reference to the module instead of just to its __dict__ in the function. The problem with that is we end up with a circular dependency of the functions in modules having a reference to the module but then the module having a reference to the functions. I tried not having the values in the module's __dict__ set to None if the reference count was above 1 and that solved this issue, but that leads to dangling references on anything in that dict that does not have a reference stored away somewhere else like encodings.search_function. Anybody have any ideas on how to deal with this short of rewriting some codecs stuff so that they don't depend on global state in the module or just telling me to just live with it? -Brett

4 12

Weekly Python Patch/Bug Summary
by Kurt B. Kaiser 20 Jan '07

20 Jan '07

Patch / Bug Summary ___________________ Patches : 423 open ( +2) / 3539 closed ( +9) / 3962 total (+11) Bugs : 960 open ( -3) / 6446 closed (+20) / 7406 total (+17) RFE : 258 open ( +3) / 249 closed ( +3) / 507 total ( +6) New / Reopened Patches ______________________ Add aliases for latin7/9/10 charsets (2007-01-13) http://python.org/sf/1634778 opened by Christoph Zwerschke htonl et al accept negative ints (2007-01-14) CLOSED http://python.org/sf/1635058 opened by Mark Roberts Logging Module - followfile patch (2006-11-17) CLOSED http://python.org/sf/1598415 reopened by cjschr CSV DictWriter Errors (2007-01-14) http://python.org/sf/1635454 opened by Mark Roberts strptime %F and %T directives (2007-01-14) CLOSED http://python.org/sf/1635473 opened by Mark Roberts File Read/Write Flushing Patch (2007-01-16) http://python.org/sf/1636874 opened by jurojin urllib: change email.Utils -> email.utils (2007-01-16) http://python.org/sf/1637157 opened by Russell Owen urllib2: email.Utils->email.utils (2007-01-16) http://python.org/sf/1637159 opened by Russell Owen smtplib email renames (2007-01-16) http://python.org/sf/1637162 opened by Russell Owen Add httponly to Cookie module (2007-01-17) http://python.org/sf/1638033 opened by Arvin Schnell compiler.pycodegen causes crashes when compiling 'with' (2007-01-17) http://python.org/sf/1638243 opened by kirat Fix to the long("123\0", 10) problem (2007-01-18) http://python.org/sf/1638879 opened by Calvin Spealman email.utils.parsedate documentation (2007-01-19) http://python.org/sf/1639973 opened by Mark Roberts Patches Closed ______________ platform.py support for IronPython (2006-09-23) http://python.org/sf/1563842 closed by lemburg Small upgrades to platform.platform() (2005-11-10) http://python.org/sf/1352731 closed by lemburg pybench support for IronPython (2006-09-23) http://python.org/sf/1563844 closed by lemburg Py3k: Fix pybench so it runs (2007-01-12) http://python.org/sf/1634499 closed by gvanrossum Bug fixes for int unification branch (2006-12-20) http://python.org/sf/1619846 closed by gvanrossum htonl et al accept negative ints (2007-01-14) http://python.org/sf/1635058 closed by gvanrossum Logging Module - followfile patch (2006-11-17) http://python.org/sf/1598415 closed by vsajip strptime %F and %T directives (2007-01-14) http://python.org/sf/1635473 closed by bcannon BSD version of ctypes.util.find_library (2006-12-07) http://python.org/sf/1610795 closed by theller New / Reopened Bugs ___________________ Problem running a subprocess (2007-01-13) http://python.org/sf/1634739 opened by Florent Rougon locale 1251 does not convert to upper case properly (2007-01-13) http://python.org/sf/1634774 opened by Ivan Dobrokotov Little mistake in docs (2007-01-14) http://python.org/sf/1635217 opened by anatoly techtonik Add registry functions to windows postinstall (2007-01-14) http://python.org/sf/1635335 opened by anatoly techtonik expanduser tests in test_posixpath fail if $HOME ends in a / (2007-01-14) CLOSED http://python.org/sf/1635353 opened by Marien Zwart Add command line help to windows unistall binary (2007-01-14) http://python.org/sf/1635363 opened by anatoly techtonik ConfigParser does not quote % (2007-01-15) http://python.org/sf/1635639 opened by Mark Roberts Interpreter seems to leak references after finalization (2007-01-15) http://python.org/sf/1635741 opened by B Sizer description of the beta distribution is incorrect (2007-01-15) CLOSED http://python.org/sf/1635892 opened by elgordo Newline skipped in "for line in file" (2007-01-16) http://python.org/sf/1636950 opened by Andy Monthei Python-2.5 segfault with tktreectrl (2007-01-16) CLOSED http://python.org/sf/1637022 opened by klappnase Python 2.5 fails to build on AIX 5.3 (xlc_r compiler) (2007-01-16) http://python.org/sf/1637120 opened by Orlando Irrazabal mailbox.py uses old email names (2007-01-16) http://python.org/sf/1637167 opened by Russell Owen make_table in difflib does not work with unicode (2007-01-18) http://python.org/sf/1637850 opened by y-unno Problem packaging wx application with py2exe. (2007-01-17) CLOSED http://python.org/sf/1637943 opened by Indy typo http://www.python.org/doc/current/tut/node10.html (2007-01-17) CLOSED http://python.org/sf/1637952 opened by jim pruett langref: missing item in numeric op list (2007-01-17) CLOSED http://python.org/sf/1637967 opened by paul rubin Incorrect documentation for random.betavariate() (2007-01-18) CLOSED http://python.org/sf/1638627 opened by Troels Walsted Hansen Bugs Closed ___________ class derived from float evaporates under += (2007-01-11) http://python.org/sf/1633630 closed by gbrandl file.encoding doesn't apply to file.write (2004-06-07) http://python.org/sf/967986 closed by lemburg platform.libc_ver() fails on Cygwin (2004-04-02) http://python.org/sf/928297 closed by lemburg subprocess swallows empty arguments under win32 (2007-01-12) http://python.org/sf/1634343 closed by astrand htonl, ntohl don't handle negative longs (2006-12-20) http://python.org/sf/1619659 closed by gvanrossum expanduser tests in test_posixpath fail if $HOME ends in a / (2007-01-14) http://python.org/sf/1635353 closed by marienz time.strftime() accepts format which time.strptime doesnt (2007-01-11) http://python.org/sf/1633628 closed by bcannon SaveConfigParser.write() doesn't quote %-Sign (2006-11-27) http://python.org/sf/1603688 closed by gbrandl logging module / wrong bytecode? (2007-01-11) http://python.org/sf/1633605 closed by vsajip description of the beta distribution is incorrect (2007-01-15) http://python.org/sf/1635892 closed by rhettinger os.path.expandvars deletes things on w32 (2001-12-18) http://python.org/sf/494589 closed by sjoerd Python-2.5 segfault with tktreectrl (2007-01-16) http://python.org/sf/1637022 closed by loewis Problem packaging wx application with py2exe. (2007-01-17) http://python.org/sf/1637943 closed by bcannon typo http://www.python.org/doc/current/tut/node10.html (2007-01-17) http://python.org/sf/1637952 closed by gbrandl langref: missing item in numeric op list (2007-01-17) http://python.org/sf/1637967 closed by gbrandl Incorrect type in PyDict_Next() example code (2007-01-05) http://python.org/sf/1629125 closed by gbrandl website issue reporter down (2007-01-03) http://python.org/sf/1627036 closed by gbrandl Incorrect documentation for random.betavariate() (2007-01-18) http://python.org/sf/1638627 closed by loewis Garbage output to file of specific size (2007-01-08) http://python.org/sf/1630894 deleted by mculbert Forwarding events and Tk.mainloop problem (2006-05-05) http://python.org/sf/1482402 closed by loewis New / Reopened RFE __________________ csv.DictWriter: Include offending name in error message (2007-01-13) http://python.org/sf/1634717 opened by Gabriel Genellina Please provide rsync-method in the urllib[2] module (2007-01-13) http://python.org/sf/1634770 opened by Matthias Klose if __name__=='__main__' missing in tutorial (2007-01-17) http://python.org/sf/1637365 opened by Gabriel Genellina Empty class 'Object' (2007-01-17) http://python.org/sf/1637926 opened by kxroberto add type defintion support (2007-01-18) CLOSED http://python.org/sf/1639002 opened by djnet RFE Closed __________ logging.RotatingFileHandler has no "infinite" backupCount (2006-09-28) http://python.org/sf/1567331 closed by vsajip Print full exceptions as they occur in logging (2006-09-06) http://python.org/sf/1553380 closed by vsajip add type defintion support (2007-01-18) http://python.org/sf/1639002 closed by gbrandl

1 0

Deletion order when leaving a scope?
by Larry Hastings 20 Jan '07

20 Jan '07

I just ran a quickie experiment and determined: when leaving a scope, variables are deleted FIFO, aka in the same order they were created. This surprised me; I'd expected them to be deleted LIFO, aka last first. Why is it thus? Is this behavior an important feature or an irrelevant side-effect? Cheers, /larry/

7 6

BangPycon 2007 - First Indian Python Conference : Call for Participation
by Anush Shetty 19 Jan '07

19 Jan '07

Hi, The BangPypers (Bangalore Python Users Group) invite you to submit a proposal to present a technical paper or tutorial, or to host a meeting or BOF session at the first Indian Python conference here at St Josephs PG College Auditorium in Bangalore,India on March 3rd,2007. There are no fixed guidelines except for the below. 1) The talk has to be on a Pythonic Topic 2) It will be restricted to 45 mts with buffer for Q& A -10 mts 3) Though not mandatory, preference will be given to relevance of the topic with the agenda of the conference being "to promote Python" Though currently the plans of having a separate tutorial section is not on, we still want to hear from you, and we can see whether something can be fitted in... There might be just 10 talks split into 2 tracks. So please hurry up and send in your proposals. Your Proposal should be mailed conference AT bangpypers DOT org BangPycon First Indian Python Conference, March 3rd 2007, St Josephs PG College, Bangalore, India. Email: conference(a)bangpypers.org http://groups.yahoo.com/group/bangpypers http://bangpypers.org -- Cheers. Anush Shetty

1 0

adding _Py prefix to names in 2.5.1?
by Neal Norwitz 19 Jan '07

19 Jan '07

http://python.org/sf/1637022 points out a problem caused by the lack of a _Py prefix on Ellipsis. Most (all?) of the new AST names are not prefixed. These are all meant to be internal names. Are there any issues with changing this? If we do so, it means that any module built with 2.5 that is using these names will fail to work in 2.5.1. No code outside the core *should* be using these names. Assuming there is no dissent, does some budding Python developer want to take on a patch? This should be pretty straightforward. Feel free to mail me off list if you prefer. Cheers, n

4 3

Re: [Python-Dev] buglet in long("123\0", 10)
by Calvin Spealman 19 Jan '07

19 Jan '07

Added a check in test_long.LongTest.test_misc() that long("123\0", 10) fails properly and adapted the patch to int_new to long_new. I get this weird feeling that if its impossible for the function (PyLong_FromString) to know if its being given bad data, having know way to know if the string is supposed to continue past the zero-byte, then doesn't it make sense to say that the function by design is broken? Anyway, here is the patch. Index: Objects/longobject.c =================================================================== --- Objects/longobject.c (revision 53406) +++ Objects/longobject.c (working copy) @@ -3287,10 +3287,27 @@ return PyLong_FromLong(0L); if (base == -909) return PyNumber_Long(x); - else if (PyString_Check(x)) + if (PyString_Check(x)) { + /* Since PyLong_FromString doesn't have a length parameter, + * check here for possible NULs in the string. */ + char *string = PyString_AS_STRING(x); + if (strlen(string) != PyString_Size(x)) { + /* create a repr() of the input string, + * just like PyLong_FromString does. */ + PyObject *srepr; + srepr = PyObject_Repr(x); + if (srepr == NULL) + return NULL; + PyErr_Format(PyExc_ValueError, + "Invalid literal for long() with base %d: %s", + base, PyString_AS_STRING(srepr)); + Py_DECREF(srepr); + return NULL; + } return PyLong_FromString(PyString_AS_STRING(x), NULL, base); + } #ifdef Py_USING_UNICODE - else if (PyUnicode_Check(x)) + if (PyUnicode_Check(x)) return PyLong_FromUnicode(PyUnicode_AS_UNICODE(x), PyUnicode_GET_SIZE(x), base); @@ -3439,3 +3456,4 @@ long_new, /* tp_new */ PyObject_Del, /* tp_free */ }; + Index: Lib/test/test_long.py =================================================================== --- Lib/test/test_long.py (revision 53406) +++ Lib/test/test_long.py (working copy) @@ -299,6 +299,8 @@ slicemin, slicemax = X()[-2L**100:2L**100] self.assertEqual(X()[slicemin:slicemax], (slicemin, slicemax)) + self.assertRaises(ValueError, long, "123\0", 10) + # ----------------------------------- tests of auto int->long conversion def test_auto_overflow(self): ### END PATCH On 1/15/07, Neal Norwitz <nnorwitz(a)gmail.com> wrote: > Calvin, > > Was Guido's answer enough or did you want some more? If you take a > look at the svn rev I sent, I think it should be pretty easy to come > up with a patch for long too. > > Getting the size won't help (exactly), since the problem is the diff > between strlen() and the size. You can see this in the int() fix. > > Let me know if you have any questions. It would be great if you could > produce a patch. > > Cheers, > n > -- > On 1/14/07, Calvin Spealman <ironfroggy(a)gmail.com> wrote: > > Is it a more general problem that null-terminated strings are used > > with data from strings we specifically allow to contain null bytes? > > Perhaps a migration of *FromString() to *FromStringAndSize() > > functions, or taking Python string object pointers, would be a more > > general solution to set as a goal, so this sort of thing can't crop up > > down the road, again. > > > > I know I'm still very uninitiated in the internals of CPython, so > > anyone please correct me if my thoughts here are against any on-going > > policy or reasoning. > > > > On 1/14/07, Neal Norwitz <nnorwitz(a)gmail.com> wrote: > > > SVN rev 52305 resolved Bug #1545497: when given an explicit base, > > > int() did ignore NULs embedded in the string to convert. > > > > > > However, the same fix wasn't applied for long(). > > > > > > n > > > > > > On 1/13/07, Guido van Rossum <guido(a)python.org> wrote: > > > > What's wrong with this session? :-) > > > > > > > > Python 2.6a0 (trunk:53416, Jan 13 2007, 15:24:17) > > > > [GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin > > > > Type "help", "copyright", "credits" or "license" for more information. > > > > >>> int('123\0') > > > > Traceback (most recent call last): > > > > File "<stdin>", line 1, in <module> > > > > ValueError: null byte in argument for int() > > > > >>> int('123\0', 10) > > > > Traceback (most recent call last): > > > > File "<stdin>", line 1, in <module> > > > > ValueError: invalid literal for int() with base 10: '123\x00' > > > > >>> long('123\0') > > > > Traceback (most recent call last): > > > > File "<stdin>", line 1, in <module> > > > > ValueError: null byte in argument for long() > > > > >>> long('123\0', 10) > > > > 123L > > > > >>> > > > > > > > > -- > > > > --Guido van Rossum (home page: http://www.python.org/~guido/) > > > > _______________________________________________ > > > > Python-Dev mailing list > > > > Python-Dev(a)python.org > > > > http://mail.python.org/mailman/listinfo/python-dev > > > > Unsubscribe: http://mail.python.org/mailman/options/python-dev/nnorwitz%40gmail.com > > > > > > > _______________________________________________ > > > Python-Dev mailing list > > > Python-Dev(a)python.org > > > http://mail.python.org/mailman/listinfo/python-dev > > > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ironfroggy%40gmail.com > > > > > > > > > -- > > Read my blog! I depend on your acceptance of my opinion! I am interesting! > > http://ironfroggy-code.blogspot.com/ > > _______________________________________________ > > Python-Dev mailing list > > Python-Dev(a)python.org > > http://mail.python.org/mailman/listinfo/python-dev > > Unsubscribe: http://mail.python.org/mailman/options/python-dev/nnorwitz%40gmail.com > > > -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/

2 2

Testers wanted: mailbox.py bugfix
by A.M. Kuchling 18 Jan '07

18 Jan '07

http://www.python.org/sf/1599254 is a bug in the mailbox.py module that affects single-file mailbox formats (mbox, Babyl, MMDF) and can cause data loss. A proposed fix, mailbox-unified-patch, is attached to the bug. If you use mailbox.py for modifying mailboxes in the above formats, can you please test the patch and report if there are any problems? The bug is pretty complicated, and such bugs in the mailbox.py (and the necessary fixes) make me nervous. Some more testing would be very reassuring. --amk

2 2

Re: [Python-Dev] [Python-3000] dict.items as attributes [Was: The bytes type]
by Jim Jewett 18 Jan '07

18 Jan '07

Other than dict.items (and .keys and .values) returning a non-list, are there any other cases where the Py3K idiom can't already be used in (or at least backported to) Py 2.x? Is there a chance that this special case could be handled by an import hook in py 2.6? For example 2.6 could have an additional LOAD_ATTR_PY3 opcode that just handles the special cases before delegating to the normal LOAD_ATTR. It would be generated instead of LOAD_ATTR whenever >>> from __future__ import py3 was in effect. Supporting 2.5 (and 2.2) would be a bit harder, but I think the new exception syntax would already require a py3->py2 translator, and these three names still seem like a fairly simple transform. So long as the "real" source is kept in py3 format, autogenerating python 2 code shouldn't be much worse than autogenerating pyc files. -jJ

4 4

Pythn 2.5 buildbots do not build anymore?
by Thomas Heller 18 Jan '07

18 Jan '07

Why don't the Python 2.5 buildbots build anymore? http://www.python.org/dev/buildbot/2.5/ Thanks, Thomas

2 1

Re: [Python-Dev] The bytes type
by Phillip J. Eby 18 Jan '07

18 Jan '07

At 07:47 AM 1/16/2007 -0800, Guido van Rossum wrote: >On 1/16/07, James Y Knight <foom(a)fuhm.net> wrote: > > On Jan 15, 2007, at 8:02 AM, Thomas Wouters wrote: > > > There seems to be rather a lot of confusion. No one is suggesting > > > Python 3.0 be anything less for the sake of backward compatibility. > > > Instead, it has been suggested Python 2.6 (and possibly 2.7) be > > > something *more* in order to provide for an easier upgrade path. No > > > compromises in Python 3.0. > > > > True: nobody is suggesting python 3.0 be anything less. But, I am > > indeed suggesting that Python 3.0 be something *more*: I am > > suggesting that people keep in mind the ease of writing of a program > > which can run on both 2.5 and 3.0. And wherever possible, act so as > > to preserve that ease. That may indeed involve a "compromise" in 3.0. > >I'm not keen on compromises in 3.0, but without specific proposals I >don't see why we're arguing. So, please, what specific thing(s) are >you proposing we do in 3.0? Please make a list of specifics rather >than attempting at specifying a general rule to match things that >could go into the list; you've tried the latter and I still don't know >what you want. I think what he's saying boils down to two things: * Don't remove any feature for which an alternative doesn't already exist in 2.5 * Don't change APIs (e.g. items()) Now, I'm not sure these goals are achievable with respect to 2.5. I think we'd be better off adding compatibility features in 2.6. To be honest, the items() and keys() thing personally baffles me. If they're supposed to be *views* on the underlying dictionary, wouldn't it make more sense for them to be *attributes* instead of methods? I.e. dict.items and dict.keys. Then, we could provide that feature in 2.6, and drop the availability of the callable forms in 3.0. Then you could write code like: for k,v in somedict.items: ... And have it work in 2.6 and 3.0. Meanwhile, .items() would still return a list in 2.6 (but be warnable about with a -3 switch), but go away entirely in 3.0. It's not a panacea, but at least would make it *possible* to write code that runs on both 3.0 and some 2.x version. Without having at least *some* 2.x version that can run 3.x code, I think there is little chance of 3.0 becoming viable. You've been comparing this to Zope 2/Zope 3, but in that world there is something called "Five" that lets you do Zope 3 things inside of Zope 2, so you have some chance of porting your code in an incremental fashion, without having to leap everything over in one go. You've cited JoS on your decision not to do 3.0 as a ground-up rewrite, so perhaps you'll find this other JoS article relevant here: """It turns out that what was stopping people from switching to Excel was that everybody else they worked with was still using Lotus 123. They didn't want a product that would create spreadsheets that nobody else could read: a classic Chicken and Egg problem. When you're the lone Excel fan in a company where everyone else is using 123, even if you love Excel, you can't switch until you can participate in the 123 ecology.""" http://www.joelonsoftware.com/articles/fog0000000052.html The analogy isn't perfect, because we are not so much trying to provide backward compatibility in "Excel" as to add *forward* compatibility to "123", but you get the idea.

5 5

Re: [Python-Dev] The bytes type
by glyph＠divmod.com 17 Jan '07

17 Jan '07

On 06:49 pm, python(a)rcn.com wrote: >I think we should draw a line in the sand and resolve not to garbage-up Py2.6. >The whole Py3.0 project is about eliminating cruft and being free of the >bonds of backwards compatibility. Adding non-essential cruft to Py2.6 >goes against that philosophy. Emotionally charged like "cruft" and "garbage" are obscuring the issue. Let's replace them with equivalents charged in the opposite direction: "I think we should draw a line in the sand and resolve not to compatibility-up Py2.6. The whole Py3.0 project is about eliminating useful libraries and being free of the bonds of working software. Adding non-essential forward-compatibility to Py2.6 goes against that philosophy." The benefit (to me, and to many others) of 3.x over 2.x is the promise of more future maintenance, not the lack of cruft. In fact, if I made a list of my current top ten problems with Python, "cruft" wouldn't even make it in. There is lots of useful software that will not work in the 3.0 series, and without forward compatibility there is no way to get there from here. As Guido said, if 3.0 is going to break compatibility, that burdens the 2.x series with the need to provide transitional functionality. The upgrade path needs to be available in one version or the other, or 2.x needs to be maintained forever. You can't have it both ways.

7 10

Re: [Python-Dev] Proposed 3.0 compatiblity module
by Phillip J. Eby 17 Jan '07

17 Jan '07

At 03:47 PM 1/16/2007 -0800, Guido van Rossum wrote: >Oh, as long as we're talking adding features to 2.6 I'm fine. I >thought you were proposing changes to the plans for 3.0, as in your >proposal" regarding the dict view API. Nah, that was a side tangent, combined with a partial lack of understanding of how the view thing was supposed to work. Having to wrap 'em with list() calls is annoying but doable. I was just throwing out an idea I didn't realize had already been shot down.

1 0

Proposed 3.0 compatiblity module
by Raymond Hettinger 17 Jan '07

17 Jan '07

ISTM, we can decide to limit 3.0's impact to a single compatibility module, essentially as an alternate set of builtins. import __new_builtins__ for name in 'bytes dict coerce'.split(): # list any new features you want to support setattr(__builtins__, name, getattr(__new_builtins__, name)) The bytes type would just be the new type. The dict would have keys=iterkeys, no has_key, etc. The coerce function would work but have a DeprecationWarning or somesuch. If you want future division, we already have a mechanism for that. If you want to eliminate backticks, just grep for them. If you don't care about 3.0 compatibilty, just don't import the module. Outside of a compatibility module, we can decide to leave Py2.6 unmolested by Py3.0, so I can stop being cranky. Raymond

4 6

Re: [Python-Dev] Proposed 3.0 compatiblity module
by Phillip J. Eby 17 Jan '07

17 Jan '07

At 03:18 PM 1/16/2007 -0800, Guido van Rossum wrote: >On 1/16/07, Phillip J. Eby <pje(a)telecommunity.com> wrote: > > The idea here being that, once 2.6 is widely-enough deployed that it can be > > assumed as a base for one's users, you can simply run the translator once, > > do any cleanup, and then have 3.0-clean code that also still runs for your > > installed base. > > > > That way, there's no chasm to leap; just a code cleanup. > >I understand; I would rather have that too, everything else being the >same. But everything else wouldn't be the same -- it would place many >more restrictions on 3.0, and the common subset would still be much >smaller. For me personally, the weight of the added restrictions to >3.0 is the killer. I don't understand; how would adding features to 2.6 restrict what you could add to 3.0?

2 1

Re: [Python-Dev] [Python-3000] Warning for 2.6 and greater
by "Martin v. Löwis" 17 Jan '07

17 Jan '07

glyph(a)divmod.com schrieb: > Also, the exact strategy I suggested could be implemented in various > ways that might be efficient. Here are a few ways it might be made more > efficient than the straw many of one extra dict lookup per call to > keys() et. al.: I'm not saying that an efficient implementation is impossible, but I still don't see it. The ones you suggest don't work that well: > * Don't make it a dictionary key; make it an attribute of the module at > the C level. Unfortunately, in a function, there is no reference to the module it came from. There is __module__ (on the function, not on the code object), but that is a string. > * Annotate the code object in some way that makes code compiled in that > scope set a global variable to enable the compatibility; protect it with > the GIL. This I don't quite understand: what has the GIL to do with that? Are you suggesting to hold the GIL while the entire function is running? That would restrict concurrency even more than the GIL already restricts it. Also, what about nested functions spanning multiple modules (with different settings)? > * These methods are already function pointers in a data structure in C; > actually have a py->py3k mode switch which swaps out the C function > pointers to the appropriate implementation for the calling code. That doesn't work: The items method is per dictionary (or actually per type, i.e. for the dict type). The same dictionary, when accessed from different modules, should have .items() behave differently. Wrt. that property: # a.py from __future__ import items_is_iter def f(d): return d.items # b.oy import a d = {} print a.f(d)() What should that print? I think I take back my earlier suggestion that I can imagine an inefficient implementation. I cannot imagine any reasonable behaviour of a per-module change of methods on dictionary objects. > Do I just suffer from having an overactive imagination? Are all of > these implementation strategies impossible for some reason, and there > are no others? Yes, and yes. Regards, Martin

3 3

Re: [Python-Dev] The bytes type
by Phillip J. Eby 17 Jan '07

17 Jan '07

At 09:50 AM 1/16/2007 -0800, Guido van Rossum wrote: >Actually it's very easy to write code using keys(), items() and >values() that works as well in 2.2 as it works in 3.0: never use the >iter* variants, and don't sweat the performance costs of creating a >list so much. If you can't afford to create a list, iterate over the >dict itself, which will give you the keys in both versions. Actually, the main reason I use items() or keys() is because I want to mutate the dictionary, or because something else is going to mutate it while I'm looping over it. But yeah, that discussion should be on Py3K. >>Without having at least *some* 2.x version that can run 3.x code, I think >>there is little chance of 3.0 becoming viable. You've been comparing this >>to Zope 2/Zope 3, but in that world there is something called "Five" that >>lets you do Zope 3 things inside of Zope 2, so you have some chance of >>porting your code in an incremental fashion, without having to leap >>everything over in one go. > >I'm not aware of compromises to 3.0's architecture that were made in >order to ease the transition though -- all of the burden was placed on >Zope 2.x (though perhaps some optional compatibility packages were >also added to Zope 3.0?). Right, which is why I don't think we should worry about 2.5 as much as 2.6. >>The analogy isn't perfect, because we are not so much trying to provide >>backward compatibility in "Excel" as to add *forward* compatibility to >>"123", but you get the idea. > >Such analogies are hard to get right; in this case he's talking about >file interchange formats. Perhaps the closest analogy is pickling. Actually, I meant the analogy for *code*. If you can't write code that runs on some 2.x + 3.x combination, you have no way to cross the chasm. If you're writing "spreadsheets for Excel" (code that runs on 3.x), you need to be able to "share them with 1-2-3 users" (run that code on 2.x).

1 0

Re: [Python-Dev] [Python-3000] Warning for 2.6 and greater
by glyph＠divmod.com 16 Jan '07

16 Jan '07

On 07:42 pm, thomas(a)python.org wrote: >On 1/10/07, Raymond Hettinger <raymond.hettinger(a)verizon.net> wrote: >> >><"Anthony Baxter"> >> > Comments? What else should get warnings? >> >>It is my strong preference that we not go down this path. >>Instead, the 2.6 vs 3.0 difference analysis should go in an >>external lint utility. >Having Python 2.6 optionally warn for >3.0-compatibility is a lot easier for the average developer than having a >separate tool or a separately compiled Python. I could not possibly agree more. Given the highly dynamic nature of Python, such a tool will *at best* catch only the most egregious uses of deprecated features. Backticks are easy enough to find, but the *only* way that I can reasonably imagine migrating a body of code like Twisted (or any non-trivial Python library) would be having a way to examine the warning output of its test suite. I am suuuuper +1 on the warnings for 2.6, as well as forward-compatibility at some point in the 2.x series for new syntax. Without the ability to bridge 2.x->3.0 during some interim period, I can say for sure that Twisted _will not_ migrate to 3.0, ever. We are really a small project and just don't have the manpower to maintain two overlapping but mutually incompatible codebases. I've been assuming for some time that the only hope for Py3k compatibility within Twisted would be using PyPy as a translation layer. With the addition of runtime compatibility warnings, it might be feasible that we could run on the "bare metal" (ha ha) of Python3's VM.

22 34

Re: [Python-Dev] [Python-3000] Warning for 2.6 and greater
by glyph＠divmod.com 16 Jan '07

16 Jan '07

On 08:19 am, martin(a)v.loewis.de wrote: >Georg Brandl schrieb: >>> If Python 3.0 was simply a release which removed deprecated features, >>> there would clearly be no issue. I would update my code in advance of >>> the 3.0 release to not use any of those features being removed, and >>> I'm all set. But that's not what I'm hearing. Python 3 is both adding >>> new ways to do things, and removing the older way, in the same >>> version, with no overlap. This makes me very anxious. >> >> It has always been planned that in those cases that allow it, the new way to do >> it will be introduced in a 2.x release too, and the old way removed only in 3.x. > >What does that mean for the example James gave: if dict.items is going >to be an iterator in 3.0, what 2.x version can make it return an >iterator, when it currently returns a list? > >There simply can't be a 2.x version that *introduces* the new way, as it >is not merely a new API, but a changed API. The API isn't changed. It's just dependent on its execution context in a confusing way. That difference right now is being reflected as "2.x VM" vs. "3.0 VM" but there are other ways to reflect it more explicitly. It would certainly be possible to have: from __future__ import items_is_iter be the same as: __py3k_compat_items_is_iter__ = True and have the 2.x series' items() method check the globals() of the calling scope to identify the return value of items() in that particular context. If the actual data structures of module dictionaries and stack objects are too expensive there are other, similar things that could be done at the C level. This implementation strategy is just the obvious thing that occurred to me after maybe 2 minutes of consideration. I'm sure someone more familiar with the internals of Python could come up with something better.

3 3

buglet in long("123\0", 10)
by Guido van Rossum 15 Jan '07

15 Jan '07

What's wrong with this session? :-) Python 2.6a0 (trunk:53416, Jan 13 2007, 15:24:17) [GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> int('123\0') Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: null byte in argument for int() >>> int('123\0', 10) Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: invalid literal for int() with base 10: '123\x00' >>> long('123\0') Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: null byte in argument for long() >>> long('123\0', 10) 123L >>> -- --Guido van Rossum (home page: http://www.python.org/~guido/)

5 5

Rewrite of import in Python source (sans docs) is complete
by Brett Cannon 15 Jan '07

15 Jan '07

Finally, after a few months worth of work, I have finally gotten far enough in my import rewrite that I am willing to stick my neck out and say it is semantically complete! You can find it in the sandbox under import_in_py. So, details of this implementation. I implemented PEP 302 importers/loaders for built-in, frozen, extension, .py, and .pyc files along with rewriting the steps __import__ goes through to do an import. I also developed an API for .py/.pyc file handling so that there is a generic filesystem importer/loader and a separate handler for .py/.pyc files. This should allow for (relatively) easy selective overriding of just how .py/.pyc files are stored (e.g., introducing a database backend) or how variants on .py/.pyc files are handled (e.g., Quixote's .ptl format). This code has extensive tests and so I am fairly confident that it does what is expected of an import rewrite. There are actually more lines in the test file than the implementation. There is also a mock implementation used for testing. Was interesting doing this in such a test-driven, XP style of only coding what I needed. I have run this code through the entire regression test suite and that is where you find out subtle differences between this implementation and the built-in import (you can see for yourself with the regrtest.sh shell script). First test_pkg will fail because currently the new import adds a __loader__ attribute on all modules (that might change for security reasons) and test_pkg is an old, stdout comparing test. Second, test_runpy fails because I have not implemented get_code on the filesystem loader which is required by runpy. Both are shallow issues that can be dealt with. Third, and the hardest difference to deal with, is that you will get some warnings that print out that you normally don't see. This is because warnings.warn and its stack_level argument don't have the effect people are used to when importing a deprecated module. Before you could set stack_level to 2 and it would look like it came from the import statement. But now, with import written in Python and thus on the call stack compared to being in C and thus not showing up, two levels back is still in the import code. I really don't know how this should be dealt with short of saying that the rule of thumb with 2 stack levels back for a warning does not work when done at the import level. It is not blazing fast at the moment. Some things, like the built-in and frozen importers/loaders could be rewritten in C without huge issue. I am also sure I have made some stupid design decisions at various points in the code. But there is benchmarking code in the sandbox called importbench and it showed up a 10x speed slowdown on a Linux box I was using in mid to late December when doing a fresh import of certain types (I don't remember exactly which kind off the top of my head). Because of this current slowness I don't know if people want to rush into trying to make this the default import implementation quite yet or if this is not too big of a thing since the common case of pulling out of sys.modules is not that much slower. I know I am currently not planning on devoting the time to bootstrap it in as I have my security work to finish first along with other Python stuff that seems more pressing to me. And since (I think) I don't need to bootstrap it in order to finish my security work I can't justify spending work time on it. But I can rearrange priorities if people really want to pursue this (especially if I can get some help with it). As with the module's name, it is currently named 'importer', but that is bad since it conflicts with the idea of importers from PEP 302. I was thinking importlib, but I wanted to wait and see what other people thought. Don't know if you guys are okay with me checking this in without having it vetted by the community first like we prefer all new modules to do. I have not done the LaTeX docs yet. I think that is all of the details that I can think of. I am still working towards implementing the security needed so that an application that embeds Python can execute arbitrary code securely. Giving a talk at PyCon on the topic for anyone interested. Special thanks needs to go to Paul Moore who I talked to through most of the design of the code. Nick Coghlan also provided some handy feedback. And Neal Norwitz for bugging about wanting something like this done. Plus thanks to everyone who has shown support. -Brett

2 2

pydoc - revised
by Laurent Gautier 15 Jan '07

15 Jan '07

Hi all, This is a somehow an update to the thread on pydoc started by Ron. Since the last entry we took the bull by the horns (so to speak), and are seriously aiming at delivering something that can qualify as: - a revision of the module pydoc - something that will facilitate the inclusion of ideas explored in efforts like the nice epydoc - something that is in general easily extensible to answer most of the need (both regarding the output that is returned, and what are the capabilities regarding the exploration of the documentation). We are probably setting the bar a little high, but we hope to get somewhere at least with the first point thanks to a number of advices on this list, some of which by pydoc's or ipython's original authors (beside that, the bar could not be elsewhere since I am not a very good limbo dancer ;-) ). There is a sourceforge page for that project, since it lets us work with SVN without risking to disturb a larger project, but we are ready to have that moved to the sandbox in the python tree (although it would be neat to wait a little until there are more tests). svn co https://pydoc-r.svn.sourceforge.net/svnroot/pydoc-r pydoc-r Still, the thought process is not over and we were thinking of having discussions recorded either in the form of posts on python-dev, or say using the mailing list on the sourceforge page. An example of discussions we had are about whether we should already name the module pydoc or an other name, Posting on python-dev would give exposure to what is discussed, but at the same could be perceived as off-topic until the module is in the sandbox. Posting on the sourceforge page would have pretty much opposite points (no exposure, but no one would feel annoyed) Any suggestion ? Thanks, Laurent

2 1

IOError errno=514 in time.sleep(1)
by Sean Reifschneider 14 Jan '07

14 Jan '07

I've been tracking down an issue with some code sporadically raising IOError with errno=514 on "time.sleep(1)". time.sleep() is implemented with the select() system call, and on Linux this is sporadically returning errno=514 (ERESTARTNOHAND). This is on 2.6.9, though I've heard reports of it happening through 2.6.17, and my review of the code indicates that 2.6.19.1 (the latest kernel) is probably also going to act the same. Now, the header file where ERESTARTNOHAND is defined says that user-space should never see this errno. Clearly, that's not as true as it could be. I'm going to ask the LKML for more information about whether this should make it to user-space, but thought I'd bring it up here as well. Some options we have: Do nothing. (I like this one :-) Catch and ignore 514. (Only on Linux? Far-reaching implications, I don't like this one). Use the libc sleep() call for integer time.sleep() values? No guarantee that libc sleep() doesn't use select, though I've seen indications that it might use alarm. I haven't looked at the libc source. Try to make time.sleep() smarter in the face of errnos. Maybe try the sleep multiple times, check time.time() before and after the sleep to do multiple sleeps. Could possibly even fail over to using /dev/rtc if select fails? In general I'm reluctant to suggest a Python fix to this, but it's fairly unexpected for time.sleep() to raise an exception... Thoughts? Sean -- I think the net needs some Viagra today. It's just not performing... -- Mike Loseke, 2000 Sean Reifschneider, Member of Technical Staff <jafo(a)tummy.com> tummy.com, ltd. - Linux Consulting since 1995: Ask me about High Availability

2 1

C standard io library, flush/read/write problem
by Emerson Clarke 14 Jan '07

14 Jan '07

The other night i was watching a google techtalk about python 3000 and Guido mentioned some problems with the C standard io library. In particular he highlighted an issue with switching between reading and writing without flushing and the fact that it caused serious errors. Not that i dont think its a good idea to write a new io library, but I wondered if it was the same problem ive encounted. It only happens on windows that i know off, but the fix is simple... Assuming you have a hanlde to the file called "Handle" and a Flush() method, the following logic for read and write will allow you to detect and prevent the problem. Add this to the Read() method before reading takes place: if ( Handle && (Handle->_flag & _IORW) && (Handle->_flag & (_IOREAD | _IOWRT)) == _IOWRT ) { Flush(); Handle->_flag |= _IOREAD; } Add this to the Write() method before writing takes place: if ( Handle && (Handle->_flag & _IORW) && (Handle->_flag & (_IOREAD | _IOWRT)) == _IOREAD ) { Flush(); Handle->_flag |= _IOWRT; } Emerson

2 1

updating Misc/NEWS
by Neal Norwitz 14 Jan '07

14 Jan '07

I've noticed a bunch of changes recently without corresponding items added to Misc/NEWS. Can everyone update NEWS especially when fixing bugs or adding new features? If you have made recent checkins, it would be great if you could go back and update Misc/NEWS if you missed it the first time. Thanks, n

1 0

file(file)
by A.M. Kuchling 14 Jan '07

14 Jan '07

Forwarded for discussion from http://www.python.org/sf/1633665. --amk [forwarded from http://bugs.debian.org/327060] Many types in Python are idempotent, so that int(1) works as expected, float(2.34)==2.34, ''.join('hello')=='hello' et cetera. Why not file()? Currently, file(open(something, 'r')) fails with "TypeError: coercing to Unicode: need string or buffer, file found." Semantically, file(fd) should be equivalent to os.fdopen(fd.fileno()) or the proposed file.fromfd() (Jp Calderone, Python-dev, 2003). You should get another independent file object that accesses the same file. What would be gained? Primarily, it would allow you to derive classes from file more easily. At present, if you want to derive like so, you're class can only work when passed a file name or buffer. class file_with_caching(file): def __init__(self, something): file.__init__(self, something) def etcetera... For instance, you have no way of creating a file_with_caching() object from the file descriptors returned from os.fork(). Also, you have no way of taking a file that is already open, and creating a file_with_caching() object from it. So, you can't use classes derived from file() on the standard input or standard output. This breaks the nice Linux OS-level definition of a file descriptor. At the Linux level, you have a nice uniform interface where all file descriptors are equally good. At the python level, some are better than others. It's a case where Python unnecessarily restricts what you can do.

7 6

Re: [Python-Dev] [Python-3000] Warning for 2.6 and greater
by glyph＠divmod.com 13 Jan '07

13 Jan '07

On 11 Jan, 08:22 pm, sluggoster(a)gmail.com wrote: >On 1/11/07, James Y Knight <foom(a)fuhm.net> wrote: >> If the goal is really to have Py 3.0 be released later this year, >There will certainly be demand for an asynchronous server in 3.0, To flip the question around: there might be a demand for Twisted in 3.0, but will there be a demand for 3.0 in Twisted? It might just be easier for everyone concerned to just continue maintaining 2.x forever. I have yet to see a reason why, other than continued maintenance, 3.0 would be a preferable development platform. >So the two projects will operate independently, and the 3.0 one may be >smaller and less ambitious than Twisted. But if the need is there it >will be written. It is quite likely that someone else will write some completely different code for python 3.0 that calls select(). I hadn't considered that the goal of 3.0 was to *discover* these people by alienating existing Python developers - that's crafty! If so, though, you'll have to figure out a way to stop Anthony from providing all this compatibility stuff. He might make it too attractive for us to continue development on future versions :). >How did Perl 4 and Perl 5 handle the situation? I basically waited >2-3 years after Perl 5 came out, then started programming the new way. > If it mattered (it didn't), I would have tied my applications >specifically to Perl 4. I handled the Perl 4 to 5 transition by dropping Perl and moving to Python, because if I was going to port all my code to another language I wanted to at least port to a better language.

2 1

Re: [Python-Dev] [Python-3000] Warning for 2.6 and greater
by glyph＠divmod.com 13 Jan '07

13 Jan '07

On 10:12 am, mwh(a)python.net wrote: >For practical reasons (we have enough work to be getting on with) PyPy >is more-or-less ignoring Python 2.5 at the moment. After funding and >so on, when there's less pressure, maybe it will seem worth it. Not >soon though. I think I know what you mean from previous conversations, but the context of the question makes the answer ambiguous. Are you ignoring 2.5 in favor of 2.4, or 3.0?

2 1

Re: [Python-Dev] file(file)
by glyph＠divmod.com 13 Jan '07

13 Jan '07

On 02:42 am, brett(a)python.org wrote: >Wrapper around open() that does proper checking of its arguments. I >will be discussing my security stuff at PyCon if you are attending and >are interested. I am both, so I guess I'll see you there :).

1 0

Weekly Python Patch/Bug Summary
by Kurt B. Kaiser 13 Jan '07

13 Jan '07

Patch / Bug Summary ___________________ Patches : 421 open ( +3) / 3530 closed ( +8) / 3951 total (+11) Bugs : 963 open ( +4) / 6426 closed (+21) / 7389 total (+25) RFE : 255 open ( +5) / 246 closed ( +1) / 501 total ( +6) New / Reopened Patches ______________________ The Unicode "lazy strings" patches (2007-01-06) http://python.org/sf/1629305 opened by Larry Hastings fast tuple[index] by inlining on BINARY_SUBSCR (2007-01-07) CLOSED http://python.org/sf/1629718 opened by Hirokazu Yamamoto Patch to add tempfile.SpooledTemporaryFile (for #415692) (2007-01-07) http://python.org/sf/1630118 opened by Dustin J. Mitchell Implement named exception cleanup (2007-01-07) CLOSED http://python.org/sf/1630248 opened by Collin Winter Fix crash when replacing sys.stdout in sitecustomize (2007-01-08) http://python.org/sf/1630975 opened by Thomas Wouters SyntaxWarning for backquotes (2007-01-09) http://python.org/sf/1631035 opened by Anthony Baxter implement warnings module in C (2007-01-08) http://python.org/sf/1631171 opened by Neal Norwitz sre module has misleading docs (2007-01-09) http://python.org/sf/1631394 opened by Tom Lynn New exception syntax (2007-01-09) CLOSED http://python.org/sf/1631942 opened by Collin Winter from __future__ import print_function (2007-01-12) http://python.org/sf/1633807 opened by Anthony Baxter Py3k: Fix pybench so it runs (2007-01-13) http://python.org/sf/1634499 opened by Larry Hastings Patches Closed ______________ Win32: Add bytesobject.c to pythoncore.vcproj (2007-01-04) http://python.org/sf/1628062 closed by loewis Win32: Fix build when you have TortoiseSVN but no .svn/* (2007-01-04) http://python.org/sf/1628061 closed by loewis fast tuple[index] by inlining on BINARY_SUBSCR (2007-01-07) http://python.org/sf/1629718 closed by loewis #1603424 subprocess.py wrongly claims 2.2 compatibility. (2006-12-05) http://python.org/sf/1609282 closed by astrand Implement named exception cleanup (2007-01-07) http://python.org/sf/1630248 closed by collinwinter (py3k) Remove the sets module (2006-06-04) http://python.org/sf/1500611 closed by gvanrossum New exception syntax (2007-01-09) http://python.org/sf/1631942 closed by collinwinter backticks will not be used at all (2007-01-03) http://python.org/sf/1627052 closed by gbrandl New / Reopened Bugs ___________________ Python 2.5 64 bit compile fails on Solaris 10/gcc 4.1.1 (2007-01-05) http://python.org/sf/1628484 opened by Bob Atkins Pydoc sets choices for doc locations incorrectly (2007-01-05) http://python.org/sf/1628895 opened by Skip Montanaro xml.dom.minidom parse bug (2007-01-05) CLOSED http://python.org/sf/1628902 opened by Pierre Imbaud clarify 80-char limit (2007-01-05) http://python.org/sf/1628906 opened by Jim Jewett inspect trouble when source file changes (2007-01-05) http://python.org/sf/1628987 opened by phil Incorrect type in PyDict_Next() example code (2007-01-05) http://python.org/sf/1629125 opened by Jason Evans email._parseaddr AddrlistClass bug (2007-01-06) http://python.org/sf/1629369 opened by Tokio Kikuchi array.array borks on deepcopy (2006-08-24) CLOSED http://python.org/sf/1545837 reopened by nnorwitz documentation of email.utils.parsedate is confusing (2007-01-06) http://python.org/sf/1629566 opened by Nicholas Riley Backward incompatibility in logging.py (2006-10-02) CLOSED http://python.org/sf/1569622 reopened by mklaas doc error for re.sub (2007-01-08) CLOSED http://python.org/sf/1630511 opened by Keith Briggs doc misleading in re.compile (2007-01-08) http://python.org/sf/1630515 opened by Keith Briggs Seg fault in readline call. (2007-01-08) http://python.org/sf/1630794 opened by gnovak fnmatch.translate undocumented (2007-01-08) http://python.org/sf/1630844 opened by Gabriel Genellina PyLong_AsLong doesn't check tp_as_number (2007-01-08) CLOSED http://python.org/sf/1630863 opened by Roger Upole Garbage output to file of specific size (2007-01-08) http://python.org/sf/1630894 opened by Michael Culbertson email.Generators does not separates headers with "\r\n" (2005-11-05) http://python.org/sf/1349106 reopened by bwarsaw Discrepancy between iterating empty and non-empty deques (2007-01-09) CLOSED http://python.org/sf/1631769 opened by Christos Georgiou logging.config.fileConfig doesn't clear logging._handlerList (2007-01-10) CLOSED http://python.org/sf/1632328 opened by Stefan H. Holek Hangs with 100% CPU load for certain regexes (2007-01-11) CLOSED http://python.org/sf/1633583 opened by Matthias Klose using locale does not display the intended behavior (2007-01-11) http://python.org/sf/1633600 opened by Matthias Klose logging module / wrong bytecode? (2007-01-12) http://python.org/sf/1633605 opened by Matthias Klose curses should reset curses.{COLS,LINES} when term. resized (2007-01-12) http://python.org/sf/1633621 opened by Matthias Klose time.strftime() accepts format which time.strptime doesnt (2007-01-12) http://python.org/sf/1633628 opened by Matthias Klose class derived from float evaporates under += (2007-01-12) http://python.org/sf/1633630 opened by Matthias Klose incomplete numerical comparisons (2007-01-12) http://python.org/sf/1633648 opened by Matthias Klose file(file) should succeed (2007-01-12) CLOSED http://python.org/sf/1633665 opened by Matthias Klose mailbox.py _fromlinepattern regexp does not support positive (2007-01-12) http://python.org/sf/1633678 opened by Matthias Klose AIX: configure ignores $CC; problems mit C++ comments (2007-01-12) http://python.org/sf/1633863 opened by Johannes Abt for line in sys.stdin: doesn't notice EOF the first time (2007-01-12) http://python.org/sf/1633941 opened by Matthias Klose re.compile("(.*$){1,4}", re.MULTILINE) fails (2007-01-12) http://python.org/sf/1633953 opened by Matthias Klose configure problem for sem_init() on HP-UX (2007-01-12) CLOSED http://python.org/sf/1634033 opened by Johannes Abt AIX: wrong flags for ld when linking standard .so modules (2007-01-12) CLOSED http://python.org/sf/1634105 opened by Johannes Abt subprocess swallows empty arguments under win32 (2007-01-12) http://python.org/sf/1634343 opened by Patrick Mézard Bugs Closed ___________ documentation error for "startswith" string method (2007-01-04) http://python.org/sf/1627956 closed by akuchling null bytes in docstrings (2006-12-26) http://python.org/sf/1622533 closed by akuchling gzip.GzipFile is slow (2003-11-25) http://python.org/sf/849046 closed by akuchling xml.dom.minidom parse bug (2007-01-05) http://python.org/sf/1628902 closed by loewis frame->f_lasti not always correct (2006-01-18) http://python.org/sf/1409443 closed by rhettinger Undocumented implicit strip() in split(None) string method (2005-01-19) http://python.org/sf/1105286 closed by rhettinger split() description not fully accurate (2005-12-14) http://python.org/sf/1380970 closed by rhettinger Underspecified behaviour of string methods split, rsplit (2006-01-25) http://python.org/sf/1414673 closed by rhettinger 32/64bit pickled Random incompatiblity (2006-04-18) http://python.org/sf/1472695 closed by rhettinger array.array borks on deepcopy (2006-08-24) http://python.org/sf/1545837 closed by nnorwitz subprocess.py (py2.5) wrongly claims py2.2 compatibility (2006-11-27) http://python.org/sf/1603424 closed by astrand subprocess.Popen closes fds for sys.stdout or sys.stderr (2006-11-28) http://python.org/sf/1604851 closed by astrand Backward incompatibility in logging.py (2006-10-02) http://python.org/sf/1569622 closed by vsajip doc error for re.sub (2007-01-08) http://python.org/sf/1630511 closed by mwh PyLong_AsLong doesn't check tp_as_number (2007-01-08) http://python.org/sf/1630863 closed by loewis RotatingFileHandler cannot recover from failed doRollover() (2007-01-04) http://python.org/sf/1627575 closed by vsajip Discrepancy between iterating empty and non-empty deques (2007-01-09) http://python.org/sf/1631769 closed by rhettinger ctypes: Returning c_void_p from callback doesn\'t work (2006-10-10) http://python.org/sf/1574593 closed by theller logging.config.fileConfig doesn't clear logging._handlerList (2007-01-10) http://python.org/sf/1632328 closed by vsajip logging's fileConfig causes KeyError on shutdown (2006-08-04) http://python.org/sf/1534765 closed by vsajip Hangs with 100% CPU load for certain regexes (2007-01-11) http://python.org/sf/1633583 closed by niemeyer configure problem for sem_init() on HP-UX (2007-01-12) http://python.org/sf/1634033 deleted by jabt AIX: wrong flags for ld when linking standard .so modules (2007-01-12) http://python.org/sf/1634105 deleted by jabt New / Reopened RFE __________________ Show "expected" token on syntax error (2007-01-12) http://python.org/sf/1634034 opened by Oliver Gramberg RFE Closed __________ file(file) should succeed (2007-01-11) http://python.org/sf/1633665 closed by akuchling

1 0

Re: [Python-Dev] file(file)
by glyph＠divmod.com 13 Jan '07

13 Jan '07

On 12:37 am, brett(a)python.org wrote: >For security reasons I might be asking for file's constructor to be >removed from the type for Python source code at some point (it can be >relocated to an extension module if desired). By forcing people to go >through open() to create a file object you can more easily control >read/write access to the file system (assuming the proper importation >of extension modules has been blocked). Not removing the constructor >allows any code that has been explicitly given a file object but not >open() to just get the class and call the constructor to open a new >file. This is a general problem with type access. Secure versions of any type should not allow access to the type period. It is hardly unique to files, and is not limited to constructors either. How do you, e.g., allow a restricted piece of code write access to only a specified area of the filesystem? More importantly, given the random behavior that open() will be growing (opening sockets? dynamic dispatch on URL scheme???) file() will likely remain a popular way to be sure you are accessing the filesystem.

2 1

Re: [Python-Dev] [Python-3000] Warning for 2.6 and greater
by glyph＠divmod.com 13 Jan '07

13 Jan '07

On 09:04 pm, tcdelaney(a)optusnet.com.au wrote: >I'm wondering if we might be going the wrong way about warning about >compatibility between 2.x and 3.x. Perhaps it might be better if the 3.0 >alpha had a 2.x compatibility mode command-line flag, which is removed late >in the beta cycle. Please, no. I don't think command-line flags are going to help much, because the problem is not the complexity of downloading and compiling py3k, it is the complexity of porting software that has modules from a variety of different sources. More importantly however, I'm not even going to be looking into porting anything to py3k until a year or so after its release - and that's only if 2.6 has given me some tools to deal with the transition. The reason I am posting to this thread is that I had *written off py3k entirely* before Anthony started making noises about forward compatibility. The support cycle of Ubuntu Dapper makes it likely that Twisted will be supporting Python 2.4 until at least 2011. I assume 2.5 will last a year after that, so assuming 2.6 has perfect forward compatibility, we'll be looking at a py3k port in early 2013. Features available in the beta releases aren't going to help me there. Plus, this defeats the whole purpose of py3k. Why break compatibility if you're going to include a bunch of gross compatibility code? Compatibility has already been broken in the py3k branch, there's no point in putting it back there. Adding it to 2.x might make sense though.

1 0

Re: [Python-Dev] Code working in both 2.x and 3.X
by Facundo Batista 12 Jan '07

12 Jan '07

Ron Adam wrote: > I thinking that the 3.0.X version be considered a try it out (alpha) release to > generate plenty of feed back, and the 3.1.X version be the first version meant > for actual development use. +1 for this approach. I think it's very clear, and everybody will understand it (ok, not everybody, we know people...). -- . Facundo . Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/

2 1

Re: [Python-Dev] [Python-3000] Warning for 2.6 and greater
by Phillip J. Eby 12 Jan '07

12 Jan '07

At 11:59 AM 1/12/2007 +0000, glyph(a)divmod.com wrote: >In order to do this, everything that has been changed in 3.0 has to have >some mechanism for working both ways in some 2.x release. I phrased this >as its "feature set" because I am not aware of any new functionality in >3.0 that simply isn't available 2.x - everything I've seen are "cleanups", >which expose basically the same features as 2.5. > >If there are some real new features, then I am in error. If I understand correctly, argument annotations, and exceptions having traceback attributes are new features.

1 0

trunk changes not suited for 3.0
by Georg Brandl 12 Jan '07

12 Jan '07

Hi, currently, changes in the trunk are merged to the p3yk branch. What about changes such as the discussed -3 warn option, or __future__ imports for stuff that'll be in 3k anyway? Can/should these revisions just be excluded from the merge? Or would it be better to apply them to a new branch? Georg

3 2

Re: [Python-Dev] [Python-3000] Warning for 2.6 and greater
by glyph＠divmod.com 12 Jan '07

12 Jan '07

On 11:22 am, anthony(a)interlink.com.au wrote: >> FWIW, I also agree with James that Python 3 shouldn't even be >> released until the 2.x series has reached parity with its feature >> set. However, if there's continuity in the version numbers >> instead of the release dates, I can at least explain to Twisted >> users that we will _pretend_ they are released in the order of >> their versions. > >I'm not sure what "parity with it's feature set" means. I don't either! "Parity with it is feature set"? I can't even parse that! ;-) By "parity with *its* feature set", though, I meant what you said here: >I do hope that it's _possible_ to work in a version of the language that works in both 2.6+ and 3.0+, even if under the hood there are differences. In order to do this, everything that has been changed in 3.0 has to have some mechanism for working both ways in some 2.x release. I phrased this as its "feature set" because I am not aware of any new functionality in 3.0 that simply isn't available 2.x - everything I've seen are "cleanups", which expose basically the same features as 2.5. If there are some real new features, then I am in error.

1 0

Re: [Python-Dev] [Python-3000] Warning for 2.6 and greater
by glyph＠divmod.com 12 Jan '07

12 Jan '07

On 07:56 am, anthony(a)interlink.com.au wrote: >Additionally, without a 2.x<->3.x upgrade path 3.x is essentially a >new language, having to build a new userbase from scratch. Worse >yet, 2.x will suffer as people have the perception "Python 2? >That's a dead/abandoned language" It's worse than that. This perception has _already_ been created. I already have heard a few folks looking for new languages to learn choose Ruby over Python and give Py3K as a reason. "Isn't Python going to be totally different in a few years anyway? I'll just wait until then, seems like a waste of time to learn it now." Given Ruby's own checkered history of compatibility, I don't think this is an _accurate_ perception, but it is the nature of perception to be inaccurate. If the plan is to provide a smooth transition, it would help a lot to have this plan of foward and backward compatibility documented somewhere very public. It's hard to find information on Py3K right now, even if you know your way around the universe of PEPs. FWIW, I also agree with James that Python 3 shouldn't even be released until the 2.x series has reached parity with its feature set. However, if there's continuity in the version numbers instead of the release dates, I can at least explain to Twisted users that we will _pretend_ they are released in the order of their versions.

2 1

Re: [Python-Dev] [Python-3000] Warning for 2.6 and greater
by glyph＠divmod.com 12 Jan '07

12 Jan '07

On 01:12 am, anthony(a)interlink.com.au wrote: >On Friday 12 January 2007 06:09, James Y Knight wrote: >> On Jan 10, 2007, at 6:46 PM, Benji York wrote: >> > Paul Moore wrote: >> >> How many other projects/packages anticipate *not* migrating to >> >> Py3K, I wonder? >> > >> > I certainly can't speak for the project as a whole, but I >> > anticipate a fair bit of work to port Zope 3 (100+ KLOC) to >> > Python 3.0. >> >> I (another Twisted developer, among other hats I wear) am also >> very worried about the Python 3.0 transition. > >I'm plan to try and make the transition as painless as possible. A >goal I have is to try and make it possible to write code that works >in both 2.6 and 3.0. Obviously 2.6 will be backwards-compatible >with previous versions, but I'd like to see it get either a command >line option or a from __future__ statement to enable compatibility >with 3.0-isms, like the dict.items change. This is all very encouraging. Please stick with a __future__ statement if at all possible though. The biggest challenge in migration is to reduce the impact so that it can be done partially in a real system. If you have a module X wants to be "3.0 safe" which imports a module Y from a different developer that is not, command line options for compatibility might not help at all. A __future__ statment in X but not in Y, though, would allow for a smooth transition. I can see how that would be tricky for things like dictionary methods, but it does seem doable. For example, have a has_key descriptor which can raise an AttributeError if the globals() of the calling stack frame has been marked in some way. >> Basically: my plea is: please don't remove the old way of doing >> things in Py 3.0 at the same time as you add the new way of doing >> things. > >If there was a way to make 2.6 as compatible as possible with 3.0, >would this make life less painful? Obviously there'd have to be >breakages in a backwards direction, but I'd hope it would make it >easier to go forward. Some things should also be OK to backport to >2.6 - for instance, I can't see an obvious reason why 2.6 can't >support "except FooError as oopsie" as an alternate spelling of >"except FooError, oopsie". You happen to have hit my favorite hot-button 3.0 issue right there :). I don't care about backticks but sometimes I _do_ have to catch exceptions. >Similarly, where the stdlib has been shuffled around, there should >be shims in 2.6 that allow people to work with the new names. This part I wouldn't even mind having to write myself. It would certainly be good to have somewhere more official though. >I don't think waiting for 2.7 to make the compatibility work is a >workable approach - we're something like 2.5-3 years away from a >2.7 release, and (optimistically) 12-18 months from a 3.0 final. >That leaves a window of 1.5-2 years where we are missing an >important tool for people. It would be nice if the versioning would actually happen in order. >> [1] Unless of course there's a perfect automated conversion >> script that can generate the 3.X compatible source code from the >> 2.X compatible source code. > >I doubt that the 2to3 script will be perfect, but hopefully it can >get most things. I can't see it easily fixing up things like > >check = mydict.has_key >... >if check(foo): Currently a common idiom, by the way, used throughout the lower levels of Twisted. >This is why I also want to add Py3kDeprecationWarning to 2.6. > >On that note, I'd like to see changes like the mass-change to the >stdlib in the 3.0 branch that changed raise A, B into raise A(B) >applied to the trunk. This makes it much easier to apply patches to >both the 3.0 branch and the trunk. Similar changes should be >applied to remove, for instance, use of <> and dict.has_key from >the stdlib. Simply put, I'd like the stdlib between 2 and 3 to be >as similar as possible. It would be nice if the stdlib could be used as a case study - if the 3 stdlib tests can pass on some version of 2 (or vice versa) that should be a minimum bar for application portability. Maybe a better way to handle the similarity is that the reorganized stdlib should simply be available as a separate piece of code on 2.x? That would allow 2.x to use any new features added and distinguish between code which had been moved to use new stdlib APIs and that which hadn't.

1 0

Re: [Python-Dev] [Python-3000] Warning for 2.6 and greater
by Anthony Baxter 12 Jan '07

12 Jan '07

cc'ing python-dev - followups should probably go there, rather than the p3yk list. So here's my latest plan: - Add a Py3KDeprecationWarning, as a subclass of DeprecationWarning (or maybe just of Warning) - Add a -W py3k shortcut, which is the same as "-W once:Py3kDeprecationWarning" - Add a C int (as in previous patch) that is also set to 1 if the command line flag is specified. This is because there's no easy or quick way to check that a warning has been set - and calling into the warnings code is expensive, even if the C code warnings module is done. We can revisit this if the C version of warnings grows some extra features to make this less awful. For 2.6, we make a single -t (mixing tabs and spaces) the default, and maybe add -T to suppress this. DeprecationWarning for 2.6: - `foo` (backticks) - input() The following are the list of things that get Py3kDeprecationWarnings (so far): integer division - make "-Q warn" the default when -W py3k specified coerce() - going away apply() - going away intern() - moving to sys (we should also move it to sys, and make intern() - call sys.intern()) file.xreadlines() - going away dict.has_key() - use 'in dict' <> - use != (aka "MakeBarryCryDeprecationWarning") __(get|set|del)slice__ __coerce__ - gone __(div|idiv|rdiv)__ - no replacement? __nonzero__ - we should add __bool__ to 2.6, and then warn people. Not sure we can rename the nb_nonzero slot, though. For the various __foo__ slots, it might be nice to warn when they're defined, rather than used, but I've not looked into how hard this would be to do, or whether it would still work with .pyc files and the like. On the other hand, warning on use might also let us pick up when C code modules use the same functions. There's other changes that are probably too hard to warn about, because there's not an easy replacement - the exec and print statements come to mind here. Comments? What else should get warnings? Anthony

9 15

Re: [Python-Dev] [Python-3000] Code working in both 2.x and 3.X
by Guido van Rossum 12 Jan '07

12 Jan '07

On 1/11/07, Ron Adam <rrr(a)ronadam.com> wrote: > Jim Jewett wrote: > > Raymond Hettinger wrote: > > > >> Also, I'm wondering if the desire for 2.6 warnings is based on the notion that > >> it will be possible to get large tools to work under both Py2.x and Py3.x. > > > > I had certainly assumed it would be possible. > > > > In fact, I had assumed that the 2->3 translator would have a mode > > which left the code running in both. > > > > This might not be the cleanest possible code for either line (parens > > in 2.x, some extra iter calls or missed shortcuts, etc), but it should > > certainly exist. If it doesn't, then people who do want to support > > both lines will themselves have to work exclusively in 2.x and > > "compile" to 3.x. > > > >> With all the module renaming/packaging, > [etc... clipped to shorten] > > There seems to be a fair amount of anxiety starting to appear about the > difficulty of trying to support both 2.x and 3.x at the same time. It's > understandable, but I think it may be the earlier than expected 3.0 release date > that is adding to that. > > Would it be reasonable to define X.0.X versions as official transition versions > which need not have both back word compatibility, or overly strict future > compatibility? > > I thinking that the 3.0.X version be considered a try it out (alpha) release to > generate plenty of feed back, and the 3.1.X version be the first version meant > for actual development use. > > 3.0.X <=> 3.alpha.X > 3.1.X <=> 3.first.X > > So the release order may be 3.0.0, 2.6.0, 3.1.0. ... > > (With minor bug fix releases in between, of course.) > > My thinking is developers won't need to support the 3.0.X version *ever*. And > even 3.1.X need not be "too strongly" constrained to be back word compatible > with version 3.0.X, as it may still have quite a few changes. Version 3.1.0 > would then be the first version that future, 3.(1+n), versions will need to > maintain back words compatibility with. > > This gives a lot more time for developers to try it, give feed back, and work > out how to do things like move stuff from 2.X to 3.X, etc... It also may give > more freedom to make changes in version 3.0 with less anxiety to third party > developers. > > Version 3.0 -> a (relatively) clean start with big changes. > > Version 2.6 -> may take into account things in version 3.0. > > Version 3.1 -> builds on 3.0 and may take into account version 2.6 This last line is news to me, and I'm not in favor of it. My plans for 3.1 are different -- to take into account user feedback on 3.0. > Isn't this the general (informal) context already being considered? -- --Guido van Rossum (home page: http://www.python.org/~guido/)

1 0

Re: [Python-Dev] [Python-3000] Warning for 2.6 and greater
by glyph＠divmod.com 11 Jan '07

11 Jan '07

On 10 Jan, 11:10 pm, p.f.moore(a)gmail.com wrote: >On 10/01/07, glyph(a)divmod.com <glyph(a)divmod.com> wrote: >>I've been assuming for some time that the only hope for Py3k compatibility >>within Twisted would be using PyPy as a translation layer. >Does this ring as many warning bells for me as it does for others? I >know very little about the current state of PyPy, but I read your >comment as implying that you expect Twisted to be unavailable (at some >level or other) for users of Py3K. Is that right? This entirely depends on the migration path available. My understanding of the situation is that Py3K is a different, incompatible language. In that sense, Twisted will only be as unavailable to users of Py3K as it is unavailable to users of Ruby, Perl, or any other dynamic-but-not-Python language. If the 2.x series grows deprecation warnings and gradually has all of py3k's features backported, such that there is a smooth transitional period of a few years where we can support one or two versions of 2.x and a version of 3.x, then eventually perhaps it will be worthwhile. Alternately, if they really aren't compatible *but* there is such a compelling featureset drawing other developers that there's really a critical mass of projects that migrate to 3.x and stop maintaining their 2.x counterparts, it might be worthwhile to jump the chasm to follow them. At a minimum, for it to be worthwhile for Twisted to attempt this if the following packages are all moved to 3.x: - pyopenssl - pygtk - pysqlite - PyObjC - win32all - zope interface (and therefore, probably, all of Zope) - pycrypto - pypam - pygame - pylucene Those projects that do move I'm sure will take the "opportunity" to incompatibly break all of their APIs as well, since no existing code will actually use them, exacerbating the problems involved in porting.

1 0

Code working in both 2.x and 3.X
by Jim Jewett 11 Jan '07

11 Jan '07

Raymond Hettinger wrote: > Also, I'm wondering if the desire for 2.6 warnings is based on the notion that > it will be possible to get large tools to work under both Py2.x and Py3.x. I had certainly assumed it would be possible. In fact, I had assumed that the 2->3 translator would have a mode which left the code running in both. This might not be the cleanest possible code for either line (parens in 2.x, some extra iter calls or missed shortcuts, etc), but it should certainly exist. If it doesn't, then people who do want to support both lines will themselves have to work exclusively in 2.x and "compile" to 3.x. > With all the module renaming/packaging, I had believed that 2.x would support both names, typically as >>> # make an deprecation warning, or log entry >>> from oldname import * So code using the new names may not work in 2.5, but should work in 2.6. If you want it to work in 2.3, you either use conditional imports or include your own forwarding modules for older pythons. > old-style classes disappearing, This is only a problem if you rely on specfically old-style semantics. Most old-style classes that I see work fine as new-style; the author just didn't want to put in the extra boilerplate (or wanted 1.5 compatibility, or just didn't bother to make a change). > encoded text objects, This one I'm less sure about, but it seems that programs which work correctly with both str and unicode in 2.x should work with 3, and that programs which (in practice) restrict themselves to ASCII will continue to work. > real division You can already get this a __future__ import, and you can explicitly request the old form (in both lines) by truncating the result. > and whatnot; that notion may be a pipe-dream. I hope not. That said, I'm not so sure about C extension modules... -jJ

1 0

Re: [Python-Dev] [Python-3000] Warning for 2.6 and greater
by Anthony Baxter 11 Jan '07

11 Jan '07

On Thursday 11 January 2007 06:32, Georg Brandl wrote: > I guess there are quite a few people who won't start moving to > Python 3.0 with 2.6, or even when 3.1 is out, as long as their > program works fine with the 2.x line. There's no need to punish > them with extra overhead. Checking a single C global int is hardly going to make a huge impact at all. -- Anthony Baxter <anthony(a)interlink.com.au> It's never too late to have a happy childhood.

1 0

Re: [Python-Dev] [Python-3000] Warning for 2.6 and greater
by Phillip J. Eby 11 Jan '07

11 Jan '07

At 11:10 PM 1/10/2007 +0000, Paul Moore wrote: >On 10/01/07, glyph(a)divmod.com <glyph(a)divmod.com> wrote: > > I've been assuming for some time that the only hope for Py3k compatibility > > within Twisted would be using PyPy as a translation layer. > >Does this ring as many warning bells for me as it does for others? I >know very little about the current state of PyPy, but I read your >comment as implying that you expect Twisted to be unavailable (at some >level or other) for users of Py3K. Is that right? > >How many other projects/packages anticipate *not* migrating to Py3K, I wonder? I think it's safe to say that any package of comparable size is going to think twice about it without transitional compatibility.

1 0

python-ideas on gmane
by Georg Brandl 11 Jan '07

11 Jan '07

Hi, is there any objection against making python-ideas available on gmane? Should I just suggest it myself, or has someone to do extra admin work on the mailing list side? Georg

3 2

Re: [Python-Dev] [Python-3000-checkins] r53335 - in python/branches/p3yk: Doc/Makefile.deps Doc/lib/lib.tex Doc/lib/libsets.tex Doc/lib/libstdtypes.tex Lib/msilib/__init__.py Lib/sets.py Lib/test/test_cookielib.py Lib/test/test_set.py Lib/test/test_sets.py
by Guido van Rossum 11 Jan '07

11 Jan '07

On 1/9/07, Brett Cannon <brett(a)python.org> wrote: > Should we bother with deprecating 'sets' in the 2.x series? I'd say so, yes. -- --Guido van Rossum (home page: http://www.python.org/~guido/)

2 1

DRAFT: pydev summary for 2006-12-01 to 2006-12-15
by Calvin Spealman 09 Jan '07

09 Jan '07

============= Announcements ============= Some of you may know that Steven Bethard, having taken over the summaries from Brett Cannon some time ago, is no longer able to keep up with them. After much holiday stress and busy days, I've come to take over this honor and archive all the goings on here. I hope I can do a good job, and allow the summaries to continue being a useful and transparent fixture. Also, after catching up with my backlog of summary work, I plan to begin the first summaries of python-3000 (and, possibly, but not likely, the python- ideas list), as more and more actual work, rather than debating and theory-talk, is going on there. Sincerly, Your new summary writer, Calvin Spealman ========= Summaries ========= ----------------------------------------------------------------------------- [NPERS] Re: a feature i'd like to see in python #2: indexing of match objects ----------------------------------------------------------------------------- Subscripting re match objects directly was proposed and the debate the waged mostly on how to deal with slicing and m.group(0) to be the entire match. The consensus seemed to be that returning a match object would break, largely because m[1:2][0] != m[1], which breaks intuition with slices. Returning a list as a slice also breaks the intuitions about the type of a slice. Contributing thread: - `a feature i'd like to see in python #2: indexing of match objects <http://mail.python.org/pipermail/python-dev/2006-December/070138.html>`__ ---------------------------------------- Python and the Linux Standard Base (LSB) ---------------------------------------- Discussions about how to properly document the expected Python environment for all LSB compliant distributions continued shortly. Focus was on how distros have a need to split the standard library up, even if the entire thing is eventually installed, and issues such as Tkinter requiring X leading to standard modules being ommitted for flexable packages. Contributing thread: - `Python and the Linux Standard Base (LSB) <http://mail.python.org/pipermail/python-dev/2006-December/070112.html>`__ ---------------------------------------------------------------------- [Python-checkins] MSI being downloaded 10x more than all other files?! ---------------------------------------------------------------------- Guido was impressed with out download numbers on Windows, but most people just attributed it to other platforms including and mirroring Python. Even taking that to consideration, the ratio of Windows to source downloads has shifted in Windows favor a good bit. The use of Python in introductory courses at MIT was mentioned as something to push these numbers further in the near future. Contributing thread: - `[Python-checkins] MSI being downloaded 10x more than all other files?! <http://mail.python.org/pipermail/python-dev/2006-December/070294.html>`__ ---------------------------------------------------------------- a feature i'd like to see in python #1: better iteration control ---------------------------------------------------------------- The request was made to allow direct deletion of items from iterables through the iterator, but the idea was deemed unscalable to iterables in general and broken with dictionaries. The solutions to work around the problem were given and generally agreed to be sufficient (such as copying all the items you want to keep, instead). Contributing thread: - `a feature i'd like to see in python #1: better iteration control <http://mail.python.org/pipermail/python-dev/2006-December/070137.html>`__ ------------------------- LSB: Binary compatibility ------------------------- The other side of the LSB issue was discussed, related to the CPython ABI, the .pyc bytecode and the marshall formats, and exactly which versions of Python may or may not gain a standard ABI, as well as how to carry this forward to future (already existing and evolving) version of Python. Some mention was made of the difficulties and conflicts being brought on by the idea of LSB standards for Python, while the benefits of this were challenged. Contributing thread: - `LSB: Binary compatibility <http://mail.python.org/pipermail/python-dev/2006-December/070191.html>`__ ---------------------------------------- Adding resume (206) support to urllib(2) ---------------------------------------- Daniel Watkins asked about adapting his HTTP resuming support into urllib, but suggestions directly moved this to urllib2 and then to httplib. The conversation quickly drifted into talking about how abstract and minimal the urllib(2) should be, while the httplib module may grow increasing complex and feature supporting. The original poster made no final comment on his intentions to follow through with the patch. Contributing thread: - `Adding resume (206) support to urllib(2) <http://mail.python.org/pipermail/python-dev/2006-December/070320.html>`__ ------------------- __str__ and unicode ------------------- The details of what an object's __str__() method must return were inquired. Must the string returned by a str or any basestring? It was announced to allow unicode objects, with no restriction on content. However, the caution that the return is likely to be encoded into the default encoding was given. Contributing thread: - `__str__ and unicode <http://mail.python.org/pipermail/python-dev/2006-December/070237.html>`__ ---------------------------------- LSB: Selection of a Python version ---------------------------------- Martin v. Lowis announced that LSB will likely move forward with Python 2.4 as the first LSB standard Python version. How to handle future bug patches, as well as the comfort of the distributions for being required to support an outdated package (2.4 is already "retired") was considered. The consensus settled on a documented support for 2.4 moving forward with security patches, and likely even 2.3, as security patches are rare enough to not cause too many cycles to be spent on them. Contributing thread: - `LSB: Selection of a Python version <http://mail.python.org/pipermail/python-dev/2006-December/070189.html>`__ ------------------------------ Threading, atexit, and logging ------------------------------ A bug (#1566280) was previously reported, related to the ordering of the atexit handlers for the logging and threading modules. Participants in the thread had a general agreement that the threading module using such a user visiable mechanism for cleanup and lifetime was a bad design in the first place to be remedied. Contributing thread: - `Threading, atexit, and logging <http://mail.python.org/pipermail/python-dev/2006-December/070233.html>`__ ----------------------------------- Creating dicts from dict subclasses ----------------------------------- A common problem of subclassing built-in types (dict, in this case) arose trying to move weakref.WeakValueDictionary to a subclass of dict, rather than UserDict. [NOTE: Could someone give a better description of this? I don't know enough of the issue here to write any content full enough. Thanks.] Contributing thread: - `Creating dicts from dict subclasses <http://mail.python.org/pipermail/python-dev/2006-December/070334.html>`__ ----------------------------------------------------------------------------- OT: World's oldest ritual discovered. Worshipped the python 70, 000 years ago ----------------------------------------------------------------------------- This was a quick and off-topic post about other instances of python symbolism in ancient humanity, including a definition of our word "pythonic" to mean "prophetic", which seems fitting enough. Contributing thread: - `OT: World's oldest ritual discovered. Worshipped the python 70, 000 years ago <http://mail.python.org/pipermail/python-dev/2006-December/070123.html>`__ ------------------ LSB: pyc stability ------------------ The LSB meetings brought up the issue of standardized bytecode formats, again. This has always been something discouraged from Python. Keeping the bytecode format free and the implementation open to ignoring old versions, has always been helpful for maintanence and development. The common reasoning of protecting IP was used, although most in-the-know consider this to be a false sense of security. Reguardless, it might be useful in legal ways, as trade secrets do require a reasonable attempt at hidding them in order to be recognized as such. The possible solution of importing encypted zip packages with a custom import hook was proposed and seems logical. Contributing thread: - `LSB: pyc stability <http://mail.python.org/pipermail/python-dev/2006-December/070187.html>`__ ---------------------------------------------------------------------- features i'd like [Python 3000?] ... #4: interpolated strings ala perl ---------------------------------------------------------------------- Perl/PHP style string interpolation was brought up, but is found unreasonable for various reasons. Cons include the use of format strings in scopes outside where they are defined, the dangers of eval, and possibly the general resisting of any idea coming from Perl. This seems a highly unlikely proposal. Contributing thread: - `features i'd like [Python 3000?] ... #4: interpolated strings ala perl <http://mail.python.org/pipermail/python-dev/2006-December/070173.html>`__ --------------------------- Polling with Pending Calls? --------------------------- Issues with the RPM library blocking interupts for long periods of time were brought up with proposed fixes by patching the rpmlib bindings to "automagically" poll for any pending calls. An existing patch was suggested but did not seem to fit the case at hand. Contributing thread: - `Polling with Pending Calls? <http://mail.python.org/pipermail/python-dev/2006-December/070181.html>`__ =============== Skipped Threads =============== - `Weekly Python Patch/Bug Summary <http://mail.python.org/pipermail/python-dev/2006-December/070119.html>`__ - `cross-compile patches <http://mail.python.org/pipermail/python-dev/2006-December/070216.html>`__ - `Windows SDK <http://mail.python.org/pipermail/python-dev/2006-December/070308.html>`__ - `help for a noob - version for a sharp ARM <http://mail.python.org/pipermail/python-dev/2006-December/070286.html>`__ - `fileinput module tests <http://mail.python.org/pipermail/python-dev/2006-December/070307.html>`__ - `Open CPython VM for type information <http://mail.python.org/pipermail/python-dev/2006-December/070328.html>`__ - `fpectl: does a better implementation make sense? <http://mail.python.org/pipermail/python-dev/2006-December/070118.html>`__ - `Small tweak to tokenize.py? <http://mail.python.org/pipermail/python-dev/2006-December/070127.html>`__ - `features i'd like [Python 3000] ... #3: fix super() <http://mail.python.org/pipermail/python-dev/2006-December/070168.html>`__ - `Makefile.pre.in Patch <http://mail.python.org/pipermail/python-dev/2006-December/070172.html>`__ - `Python and LSB: Meeting Report <http://mail.python.org/pipermail/python-dev/2006-December/070186.html>`__ - `Virtual Python (was Re: Python and the Linux Standard Base (LSB)) <http://mail.python.org/pipermail/python-dev/2006-December/070194.html>`__ - `DRAFT: python-dev summary for 2006-11-16 through 2006-11-30 <http://mail.python.org/pipermail/python-dev/2006-December/070229.html>`__ - `Looking for new Python-Dev Summarizers <http://mail.python.org/pipermail/python-dev/2006-December/070232.html>`__ - `Adding C99 bool to struct module <http://mail.python.org/pipermail/python-dev/2006-December/070276.html>`__ - `Spam and semi-spam and neo-spam <http://mail.python.org/pipermail/python-dev/2006-December/070284.html>`__ - `[NPERS] Re: a feature i'd like to see in python #1: betteriteration control <http://mail.python.org/pipermail/python-dev/2006-December/070171.html>`__ - `Next PyPy sprint: Leysin 8-14th January 2007 <http://mail.python.org/pipermail/python-dev/2006-December/070217.html>`__ - `Distribution tools: What I would like to see <http://mail.python.org/pipermail/python-dev/2006-December/070234.html>`__ - `RealClearPolitics - Articles - Exit Rumsfeld, Smiling <http://mail.python.org/pipermail/python-dev/2006-December/070283.html>`__

2 1

Apologies from the CBS autoresponder
by Tim Golden 09 Jan '07

09 Jan '07

My many apologies to those who had their New Year interrupted by my company's inane autoresponder. I'm afraid that I went on leave for a week without suspending list delivery just as my company changed its name and sent out (unknown to me) an automatic message to *everything* coming in to the old address. I've pulled the plug on that address's subscription, altho' I suspect some list admin got there before me. Apologies again. Tim Golden

1 0

Bug or not? Different behaviour iterating list and collections.deque
by Christos Georgiou 08 Jan '07

08 Jan '07

Hello, people. I am not sure whether this is a bug or intentional, so I thought checking it with you before opening a bug. I will explain this issue, but please understand this is not a question for help to change the algorithm (this has been done already), so it's not a question of c.l.py. It's a matter of discrepancy. A list that changes while iterating produces no errors, while a deque fails. Given the following example code: #code start import itertools, collections def item_is_special(item): "Just a dummy check in this example" return item % 3 == 0 def item_products(item): "Also a dummy function for the example" return [item*20+1, item*30+1] def process_list(items, type_of_que, special_case): # we want to process some items, but some of them # produce other items to be processed by the # same algorithm products= type_of_que() if special_case: products.append(-1) for item in itertools.chain(items, products): if item_is_special(item): for product in item_products(item): products.append(product) else: print "got", item #code end we have the following cases: >>> process_list(numbers, list, False) got 1 got 2 got 61 got 91 List works as expected. >>> process_list(numbers, collections.deque, False) got 1 got 2 deque does not work, most probably because deque.__iter__ of an empty deque ignores later changes. For this reason the `special_case' parameter was inserted in the code above, so that there is at least an item when itertools.chain calls iter on the deque: >>> process_list(numbers, collections.deque, True) got 1 got 2 Traceback (most recent call last): File "<stdin>", line 1, in <module> File "testdequeiter.py", line 17, in process_list for item in itertools.chain(items, products): RuntimeError: deque mutated during iteration Is that intentional? If not, let me know so that I open a bug.

4 6

Re: [Python-Dev] Pydoc Improvements / Rewrite
by Laurent Gautier 08 Jan '07

08 Jan '07

Ron, I agree that pydoc could benefit a bit from some cleanup. As you point it out, the ability to write quick viewers would be very helpful. I came across that when wanting to develop script on a remote web server for which I only had FTP access: I ended up having to study pydoc more than I wanted in order to be able to build a display-the-doc cgi. However having two different modules might not be needed. Introspection is probably already available in the separate module 'inspect', and what a code pydoc would have to do is model the documentation (as a tree) and offer convenience function to navigate the data. Beside that, there would be sub-modules for the different viewers for the documentation data - the interactive console being just one of the viewers. Finally, I would suspect that an API-breaking modification of the module would need time to be accepted. May be the original author of pydoc is considering changes as well, and joining effort would be possible ? L. PS: I would also not go for a module name deliberately prefixed with "_" (as some people might associate that with protected or private objects).

9 19

Pydoc Improvements / Rewrite
by Ron Adam 08 Jan '07

08 Jan '07

I started to discuss this on the new python-ideas list because I thought it may need a bit more completion before bringing it here. But It was suggested that it really does belong here as it is something that would be nice to have in python 2.6. So I'm reposting the introduction here. The still very rough source files can be downloaded from: http://ronadam.com/dl/_pydoc.zip There is still much to do, but I think having some experienced feed back on where it should go is important. Cheers, Ron Adam ps.. Please disregard the website for now, it's purpose was to share with family and friends, although I will probably share the python source to the ledger program there and other python toys and experiments (for fun) in the near future. [Introduction From python-ideas list] Improving pydoc has been suggested before by me and others. I've been working on a version that is probably 80% done and would like to get feed back at this point to determine if I'm approaching this in the best way. Basically pydoc.py consists of over 2,000 lines of python code and is not well organized inside which means (my thoughts) it will pretty much stay the way it is and not be as useful as it could be. Currently pydoc.py has the following uses. * It is imported and used as pythons console help function. * It can be used to generate help text files. * It can open a tkinter search index and from that launch a web server and a browser to veiw a html help page. * It can be used to generate static html help files. It looks (to me) like splitting it into two modules would be good. One module for just the text help and introspection functions, and the other for the html server and html output stuff. [It was suggested on python-ideas that making it a package may be good.] 1. pyhelp.py - Pythons help function from inside the console, and running it directly would open an interactive text help session. 2. _pydoc.py - Python html help browser. This starts an html server and opens a web page with a modules/package index. The html page headers would contain the current Python version info and the following inputs fields. * get field - directly bring up help on an object/module/or package. * Search field - returns a search results index. * Modules button - modules index page * Keywords button - keywords index page * Help button - pydoc Help instructions, version, and credits info. Note: The leading underscore "_pydoc.py" is to keep it from clashing with the current pydoc version. It probably should be something else. An additional feature is clicking on a filename opens up a numbered source listing. Which is nice for source code browsing and for referencing specific lines of code in python list discussions. ;-) The colors, fonts and general appearance can be changed by editing the style sheet. The output is readable as plain (outline form) text if the style sheet is ignored by the browser. _pydoc.py imports pyhelp and uses it to do the introspection work and extends classes in pyhelp to produce html output. I've tried to make pyhelp.py useful in a general way so that it can more easily be used as a base that other output formats can be built from. That's something that can't be done presently. These improvements to pydoc mean you can browse pythons library dynamically without ever leaving the web browser. Currently you switch back and forth between the browser and a tkinter index window. Something I found to be annoying enough to discourage me from using pydoc. The version I'm working on is split up into eight python files, each addressing a particular function of pydoc. That was to help me organize things better. These will be recombined into fewer files. Some parts of it could be moved to other modules if they seem more generally useful. For example, the console text pager could be used in many other console applications. Things that still need to be done are adding the object resolution order output back in. And adding inter-page html links back in. And a few other things I just haven't gotten to yet. I got a bit burned out on this project a while back, and then moved to a new house.. etc.. etc.. But I'm starting to have more time, and with the current discussion s turning on to refining pythons library this seems like it would be a useful tool in that effort. Any comments on this general approach? Any suggestions, questions, or comments? I'll post a link to the zipped files in the next day or two and announce it here. I need to look into a glitch on my computer first that's probably a windows path/name problem. I don't think it's anything that needs to be fixed in the files but I want to be sure. Cheers, Ron Adam

9 13

smarter temporary file object (SF #415692)
by dustin＠v.igoro.us 08 Jan '07

08 Jan '07

I came across a complaint that PEP 0042 had become a graveyard of neglected ideas, and decided to have a look through and implement something. Creating a smarter temporary file object seemed simple enough. Oddly, even after GvR re-opened it, I can't post an attachment to that tracker item (it's under "Feature Requests" -- does it need to get moved to "Patches" first?), but the implementation is short, so it's included below. This is intended to be appended to Lib/tempfile.py (and thus assumes that module's globals are present). I would appreciate it if the gurus of python-dev could take a peek and let me know if this is unsuitable or incorrect for any reason. It's not the most straightforward implementatio -- I used the optimization techniques I found in TemporaryFile. If this looks good, I'll prepare a patch against trunk, including an additional chunk of documentation and a unit test. Dustin -----cut-here----- try: from cStringIO import StringIO except: from StringIO import StringIO class SpooledTemporaryFile: """Temporary file wrapper, specialized to switch from StringIO to a real file when it exceeds a certain size or when a fileno is needed. """ _rolled = False def __init__(self, max_size=0, mode='w+b', bufsize=-1, suffix="", prefix=template, dir=None): self._file = StringIO() self._max_size = max_size self._TemporaryFileArgs = (mode, bufsize, suffix, prefix, dir) def _check(self, file): if self._rolled: return if file.tell() > self.__dict__['_max_size']: self._rollover(file) def _rollover(self, file): args = self.__dict__['_TemporaryFileArgs'] self.__dict__.clear() # clear attributes cached by __getattr__ newfile = self._file = TemporaryFile(*args) newfile.write(file.getvalue()) newfile.seek(file.tell(), 0) self._rolled = True # replace patched functions with the new file's methods self.write = newfile.write self.writelines = newfile.writelines self.fileno = newfile.fileno def write(self, s): file = self.__dict__['_file'] rv = file.write(s) self._check(file) return rv def writelines(self, iterable): file = self.__dict__['_file'] rv = file.writelines(iterable) self._check(file) return rv def fileno(self): self._rollover(self.__dict__['_file']) return self.fileno() def __getattr__(self, name): file = self.__dict__['_file'] a = getattr(file, name) if type(a) != type(0): setattr(self, name, a) return a

2 2

features i'd like [Python 3000] ... #3: fix super()
by Ben Wing 06 Jan '07

06 Jan '07

i don't like the current super() at all. having to type super(Foo, self).__init__(...) is messy, hard to remember, and error-prone. it also introduces an unfortunate dependency in that the name of the class (Foo) has to be hard-coded in the call, and if you change the class name you also have to go and change all super() calls -- more chances for errors. as a result, i imagine there's a strong urge to just hardcode the name of the parent -- a bad idea, and the reason why super() was introduced in the first place. instead, `super' should work like in other languages. a couple of ideas: -- super.meth(args) calls the superclass method `meth', in the same way that super(Foo, self).meth(args) currently works. (this is the same syntax as in java. this probably requires making `super' be a keyword, although it might be possible to hack this by examining the current call stack.) -- as an alternative or in addition, super(args) would work like super.meth(args) if we're currently inside of `meth' in a subclass. i suspect that 90% of the time or more, `super' is used to chain to the superclass version of an overridden method, and this optimizes for the common case. it makes for one less possibility of misspelling `__init__' and one less thing that requires renaming if a method is renamed or code copied to another method (granted, this isn't such a big deal as in the case of class renaming). if the form `super.meth(args)' isn't also supported, then a call of this sort would have to be made through the introspection API. (i think it would be a good idea to have this functionality available through the introspection API, in any case. currently i don't see any obvious way in module `inspect' to find out which class a call to `super(...).meth(...)' is invoked on, although maybe a loop with inspect.getmro() and hasattr() would work. it would be nice to have a function like inspect.getsuper(class, 'meth'), i think.) ben

4 3

Re: [Python-Dev] 2.5.1 plans
by Ralf W. Grosse-Kunstleve 05 Jan '07

05 Jan '07

It would be nice if this simple fix could be included (main branch and 2.5.1): https://sourceforge.net/tracker/?func=detail&aid=1598181&group_id=5470&atid… [ 1598181 ] subprocess.py: O(N**2) bottleneck I submitted the trivial fix almost two months ago, but apparently nobody feels responsible... Ralf ----- Original Message ---- From: Neal Norwitz <nnorwitz(a)gmail.com> To: Python Dev <python-dev(a)python.org> Sent: Wednesday, January 3, 2007 10:59:15 PM Subject: Re: [Python-Dev] 2.5.1 plans The current schedule looks like it's shaping up to be: Wed, Jan 24 for 2.5.1c1 Wed Jan 31 for 2.5.1 It would be great if you could comment on some of the bug reports below. I think several already have patches/suggested fixes. These looks like they would be nice to fix:: http://python.org/sf/1579370 Segfault provoked by generators and exceptions http://python.org/sf/1377858 segfaults when using __del__ and weakrefs There is one outstanding issue I was notified of in private mail:: http://python.org/sf/1568240 Tix is not included in 2.5 for Windows It's not clear to me if this should be fixed, but it's got a high priority:: http://python.org/sf/1467929 %-formatting and dicts n __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com

4 4

kill the cbsoutdoor.co.uk autoresponder
by Gregory P. Smith 05 Jan '07

05 Jan '07

Whoever is subscribed to python-dev with a broken corporate autoresponder that sends everyone who posts to the list this useless response multiple times please unsubscribe yourself. Its highly annoying and entirely useless since its not even identifying the list subscriber(s) deserving the blame. -g On Fri, Jan 05, 2007 at 06:28:35AM +0000, noreply(a)cbsoutdoor.co.uk wrote: > Dear Recipient, > We have now changed our company name to CBS Outdoor Limited. My new email address is therefore firstname.surname(a)cbsoutdoor.co.uk Your email has been redirected but please amend your address book accordingly. > > Thank you. > > > > > CBS Outdoor Limited > > > > noreply(a)cbsoutdoor.co.uk > www.cbsoutdoor.co.uk > > The contents of this e-mail are confidential to the ordinary user of the e-mail address to which it was addressed, and may also be privileged. If you are not the addressee of this e-mail you may not copy, forward, disclose or otherwise use it or any part of it in any form whatsoever. If you have received this e-mail in error, please e-mail the sender by replying to this message. CBS Outdoor Ltd reserves the right to monitor e-mail communications from external/internal sources for the purposes of ensuring correct and appropriate use of CBS Outdoor facilities.

3 3

crashing on shutdown due to warning at shutdown
by Neal Norwitz 05 Jan '07

05 Jan '07

I fixed the crash that was due to raising a warning on shutdown. I have heard about crashes at shutdown and wonder if this was the cause. There might be similar bugs lurking that assume PyModule_GetDict() always returns a valid pointer. It doesn't, it can return NULL. I'm not sure if the original problem existed or not, but after this fix on shutdown, I see: warning: DBTxn aborted in destructor. No prior commit() or abort(). I don't know if this is a real problem, a test problem, or what. n

2 1

Weekly Python Patch/Bug Summary
by Kurt B. Kaiser 05 Jan '07

05 Jan '07

Patch / Bug Summary ___________________ Patches : 418 open ( +5) / 3522 closed ( +1) / 3940 total ( +6) Bugs : 959 open (+13) / 6405 closed ( +5) / 7364 total (+18) RFE : 250 open ( +2) / 245 closed ( -1) / 495 total ( +1) New / Reopened Patches ______________________ update to PEP 344 - exception attributes (2007-01-02) CLOSED http://python.org/sf/1626538 opened by Jim Jewett backticks will not be used at all (2007-01-03) http://python.org/sf/1627052 opened by Jim Jewett Fix for #1601399 (urllib2 does not close sockets properly) (2007-01-03) http://python.org/sf/1627441 opened by John J Lee Win32: Fix build when you have TortoiseSVN but no .svn/* (2007-01-04) http://python.org/sf/1628061 opened by Larry Hastings Win32: Add bytesobject.c to pythoncore.vcproj (2007-01-04) http://python.org/sf/1628062 opened by Larry Hastings socket.readline() interface doesn't handle EINTR properly (2007-01-04) http://python.org/sf/1628205 opened by Maxim Sobolev Patches Closed ______________ update to PEP 344 - exception attributes (2007-01-02) http://python.org/sf/1626538 closed by ping New / Reopened Bugs ___________________ sqlite3 documentation omits: close(), commit(), autocommit (2006-12-30) http://python.org/sf/1625205 opened by kitbyaydemir re module documentation on search/match is unclear (2006-12-31) http://python.org/sf/1625381 opened by Richard Boulton 'imp' documentation does not mention that lock is re-entrant (2006-12-31) http://python.org/sf/1625509 opened by Dustin J. Mitchell add ability to specify name to os.fdopen (2007-01-01) http://python.org/sf/1625576 opened by Mark Diekhans 'Installing Python Modules' does not work for Windows (2007-01-02) http://python.org/sf/1626300 opened by Christopher Lambacher Would you mind renaming object.h to pyobject.h? (2007-01-02) http://python.org/sf/1626545 opened by Anton Tropashko posixmodule.c leaks crypto context on Windows (2007-01-03) http://python.org/sf/1626801 opened by Yitz Gale website issue reporter down (2007-01-03) http://python.org/sf/1627036 opened by Jim Jewett mention side-lists from python-dev description (2007-01-03) http://python.org/sf/1627039 opened by Jim Jewett xml.dom.minidom parse bug (2007-01-03) CLOSED http://python.org/sf/1627096 opened by Pierre Imbaud xml.dom.minidom parse bug (2007-01-03) CLOSED http://python.org/sf/1627244 opened by Pierre Imbaud an extra comma in condition command crashes pdb (2007-01-03) http://python.org/sf/1627316 opened by Ilya Sandler Typo in module index for Carbon.CarbonEvt (2007-01-03) CLOSED http://python.org/sf/1627373 opened by Brett Cannon Status bar on OSX garbled (2007-01-03) http://python.org/sf/1627543 opened by sigzero RotatingFileHandler cannot recover from failed doRollover() (2007-01-03) http://python.org/sf/1627575 opened by Forest Wilkinson documentation error for "startswith" string method (2007-01-04) CLOSED http://python.org/sf/1627690 opened by Keith Briggs plat-mac videoreader.py auido format info (2007-01-04) http://python.org/sf/1627952 opened by Ryan Owen documentation error for "startswith" string method (2007-01-04) http://python.org/sf/1627956 opened by Keith Briggs Bugs Closed ___________ test_logging hangs on cygwin (2006-04-06) http://python.org/sf/1465643 closed by sf-robot xml.dom.minidom parse bug (2007-01-03) http://python.org/sf/1627096 closed by loewis xml.dom.minidom parse bug (2007-01-03) http://python.org/sf/1627244 closed by nnorwitz Typo in module index for Carbon.CarbonEvt (2007-01-03) http://python.org/sf/1627373 closed by nnorwitz documentation error for "startswith" string method (2007-01-04) http://python.org/sf/1627690 closed by loewis Logging problem on Windows XP (2006-09-27) http://python.org/sf/1566280 closed by loewis New / Reopened RFE __________________ smarter temporary file object (2001-04-12) http://python.org/sf/415692 reopened by gvanrossum optparse "store" action should not gobble up next option (2007-01-03) http://python.org/sf/1627266 opened by Raghuram Devarakonda

1 0

Re: [Python-Dev] pep-3108.txt
by M.-A. Lemburg 05 Jan '07

05 Jan '07

On 2007-01-02 01:02, brett.cannon wrote: > Author: brett.cannon > Date: Tue Jan 2 01:02:41 2007 > New Revision: 53204 > > Added: > peps/trunk/pep-3108.txt (contents, props changed) > Modified: > peps/trunk/pep-0000.txt > Log: > Add PEP 3108: Standard Library Reorganization. > >... > > +Open Issues > +=========== > + > +Consolidate dependent modules together into a single module or package? > ... > +Consolidate certain modules with similar themes together in a package? > +---------------------------------------------------------------------- > ... If you do follow this route, please take the chance to place the whole Python stdlib under a single package. That way we'll avoid name clashes with existing packages and modules now and in the future. Together with absolute imports this also improves the readability of modules since it becomes immediately clear where the imported code is coming from. Note that as side-effect of this it becomes a lot harder to manipulate PYTHONPATH to trick Python into loading a standard module from a non-standard location, improving security and robustness of the Python installations. > +Packages are often used to group together modules that have a similar > +theme but do not have any direct relationship or dependency upon each > +other. For Python 3.0 obvious groupings could be done since renaming > +of various modules is already occurring. > + > +* collections > + + heapq > + + Queue > + + sets > + + UserDist > + + UserList > + + What to do with UserString? > + - Have a package for Python implementations of built-in types > + instead of putting the User* modules into 'collections'? > +* mac > + + Various Mac-specific modules. > + + Same can be done for other platform-specific code. > +* Profiling > + + cProfile > + + profile > + + hotshot > + + pstats > +* email > + + mailbox > + + mhlib > +* Databases > + + anydbm > + + dbhash > + + dbm > + + bsddb > + + dumbdbm > + + gdbm > + + whichdb > +* Audio > + + aifc > + + audioop > + + chunk > + + ossaudiodev > + + sunau > + + wave > + + winsound > +* Servers > + + BaseHTTPServer > + + CGIHTTPServer > + + DocXMLRPCServer > + + SimpleHTTPServer > + + SimpleXMLRPCServer > + + SocketServer The package names should probably be converted to lower-case to follow PEP 8. Thanks and Happy New Year, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 02 2007) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

4 10

2.5.1 plans
by Neal Norwitz 04 Jan '07

04 Jan '07

I don't have a schedule in mind for 2.5.1 yet, however we should start preparing for it. The release will probably happen sometime in January if everyone is available. The branch has been pretty quiet, so I'm not expecting too many problems. Once we figure out a date for release I'll follow up here. If you have any bugs or patches that should be addressed prior to release, please assign them to me and bump the priority to 9. I'm not sure that there are any bugs which absolutely must be fixed before release, though there are a few that would be nice. Any other discussion for 2.5.1 necessary? n

2 2