Mailman 3 June 2009 - Python-Dev

PEP 1, PEP Purpose and Guidelines
by barry＠zope.com 18 May '21

18 May '21

It has been a while since I posted a copy of PEP 1 to the mailing lists and newsgroups. I've recently done some updating of a few sections, so in the interest of gaining wider community participation in the Python development process, I'm posting the latest revision of PEP 1 here. A version of the PEP is always available on-line at http://www.python.org/peps/pep-0001.html Enjoy, -Barry -------------------- snip snip -------------------- PEP: 1 Title: PEP Purpose and Guidelines Version: $Revision: 1.36 $ Last-Modified: $Date: 2002/07/29 18:34:59 $ Author: Barry A. Warsaw, Jeremy Hylton Status: Active Type: Informational Created: 13-Jun-2000 Post-History: 21-Mar-2001, 29-Jul-2002 What is a PEP? PEP stands for Python Enhancement Proposal. A PEP is a design document providing information to the Python community, or describing a new feature for Python. The PEP should provide a concise technical specification of the feature and a rationale for the feature. We intend PEPs to be the primary mechanisms for proposing new features, for collecting community input on an issue, and for documenting the design decisions that have gone into Python. The PEP author is responsible for building consensus within the community and documenting dissenting opinions. Because the PEPs are maintained as plain text files under CVS control, their revision history is the historical record of the feature proposal[1]. Kinds of PEPs There are two kinds of PEPs. A standards track PEP describes a new feature or implementation for Python. An informational PEP describes a Python design issue, or provides general guidelines or information to the Python community, but does not propose a new feature. Informational PEPs do not necessarily represent a Python community consensus or recommendation, so users and implementors are free to ignore informational PEPs or follow their advice. PEP Work Flow The PEP editor, Barry Warsaw <peps(a)python.org>, assigns numbers for each PEP and changes its status. The PEP process begins with a new idea for Python. It is highly recommended that a single PEP contain a single key proposal or new idea. The more focussed the PEP, the more successfully it tends to be. The PEP editor reserves the right to reject PEP proposals if they appear too unfocussed or too broad. If in doubt, split your PEP into several well-focussed ones. Each PEP must have a champion -- someone who writes the PEP using the style and format described below, shepherds the discussions in the appropriate forums, and attempts to build community consensus around the idea. The PEP champion (a.k.a. Author) should first attempt to ascertain whether the idea is PEP-able. Small enhancements or patches often don't need a PEP and can be injected into the Python development work flow with a patch submission to the SourceForge patch manager[2] or feature request tracker[3]. The PEP champion then emails the PEP editor <peps(a)python.org> with a proposed title and a rough, but fleshed out, draft of the PEP. This draft must be written in PEP style as described below. If the PEP editor approves, he will assign the PEP a number, label it as standards track or informational, give it status 'draft', and create and check-in the initial draft of the PEP. The PEP editor will not unreasonably deny a PEP. Reasons for denying PEP status include duplication of effort, being technically unsound, not providing proper motivation or addressing backwards compatibility, or not in keeping with the Python philosophy. The BDFL (Benevolent Dictator for Life, Guido van Rossum) can be consulted during the approval phase, and is the final arbitrator of the draft's PEP-ability. If a pre-PEP is rejected, the author may elect to take the pre-PEP to the comp.lang.python newsgroup (a.k.a. python-list(a)python.org mailing list) to help flesh it out, gain feedback and consensus from the community at large, and improve the PEP for re-submission. The author of the PEP is then responsible for posting the PEP to the community forums, and marshaling community support for it. As updates are necessary, the PEP author can check in new versions if they have CVS commit permissions, or can email new PEP versions to the PEP editor for committing. Standards track PEPs consists of two parts, a design document and a reference implementation. The PEP should be reviewed and accepted before a reference implementation is begun, unless a reference implementation will aid people in studying the PEP. Standards Track PEPs must include an implementation - in the form of code, patch, or URL to same - before it can be considered Final. PEP authors are responsible for collecting community feedback on a PEP before submitting it for review. A PEP that has not been discussed on python-list(a)python.org and/or python-dev(a)python.org will not be accepted. However, wherever possible, long open-ended discussions on public mailing lists should be avoided. Strategies to keep the discussions efficient include, setting up a separate SIG mailing list for the topic, having the PEP author accept private comments in the early design phases, etc. PEP authors should use their discretion here. Once the authors have completed a PEP, they must inform the PEP editor that it is ready for review. PEPs are reviewed by the BDFL and his chosen consultants, who may accept or reject a PEP or send it back to the author(s) for revision. Once a PEP has been accepted, the reference implementation must be completed. When the reference implementation is complete and accepted by the BDFL, the status will be changed to `Final.' A PEP can also be assigned status `Deferred.' The PEP author or editor can assign the PEP this status when no progress is being made on the PEP. Once a PEP is deferred, the PEP editor can re-assign it to draft status. A PEP can also be `Rejected'. Perhaps after all is said and done it was not a good idea. It is still important to have a record of this fact. PEPs can also be replaced by a different PEP, rendering the original obsolete. This is intended for Informational PEPs, where version 2 of an API can replace version 1. PEP work flow is as follows: Draft -> Accepted -> Final -> Replaced ^ +----> Rejected v Deferred Some informational PEPs may also have a status of `Active' if they are never meant to be completed. E.g. PEP 1. What belongs in a successful PEP? Each PEP should have the following parts: 1. Preamble -- RFC822 style headers containing meta-data about the PEP, including the PEP number, a short descriptive title (limited to a maximum of 44 characters), the names, and optionally the contact info for each author, etc. 2. Abstract -- a short (~200 word) description of the technical issue being addressed. 3. Copyright/public domain -- Each PEP must either be explicitly labelled as placed in the public domain (see this PEP as an example) or licensed under the Open Publication License[4]. 4. Specification -- The technical specification should describe the syntax and semantics of any new language feature. The specification should be detailed enough to allow competing, interoperable implementations for any of the current Python platforms (CPython, JPython, Python .NET). 5. Motivation -- The motivation is critical for PEPs that want to change the Python language. It should clearly explain why the existing language specification is inadequate to address the problem that the PEP solves. PEP submissions without sufficient motivation may be rejected outright. 6. Rationale -- The rationale fleshes out the specification by describing what motivated the design and why particular design decisions were made. It should describe alternate designs that were considered and related work, e.g. how the feature is supported in other languages. The rationale should provide evidence of consensus within the community and discuss important objections or concerns raised during discussion. 7. Backwards Compatibility -- All PEPs that introduce backwards incompatibilities must include a section describing these incompatibilities and their severity. The PEP must explain how the author proposes to deal with these incompatibilities. PEP submissions without a sufficient backwards compatibility treatise may be rejected outright. 8. Reference Implementation -- The reference implementation must be completed before any PEP is given status 'Final,' but it need not be completed before the PEP is accepted. It is better to finish the specification and rationale first and reach consensus on it before writing code. The final implementation must include test code and documentation appropriate for either the Python language reference or the standard library reference. PEP Template PEPs are written in plain ASCII text, and should adhere to a rigid style. There is a Python script that parses this style and converts the plain text PEP to HTML for viewing on the web[5]. PEP 9 contains a boilerplate[7] template you can use to get started writing your PEP. Each PEP must begin with an RFC822 style header preamble. The headers must appear in the following order. Headers marked with `*' are optional and are described below. All other headers are required. PEP: <pep number> Title: <pep title> Version: <cvs version string> Last-Modified: <cvs date string> Author: <list of authors' real names and optionally, email addrs> * Discussions-To: <email address> Status: <Draft | Active | Accepted | Deferred | Final | Replaced> Type: <Informational | Standards Track> * Requires: <pep numbers> Created: <date created on, in dd-mmm-yyyy format> * Python-Version: <version number> Post-History: <dates of postings to python-list and python-dev> * Replaces: <pep number> * Replaced-By: <pep number> The Author: header lists the names and optionally, the email addresses of all the authors/owners of the PEP. The format of the author entry should be address(a)dom.ain (Random J. User) if the email address is included, and just Random J. User if the address is not given. If there are multiple authors, each should be on a separate line following RFC 822 continuation line conventions. Note that personal email addresses in PEPs will be obscured as a defense against spam harvesters. Standards track PEPs must have a Python-Version: header which indicates the version of Python that the feature will be released with. Informational PEPs do not need a Python-Version: header. While a PEP is in private discussions (usually during the initial Draft phase), a Discussions-To: header will indicate the mailing list or URL where the PEP is being discussed. No Discussions-To: header is necessary if the PEP is being discussed privately with the author, or on the python-list or python-dev email mailing lists. Note that email addresses in the Discussions-To: header will not be obscured. Created: records the date that the PEP was assigned a number, while Post-History: is used to record the dates of when new versions of the PEP are posted to python-list and/or python-dev. Both headers should be in dd-mmm-yyyy format, e.g. 14-Aug-2001. PEPs may have a Requires: header, indicating the PEP numbers that this PEP depends on. PEPs may also have a Replaced-By: header indicating that a PEP has been rendered obsolete by a later document; the value is the number of the PEP that replaces the current document. The newer PEP must have a Replaces: header containing the number of the PEP that it rendered obsolete. PEP Formatting Requirements PEP headings must begin in column zero and the initial letter of each word must be capitalized as in book titles. Acronyms should be in all capitals. The body of each section must be indented 4 spaces. Code samples inside body sections should be indented a further 4 spaces, and other indentation can be used as required to make the text readable. You must use two blank lines between the last line of a section's body and the next section heading. You must adhere to the Emacs convention of adding two spaces at the end of every sentence. You should fill your paragraphs to column 70, but under no circumstances should your lines extend past column 79. If your code samples spill over column 79, you should rewrite them. Tab characters must never appear in the document at all. A PEP should include the standard Emacs stanza included by example at the bottom of this PEP. A PEP must contain a Copyright section, and it is strongly recommended to put the PEP in the public domain. When referencing an external web page in the body of a PEP, you should include the title of the page in the text, with a footnote reference to the URL. Do not include the URL in the body text of the PEP. E.g. Refer to the Python Language web site [1] for more details. ... [1] http://www.python.org When referring to another PEP, include the PEP number in the body text, such as "PEP 1". The title may optionally appear. Add a footnote reference that includes the PEP's title and author. It may optionally include the explicit URL on a separate line, but only in the References section. Note that the pep2html.py script will calculate URLs automatically, e.g.: ... Refer to PEP 1 [7] for more information about PEP style ... References [7] PEP 1, PEP Purpose and Guidelines, Warsaw, Hylton http://www.python.org/peps/pep-0001.html If you decide to provide an explicit URL for a PEP, please use this as the URL template: http://www.python.org/peps/pep-xxxx.html PEP numbers in URLs must be padded with zeros from the left, so as to be exactly 4 characters wide, however PEP numbers in text are never padded. Reporting PEP Bugs, or Submitting PEP Updates How you report a bug, or submit a PEP update depends on several factors, such as the maturity of the PEP, the preferences of the PEP author, and the nature of your comments. For the early draft stages of the PEP, it's probably best to send your comments and changes directly to the PEP author. For more mature, or finished PEPs you may want to submit corrections to the SourceForge bug manager[6] or better yet, the SourceForge patch manager[2] so that your changes don't get lost. If the PEP author is a SF developer, assign the bug/patch to him, otherwise assign it to the PEP editor. When in doubt about where to send your changes, please check first with the PEP author and/or PEP editor. PEP authors who are also SF committers, can update the PEPs themselves by using "cvs commit" to commit their changes. Remember to also push the formatted PEP text out to the web by doing the following: % python pep2html.py -i NUM where NUM is the number of the PEP you want to push out. See % python pep2html.py --help for details. Transferring PEP Ownership It occasionally becomes necessary to transfer ownership of PEPs to a new champion. In general, we'd like to retain the original author as a co-author of the transferred PEP, but that's really up to the original author. A good reason to transfer ownership is because the original author no longer has the time or interest in updating it or following through with the PEP process, or has fallen off the face of the 'net (i.e. is unreachable or not responding to email). A bad reason to transfer ownership is because you don't agree with the direction of the PEP. We try to build consensus around a PEP, but if that's not possible, you can always submit a competing PEP. If you are interested assuming ownership of a PEP, send a message asking to take over, addressed to both the original author and the PEP editor <peps(a)python.org>. If the original author doesn't respond to email in a timely manner, the PEP editor will make a unilateral decision (it's not like such decisions can be reversed. :). References and Footnotes [1] This historical record is available by the normal CVS commands for retrieving older revisions. For those without direct access to the CVS tree, you can browse the current and past PEP revisions via the SourceForge web site at http://cvs.sourceforge.net/cgi-bin/cvsweb.cgi/python/nondist/peps/?cvsroot=… [2] http://sourceforge.net/tracker/?group_id=5470&atid=305470 [3] http://sourceforge.net/tracker/?atid=355470&group_id=5470&func=browse [4] http://www.opencontent.org/openpub/ [5] The script referred to here is pep2html.py, which lives in the same directory in the CVS tree as the PEPs themselves. Try "pep2html.py --help" for details. The URL for viewing PEPs on the web is http://www.python.org/peps/ [6] http://sourceforge.net/tracker/?group_id=5470&atid=305470 [7] PEP 9, Sample PEP Template http://www.python.org/peps/pep-0009.html Copyright This document has been placed in the public domain. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 End:

8 14

Reviving restricted mode?
by Guido van Rossum 14 Aug '14

14 Aug '14

I've received some enthusiastic emails from someone who wants to revive restricted mode. He started out with a bunch of patches to the CPython runtime using ctypes, which he attached to an App Engine bug: http://code.google.com/p/googleappengine/issues/detail?id=671 Based on his code (the file secure.py is all you need, included in secure.tar.gz) it seems he believes the only security leaks are __subclasses__, gi_frame and gi_code. (I have since convinced him that if we add "restricted" guards to these attributes, he doesn't need the functions added to sys.) I don't recall the exploits that Samuele once posted that caused the death of rexec.py -- does anyone recall, or have a pointer to the threads? -- --Guido van Rossum (home page: http://www.python.org/~guido/)

19 35

PEP 382: Namespace Packages
by "Martin v. Löwis" 04 Nov '09

04 Nov '09

I propose the following PEP for inclusion to Python 3.1. Please comment. Regards, Martin Abstract ======== Namespace packages are a mechanism for splitting a single Python package across multiple directories on disk. In current Python versions, an algorithm to compute the packages __path__ must be formulated. With the enhancement proposed here, the import machinery itself will construct the list of directories that make up the package. Terminology =========== Within this PEP, the term package refers to Python packages as defined by Python's import statement. The term distribution refers to separately installable sets of Python modules as stored in the Python package index, and installed by distutils or setuptools. The term vendor package refers to groups of files installed by an operating system's packaging mechanism (e.g. Debian or Redhat packages install on Linux systems). The term portion refers to a set of files in a single directory (possibly stored in a zip file) that contribute to a namespace package. Namespace packages today ======================== Python currently provides the pkgutil.extend_path to denote a package as a namespace package. The recommended way of using it is to put:: from pkgutil import extend_path __path__ = extend_path(__path__, __name__) int the package's ``__init__.py``. Every distribution needs to provide the same contents in its ``__init__.py``, so that extend_path is invoked independent of which portion of the package gets imported first. As a consequence, the package's ``__init__.py`` cannot practically define any names as it depends on the order of the package fragments on sys.path which portion is imported first. As a special feature, extend_path reads files named ``*.pkg`` which allow to declare additional portions. setuptools provides a similar function pkg_resources.declare_namespace that is used in the form:: import pkg_resources pkg_resources.declare_namespace(__name__) In the portion's __init__.py, no assignment to __path__ is necessary, as declare_namespace modifies the package __path__ through sys.modules. As a special feature, declare_namespace also supports zip files, and registers the package name internally so that future additions to sys.path by setuptools can properly add additional portions to each package. setuptools allows declaring namespace packages in a distribution's setup.py, so that distribution developers don't need to put the magic __path__ modification into __init__.py themselves. Rationale ========= The current imperative approach to namespace packages has lead to multiple slightly-incompatible mechanisms for providing namespace packages. For example, pkgutil supports ``*.pkg`` files; setuptools doesn't. Likewise, setuptools supports inspecting zip files, and supports adding portions to its _namespace_packages variable, whereas pkgutil doesn't. In addition, the current approach causes problems for system vendors. Vendor packages typically must not provide overlapping files, and an attempt to install a vendor package that has a file already on disk will fail or cause unpredictable behavior. As vendors might chose to package distributions such that they will end up all in a single directory for the namespace package, all portions would contribute conflicting __init__.py files. Specification ============= Rather than using an imperative mechanism for importing packages, a declarative approach is proposed here, as an extension to the existing ``*.pkg`` mechanism. The import statement is extended so that it directly considers ``*.pkg`` files during import; a directory is considered a package if it either contains a file named __init__.py, or a file whose name ends with ".pkg". In addition, the format of the ``*.pkg`` file is extended: a line with the single character ``*`` indicates that the entire sys.path will be searched for portions of the namespace package at the time the namespace packages is imported. Importing a package will immediately compute the package's __path__; the ``*.pkg`` files are not considered anymore after the initial import. If a ``*.pkg`` package contains an asterisk, this asterisk is prepended to the package's __path__ to indicate that the package is a namespace package (and that thus further extensions to sys.path might also want to extend __path__). At most one such asterisk gets prepended to the path. extend_path will be extended to recognize namespace packages according to this PEP, and avoid adding directories twice to __path__. No other change to the importing mechanism is made; searching modules (including __init__.py) will continue to stop at the first module encountered. Discussion ========== With the addition of ``*.pkg`` files to the import mechanism, namespace packages can stop filling out the namespace package's __init__.py. As a consequence, extend_path and declare_namespace become obsolete. It is recommended that distributions put a file <distribution>.pkg into their namespace packages, with a single asterisk. This allows vendor packages to install multiple portions of namespace package into a single directory, with no risk of overlapping files. Namespace packages can start providing non-trivial __init__.py implementations; to do so, it is recommended that a single distribution provides a portion with just the namespace package's __init__.py (and potentially other modules that belong to the namespace package proper). The mechanism is mostly compatible with the existing namespace mechanisms. extend_path will be adjusted to this specification; any other mechanism might cause portions to get added twice to __path__. Copyright ========= This document has been placed in the public domain.

31 110

pthreads, fork, import, and execvp
by Rotem Yaari 16 Sep '09

16 Sep '09

Hello everyone! We have been encountering several deadlocks in a threaded Python application which calls subprocess.Popen (i.e. fork()) in some of its threads. This has occurred on Python 2.4.1 on a 2.4.27 Linux kernel. Preliminary analysis of the hang shows that the child process blocks upon entering the execvp function, in which the import_lock is acquired due to the following line: def _ execvpe(file, args, env=None): from errno import ENOENT, ENOTDIR ... It is known that when forking from a pthreaded application, acquisition attempts on locks which were already locked by other threads while fork() was called will deadlock. Due to these oddities we were wondering if it would be better to extract the above import line from the execvpe call, to prevent lock acquisition attempts in such cases. Another workaround could be re-assigning a new lock to import_lock (such a thing is done with the global interpreter lock) at PyOS_AfterFork or pthread_atfork. We'd appreciate any opinions you might have on the subject. Thanks in advance, Yair and Rotem

12 29

compiling python2.5 on linux under wine
by Luke Kenneth Casson Leighton 08 Sep '09

08 Sep '09

hey, has anyone investigated compiling python2.5 using winegcc, under wine? i'm presently working my way through it, just for kicks, and was wondering if anyone would like to pitch in or stare at the mess under a microscope. it's not as crazed as it sounds. cross-compiling python2.5 for win32 with mingw32 is an absolute miserable bitch of a job that goes horribly wrong when you actually try to use the minimalist compiler to do any real work. so i figured that it would be easier to get python compiled using wine. i _have_ got some success - a python script and a python.exe.so (which is winegcc's friendly way of telling you you have something that stands a chance of working) as well as a libpython25.dll.so. what i _don't_ yet have is an _md5.dll (or should it be _md5.lib?) i.e. the standard modules are a bit... iffy. the _winreg.o is compiled; the _md5.o is compiled; the winreg.lib is not. whoops. plus, it's necessary to enable nt_dl.c which is in PC/ _not_ in Modules/. one of the key issues that's a bit of a bitch is that python is compiled up for win32 with a hard-coded pyconfig.h which someone went to a _lot_ of trouble to create by hand instead of using autoconf. oh - and it uses visualstudio so there's not even a Makefile. ignoring that for the time-being was what allowed me to get as far as actually having a python interpreter (with no c-based modules). so there's a whole _stack_ of stuff that needs dragging kicking and screaming into the 21st century. there _is_ a reason why i want to do this. actually, there's two. firstly, i sure as shit do _not_ want to buy, download, install _or_ run visual studio. i flat-out refuse to run an MS os and visual studio runs like a dog under wine. secondly, i want a python25.lib which i can use to cross-compile modules for poor windows users _despite_ sticking to my principles and keeping my integrity as a free software developer. thirdly i'd like to cross-compile pywebkitgtk for win32 fourthly i'd like to compile and link applications to the extremely successful and well wicked MSHTML.DLL... in the _wine_ project :) not the one in windows (!) i want to experiment with DOM model manipulation - from python - similar to the OLPC HulaHop project - _but_ i want to compile or cross-compile everything from linux, not windows (see 1 above) fifthly i'd like to see COM (DCOM) working and pywin32 compiled and useable under wine, even if it means having to get a license to use dcom98 and oleauth.lib and oleauth.h etc. and all the developer files needed to link DCOM applications under windows. actually what i'd _really_ like to see is FreeDCE's DCOM work actually damn well finished, it's only been eight years since wez committed the first versions of the IDL and header files, and it's only been over fifteen years since microsoft began its world domination using COM and DCOM. ... but that's another story :) so that's ... five reasons not two. if anyone would like to collaborate on a crazed project with someone who can't count, i'm happy to make available what i've got up to so far, on github.org. l.

6 12

python sendmsg()/recvmsg() implementation
by Kálmán Gergely 24 Jul '09

24 Jul '09

Hello, my name is Greg. I've just started using python after many years of C programming, and I'm also new to the list. I wanted to clarify this first, so that maybe I will get a little less beating for my stupidity :) I use python3 and Linux on arch x86 (production will be on x86_64, though this shouldn't matter much). The application that I'm presently working on is a network server. It would use separate processes to accept the connections, and to do the work (much like how apache prefork does). One process accept()s on the original socket and the received socket (client socket) will be read for a request. After the request is received and parsed this process (the controller) will choose one from its' children that is most capable of handling the said request. It would then pass the file descriptor through a socketpair to the appropriate children and go handle the next client. All works fine and smooth, but I realized that I need sendmsg()/recvmsg() to pass the FD. Since these are not implemented in the python socket module, and Linux has no other way to do this, I'm stuck. Fell flat on my face, too :) Browsing the net I've found a patch to the python core (http://bugs.python.org/issue1194378), dated 2005. First of all, I would like to ask you guys, whether you know of any way of doing this FD passing magic, or that you know of any 3rd party module / patch / anything that can do this for me. Since I'm fairly familiar with C and (not that much, but I feel the power) python, I would take the challenge of writing it, given that the above code is still somewhat usable. If all else fails I would like to have your help to guide me through this process. Thanks Kalman Gergely

5 7

GSoC: Core Python development tools
by Daniel (ajax) Diniz 23 Jul '09

23 Jul '09

Hi, I'd like to bring up the general idea of using a PSF slot in GSoC2009 to improve the Python development infrastructure. I also happen to have two concrete proposals for that (such a coincidence!). But I assure you the general idea is more important than my proposals :) General: Solving issues that get in core devs' way when they work on Python development could be a nice PSF GSoC project. I believe there are enough code related tasks that would greatly improve Python development workflow but lack manpower to complete. E.g., ISTM that a student working on svnmerge in past SoC editions would've been able to mitigate some painful shortcomings. If the core developers could come up with a list of peeves that would be solvable in code (in existing tools), that would allow for a very useful GSoC proposal. Proposals: These might fit the description above, and they're both tracker related (yep, one-trick-pony here). The upside is that at least one potential mentor is available for each, and I'd be able to offer support to the student. First, the PSF could invest a slot on integrating different tools of the development workflow. Variations of 'file issue at bug tracker, submit code for review' or 'branch locally to virtualenv, upload failing testcase to tracker, upload patch to tracker' command line utils. If these could be developed as general tools (i.e., not deeply tied to Python dev infrastructure), Ali Afshar from PIDA is willing to mentor it. I'd be available to help with Roundup and Rietveld (integration in progress), but would like to see it work with Launchpad, Bugzilla, Google Code and Review Board :) The other proposal is to use a slot in Roundup proper and the Python tracker instance. This could have a core Roundup developer as mentor, under the condition it's focused on Roundup itself. IMO, are some missing/broken core features in Roundup that would have a positive impact on our tracker, mostly those relating to searches, security and UI. I'd have a lot to contribute to the Python tracker part, based on ongoing work. Even if neither is considered worthy, I'll keep them on my to-do list and hope to slowly and hackishly work towards both proposals' goals. Barring feedback saying that they're out of scope, stupid and downright offensive, I'll post details on each to this thread very soon. So, if the PSF was to use a slot on improving the way you work on Python development, what would you like to see fixed or implemented? Best regards, Daniel

13 31

Re: subprocess and EINTR errnos
by Peter Astrand 06 Jul '09

06 Jul '09

On Wed, 10 Nov 2004, John P Speno wrote: Hi, sorry for the delayed response. > While using subprocess (aka popen5), I came across one potential gotcha. I've had > exceptions ending like this: > > File "test.py", line 5, in test > cmd = popen5.Popen(args, stdout=PIPE) > File "popen5.py", line 577, in __init__ > data = os.read(errpipe_read, 1048576) # Exceptions limited to 1 MB > OSError: [Errno 4] Interrupted system call > > (on Solaris 9) > > Would it make sense for subprocess to use a more robust read() function > which can handle these cases, i.e. when the parent's read on the pipe > to the child's stderr is interrupted by a system call, and returns EINTR? > I imagine it could catch EINTR and EAGAIN and retry the failed read(). I assume you are using signals in your application? The os.read above is not the only system call that can fail with EINTR. subprocess.py is full of other system calls that can fail, and I suspect that many other Python modules are as well. I've made a patch (attached) to subprocess.py (and test_subprocess.py) that should guard against EINTR, but I haven't committed it yet. It's quite large. Are Python modules supposed to handle EINTR? Why not let the C code handle this? Or, perhaps the signal module should provide a sigaction function, so that users can use SA_RESTART. Index: subprocess.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/subprocess.py,v retrieving revision 1.8 diff -u -r1.8 subprocess.py --- subprocess.py 7 Nov 2004 14:30:34 -0000 1.8 +++ subprocess.py 17 Nov 2004 19:42:30 -0000 @@ -888,6 +888,50 @@ pass + def _read_no_intr(self, fd, buffersize): + """Like os.read, but retries on EINTR""" + while True: + try: + return os.read(fd, buffersize) + except OSError, e: + if e.errno == errno.EINTR: + continue + else: + raise + + + def _read_all(self, fd, buffersize): + """Like os.read, but retries on EINTR, and reads until EOF""" + all = "" + while True: + data = self._read_no_intr(fd, buffersize) + all += data + if data == "": + return all + + + def _write_no_intr(self, fd, s): + """Like os.write, but retries on EINTR""" + while True: + try: + return os.write(fd, s) + except OSError, e: + if e.errno == errno.EINTR: + continue + else: + raise + + def _waitpid_no_intr(self, pid, options): + """Like os.waitpid, but retries on EINTR""" + while True: + try: + return os.waitpid(pid, options) + except OSError, e: + if e.errno == errno.EINTR: + continue + else: + raise + def _execute_child(self, args, executable, preexec_fn, close_fds, cwd, env, universal_newlines, startupinfo, creationflags, shell, @@ -963,7 +1007,7 @@ exc_value, tb) exc_value.child_traceback = ''.join(exc_lines) - os.write(errpipe_write, pickle.dumps(exc_value)) + self._write_no_intr(errpipe_write, pickle.dumps(exc_value)) # This exitcode won't be reported to applications, so it # really doesn't matter what we return. @@ -979,7 +1023,7 @@ os.close(errwrite) # Wait for exec to fail or succeed; possibly raising exception - data = os.read(errpipe_read, 1048576) # Exceptions limited to 1 MB + data = self._read_all(errpipe_read, 1048576) # Exceptions limited to 1 MB os.close(errpipe_read) if data != "": child_exception = pickle.loads(data) @@ -1003,7 +1047,7 @@ attribute.""" if self.returncode == None: try: - pid, sts = os.waitpid(self.pid, os.WNOHANG) + pid, sts = self._waitpid_no_intr(self.pid, os.WNOHANG) if pid == self.pid: self._handle_exitstatus(sts) except os.error: @@ -1015,7 +1059,7 @@ """Wait for child process to terminate. Returns returncode attribute.""" if self.returncode == None: - pid, sts = os.waitpid(self.pid, 0) + pid, sts = self._waitpid_no_intr(self.pid, 0) self._handle_exitstatus(sts) return self.returncode @@ -1049,27 +1093,33 @@ stderr = [] while read_set or write_set: - rlist, wlist, xlist = select.select(read_set, write_set, []) + try: + rlist, wlist, xlist = select.select(read_set, write_set, []) + except select.error, e: + if e[0] == errno.EINTR: + continue + else: + raise if self.stdin in wlist: # When select has indicated that the file is writable, # we can write up to PIPE_BUF bytes without risk # blocking. POSIX defines PIPE_BUF >= 512 - bytes_written = os.write(self.stdin.fileno(), input[:512]) + bytes_written = self._write_no_intr(self.stdin.fileno(), input[:512]) input = input[bytes_written:] if not input: self.stdin.close() write_set.remove(self.stdin) if self.stdout in rlist: - data = os.read(self.stdout.fileno(), 1024) + data = self._read_no_intr(self.stdout.fileno(), 1024) if data == "": self.stdout.close() read_set.remove(self.stdout) stdout.append(data) if self.stderr in rlist: - data = os.read(self.stderr.fileno(), 1024) + data = self._read_no_intr(self.stderr.fileno(), 1024) if data == "": self.stderr.close() read_set.remove(self.stderr) Index: test/test_subprocess.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_subprocess.py,v retrieving revision 1.14 diff -u -r1.14 test_subprocess.py --- test/test_subprocess.py 12 Nov 2004 15:51:48 -0000 1.14 +++ test/test_subprocess.py 17 Nov 2004 19:42:30 -0000 @@ -7,6 +7,7 @@ import tempfile import time import re +import errno mswindows = (sys.platform == "win32") @@ -35,6 +36,16 @@ fname = tempfile.mktemp() return os.open(fname, os.O_RDWR|os.O_CREAT), fname + def read_no_intr(self, obj): + while True: + try: + return obj.read() + except IOError, e: + if e.errno == errno.EINTR: + continue + else: + raise + # # Generic tests # @@ -123,7 +134,7 @@ p = subprocess.Popen([sys.executable, "-c", 'import sys; sys.stdout.write("orange")'], stdout=subprocess.PIPE) - self.assertEqual(p.stdout.read(), "orange") + self.assertEqual(self.read_no_intr(p.stdout), "orange") def test_stdout_filedes(self): # stdout is set to open file descriptor @@ -151,7 +162,7 @@ p = subprocess.Popen([sys.executable, "-c", 'import sys; sys.stderr.write("strawberry")'], stderr=subprocess.PIPE) - self.assertEqual(remove_stderr_debug_decorations(p.stderr.read()), + self.assertEqual(remove_stderr_debug_decorations(self.read_no_intr(p.stderr)), "strawberry") def test_stderr_filedes(self): @@ -186,7 +197,7 @@ 'sys.stderr.write("orange")'], stdout=subprocess.PIPE, stderr=subprocess.STDOUT) - output = p.stdout.read() + output = self.read_no_intr(p.stdout) stripped = remove_stderr_debug_decorations(output) self.assertEqual(stripped, "appleorange") @@ -220,7 +231,7 @@ stdout=subprocess.PIPE, cwd=tmpdir) normcase = os.path.normcase - self.assertEqual(normcase(p.stdout.read()), normcase(tmpdir)) + self.assertEqual(normcase(self.read_no_intr(p.stdout)), normcase(tmpdir)) def test_env(self): newenv = os.environ.copy() @@ -230,7 +241,7 @@ 'sys.stdout.write(os.getenv("FRUIT"))'], stdout=subprocess.PIPE, env=newenv) - self.assertEqual(p.stdout.read(), "orange") + self.assertEqual(self.read_no_intr(p.stdout), "orange") def test_communicate(self): p = subprocess.Popen([sys.executable, "-c", @@ -305,7 +316,8 @@ 'sys.stdout.write("\\nline6");'], stdout=subprocess.PIPE, universal_newlines=1) - stdout = p.stdout.read() + + stdout = self.read_no_intr(p.stdout) if hasattr(open, 'newlines'): # Interpreter with universal newline support self.assertEqual(stdout, @@ -343,7 +355,7 @@ def test_no_leaking(self): # Make sure we leak no resources - max_handles = 1026 # too much for most UNIX systems + max_handles = 10 # too much for most UNIX systems if mswindows: max_handles = 65 # a full test is too slow on Windows for i in range(max_handles): @@ -424,7 +436,7 @@ 'sys.stdout.write(os.getenv("FRUIT"))'], stdout=subprocess.PIPE, preexec_fn=lambda: os.putenv("FRUIT", "apple")) - self.assertEqual(p.stdout.read(), "apple") + self.assertEqual(self.read_no_intr(p.stdout), "apple") def test_args_string(self): # args is a string @@ -457,7 +469,7 @@ p = subprocess.Popen(["echo $FRUIT"], shell=1, stdout=subprocess.PIPE, env=newenv) - self.assertEqual(p.stdout.read().strip(), "apple") + self.assertEqual(self.read_no_intr(p.stdout).strip(), "apple") def test_shell_string(self): # Run command through the shell (string) @@ -466,7 +478,7 @@ p = subprocess.Popen("echo $FRUIT", shell=1, stdout=subprocess.PIPE, env=newenv) - self.assertEqual(p.stdout.read().strip(), "apple") + self.assertEqual(self.read_no_intr(p.stdout).strip(), "apple") def test_call_string(self): # call() function with string argument on UNIX @@ -525,7 +537,7 @@ p = subprocess.Popen(["set"], shell=1, stdout=subprocess.PIPE, env=newenv) - self.assertNotEqual(p.stdout.read().find("physalis"), -1) + self.assertNotEqual(self.read_no_intr(p.stdout).find("physalis"), -1) def test_shell_string(self): # Run command through the shell (string) @@ -534,7 +546,7 @@ p = subprocess.Popen("set", shell=1, stdout=subprocess.PIPE, env=newenv) - self.assertNotEqual(p.stdout.read().find("physalis"), -1) + self.assertNotEqual(self.read_no_intr(p.stdout).find("physalis"), -1) def test_call_string(self): # call() function with string argument on Windows /Peter Åstrand <astrand(a)lysator.liu.se>

5 5

PEP 376
by Tarek Ziadé 04 Jul '09

04 Jul '09

Hello, If no one objects, I'd like to push PEP 376 in the "accepted" status and go ahead with its implementation, with continuous feedback at Distutils-SIG as we did to build it. The next PEPs that are being discussed at Distutils-SIG are : - the new version of PEP 345 for the inclusion of fields like "installed_requires" (dependencies) - PEP 386, that talks about distribution version numbers (because it's needed to describe dependencies) While they are still under heavy discussion, I have good hope that they will both make it for 2.7 and 3.2. Regards Tarek -- Tarek Ziadé | http://ziade.org

15 52

PEP 376
by Tarek Ziadé 03 Jul '09

03 Jul '09

Hello, We have polished out PEP 376 and its code prototype at Distutils-SIG. It seems to fullfill now all the requirements, so I am mailing it here again, for a new round of feedback, if needed. - the pep : http://svn.python.org/projects/peps/trunk/pep-0376.txt - the code prototype : http://bitbucket.org/tarek/pep376/src/tip/pkgutil.py Notice that if the PEP is accepted at this point, I will : - focus on making the code work as fast as possible, for directories browsing - work on the backport and the required patches for setuptools and pip at the same time, and see if I can get some beta-testers that are willing to switch to this new version to test it extensively before 2.7/3.2 are out. Regards Tarek -- Tarek Ziadé | http://ziade.org

8 16

pthread sem PyThread_acquire_lock
by Florian Mayer 03 Jul '09

03 Jul '09

Hello. As this is my first post I will try to introduce myself as requested in the welcome email. If you aren't interested in my person, just continue reading at the next paragraph. I'm a student from Vienna/Austria. I attend what would match high school in the United States. I have been writing Python for about 3 years now and have just began to dig into the implementation of CPython. Now to my real question. I have noticed that PyThread_acquire_lock swallows all signals with pthread using sems. Looking at the source code it appears that this is intended, but I cannot see the reason for that. It seems the pthread sem implementation is the only one doing so. Can any of you tell me the reason why it swallows signals? I have already prepared a patch that introduces a new _noswallow function if sem are used and uses this in threadmodule.c. But if there isn't any real reason for it to swallow exceptions, I think it would be wisest to change that. Best regards, Florian Mayer

6 8

Exception for setting attributes of built-in type
by Seo Sanghyeon 02 Jul '09

02 Jul '09

Exception for setting attributes of built-in type differs between CPython and IronPython. This is not purely theoretical, as zope.interface tries to set Implements declaration as __implemented__ attribute of built-in type object, and excepts TypeError. Python 2.6.1 >>> object.flag = True TypeError: can't set attributes of built-in/extension type 'object' IronPython 2.6 >>> object.flag = True AttributeError: 'object' object has no attribute 'flag' I was surprised that CPython raises TypeError. Library Reference seems to mention it here: exception AttributeError Raised when an attribute reference or assignment fails. (When an object does not support attribute references or attribute assignments at all, TypeError is raised.) http://docs.python.org/library/exceptions.html What does it mean that "an object does not support attribute references or attribute assignments at all"? -- Seo Sanghyeon

8 12

Re: [Python-Dev] PEP 376
by P.J. Eby 01 Jul '09

01 Jul '09

At 07:57 PM 6/29/2009 +0200, Tarek Ziadé wrote: >Hello, > >If no one objects, I'd like to push PEP 376 in the "accepted" status >and go ahead with its implementation, >with continuous feedback at Distutils-SIG as we did to build it. I do have a question about the current draft... Do zipped distributions use EGG-INFO or a project-version.egg-info? This isn't spelled out in the PEP, although I get the general idea that the EGG-INFO format isn't supported, and thus the PEP and API do not support existing .egg files. This should probably be made clear, if that's the intention.

3 5

Flow of control - a new way - Idea
by vin vin 30 Jun '09

30 Jun '09

HI all I am not the too technical guy, but thinking about the new way of controlling the flow instead of throwing an error. as of now if we need to break a control and go back, exceptions helps, but it is not a actual way. it would be great if we have a control over the frames execution, I mean A calls B which calls C which calls D at that point if we think to move directly to B (what error handler do if that B has the handler defined of the error), changing the frames instruction pointer to back to the B's position (in python code without raising an exception) is a really useful for this web applications. ----------------------------- How we can achieve this? a simple way? Actually python interpreter currently doing this. How? method *A* method *B* <== has exception handler try: except: method *C* method *D* <========= got exception here.. now exception handler checks that which caller methods has the exception handler provided for this exception. in our case method B has this. so, the instruction pointer has moved back to that method B with the exception. ------------------- We can have this same functionality* BUT WITHOUT THE USE OF EXCEPTION*. where it can help? so many places, & it gives the full power of execution control to the user. A simple usage of this is : Currently most of the WSGI frameworks breaking the flow if needed to go on another route, it simple raising the exception, But if any of the called modules handled those exceptions it fails the flow. and we can't guide the extension modules to don't use full try except as it is the pythons power. What you say? Please excuse me if we have this control already, (can u explain?) I hope, currently no languages gives such full control over coders. -- Thanks Vinoth

4 3

Anyone against having a loop option for regrtest?
by Jesse Noller 30 Jun '09

30 Jun '09

Something that's been helping me squirrel out "wacky" and "fun" bugs in multiprocessing is running the tests in a loop - sometimes hundreds of times. Right now, I hack this up with a bash script, but I'm sitting here wondering if adding a "loop for x iterations" option to regrtest.py would be useful to others as well. Any thoughts? Does anyone hate this idea with the power of a thousand suns? -jesse

3 3

Re: [Python-Dev] ndPython: I NEED TO TALK WITH ONE OF THE PYTHON CORE
by Filippo Battaglia 29 Jun '09

29 Jun '09

Thanks for your answers. Sorry for the title in upper case. I didn't want to create troubles. :) I've an important question for you: is it possible that a large python module, created using SWIG and with a hundred of routines, makes slower the execution (i.e. the job of ceval.c) of the Python interpreter ? We've observed that, if we don't import ndpsp.pyc at startup, the time of execution of a loop containing the pass instruction becomes near normal. How Python recalls the C functions in a C wrapper ? Thanks for your very important help. Filippo Battaglia

3 2

2.4.7: Final 2.4 release
by "Martin v. Löwis" 29 Jun '09

29 Jun '09

Python 2.4 will become 5 years old this November. I plan to make the final security release this month or next month. If you want to see additional patches in Python 2.4, please let us now, or commit them yourself if you can. Remember that only security fixes can be considered for inclusion, and preferably only fixes that have already been applied to later releases of Python. Regards, Martin

1 0

[RELEASED] Python 3.1 final
by Benjamin Peterson 28 Jun '09

28 Jun '09

On behalf of the Python development team, I'm thrilled to announce the first production release of Python 3.1. Python 3.1 focuses on the stabilization and optimization of the features and changes that Python 3.0 introduced. For example, the new I/O system has been rewritten in C for speed. File system APIs that use unicode strings now handle paths with undecodable bytes in them. Other features include an ordered dictionary implementation, a condensed syntax for nested with statements, and support for ttk Tile in Tkinter. For a more extensive list of changes in 3.1, see http://doc.python.org/3.1/whatsnew/3.1.html or Misc/NEWS in the Python distribution. To download Python 3.1 visit: http://www.python.org/download/releases/3.1/ The 3.1 documentation can be found at: http://docs.python.org/3.1 Bugs can always be reported to: http://bugs.python.org Enjoy! -- Benjamin Peterson Release Manager benjamin at python.org (on behalf of the entire python-dev team and 3.1's contributors)

6 5

Summary of Python tracker Issues
by Python tracker 26 Jun '09

26 Jun '09

ACTIVITY SUMMARY (06/19/09 - 06/26/09) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 2242 open (+17) / 15925 closed (+16) / 18167 total (+33) Open issues with patches: 889 Average duration of open issues: 657 days. Median duration of open issues: 408 days. Open Issues Breakdown open 2212 (+17) pending 30 ( +0) Issues Created Or Reopened (33) _______________________________ test_with.py has a couple minor mistakes 06/19/09 CLOSED http://bugs.python.org/issue6313 created exarkun patch logging.basicConfig(level='DEBUG', ... 06/20/09 CLOSED http://bugs.python.org/issue6314 created alexl locale._build_localename(locale.getdefaultlocale()) returns 'C.m 06/20/09 CLOSED http://bugs.python.org/issue6315 created sjt format, str.format don't work well with datetime, date object 06/20/09 CLOSED http://bugs.python.org/issue6316 created falsetru winsound.PlaySound doesn't accept non-unicode string 06/21/09 http://bugs.python.org/issue6317 created ocean-city patch HTMLParser Attributes Containing Escaped Quotes 06/21/09 CLOSED http://bugs.python.org/issue6318 created ericryk bdist_msi runs out of memory for large packages 06/21/09 CLOSED http://bugs.python.org/issue6319 created bethard patch Standard string encodings should include GSM0.38 06/21/09 http://bugs.python.org/issue6320 created jwishnie Reload Python modules when running programs 06/22/09 http://bugs.python.org/issue6321 created samwyse Pdb breakpoints don't work on lines without bytecode 06/22/09 http://bugs.python.org/issue6322 created inducer Py3.1 pdb doesn't deal well with syntax errors 06/22/09 http://bugs.python.org/issue6323 created inducer patch, needs review "in" expression falls back to __iter__ before __getitem__ 06/22/09 http://bugs.python.org/issue6324 created afoglia robotparser doesn't handle URL's with query strings 06/23/09 http://bugs.python.org/issue6325 created skybrian Add a "swap" method to list 06/23/09 http://bugs.python.org/issue6326 created krisvale patch, patch [mimetext] long lines get cut with exclamation mark and newline 06/23/09 http://bugs.python.org/issue6327 created martijntje login() function failed in smtplib with message "argument 1 must 06/23/09 CLOSED http://bugs.python.org/issue6328 created hdvision Fix iteration for memoryviews 06/23/09 CLOSED http://bugs.python.org/issue6329 created rhettinger patch trunk does not build with --enable-unicode=ucs4 06/23/09 http://bugs.python.org/issue6330 created eric.smith easy Add unicode script info to the unicode database 06/23/09 http://bugs.python.org/issue6331 created doerwalter patch typo on man page warning control 06/23/09 CLOSED http://bugs.python.org/issue6332 created lieryan patch logging: ValueError: I/O operation on closed file 06/24/09 http://bugs.python.org/issue6333 created srid 3.0/3.1: Bad bug in range() computation (or possible Integer pro 06/24/09 CLOSED http://bugs.python.org/issue6334 created mfxmfx patch Add support for mingw 06/24/09 http://bugs.python.org/issue6335 created smartmobili patch nb_divide missing in docs 06/24/09 http://bugs.python.org/issue6336 created flub multiprocessing module: Double close of sys.stdin - ID: 2811568 06/24/09 CLOSED http://bugs.python.org/issue6337 created subdir patch Error message displayed on stderr when no C compiler is present 06/24/09 http://bugs.python.org/issue6338 created ehuss patch Some functional errors in turtle.py documentation (missing links 06/24/09 CLOSED http://bugs.python.org/issue6339 created gregorlingl replace tdemo_chaos.py 06/24/09 CLOSED http://bugs.python.org/issue6340 created gregorlingl io.path.ismount gives "local variable 'p' referenced before ass 06/25/09 CLOSED http://bugs.python.org/issue6341 created g.moralis io.path.ismount gives "local variable 'p' referenced before ass 06/25/09 CLOSED http://bugs.python.org/issue6342 created g.moralis TimedRotatingFileHandler permission denied rename failure on Win 06/25/09 http://bugs.python.org/issue6343 created dnagata mmap.read() crashes when passed a negative argument 06/25/09 http://bugs.python.org/issue6344 created amaury.forgeotdarc extra characters displayed when writing to sys.stdout.buffer 06/26/09 CLOSED http://bugs.python.org/issue6345 created metolone Issues Now Closed (31) ______________________ zlib.crc32() and adler32() return value 263 days http://bugs.python.org/issue1202 gregory.p.smith 64bit Tix HList class missing method implementation for info_bbox 627 days http://bugs.python.org/issue1230 gpolo xml/sax/expatreader.py raises AttributeError when run 202 days http://bugs.python.org/issue4490 amaury.forgeotdarc patch test_tcl testLoadTk fails if DISPLAY defined but connect fails, 105 days http://bugs.python.org/issue5450 gpolo patch pyexpat defines global symbol template_string 86 days http://bugs.python.org/issue5590 doko SimpleXMLRPCServer not suitable for HTTP/1.1 keep-alive 27 days http://bugs.python.org/issue6096 loewis patch, patch, easy, needs review HTTP/1.1 with keep-alive support for xmlrpclib.ServerProxy 27 days http://bugs.python.org/issue6099 loewis patch, patch, needs review Backport the IO lib to trunk 15 days http://bugs.python.org/issue6215 pitrou patch subprocess.Popen() may leak file descriptors 7 days http://bugs.python.org/issue6274 facundobatista patch Update contextlib.nested docs in light of deprecation 6 days http://bugs.python.org/issue6288 ncoghlan virtual memory error with archivemail 0 days http://bugs.python.org/issue6311 loewis test_with.py has a couple minor mistakes 0 days http://bugs.python.org/issue6313 benjamin.peterson patch logging.basicConfig(level='DEBUG', ... 2 days http://bugs.python.org/issue6314 vsajip locale._build_localename(locale.getdefaultlocale()) returns 'C.m 5 days http://bugs.python.org/issue6315 r.david.murray format, str.format don't work well with datetime, date object 0 days http://bugs.python.org/issue6316 falsetru HTMLParser Attributes Containing Escaped Quotes 0 days http://bugs.python.org/issue6318 georg.brandl bdist_msi runs out of memory for large packages 0 days http://bugs.python.org/issue6319 bethard patch login() function failed in smtplib with message "argument 1 must 0 days http://bugs.python.org/issue6328 r.david.murray Fix iteration for memoryviews 0 days http://bugs.python.org/issue6329 rhettinger patch typo on man page warning control 0 days http://bugs.python.org/issue6332 georg.brandl patch 3.0/3.1: Bad bug in range() computation (or possible Integer pro 1 days http://bugs.python.org/issue6334 mfxmfx patch multiprocessing module: Double close of sys.stdin - ID: 2811568 0 days http://bugs.python.org/issue6337 jnoller patch Some functional errors in turtle.py documentation (missing links 1 days http://bugs.python.org/issue6339 r.david.murray replace tdemo_chaos.py 1 days http://bugs.python.org/issue6340 r.david.murray io.path.ismount gives "local variable 'p' referenced before ass 0 days http://bugs.python.org/issue6341 r.david.murray io.path.ismount gives "local variable 'p' referenced before ass 0 days http://bugs.python.org/issue6342 amaury.forgeotdarc extra characters displayed when writing to sys.stdout.buffer 0 days http://bugs.python.org/issue6345 benjamin.peterson curselection() in Tkinter.py should return ints 1997 days http://bugs.python.org/issue869780 gpolo patch Add XML-RPC Fault Interoperability to XMLRPC libraries 1310 days http://bugs.python.org/issue1360243 loewis patch, easy Tix ComboBox entry is blank when not editable 944 days http://bugs.python.org/issue1600182 gpolo syslog syscall support for SysLogLogger 782 days http://bugs.python.org/issue1711603 LwarX patch Top Issues Most Discussed (10) ______________________________ 9 Native (and default) tarfile support for setup.py sdist in dist 9 days open http://bugs.python.org/issue6296 9 ElementTree (py3k) doesn't properly encode characters that can' 19 days open http://bugs.python.org/issue6233 9 Python 2.6 makes .pyc/.pyo bytecode files executable 37 days open http://bugs.python.org/issue6070 7 3.0/3.1: Bad bug in range() computation (or possible Integer pr 1 days closed http://bugs.python.org/issue6334 6 cElementTree.iterparse & ElementTree.iterparse return different 15 days open http://bugs.python.org/issue6266 5 Add a "swap" method to list 3 days open http://bugs.python.org/issue6326 5 Py3.1 pdb doesn't deal well with syntax errors 4 days open http://bugs.python.org/issue6323 5 os.popen causes illegal seek on AIX in Python 3.1rc 19 days open http://bugs.python.org/issue6236 5 test_tcl testLoadTk fails if DISPLAY defined but connect fails, 105 days closed http://bugs.python.org/issue5450 5 Document auto __ne__ generation; provide a use case for non-tri 215 days open http://bugs.python.org/issue4395

1 0

Re: [Python-Dev] [Python-checkins] r73569 - peps/trunk/pep-0101.txt
by Barry Warsaw 26 Jun '09

26 Jun '09

On Jun 26, 2009, at 8:49 AM, benjamin.peterson wrote: > Author: benjamin.peterson > Date: Fri Jun 26 14:48:55 2009 > New Revision: 73569 > > Log: > update release candidate shorthand > > Modified: > peps/trunk/pep-0101.txt > > Modified: peps/trunk/pep-0101.txt > = > = > = > = > = > = > = > = > ====================================================================== > --- peps/trunk/pep-0101.txt (original) > +++ peps/trunk/pep-0101.txt Fri Jun 26 14:48:55 2009 > @@ -66,7 +66,7 @@ > > We use the following conventions in the examples below. Where a > release > number is given, it is of the form X.YaZ, e.g. 2.6a3 for Python > 2.6 alpha > - 3, where "a" == alpha, "b" == beta, "c" == release candidate. > + 3, where "a" == alpha, "b" == beta, "rc" == release candidate. > > Final releases are named "releaseXY". The branch tag is > "releaseXY-maint" > because this will point to the long lived maintenance branch. > The fork I'm sure this has been discussed but I missed it. Why was this change made? If nothing else, it breaks many years of tradition. -Barry

3 2

ndPython: I NEED TO TALK WITH ONE OF THE PYTHON CORE DEVELOPERS
by Filippo Battaglia 25 Jun '09

25 Jun '09

I need help with an implementation of your interpreter under Nanodesktop PSPE/PSP. I'm working to a version of Stackless Python interpreter (2.5.2) called ndPython. It is able to work under PSPE and PSP and it shall be the most powerful interpreter ever realized before on these platforms. I need to know something about the C functions that are recalled by the interpreter when it executes a .pyc file. Our version of ndPython is very slow in execution respect to Carlos's StackLess Python (another product for PSP). We believe that the trouble is in any routine of our Nanodesktop libc that could be a bottleneck. But we don't know which can be the interested routine (string ? memory allocation ?) Can you tell me which modules provide to decode and execute a .pyc file ? The number of modules that are statically linked in the interpreter can affect its performance ? (for example, for a larger number of strcmp between the keywords). The execution of a .pyc file is a memory allocation intensive task ? I've already enable the PYMEM option and I've see that the speed is improved (but Carlos's StackLess Python remains faster). We've done these tests: http://www.psp-ita.com/forum/viewtopic.php?t=28323&start=20 from ndpsp import * x = time.clock () for i in range(1000): pass print >> clock, "1.", time.clock()-t We've the results: stackless: 0.0037279 ndPython: 0.1015625 We don't understand why ndPython is so slower. Please, help us. The product is complete, but we cannot release cause this problem. Thank you in advance. Filippo Battaglia -- Caselle da 1GB, trasmetti allegati fino a 3GB e in piu' IMAP, POP3 e SMTP autenticato? GRATIS solo con Email.it http://www.email.it/f Sponsor: Vieni in vacanza nei Riccione Family Hotels! Ti aspettano servizi per bambini, mini club e tanto divertimento! Prenota on-line la tua vacanza! Clicca qui: http://adv.email.it/cgi-bin/foclick.cgi?mid=9206&d=24-6

5 5

Syntax for parsing tuples/lists into C arrays
by Yingjie Lan 25 Jun '09

25 Jun '09

The current C API does not support directly parsing into C arrays. Parsing tuples into C arrays can be greatly facilitated with the introduction of a natural extension of the current parsing syntax. I'd like to propose a syntax extension to facilitate that kind of tasks, and would appreciate your feedbacks. The extension comes from the way strings are parsed in current C API: { const char * s; int i; PyArg_ParseTyple(args, "s#", &s, &i); } Consider for a moment the string as a sequence of char's, which by the example above is parsed into an array of char's. Note that in python, a tuple of some type (e.g. int's), is also a sequence of that type, This perspective provides a natural way to extend the the use of "#" symbol -- what is missing is how to signal a tuple as a sequence, without interfering with current fixed-length tuple parsing syntax. To signal a tuple as a sequence of a given type of objects, "(type:)" can be used. For example, "(i:)" signals a tuple of integers as array of integers. The ":" is chosen to denote possible repetition of the whole pattern inside the parentheses. Now consider 3 different scenarios of parsing tuples into arrays. Scenario (1) Unkown size. Here are some examples: { //parsing a tuple and its size int i, *t; PyArg_ParseTuple(args, "(i:)#", &t, &i); } { //parsing a matrix of i rows and j columns: int i, j, **t; PyArg_ParseTuple(args, "((i:):)##", &t, &i, &j); } Here is an informal explanation of "((i:):)##": the '#' can be seen as a peeling operator, which peels away the outer most "(...:)" that denote a "free sequence" (i.e. sequence of unknown size) with some pattern "...". For types like 's' or its similar types, pretend as if there is an invisible pair of parentheses around it. For example, "(s:)##" matches a tuple of strings of some fixed length. Here the first '#' peels away the parentheses, leaving only "s#", which is equivalent to "(*char*:)#" in concept, where "*char*" denote the character type (not exist in Python). As the second '#' is outside the tuple, it is shared by all strings inside, so they _must_ have the same length. If there is nothing to peel for a '#', it is considered a dangling '#', an error. Now consider some more interesting examples: { int i, *t, *v; PyArg_ParseTuple(args, "(ii:)#", &t, &v, &i); //args=(1,2,3,4,5,6) //t={1,3,5} //v={2,4,6} //i=3 } In that example, a pattern is provided inside the sequence "(ii:)", which can repeat itself multiple times. The repeated times is extracted by the '#' symbol. Below is another example: { int i,j, *t, **v; PyArg_ParseTuple(args, "(i(i:):)##", &t, &v, &i, &j); //args=(1,(3,),5,(7,)) //t={1,5} //v={{3},{7}} //i=2, j=1 } And another one: { int i,j, **t, **v; PyArg_ParseTuple(args, "((i:)(i:):)##", &t, &v, &i, &j); //args=((1,),(3,),(5,),(7,)) //t={{1},{5}} //v={{3},{7}} //i=2, j=1 } Note the second '#' is applied to both the free sequences inside the outer tuple. This indicates that all the inside tuples must be of the same length. What if they have their own lengths? We provide a separate matching '#' for each inside tuple: { int i,j,k, **t, **v; PyArg_ParseTuple(args, "((i:)(i:):)#(##)", &t, &v, &i, &j,&k); //args=((1,),(3,4),(5,),(7,8)) //t={{1},{5}} //v={{3,4},{7,8}} //i=2, j=1, k=2 } The syntax can also deal with more complicated situations, such as an irregular 2-dimensional array, where each row has a different row length: { int i, *j, **t; PyArg_ParseTuple(args, "((i:)#:)#", &t, &j, &i); for(a=0; a) //t={{{1,2},{1,2}},{{1,2}}} //i=2, k=2 //j={2,1} } This syntax can be used for parsing a tuple of strings of different lengths: { int i, *j; const char ** s; PyArg_ParseTuple(args, "(s#:)#", &s, &j, &i); //args=("ab", "cde") //s={"ab", "cde"} //j={2, 3} //i=2 } Scenario (2) If you know the size of a tuple before hand, use this syntax: "(*type*integer)", meaning there are exactly 'integer' number of *type* objects in the tuple. { int *t; // PyArg_ParseTuple(args, "(i5)", &t); //args = (1,2,3,4,5) //t = {1,2,3,4,5} } A complicated case: { int **t; PyArg_ParseTuple(args, "((i5)2)", &t); //args=((1,2,3,4,5),(5,6,7,8,9)) //t={{1,2,3,4,5},{5,6,7,8,9}} } Another complicated case: { int **t, **v, i; PyArg_ParseTuple(args, "(i3i2:)#", &t,&v,&i); //args=(1,2,3,4,5,6,7,8,9,10) //t={{1,2,3},{6,7,8}} //v={{4,5},{9,10}} //i=2 } One more thing: the difference between "(ii)" and "(i2)" is that the former yields two single integers, while the latter gives an array of length two. Scenario (3) Concatenating arrays while parsing. Sometimes you would like to have a matrix parsed into a single array. We can use a "(+" to signal that the arrays parsed from the elements in that tuple should be concatenated into a single array. { int i,j, *t; PyArg_ParseTuple(args, "(+(i))##", &t, &i, &j); //args=((1,2,3),(4,5,6)) //t={1,2,3,4,5,6} //i=2, j=3 } Another example: { int i, *t; PyArg_ParseTuple(args, "(+(i2))#", &t, &i); } Still another example, with strings: { int i, *j; const char * t; //string PyArg_ParseTuple(args, "(+(+s#2))#", &t, &j, &i); //args=(("ab","c"),("d","efg")) //t="abcdefg" //i=2 ////notice the size arrays are concatenated similarly //j={2,1,1,3} //without '+': j={{2,1},{1,3}} const char ** sa; //The '*' in place of '#' to discard the size info PyArg_ParseTuple(args, "(+(s*2))", &sa); //args=(("ab","c"),("d","efg")) //sa = {"ab","c","d","efg"} PyArg_ParseTuple(args, "((+s*2))", &sa); //args=(("ab","c"),("d","efg")) //sa = { "abc", "defg" } } And finally, an example with multi-item pattern: { int *t, *v, i; PyArg_ParseTuple(args, "(+i1i2:)#", &t, &v, &i); //args=(1,2,3,4,5,6) //t={1,4} //v={2,3,5,6} //i=2 } Epilogue: This syntax can be applied to lists: just use "[...:]" instead of "(...:)". It can also be used for building tuples or lists out of arrays. That's it. Comments? Yingjie

3 2

trace module options.
by Edward Peschko 25 Jun '09

25 Jun '09

All, I've been looking at the trace module, and although it looks useful, I'm surprised that there aren't a couple of features that I would have thought would be fairly basic. So, does trace support (for the --trace option): - indentation tracking stacklevel (where each function is prefixed by tabs equal to the number of stacklevels deep in the program) - output to something other than sys.stdout (eg. output to a file specified either by environmental variable or by parameter). - mult-threaded programs going to multiple output handles, especially in light of the above - fully qualified python modules in path: (eg: /path/to/module/my_module.py(1): print "HERE" instead of my_module.py(1): print "HERE". Ultimately, I'd like to be able to look at two runs of a program and be able to pinpoint the very first difference between thembased on the output of their trace runs. As it stands, I really can't do this. Of course I could program the above, but I was hoping to avoid duplicated effort if someone has already programmed options like this.. Ed

2 2

Python for Windows??
by David H. Burns 25 Jun '09

25 Jun '09

I downloaded what claims to be Python for Windows (3.01). The tutorial brags a lot about how easy it is to learn, but the tutorials and instruction seem to be for a Linux or Unix version. There are three executable programs in the Python directory and no indication which should be used to start Python. One opens a Dos-like window presumably for "command-line" entry. I can't make anything of it. I did a straight install on a XP system. Any help would be appreciated. Have a good day, David H. Burns

4 3

Re: [Python-Dev] PEP 376
by P.J. Eby 24 Jun '09

24 Jun '09

At 10:38 AM 6/23/2009 +0200, Tarek Ziadé wrote: >What about using the same names used in Python's site module: >"sitedir" is the name used for >a directory we named DistributionDirectory. No, a site dir is a Python-defined directory for site-installed packages, and/or a directory where .pth files are processed. Wrong connotations entirely, since packages may be installed to other directories, and typically are in e.g. shared hosting environments. DistributionDirectory is fine by me. DistributionDirectories sounds like what setuptools calls an Environment.

1 0