Mailman 3 October 2009 - Python-Dev

Retrieve an arbitrary element from a set without removing it
by average 28 Oct '09

28 Oct '09

[I wrote:] > If Python3 were to have this feature it would make it worth > migrating to Sorry that may have sounded more harsh than I expected. If I had more resources, I'd propose (and volunteer) a python3000 branch where any and all who were disappointed at the *lack* of compatability changes could continue working on the core language. (Moratorium controversy solved and quaranteened--any desireable features for the working branch would be back-propagated via the time machine as BDFL permits.) marcos

3 3

Where is `ctypes.com` ?
by Olemis Lang 28 Oct '09

28 Oct '09

Hello ! Recently I found a code snippet [1]_ illustrating integration between Python and COM technology in Win32 systems. I tried to reproduce it and I can't import module `ctypes.com`. Q: - Is it (`ctypes.com`) distributed with stdlib ? If that's true then I'm affraid that those py files were removed somehow (accidentally ? who knows ? ) ... Thnx in advance ! .. [1] ctypes.win32.com.server @ Py (http://svn.python.org/projects/ctypes/tags/release_0_6_2/ctypes/win32/com/s…) -- Regards, Olemis. Blog ES: http://simelo-es.blogspot.com/ Blog EN: http://simelo-en.blogspot.com/ Featured article: [visualization-api] ANN: TracGViz 1.3.4 released: msg#00244 ... - http://feedproxy.google.com/~r/TracGViz-full/~3/OBfKGS_sy2E/msg00244.html

2 2

Python 2.6.4rc2
by Barry Warsaw 28 Oct '09

28 Oct '09

Hello everyone. The source tarballs and Windows installers for Python 2.6.4rc2 are now available: http://www.python.org/download/releases/2.6.4/ Please download them, install them, and try to use them with your projects and environments. Let's make 2.6.4 a rock solid release! If there are no more regressions found, we'll do the final release in one week, on 25-October. Enjoy, -Barry

11 18

Set methods for mapping views
by Raymond Hettinger 28 Oct '09

28 Oct '09

[GvR] > I still wish we could go back in time and unify sets and dicts, if > only to find out how that experiment would turn out. I'm curious about the outcome of another experiment along those lines. Is anyone seeing uptake for the set methods on mapping views in Py3.x? I haven't seen any code using it, nor any bug reports or documentation requests, nor any code in the ASPN cookbook, nor mention of it on the newsgroup or python-help. Has anyone here seen any hints about how this is faring in the wild? Raymond

7 9

RELEASED Python 2.6.4
by Barry Warsaw 27 Oct '09

27 Oct '09

On behalf of the Python community, I'm happy to announce the availability of Python 2.6.4. This is the latest production-ready version in the Python 2.6 series. We had a little trouble with the Python 2.6.3 release; a number of unfortunate regressions were introduced. I take responsibility for rushing it out, but the good news is that Python 2.6.4 fixes the known regressions in 2.6.3. We've had a lengthy release candidate cycle this time, and are confident that 2.6.4 is a solid release. We highly recommend you upgrade to Python 2.6.4. Please see the NEWS file for all the gory details. http://www.python.org/download/releases/2.6.4/NEWS.txt Source tarballs and the Windows installers can be downloaded from the Python 2.6.4 page. The Mac OS X disk image will be uploaded soon. http://www.python.org/download/releases/2.6.4/ For more information on Python 2.6 in general, please see http://docs.python.org/whatsnew/2.6.html Please report bugs for any Python version in the Python tracker. http://bugs.python.org Enjoy, -Barry Barry Warsaw barry(a)python.org Python 2.6 Release Manager (on behalf of the entire python-dev team)

2 1

One obvious way to do interning [Was: Retrieve an arbitrary element from a set without removing it]
by Alexander Belopolsky 26 Oct '09

26 Oct '09

Changing the subject to reflect branched discussion and forwarding to python-ideas where it probably belongs. On Mon, Oct 26, 2009 at 12:02 PM, Terry Reedy <tjreedy(a)udel.edu> wrote: > Alexander Belopolsky wrote: > >> Here is an alternative idea on how storing interned objects in a set >> can be supported. Currently set.add method returns None and has no >> effect when set already has an object equal to the one being added. I >> propose to consider changing that behavior to make set.add return the >> added object or the set member that is equal to the object being >> added. It is unlikely that many programs rely on the return value >> being None (with doctests being a probable exception), so adding this >> feature is unlikely to cause much grief. > > I had exactly the same idea, but did not post because it violates the > general rule that mutators return None. Is there such a rule? What about set/dict pop? > On the other hand, the returned > value here would not be the mutated collection, so no chaining is possible. I assume you refer to chaining as in s.add(1).add(2) which would be enabled if s.add(..) returned s. My proposal would enable a different type of "chaining" which I would find useful, but ready to hear objections: v = compute_value() s.add(v) # use v can, with my proposal, be rewritten as v = s.add(compute_value()) with an added benefit that v that is used is the "canonical" value. > And 'add' is clearly intended to change something. > Is this an argument for or against the proposal? > On the other hand, frozensets do not have an add method. However, PySet_Add "works with frozenset instances (like PyTuple_SetItem() it can be used to fill-in the values of brand new frozensets before they are exposed to other code). " <http://docs.python.org/3.1/c-api/set.html#PySet_Add> I will experiment with changing PySet_Add to see how much it would break and whether it will be helpful in implementing set-based interning of python strings.

3 2

Clean up Python/thread_*.h ?
by Antoine Pitrou 26 Oct '09

26 Oct '09

Hello, I am wondering which of the files in Python/thread_*.h are really necessary today. Looking at these files, I think most of them could perhaps be removed in py3k. I've identified three categories of potentially removable files: * The unused file: thread_wince.h Windows CE has actually been using thread_nt.h since January 2009 (see http://bugs.python.org/issue4893 ). thread_wince.h is still there, but it's unused and grep brings no instance of it being mentioned anywhere in the source tree. * The (unsupported, untested?) files: thread_atheos.h, thread_cthread.h, thread_lwp.h, thread_os2.h, thread_pth.h, thread_sgi.h. These files refer to architectures which we probably have stopped caring about. It is not even sure whether these files compile and work ok. Most of them have not seen maintenance for years, except for the OS/2 file. In any case, it is obvious they haven't received the level of attention and support that the pthreads and NT versions have. (also, according to http://www.lesstif.org/~amai/os2/html/porting.html, there is a pthreads interface available for OS/2) * The questionable files: thread_pth.h, thread_solaris.h. GNU pth is probably obsolete on all platforms and superseced by the pthreads API. According to http://www.gnu.org/software/pth/, the last pth version was released in 2006 (and the changelog is rather unimpressive). As for Solaris, according to the Sun website it should provide the pthreads API. It was already documented in a 1999 manpage for SunOS 5.9: http://docs.sun.com/app/docs/doc/816-0216/6m6ngupon?a=view . Making a decision and removing the files considered unnecessary would clarify what platforms still are/should be supported. Having less supported platforms would also make improvements easier. Right now making an addition to the core threading primitives is almost impossible since it would require to provide all those platform-specific versions which almost nobody is competent for. Regards Antoine.

5 5

Buildbot alternate renderings
by "Martin v. Löwis" 25 Oct '09

25 Oct '09

Triggered by the recent discussion, I looked at the changes in status display that the buildbot people have implemented recently. I added a few links into alternate renderings; see http://www.python.org/dev/buildbot/ Regards, Martin

1 0

updated PEP 389: argparse
by Steven Bethard 24 Oct '09

24 Oct '09

Sorry for the delay, but I've finally updated PEP 389, the argparse PEP, based on all the feedback from python-dev. The full PEP is below, but in short, the important changes are: * The getopt module will *not* be deprecated. * In Python 2, the -3 flag will cause deprecation warnings to be issued for optparse. * No casually visible deprecation warnings for optparse are expected until Jun 2013. * Support for {}-format strings will be added when one of the converters is approved. This can be done at any point though, so it shouldn't hold up inclusion of argparse. Steve ---------------------------------------------------------------------- PEP: 389 Title: argparse - new command line parsing module Version: $Revision: 75674 $ Last-Modified: $Date: 2009-10-24 12:01:49 -0700 (Sat, 24 Oct 2009) $ Author: Steven Bethard <steven.bethard(a)gmail.com> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 25-Sep-2009 Python-Version: 2.7 and 3.2 Post-History: 27-Sep-2009, 24-Oct-2009 Abstract ======== This PEP proposes inclusion of the argparse [1]_ module in the Python standard library in Python 2.7 and 3.2. Motivation ========== The argparse module is a command line parsing library which provides more functionality than the existing command line parsing modules in the standard library, getopt [2]_ and optparse [3]_. It includes support for positional arguments (not just options), subcommands, required options, options syntaxes like "/f" and "+rgb", zero-or-more and one-or-more style arguments, and many other features the other two lack. The argparse module is also already a popular third-party replacement for these modules. It is used in projects like IPython (the Scipy Python shell) [4]_, is included in Debian testing and unstable [5]_, and since 2007 has had various requests for its inclusion in the standard library [6]_ [7]_ [8]_. This popularity suggests it may be a valuable addition to the Python libraries. Why aren't getopt and optparse enough? ====================================== One argument against adding argparse is that thare are "already two different option parsing modules in the standard library" [9]_. The following is a list of features provided by argparse but not present in getopt or optparse: * While it is true there are two *option* parsing libraries, there are no full command line parsing libraries -- both getopt and optparse support only options and have no support for positional arguments. The argparse module handles both, and as a result, is able to generate better help messages, avoiding redundancies like the ``usage=`` string usually required by optparse. * The argparse module values practicality over purity. Thus, argparse allows required options and customization of which characters are used to identify options, while optparse explicitly states "the phrase 'required option' is self-contradictory" and that the option syntaxes ``-pf``, ``-file``, ``+f``, ``+rgb``, ``/f`` and ``/file`` "are not supported by optparse, and they never will be". * The argparse module allows options to accept a variable number of arguments using ``nargs='?'``, ``nargs='*'`` or ``nargs='+'``. The optparse module provides an untested recipe for some part of this functionality [10]_ but admits that "things get hairy when you want an option to take a variable number of arguments." * The argparse module supports subcommands, where a main command line parser dispatches to other command line parsers depending on the command line arguments. This is a common pattern in command line interfaces, e.g. ``svn co`` and ``svn up``. Why isn't the functionality just being added to optparse? ========================================================= Clearly all the above features offer improvements over what is available through optparse. A reasonable question then is why these features are not simply provided as patches to optparse, instead of introducing an entirely new module. In fact, the original development of argparse intended to do just that, but because of various fairly constraining design decisions of optparse, this wasn't really possible. Some of the problems included: * The optparse module exposes the internals of its parsing algorithm. In particular, ``parser.largs`` and ``parser.rargs`` are guaranteed to be available to callbacks [11]_. This makes it extremely difficult to improve the parsing algorithm as was necessary in argparse for proper handling of positional arguments and variable length arguments. For example, ``nargs='+'`` in argparse is matched using regular expressions and thus has no notion of things like ``parser.largs``. * The optparse extension APIs are extremely complex. For example, just to use a simple custom string-to-object conversion function, you have to subclass ``Option``, hack class attributes, and then specify your custom option type to the parser, like this:: class MyOption(Option): TYPES = Option.TYPES + ("mytype",) TYPE_CHECKER = copy(Option.TYPE_CHECKER) TYPE_CHECKER["mytype"] = check_mytype parser = optparse.OptionParser(option_class=MyOption) parser.add_option("-m", type="mytype") For comparison, argparse simply allows conversion functions to be used as ``type=`` arguments directly, e.g.:: parser = argparse.ArgumentParser() parser.add_option("-m", type=check_mytype) But given the baroque customization APIs of optparse, it is unclear how such a feature should interact with those APIs, and it is quite possible that introducing the simple argparse API would break existing custom Option code. * Both optparse and argparse parse command line arguments and assign them as attributes to an object returned by ``parse_args``. However, the optparse module guarantees that the ``take_action`` method of custom actions will always be passed a ``values`` object which provides an ``ensure_value`` method [12]_, while the argparse module allows attributes to be assigned to any object, e.g.:: foo_object = ... parser.parse_args(namespace=foo_object) foo_object.some_attribute_parsed_from_command_line Modifying optparse to allow any object to be passed in would be difficult because simply passing the ``foo_object`` around instead of a ``Values`` instance will break existing custom actions that depend on the ``ensure_value`` method. Because of issues like these, which made it unreasonably difficult for argparse to stay compatible with the optparse APIs, argparse was developed as an independent module. Given these issues, merging all the argparse features into optparse with no backwards incompatibilities seems unlikely. Deprecation of optparse ======================= Because all of optparse's features are available in argparse, the optparse module will be deprecated. The following very conservative deprecation strategy will be followed: * Python 2.7+ and 3.2+ -- The following note will be added to the optparse documentation: The optparse module is deprecated, and has been replaced by the argparse module. * Python 2.7+ -- If the Python 3 compatibility flag, ``-3``, is provided at the command line, then importing optparse will issue a DeprecationWarning. Otherwise no warnings will be issued. * Python 3.2 (estimated Jun 2010) -- Importing optparse will issue a PendingDeprecationWarning, which is not displayed by default. * Python 3.3 (estimated Jan 2012) -- Importing optparse will issue a PendingDeprecationWarning, which is not displayed by default. * Python 3.4 (estimated Jun 2013) -- Importing optparse will issue a DeprecationWarning, which *is* displayed by default. * Python 3.5 (estimated Jan 2015) -- The optparse module will be removed. Though this is slower than the usual deprecation process, with two releases of PendingDeprecationWarnings instead of the usual one, it seems prudent to avoid producing any casually visible Deprecation warnings until Python 3.X has had some additional time to attract developers. Updates to getopt documentation =============================== The getopt module will not be deprecated. However, its documentation will be updated to point to argparse in a couple of places. At the top of the module, the following note will be added: The getopt module is a parser for command line options whose API is designed to be familiar to users of the C getopt function. Users who are unfamiliar with the C getopt function or who would like to write less code and get better help and error messages should consider using the argparse module instead. Additionally, after the final getopt example, the following note will be added: Note that an equivalent command line interface could be produced with less code by using the argparse module:: import argparse if __name__ == '__main__': parser = argparse.ArgumentParser() parser.add_argument('-o', '--output') parser.add_argument('-v', dest='verbose', action='store_true') args = parser.parse_args() # ... do something with args.output ... # ... do something with args.verbose .. Deferred: string formatting =========================== The argparse module supports Python from 2.3 up through 3.2 and as a result relies on traditional ``%(foo)s`` style string formatting. It has been suggested that it might be better to use the new style ``{foo}`` string formatting [13]_. There was some discussion about how best to do this for modules in the standard library [14]_ and several people are developing functions for automatically converting %-formatting to {}-formatting [15]_ [16]_. When one of these is added to the standard library, argparse will use them to support both formatting styles. Rejected: getopt compatibility methods ====================================== Previously, when this PEP was suggesting the deprecation of getopt as well as optparse, there was some talk of adding a method like:: ArgumentParser.add_getopt_arguments(options[, long_options]) However, this method will not be added for a number of reasons: * The getopt module is not being deprecated, so there is less need. * This method would not actually ease the transition for any getopt users who were already maintaining usage messages, because the API above gives no way of adding help messages to the arguments. * Some users of getopt consider it very important that only a single function call is necessary. The API above does not satisfy this requirement because both ``ArgumentParser()`` and ``parse_args()`` must also be called. Out of Scope: Various Feature Requests ====================================== Several feature requests for argparse were made in the discussion of this PEP: * Support argument defaults from environment variables * Support argument defaults from configuration files * Support "foo --help subcommand" in addition to the currently supported "foo subcommand --help" These are all reasonable feature requests for the argparse module, but are out of the scope of this PEP, and have been redirected to the argparse issue tracker. Discussion: sys.err and sys.exit ================================ There were some concerns that argparse by default always writes to ``sys.err`` and always calls ``sys.exit`` when invalid arguments are provided. This is the desired behavior for the vast majority of argparse use cases which revolve around simple command line interfaces. However, in some cases, it may be desirable to keep argparse from exiting, or to have it write its messages to something other than ``sys.err``. These use cases can be supported by subclassing ``ArgumentParser`` and overriding the ``exit`` or ``_print_message`` methods. The latter is an undocumented implementation detail, but could be officially exposed if this turns out to be a common need. References ========== .. [1] argparse (http://code.google.com/p/argparse/) .. [2] getopt (http://docs.python.org/library/getopt.html) .. [3] optparse (http://docs.python.org/library/optparse.html) .. [4] argparse in IPython (http://mail.scipy.org/pipermail/ipython-dev/2009-April/005102.html) .. [5] argparse in Debian (http://packages.debian.org/search?keywords=python-argparse) .. [6] 2007-01-03 request for argparse in the standard library (http://mail.python.org/pipermail/python-list/2007-January/592646.html) .. [7] 2009-06-09 request for argparse in the standard library (http://bugs.python.org/issue6247) .. [8] 2009-09-10 request for argparse in the standard library (http://mail.python.org/pipermail/stdlib-sig/2009-September/000342.html) .. [9] Fredrik Lundh response to [6]_ (http://mail.python.org/pipermail/python-list/2007-January/592675.html) .. [10] optparse variable args (http://docs.python.org/library/optparse.html#callback-example-6-variable-ar…) .. [11] parser.largs and parser.rargs (http://docs.python.org/library/optparse.html#how-callbacks-are-called) .. [12] take_action values argument (http://docs.python.org/library/optparse.html#adding-new-actions) .. [13] use {}-formatting instead of %-formatting (http://bugs.python.org/msg89279) .. [14] transitioning from % to {} formatting (http://mail.python.org/pipermail/python-dev/2009-September/092326.html) .. [15] Vinay Sajip's %-to-{} converter (http://gist.github.com/200936) .. [16] Benjamin Peterson's %-to-{} converter (http://bazaar.launchpad.net/~gutworth/+junk/mod2format/files) Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End:

3 3

First shot at some_set.get()
by Willi Richert 24 Oct '09

24 Oct '09

Hi, here is the first shot to provide a faster means of retrieving an arbitrary element from a set without removing it. The times for ========================= from timeit import * stat1 = "for i in xrange(100): iter(s).next()" stat2 = "for i in xrange(100): s.get()" for stat in [stat1, stat2]: t = Timer(stat, setup="s=set(range(10000))") # outside the try/except try: print t.timeit(number=1000) except: t.print_exc() ========================= are stat1: 0.0451760292053 stat2: 0.0148510932922 The patch is attached. Feel free to criticize. I would love to see something like that in the standard Python interpreter. Regards, wr

9 10