Mailman 3 February 2012 - Python-Dev

State of PEP-3118 (memoryview part)
by Stefan Krah 01 Mar '12

01 Mar '12

State of PEP-3118 (memoryview part) Hello, In Python 3.3 most issues with the memoryview object have been fixed in a recent commit (3f9b3b6f7ff0). Many features have been added, see: http://docs.python.org/dev/whatsnew/3.3.html The underlying problems with memoryview were intricate and required a long discussion (issue #10181) that led to a complete rewrite of memoryobject.c. We have several options with regard to 2.7 and 3.2: 1) Won't fix. 2) Backport the changes and disable as much of the new functionality as possible. 3) Backport all of it (this would be the least amount of work and could be done relatively quickly). 4) Nick suggested another option: put a module with the new functionality on PyPI. This would be quite a bit of work, and personally I don't have time for that. Options 2) and 3) would ideally entail one backwards incompatible bugfix: In 2.7 and 3.2 assignment to a memoryview with format 'B' rejects integers but accepts byte objects, but according to the struct syntax mandated by the PEP it should be the other way round. It would be nice to get some opinions and ideas, especially of course from the release managers. Stefan Krah

7 12

Backporting PEP 414
by Ethan Furman 01 Mar '12

01 Mar '12

Here's what I know: We don't add features to bug-fix releases. u'' is considered a feature. By not backporting to 3.1 and 3.2 we are not easing the migration pains from 2.x. Here's what I don't know: Why is readding u'' a feature and not a bug? (Just had a thought about this -- because the removal of u'' is documented.) To take a different example: callable() had been removed from 3.0, and was added back in 3.2. callable() is not a big deal as you can roll your own quite easily -- and that is the huge difference: a user *cannot* add u'' back to 3.0/3.1 (at least, not without modifying and rebuilding the Python interpreter source). If there is already a FAQ entry feel free to point me to it, but I would still be curious why, in this instance, practicality does not beat purity? My apologies if this type of question has been rehashed before. ~Ethan~

15 15

PEP 411: Provisional packages in the Python standard library
by Eli Bendersky 29 Feb '12

29 Feb '12

Hi all, Following the intensive and fruitful discussion of the (now rejected) PEP 408 (http://mail.python.org/pipermail/python-dev/2012-January/115850.html), we've drafted PEP 411 to summarize the conclusions with regards to the process of marking packages provisional. Note that this is an informational PEP, and that for the sake of completeness it duplicates some of the contents of PEP 408. It is pasted below, as well as online at http://www.python.org/dev/peps/pep-0411/. Comments are welcome. Eli ------------------------------------------------ PEP: 411 Title: Provisional packages in the Python standard library Version: $Revision$ Last-Modified: $Date$ Author: Nick Coghlan <ncoghlan(a)gmail.com>, Eli Bendersky <eliben(a)gmail.com> Status: Draft Type: Informational Content-Type: text/x-rst Created: 2012-02-10 Python-Version: 3.3 Post-History: 2012-02-10 Abstract ======== The process of including a new package into the Python standard library is hindered by the API lock-in and promise of backward compatibility implied by a package being formally part of Python. This PEP describes a methodology for marking a standard library package "provisional" for the period of a single minor release. A provisional package may have its API modified prior to "graduating" into a "stable" state. On one hand, this state provides the package with the benefits of being formally part of the Python distribution. On the other hand, the core development team explicitly states that no promises are made with regards to the the stability of the package's API, which may change for the next release. While it is considered an unlikely outcome, such packages may even be removed from the standard library without a deprecation period if the concerns regarding their API or maintenante prove well-founded. Proposal - a documented provisional state ========================================= Whenever the Python core development team decides that a new package should be included into the standard library, but isn't entirely sure about whether the package's API is optimal, the package can be included and marked as "provisional". In the next minor release, the package may either be "graduated" into a normal "stable" state in the standard library, or be rejected and removed entirely from the Python source tree. If the package ends up graduating into the stable state after being provisional for a minor release, its API may be changed according to accumulated feedback. The core development team explicitly makes no guarantees about API stability and backward compatibility of provisional packages. Marking a package provisional ----------------------------- A package will be marked provisional by including the following paragraph as a note at the top of its documentation page: The <X> package has been included in the standard library on a provisional basis. While major changes are not anticipated, as long as this notice remains in place, backwards incompatible changes are permitted if deemed necessary by the standard library developers. Such changes will not be made gratuitously - they will occur only if serious API flaws are uncovered that were missed prior to inclusion of the package. Moving a package from the provisional to the stable state simply implies removing this note from its documentation page. Which packages should go through the provisional state ------------------------------------------------------ We expect most packages proposed for addition into the Python standard library to go through a minor release in the provisional state. There may, however, be some exceptions, such as packages that use a pre-defined API (for example ``lzma``, which generally follows the API of the existing ``bz2`` package), or packages with an API that has wide acceptance in the Python development community. In any case, packages that are proposed to be added to the standard library, whether via the provisional state or directly, must fulfill the acceptance conditions set by PEP 2. Criteria for "graduation" ------------------------- In principle, most provisional packages should eventually graduate to the stable standard library. Some reasons for not graduating are: * The package may prove to be unstable or fragile, without sufficient developer support to maintain it. * A much better alternative package may be found during the preview release. Essentially, the decision will be made by the core developers on a per-case basis. The point to emphasize here is that a packages's inclusion in the standard library as "provisional" in some release does not guarantee it will continue being part of Python in the next release. Rationale ========= Benefits for the core development team -------------------------------------- Currently, the core developers are really reluctant to add new interfaces to the standard library. This is because as soon as they're published in a release, API design mistakes get locked in due to backward compatibility concerns. By gating all major API additions through some kind of a provisional mechanism for a full release, we get one full release cycle of community feedback before we lock in the APIs with our standard backward compatibility guarantee. We can also start integrating provisional packages with the rest of the standard library early, so long as we make it clear to packagers that the provisional packages should not be considered optional. The only difference between provisional APIs and the rest of the standard library is that provisional APIs are explicitly exempted from the usual backward compatibility guarantees. Benefits for end users ---------------------- For future end users, the broadest benefit lies in a better "out-of-the-box" experience - rather than being told "oh, the standard library tools for task X are horrible, download this 3rd party library instead", those superior tools are more likely to be just be an import away. For environments where developers are required to conduct due diligence on their upstream dependencies (severely harming the cost-effectiveness of, or even ruling out entirely, much of the material on PyPI), the key benefit lies in ensuring that all packages in the provisional state are clearly under python-dev's aegis from at least the following perspectives: * Licensing: Redistributed by the PSF under a Contributor Licensing Agreement. * Documentation: The documentation of the package is published and organized via the standard Python documentation tools (i.e. ReST source, output generated with Sphinx and published on http://docs.python.org). * Testing: The package test suites are run on the python.org buildbot fleet and results published via http://www.python.org/dev/buildbot. * Issue management: Bugs and feature requests are handled on http://bugs.python.org * Source control: The master repository for the software is published on http://hg.python.org. Candidates for provisional inclusion into the standard library ============================================================== For Python 3.3, there are a number of clear current candidates: * ``regex`` (http://pypi.python.org/pypi/regex) - approved by Guido [#]_. * ``daemon`` (PEP 3143) * ``ipaddr`` (PEP 3144) Other possible future use cases include: * Improved HTTP modules (e.g. ``requests``) * HTML 5 parsing support (e.g. ``html5lib``) * Improved URL/URI/IRI parsing * A standard image API (PEP 368) * Encapsulation of the import state (PEP 368) * Standard event loop API (PEP 3153) * A binary version of WSGI for Python 3 (e.g. PEP 444) * Generic function support (e.g. ``simplegeneric``) Rejected alternatives and variations ==================================== See PEP 408. References ========== .. [#] http://mail.python.org/pipermail/python-dev/2012-January/115962.html Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End:

11 19

PEP 415: Implementing PEP 409 differently
by Benjamin Peterson 28 Feb '12

28 Feb '12

PEP: 415 Title: Implementing PEP 409 differently Version: $Revision$ Last-Modified: $Date$ Author: Benjamin Peterson <benjamin(a)python.org> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 26-Feb-2012 Post-History: 26-Feb-2012 Abstract ======== PEP 409 allows PEP 3134 exception contexts and causes to be suppressed when the exception is printed. This is done using the ``raise exc from None`` syntax. This PEP proposes to implement context and cause suppression differently. Rationale ========= PEP 409 changes ``__cause__`` to be ``Ellipsis`` by default. Then if ``__cause__`` is set to ``None`` by ``raise exc from None``, no context or cause will be printed should the exception be uncaught. The main problem with this scheme is it complicates the role of ``__cause__``. ``__cause__`` should indicate the cause of the exception not whether ``__context__`` should be printed or not. This use of ``__cause__`` is also not easily extended in the future. For example, we may someday want to allow the programmer to select which of ``__context__`` and ``__cause__`` will be printed. The PEP 409 implementation is not amendable to this. The use of ``Ellipsis`` is a hack. Before PEP 409, ``Ellipsis`` was used exclusively in extended slicing. Extended slicing has nothing to do with exceptions, so it's not clear to someone inspecting an exception object why ``__cause__`` should be set to ``Ellipsis``. Using ``Ellipsis`` by default for ``__cause__`` makes it asymmetrical with ``__context__``. Proposal ======== A new attribute on ``BaseException``, ``__suppress_context__``, will be introduced. The ``raise exc from None`` syntax will cause ``exc.__suppress_context__`` to be set to ``True``. Exception printing code will check for the attribute to determine whether context and cause will be printed. ``__cause__`` will return to its original purpose and values. There is precedence for ``__suppress_context__`` with the ``print_line_and_file`` exception attribute. Patches ======= There is a patch on `Issue 14133`_. References ========== .. _issue 14133: http://bugs.python.org/issue6210 Copyright ========= This document has been placed in the public domain.

4 6

Porting Stories (was PEP 414 - Unicode Literals for Python 3)
by Brian Curtin 28 Feb '12

28 Feb '12

On Tue, Feb 28, 2012 at 09:04, Barry Warsaw <barry(a)python.org> wrote: > We really need to stop saying that porting to Python 3 is hard, or should be > delayed. It's not in the vast majority of cases. Yes, there are warts, and > we should continue to improve Python 3 so it gets easier, but by no means is > it impossible for most code to be working very nicely on Python 3 today. I've been singing and dancing about the ease of porting for a while now, but it's mostly thanks to the fact that I never had to do any Unicode tomfoolery. Now with this PEP, the game gets easier for a lot more people. Does anyone have a good porting experience they'd like to share, which I could maybe use as a PR effort for us? Barry, I know you wrote some pretty solid coverage of your DBus port. Anyone else? Personal projects or work stuff (assuming its ok to share). blog.python.org has been asleep for a while and a good porting testimonial might be a way to jumpstart it, or I can get it on the python.org front page. If you have anything to share on that front, please contact me directly.

1 0

Marking packaging-related PEPs as Finished after fixing some bugs in them
by Éric Araujo 28 Feb '12

28 Feb '12

Hello, The three packaging-related PEPs that were written by the distutils SIG and approved two years ago are still marked as Accepted, not Finished: SA 345 Metadata for Python Software Packages 1.2 Jones SA 376 Database of Installed Python Distributions Ziadé SA 386 Changing the version comparison module in Distutils Ziadé They’re all implemented in packaging/distutils2. Sadly, all of them have rather serious issues, so I wanted to ask what the process should be to solve the problems and mark the PEPs final. In PEP 345, legal values for project names are not defined, so for example “Foo (>=0.5)” could legally be a project named Foo without version predicate, or a project named “Foo (>=0.5)”. I don’t have a solution to propose, as I did not run a poll or check existing projects. Second, the Provides-Dist allows values like “Name (<= version)”, which does not make sense: it should allow only unversioned names and names with one version (without operators). Finally, some of the new metadata fields are also misnamed, namely the ones ending in -Dist like Requires-Dist, whose value is a release (i.e. a name + optional version specifier), not a distribution (i.e. a specific archive or installer for a release), and therefore should be Requires-Release or something prettier. (Remember that it cannot be just Requires, which is taken by PEP 314, contains module names instead of project names, and is not used by anyone TTBOMK.) packaging.database, which implements PEP 376, has a few known bugs; I don’t know if that should prevent the PEP from being marked Finished. It could be that finishing the implementation shows issues in the PEP, like for the other two. In PEP 386, the rule that versions using an 'rc' marker should sort after 'c' is buggy: I don’t think anyone will disagree that 1.0rc1 == 1.0c1 and 1.0rc1 < 1.0c2. The 'rc' marker was added by Tarek shortly before the PEP was accepted (see http://mail.python.org/pipermail/python-dev/2010-January/097041.html), and was not discussed. My preferred solution would be to accept 'rc' when parsing but then always use 'c' internally and in all output (file names, METADATA file, etc.). If it is judged important that projects be able to use 'rc' and see it in output, then I’ll have to add special cases in __eq__ and __hash__ methods, which is feasible if inelegant. (Issues found by Josip Djolonga, Alexis Métaireau and I.) Cheers

6 5

Re: [Python-Dev] cpython: Close issue #6210: Implement PEP 409
by Antoine Pitrou 27 Feb '12

27 Feb '12

On Sun, 26 Feb 2012 09:02:59 +0100 nick.coghlan <python-checkins(a)python.org> wrote: > + > +No debugging capability is lost, as the original exception context remains > +available if needed (for example, if an intervening library has incorrectly > +suppressed valuable underlying details):: That's debatable, since you now have to *add* code if you want to print the original exception as well. The new capability improves the terseness and clarity of error messages at the expense of debuggability. > + def prepare_subprocess(): > + # don't create core file > + try: > + setrlimit(RLIMIT_CORE, (0, 0)) > + except (ValueError, resource_error): > + pass Really? This sounds quite wrong, but it should *at least* explain why a test of the "raise" statement would produce a core file! (but I think you should consider removing this part) > + def get_output(self, code, filename=None): > + """ > + Run the specified code in Python (in a new child process) > and read the > + output from the standard error or from a file (if filename > is set). > + Return the output lines as a list. > + """ We already have assert_python_ok and friends. It's not obvious what this additional function achieves. Also, the "filename" argument is never used. > + output = re.sub('Current thread 0x[0-9a-f]+', > + 'Current thread XXX', > + output) This looks like output from the faulthandler module. Why would faulthandler kick in here? Regards Antoine.

3 2

Re: [Python-Dev] [Python-checkins] cpython (3.2): Updated logging cookbook with info on alternative format styles.
by Nick Coghlan 27 Feb '12

27 Feb '12

On Mon, Feb 27, 2012 at 9:04 PM, vinay.sajip <python-checkins(a)python.org> wrote: > +There is, however, a way that you can use {}- and $- formatting to construct > +your individual log messages. Recall that for a message you can use an > +arbitrary object as a message format string, and that the logging package will > +call ``str()`` on that object to get the actual format string. Consider the > +following two classes:: > + > + class BraceMessage(object): > + def __init__(self, fmt, *args, **kwargs): > + self.fmt = fmt > + self.args = args > + self.kwargs = kwargs > + > + def __str__(self): > + return self.fmt.format(*self.args, **self.kwargs) > + > + class DollarMessage(object): > + def __init__(self, fmt, **kwargs): > + self.fmt = fmt > + self.kwargs = kwargs > + > + def __str__(self): > + from string import Template > + return Template(self.fmt).substitute(**self.kwargs) > + > +Either of these can be used in place of a format string, to allow {}- or > +$-formatting to be used to build the actual "message" part which appears in the > +formatted log output in place of "%(message)s" or "{message}" or "$message". > +It's a little unwieldy to use the class names whenever you want to log > +something, but it's quite palatable if you use an alias such as __ (double > +underscore – not to be confused with _, the single underscore used as a > +synonym/alias for :func:`gettext.gettext` or its brethren). This is the part I was thinking might be simplified by allowing a "style" parameter to be passed to getLogger(). Consider StrFormatLogger and StringTemplateLogger classes that were just wrappers around an ordinary Logger instance, and made the relevant conversions to StrFormatMessage or StringTemplateMessage on the caller's behalf. Then (assuming the current getLogger() is available as _getLogger()), you could just do something like: _LOGGER_STYLES = { "%": lambda x: x, "{": StrFormatLogger, "$": StringTemplateLogger, } def getLogger(name, style='%'): if style not in _STYLES: raise ValueError('Style must be one of: %s' % ','.join(_LOGGER_STYLES.keys())) return _LOGGER_STYLES[style](_getLogger()) Since each module should generally be doing its own getLogger() call (or else should be documenting that it accepts an ordinary logger instance as a parameter), it seems like this would allow fairly clean use of the alternate styles without complicating each individual logging operation. (The xyzStyle approach used by formatters here won't work, since we want different modules to be able to use different formatting styles. However, ordinary inheritance should allow StrFormatLogger and StringTemplateLogger to share most of their implementation) Cheers, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

1 0

New-style formatting in the logging module (was Re: cpython (3.2): Issue #14123: Explicitly mention that old style % string formatting has caveats)
by Nick Coghlan 27 Feb '12

27 Feb '12

On Mon, Feb 27, 2012 at 10:44 AM, Larry Hastings <larry(a)hastings.org> wrote: > On 02/26/2012 03:33 PM, Guido van Rossum wrote: >> >> One thing I'd like to see happening regardless is support for >> new-style formatting in the logging module. It's a little tricky to >> think how that would work, alas -- should this be a property of the >> logger or of the call? > > > There already is some support. logging.Formatter objects can be initialized > with a "style" parameter, making this a property of the logger. (New in > 3.2.) > > http://docs.python.org/py3k/library/logging.html#formatter-objects > > Is that what you had in mind? It's half the puzzle (since composing the event fields into the actual log output is a logger action, you know the style when you supply the format string). The other half is that logging's lazy formatting currently only supporting printf-style format strings - to use brace formatting you currently have to preformat the messages, so you incur the formatting cost even if the message gets filtered out by the logging configuration. For that, a logger setting doesn't work, since one logger may be shared amongst multiple modules, some of which may use printf formatting, others brace formatting. It could possibly be a flag to getLogger() though - provide a facade on the existing logger type that sets an additional event property to specify the lazy formatting style. Cheers, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

2 1

Re: [Python-Dev] cpython (3.2): Issue #14123: Explicitly mention that old style % string formatting has caveats
by Antoine Pitrou 27 Feb '12

27 Feb '12

On Sun, 26 Feb 2012 19:50:10 +0100 martin(a)v.loewis.de wrote: > >> - The formatting operations described here are obsolete and may go away > >> in future > >> - versions of Python. Use the new :ref:`string-formatting` in new code. > >> + The formatting operations described here are modelled on C's printf() > >> + syntax. They only support formatting of certain builtin types. The > >> + use of a binary operator means that care may be needed in order to > >> + format tuples and dictionaries correctly. As the new > >> + :ref:`string-formatting` syntax is more flexible and handles tuples and > >> + dictionaries naturally, it is recommended for new code. However, there > >> + are no current plans to deprecate printf-style formatting. > >> > > > > Please consider just deleting the last sentence. Documentation is meant for > > users (often new users) and not core devs. As such, I just don't see what > > it adds. If the aim to to document this intent somewhere, a PEP would be a > > better place than the formal documentation. > > I'd rather leave the last sentence, and delete the penultimate sentence. > The last sentence is useful information to the end user ("we will not > deprecate printf-style formatting, so there is no need to change existing > code"). I'd drop the penultimate sentence because there is no consensus > that it is a useful recommendation (and it is certainly not a statement > of fact). It would be nice to call it something else than "printf-style formatting". While it is certainly modelled on printf(), knowledge of C or printf is not required to understand %-style formatting, nor even to appreciate it. Regards Antoine.

10 11