Mailman 3 February 2012 - Python-Dev

Re: [Python-Dev] [Python-checkins] cpython: Issue #14080: fix sporadic test_imp failure. Patch by Stefan Krah.
by Brett Cannon 26 Feb '12

26 Feb '12

On Sun, Feb 26, 2012 at 12:13, antoine.pitrou <python-checkins(a)python.org>wrote: > http://hg.python.org/cpython/rev/1d7472b015f0 > changeset: 75296:1d7472b015f0 > user: Antoine Pitrou <solipsis(a)pitrou.net> > date: Sun Feb 26 18:09:50 2012 +0100 > summary: > Issue #14080: fix sporadic test_imp failure. Patch by Stefan Krah. > > files: > Lib/test/test_imp.py | 1 + > 1 files changed, 1 insertions(+), 0 deletions(-) > > > diff --git a/Lib/test/test_imp.py b/Lib/test/test_imp.py > --- a/Lib/test/test_imp.py > +++ b/Lib/test/test_imp.py > @@ -325,6 +325,7 @@ > self.addCleanup(cleanup) > # Touch the __init__.py file. > support.create_empty_file('pep3147/__init__.py') > + importlib.invalidate_caches() > expected___file__ = os.sep.join(('.', 'pep3147', '__init__.py')) > m = __import__('pep3147') > self.assertEqual(m.__file__, expected___file__, (m.__file__, > m.__path__)) Should that just go into support.create_empty_file()? Since it's just a performance issue I don't see it causing unexpected test failures and it might help with any future issues.

2 1

Re: [Python-Dev] [Python-checkins] cpython (3.2): Issue #14123: Explicitly mention that old style % string formatting has caveats
by Terry Reedy 26 Feb '12

26 Feb '12

On 2/26/2012 1:50 PM, martin(a)v.loewis.de wrote: > > Zitat von Eli Bendersky <eliben(a)gmail.com>: > >>> >>> - The formatting operations described here are obsolete and may go away >>> in future >>> - versions of Python. Use the new :ref:`string-formatting` in new code. >>> + The formatting operations described here are modelled on C's printf() >>> + syntax. They only support formatting of certain builtin types. The >>> + use of a binary operator means that care may be needed in order to >>> + format tuples and dictionaries correctly. As the new >>> + :ref:`string-formatting` syntax is more flexible and handles tuples >>> and >>> + dictionaries naturally, it is recommended for new code. However, there >>> + are no current plans to deprecate printf-style formatting. >>> >> >> Please consider just deleting the last sentence. Documentation is >> meant for >> users (often new users) and not core devs. As such, I just don't see what >> it adds. If the aim to to document this intent somewhere, a PEP would >> be a >> better place than the formal documentation. > > I'd rather leave the last sentence, and delete the penultimate sentence. > The last sentence is useful information to the end user ("we will not > deprecate printf-style formatting, so there is no need to change existing > code"). I'd drop the penultimate sentence because there is no consensus > that it is a useful recommendation (and it is certainly not a statement > of fact). I agree that the 'recommendation' is subjective, even though I strongly agree with it *for new Python programmers who are not already familiar with printf style formatting*. However, that sort of nuanced recommendation goes better in a HowTo. Statements about non-deprecation are also out of place as that is the default. So I agree with both of you. Let us drop both of the last two sentences. Then we can all be happy. There is a difference between 'There are no current plans to ...' and 'We will never ...'. However, '...' should not be discussed or even proposed or even mentioned until there is a bug-free automatic converter. I think the recent rehashing was mostly a needless irritation except as it prompted a doc update. --- Terry Jan Reedy

1 0

Proposing an alternative to PEP 410
by Larry Hastings 26 Feb '12

26 Feb '12

I've been meditating on the whole os.stat mtime representation thing. Here's a possible alternative approach. * Improve datetime.datetime objects so they support nanosecond resolution, in such a way that it's 100% painless to make them even more precise in the future. * Add support to datetime objects that allows adding and subtracting ints and floats as seconds. This behavior is controllable with a flag on the object--by default this behavior is off. * Support accepting naive datetime.datetime objects in all functions that accept a timestamp in os (utime etc). * Change the result of os.stat to be a custom class rather than a PyStructSequence. Support the sequence protocol on the custom class but mark it PendingDeprecation, to be removed completely in 3.5. (I can't take credit for this idea; MvL suggested it to me once while we were talking about this issue. Now that the os.stat object has named fields, who uses the struct unpacking anymore?) * Add support for setting "stat_float_times=2" (or perhaps "stat_float_times=datetime.datetime" ?) to enable returning st_[acm]time as naive datetime.datetime objects--specifically, ones that allow addition and subtraction of ints and floats. The value would be similar to calling datetime.datetime.fromdatetime() on the current float timestamp, but would preserve all available precision. * Add a new parameter to functions that produce stat-like timestamps to explicitly specify the type of the timestamps (float or datetime), as proposed in PEP 410. I realize datetime objects aren't a drop-in replacement for floats (or ints). In particular their str/repr representations are much more ornate. So I'd expect some breakage. Personally I think the adding/subtracting ints change is a tiny bit smelly--but this is a practicality beating purity thing. I propose making it non-default behavior just to minimize the effects of the change. Similarly, I realize os.stat_float_times was always a bit of a hack, what with it being global state and all. However the approach has the virtue of having worked in the past. I disagree with PEP 410's conclusions about the suitability of datetime as a timestamp object. I think "naive" datetime objects are a perfect fit. Specficially addressing PEP 410's concerns: * I don't propose doing anything about the other functions that have no explicit start time; I'm only proposing changing the functions that deal with timestamps. (Perhaps the right thing for epoch-less times like time.clock would be timedelta? But I think we can table this discussion for now.) * "You can't compare naive and non-naive datetimes." So what? The existing timestamp from os.stat is a float, and you can't compare floats and non-naive datetimes. How is this an issue? Perhaps someone else can propose something even better, //arry/

7 14

Status regarding Old vs. Advanced String Formating
by Tshepang Lekhonkhobe 26 Feb '12

26 Feb '12

Hi, I was of the thought that Old String Formatting |"%s" % foo| was to be phased out by Advanced String Formatting |"{}.format(foo)|. I however keep seeing new code committed into the main VCS using the old style. Is this okay? Is there a policy? I ask also because I expect CPython to lead by example. On another note, will the old format ever be deprecated? Is there a date?

17 28

struct.pack inconsistencies between platforms
by pmon mail 26 Feb '12

26 Feb '12

Hi I have found myself in the following troubling situation. I'm running the following code on a Python 2.6.5 on Linux x86: Python 2.6.5 (r265:79063, Apr 16 2010, 13:09:56) [GCC 4.4.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import struct >>> len(struct.pack('L',0)) 4 Works as expected and documented (http://docs.python.org/library/struct.html ). I'm running the same code on a MacPro (OS X 10.7.3) and I'm getting the following: Python 2.7.1 (r271:86832, Jun 16 2011, 16:59:05) [GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import struct >>> len(struct.pack('L',0)) 8 Documentation clearly states that the 'L' is a 4 byte integer. Is this a bug? I'm I missing something? Thanks PMon

3 5

Re: [Python-Dev] [Python-checkins] cpython (3.2): Issue #14123: Explicitly mention that old style % string formatting has caveats
by Eli Bendersky 26 Feb '12

26 Feb '12

> > - The formatting operations described here are obsolete and may go away > in future > - versions of Python. Use the new :ref:`string-formatting` in new code. > + The formatting operations described here are modelled on C's printf() > + syntax. They only support formatting of certain builtin types. The > + use of a binary operator means that care may be needed in order to > + format tuples and dictionaries correctly. As the new > + :ref:`string-formatting` syntax is more flexible and handles tuples and > + dictionaries naturally, it is recommended for new code. However, there > + are no current plans to deprecate printf-style formatting. > Please consider just deleting the last sentence. Documentation is meant for users (often new users) and not core devs. As such, I just don't see what it adds. If the aim to to document this intent somewhere, a PEP would be a better place than the formal documentation. Eli

1 0

folding cElementTree behind ElementTree in 3.3
by Eli Bendersky 26 Feb '12

26 Feb '12

Hello, Here's a note from "What's new in Python 3.0": """A common pattern in Python 2.x is to have one version of a module implemented in pure Python, with an optional accelerated version implemented as a C extension; for example, pickle and cPickle. This places the burden of importing the accelerated version and falling back on the pure Python version on each user of these modules. In Python 3.0, the accelerated versions are considered implementation details of the pure Python versions. Users should always import the standard version, which attempts to import the accelerated version and falls back to the pure Python version. The pickle / cPickle pair received this treatment. The profile module is on the list for 3.1. The StringIO module has been turned into a class in the io module.""" Is there a good reason why xml.etree.ElementTree / xml.etree.cElementTree did not "receive this treatment"? In the case of this module, it's quite unfortunate because: 1. The accelerated module is much faster and memory efficient (see recent benchmarks here: http://bugs.python.org/issue11379), and XML processing is an area where processing matters 2. The accelerated module implements the same API 3. It's very hard to even find out about the existence of the accelerated module. Its sole mention in the docs is this un-emphasized line in http://docs.python.org/dev/py3k/library/xml.etree.elementtree.html: "A C implementation of this API is available as xml.etree.cElementTree." Even to an experienced user who carefully reads the whole documentation it's not easy to notice. For the typical user who just jumps around to functions/methods he's interested in, it's essentially invisible. Eli

20 75

Rejecting PEP 410 (Use decimal.Decimal type for timestamps)
by Guido van Rossum 26 Feb '12

26 Feb '12

After an off-list discussion with Victor I have decided to reject PEP 410. Here are my reasons for rejecting the PEP. (Someone please copy this to the PEP or reference this message in the archives on mail.python.org.) 1. I have a general dislike of APIs that take a flag parameter which modifies the return type. But I also don't like having to add variants that return Decimal for over a dozen API functions (stat(), fstat(), etc.). I really think that this PEP would add a lot of complexity that we don't need. 2. The Decimal type is somewhat awkward to use; it doesn't mix with floats, there's a context that sets things like precision and rounding, it's still a floating point type that may lose precision (something which many people don't get when they first see it). 3. There are *very* few clocks in existance (if any) that actually measure time with more than 56 bits of accuracy. Sure, for short time periods we can measure nanoseconds. But a Python (64-bit) float can represent quite a large number of nanoseconds exactly (many millions of seconds), so if you're using some kind of real-time timer that reset e.g. at the start of the current process you should be fine using a float to represent the time with great precision and accuracy. On the other hand, if you're measuring the time of day expressed in seconds (and fractions) since 1/1/1970, you should consider yourself lucky if your clock is accurate within 1 second. (Especially since POSIX systems aren't allowed to admit the existence of leap seconds in their timestamps -- around a leap second you must adjust your clock, either gradually or abruptly.) I'll give you that some people might have clocks accurate to a microsecond. Such timestamps can be represented exactly as floats (at least until some point in the very distant future, when hopefully we'll have 128-bit floats). 4. I don't expect that timestamps with even greater precision than nanoseconds will ever become fashionable. Given that light travels about 30 cm in a nanosecond, there's not much use for more accurate time measurements in daily life. Given relativity theory, at such a timescale simultaneity of events is iffy at best. 5. I see only one real use case for nanosecond precision: faithful copying of the mtime/atime recorded by filesystems, in cases where the filesystem (like e.g. ext4) records these times with nanosecond precision. Even if such timestamps can't be trusted to be accurate, converting them to floats and back loses precision, and verification using tools not written in Python will flag the difference. But for this specific use case a much simpler set of API changes will suffice; only os.stat() and os.utime() need to change slightly (and variants of os.stat() like os.fstat()). 6. If you worry about systems where float has fewer bits: I don't think there are any relevant systems that have both a smaller float type and nanosecond clocks. So far the rejection note. As to the changes alluded to in #5: Let os.stat() and friends return extra fields st_atime_ns (etc.) that give the timestamps in nanoseconds as a Python (long) integer, such that e.g. (in close approximation) st_atime == st_atime_ns * 1e-9. Let os.utime() take an optional keyword argument ns=(atime_ns, mtime_ns). Details of the actual design of this API, such as the actual field and parameter names, may change; this is just a suggestion. We don't need a PEP for this proposal; we can just open a tracker issue and hash out the details during the code review. I'm also in favor of getting rid of os.stat_float_times(). As proposed in another thread, we may deprecated it in 3.3 and delete it in 3.5. I'm not excited about adding more precision to datetime and timedelta. -- --Guido van Rossum (python.org/~guido)

2 1

C-API functions for reading/writing tstate->exc_* ?
by Stefan Behnel 26 Feb '12

26 Feb '12

Hi, the Cython and PyPy projects are currently working on getting Cython implemented extensions to build and run in PyPy, interfacing at the C-API level for now. One problem we encountered was that there is currently no "abstract" way to access tstate->exc_type and friends, i.e. the last exception that was caught, aka. sys.exc_info(). Apparently, PyPy stores them at a frame level, whereas CPython makes them available in thread local storage as bare struct fields. Even if PyPy put them there on request, any changes to these fields would pass unnoticed. And Cython needs to set them in order to properly implement the semantics of a try-except statement (and in some other places where exception state is scoped). When compiling for PyPy, Cython therefore needs a way to tell PyPy about any changes. For the tstate->curexc_* fields, there are the two functions PyErr_Fetch() and PyErr_Restore(). Could we have two similar "official" functions for the exc_* fields? Maybe PyErr_FetchLast() and PyErr_RestoreLast()? Note that Cython would not have a reason to actually use them in CPython, and it should be uncommon for non-Cython extension modules to care about the exc_* fields at all. So these functions won't be of much use if actually implemented in CPython (although I wouldn't mind doing that). The question is just if we could have two officially named functions that PyPy (and maybe other Pythons) could implement in order to access the last raised exception in a way that does not depend on implementation details. Stefan

4 6

Versioning proposal: syntax.stdlib.bugfix
by Terry Reedy 26 Feb '12

26 Feb '12

We have two similar proposals, PEPs 407 and 413, to speed up the release of at least library changes. To me, both have major problems with version numbering. I think the underlying problem is starting with a long-term fixed leading '3', which conveys no information about current and future changes (at least for another decade). So I propose for consideration that we use the first digit to indicate a version of core python with fixed grammar/syntax and corresponding semantics. I would have this be stable for at least two years. It seems that most current syntax proposals amount to duplication of current function to suite someone's or some people's stylistic preference. My current view is that current syntax in mostly good enough, the implementation thereof is close to bug-free, and we should think carefully about changes. We could then use the second digit to indicate library version. The .0 library version would be for a long-term support version. The library version could change every six months, but I would not necessarily fix it at any particular interval. If we have some important addition or upgrade at four months, release it. If we need another month to include an important change, perhaps wait. The third digit would be for initial (.0) and bugfix releases, as at present. Non .0 bugfix releases would mostly be for x.0 long-term syntax+library versions. x.(y!=0).0 library-change-only releases would only get x.(y!=0).1 bugfix releases on an 'emergency' basis. How this would work: Instead of 3.3.0, release 4.0.0. That would be followed by 4.0.1, 4.0.2, etc, bugfixes, however often we feel like it, until 5.0.0 is released. 4.0.0 would also be followed by 4.1.0 with updated stdlib in about 6 months, then barring mistakes, 4.2.0, etc, again until 5.0.0. A variation of this proposal would be to prefix '3.' to core.lib.fix. I disfavor that for 3 reasons. 1. It is not needed to indicate 'not Python 2' as *any* leading digit greater than 2 says the same. 2. It makes for a more awkward 4 level number. 3. It presupposes a 3 to 4 transition something like the 2 to 3 transition. However, I am dubious about for more than one reason (another topic for another post). -- Terry Jan Reedy

3 3