Python-Dev February 2014

python-dev@python.org

106 participants
76 discussions

_PyUnicode_CheckConsistency() too strict?

by Phil Thompson

_PyUnicode_CheckConsistency() checks that the contents of the string matches the _KIND of the string. However it does this in a very strict manner, ie. that the contents *exactly* match the _KIND rather than just detecting an inconsistency between the contents and the _KIND. For example, a string created with a maxchar of 255 (ie. a Latin-1 string) must contain at least one character in the range 128-255 otherwise you get an assertion failure. As it stands, when converting Latin-1 strings in my C extension module I must first check each character and specify a maxchar of 127 if the strings happens to only contain ASCII characters. What is the reasoning behind the checks being so strict? Phil

6 years, 7 months

Guidance regarding what counts as breaking backwards compatibility

by Steven D'Aprano

Hi all, Over on the Python-ideas list, there's a thread about the new statistics module, and as the author of that module, I'm looking for a bit of guidance regarding backwards compatibility. Specifically two issues: (1) With numeric code, what happens if the module become more[1] accurate in the future? Does that count as breaking backwards compatibility? E.g. Currently I use a particular algorithm for calculating variance. Suppose that for a particular data set, this algorithm is accurate to (say) seven decimal places: # Python 3.4 variance(some_data) == 1.23456700001 Later, I find a better algorithm, which improves the accuracy of the result: # Python 3.5 or 3.6 variance(some_data) == 1.23456789001 Would this count as breaking backwards compatibility? If so, how should I handle this? I don't claim that the current implementation of the statistics module is optimal, as far as precision and accuracy is concerned. It may improve in the future. Or would that count as a bug-fix? "Variance function was inaccurate, now less wrong", perhaps. I suppose the math module has the same issue, except that it just wraps the C libraries, which are mature and stable and unlikely to change. The random module has a similar issue: http://docs.python.org/3/library/random.html#notes-on-reproducibility (2) Mappings[2] are iterable. That means that functions which expect sequences or iterators may also operate on mappings by accident. For example, sum({1: 100, 2: 200}) returns 3. If one wanted to reserve the opportunity to handle mappings specifically in the future, without being locked in by backwards-compatibility, how should one handle it? a) document that behaviour with mappings is unsupported and may change in the future; b) raise a warning when passed a mapping, but still iterate over it; c) raise an exception and refuse to iterate over the mapping; d) something else? Question (2) is of course a specific example of a more general question, to what degree is the library author responsible for keeping backwards compatibility under circumstances which are not part of the intended API, but just work by accident? [1] Or, for the sake of the argument, less accurate. [2] And sets. -- Steven

6 years, 7 months

News from asyncio

by Victor Stinner

Hi, I'm working for eNovance on the asyncio module, the goal is to use it in the huge OpenStack project (2.5 millions line of code) which currently uses eventlet. I'm trying to fix remaining issues in the asyncio module before Python 3.4 final. The asyncio project is very active but discussions are splitted between its own dedicated mailing list (python-tulip Google group), Tulip bug tracker and Python bug tracker. Please join Tulip mailing list if you are interested to contribute. http://code.google.com/p/tulip/ I would like to share with you the status of the module. Many bugs have been fixed recently. I suppose that new bugs are found because new developers started to play with asyncio since Python 3.4 beta 1. asyncio issues fixed in Python 3.4 beta 3, in a random order: - I wrote most of the asyncio documentation, please help me to improve it! I tried to add many short examples, each time explaining one feature or concept (ex: callbacks, signals, futures, etc.): http://docs.python.org/dev/library/asyncio.html - Characters devices (TTY/PTY) are now supported, useful to control real terminal (not pipes) for subprocesses. On Mac OS X older than Maverick (10.9), the SelectSelector should be used instead of KqueueSelector (kqueue didnd't support character devices) - Tulip #111: StreamReader.readexactly() now raises an IncompleteReadError if the end of stream is reached before we received enough bytes, instead of returning less bytes than requested. - Python #20311: asyncio had a performance issue related to the resolution of selectors and clocks. For example, selectors.EpollSelector has a resolution of 1 millisecond (10^-3), whereas asyncio uses arbitrary timestamps. The issue was fixed by adding a resolution attribute to selectors and a private granularity attribute to asyncio.BaseEventLoop, and use the granularity in asyncio event loop to round times. - New Task.current_task() class method - Guido wrote a web crawler, see examples/crawl.py in Tulip - More symbols are exported in the main asyncio module (ex: Queue, iscouroutine(), etc.) - Charles-François improved the signal handlers: SA_RESTART flag is now set to limit EINTR errors in syscalls - Some optimizations (ex: don't call logger.log() when it's not needed) - Many bugfixes - (sorry if I forgot other changes, see also Tulip history and history of the asyncio module in Python) I also would like to change asyncio to support a "stream-like" API for subprocesses, see Tulip issue #115 (and Python issue #20400): http://code.google.com/p/tulip/issues/detail?id=115 I ported ayncio on Python 2.6 and 2.7, because today OpenStack only uses these Python versions. I created a new project called "Trollius" (instead of "Tulip") because the syntax is a little bit different. "yield from" becomes "yield", and "return x" becomes "raise Return(x)": https://bitbucket.org/enovance/trollius https://pypi.python.org/pypi/trollius If you are interested by the OpenStack part, see my blueprint (something similar to PEPs but for smaller changes) for Oslo Messaing: https://wiki.openstack.org/wiki/Oslo/blueprints/asyncio There is an ongoing effort to port OpenStack to Python 3, eNovance is also working on the portage: https://wiki.openstack.org/wiki/Python3 Victor

6 years, 7 months

Re: [Python-Dev] [Python-checkins] cpython: whatsnew: move of reload, update new windows-only ssl functions entry.

by Brett Cannon

On Sat, Feb 1, 2014 at 12:27 PM, r.david.murray <python-checkins(a)python.org>wrote: > http://hg.python.org/cpython/rev/b3f034f5000f > changeset: 88884:b3f034f5000f > parent: 88882:19d81cc213d7 > user: R David Murray <rdmurray(a)bitdance.com> > date: Sat Feb 01 12:27:07 2014 -0500 > summary: > whatsnew: move of reload, update new windows-only ssl functions entry. > > files: > Doc/whatsnew/3.4.rst | 10 ++++++++-- > 1 files changed, 8 insertions(+), 2 deletions(-) > > > diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst > --- a/Doc/whatsnew/3.4.rst > +++ b/Doc/whatsnew/3.4.rst > @@ -765,6 +765,10 @@ > it will normally be desirable to override the default implementation > for performance reasons. (Contributed by Brett Cannon in :issue:`18072`.) > > +The :func:`~importlib.reload` function has been moved from :mod:`imp` > +to :mod:`importlib`. The :func:`mod.reload` name is retained for > +backward compatibility, but is deprecated. > + > That wording seems confusing to me. It makes it seem like importlib.reload is deprecated when in fact it's the imp module itself. -Brett > > inspect > ------- > @@ -1062,8 +1066,10 @@ > list of the loaded ``CA`` certificates. (Contributed by Christian Heimes > in > and :issue:`18147`.) > > -Add :func:`ssl.enum_cert_store` to retrieve certificates and CRL from > Windows' > -cert store. (Contributed by Christian Heimes in :issue:`17134`.) > +Two new windows-only functions, :func:`~ssl.enum_certificates` and > +:func:`~ssl.enum_crls` provide the ability to retrieve certificates, > +certificate information, and CRLs from the Windows cert store. > (Contributed > +by Christian Heimes in :issue:`17134`.) > > Support for server-side SNI using the new > :meth:`ssl.SSLContext.set_servername_callback` method. > > -- > Repository URL: http://hg.python.org/cpython > > _______________________________________________ > Python-checkins mailing list > Python-checkins(a)python.org > https://mail.python.org/mailman/listinfo/python-checkins > >

6 years, 7 months

Negative times behaviour in itertools.repeat for Python maintenance releases (2.7, 3.3 and maybe 3.4)

by Vajrasky Kok

Dear comrades, I would like to bring to your attention my disagreement with Larry Hastings in this ticket: http://bugs.python.org/issue19145 (Inconsistent behaviour in itertools.repeat when using negative times). Let me give you the context: >>> from itertools import repeat >>> repeat('a') repeat('a') >>> repeat('a', times=-1) repeat('a') >>> repeat('a', -1) repeat('a', 0) >>> repeat('a', times=-4) repeat('a', -4) >>> repeat('a', -4) repeat('a', 0) Right now, the only way you can tell repeat to do endless repetitions is to omit the `times` argument or by setting `times` argument to -1 via keyword. Larry intends to fix this in Python 3.5 by making None value to `times` argument as a representative of unlimited repetitions and negative `times` argument (either via keyword or positional) ALWAYS means 0 repetitions. This will ensure repeat has the appropriate signature. I have no qualms about it. All is well. My disagreement is related to Larry's decision not to fix this bug in Python 2.7, 3.3, and 3.4. Both of us agree that we should not let Python 2.7, 3.3, and 3.4 happily accepts None value because that is more than bug fix. What we don't agree is whether we should make negative `times` argument via keyword behaviour needs to be changed or not. He prefer let it be. I prefer we change the behaviour so that negative `times` argument in Python 2.7, 3.3, and 3.4 ALWAYS means 0 repetitions. My argument is that, on all circumstances, argument sent to function via positional or keyword must mean the same thing. Let's consider this hypothetical code: # 0 means draw, positive int means win, negative int means lose result = goals_result_of_the_match() # For every goal of the winning match, we donate money to charity. # Every donation consists of 10 $. import itertools itertools.repeat(donate_money_to_charity(), result) Later programmer B refactor this code: # 0 means draw, positive int means win, negative int means lose result = goals_result_of_the_match() # For every goal of the winning match, we donate money to charity. # Every donation consists of 10 $ from itertools import repeat repeat(object=donate_money_to_charity(), times=result) They use Python 2.7 (remember Python 2.7 is here to stay for a long long time). And imagine the match is lost 0-1 or 1-2 or 2-3 (so the goal difference is negative one / -1). It means they donate money to charity endlessly!!! They can go bankrupt. So I hope my argument is convincing enough. We need to fix this bug in Python 2.7, 3.3, and 3.4, by making `times` argument sent via positional or keyword in itertools.repeat ALWAYS means the same thing, which is 0 repetitions. If this is not possible, at the very least, we need to warn this behaviour in the doc. Whatever decision that comes up from this discussion, I will make peace with it. Vajrasky

6 years, 8 months

Add PyType_GetSlot

by "Martin v. Löwis"

I'd like to resolve a long-standing issue of the stable ABI in 3.4: http://bugs.python.org/issue17162 The issue is that, since PyTypeObject is opaque, module authors cannot get at tp_free, which they may need to in order to implement tp_dealloc properly. Rather than providing the proposed specific wrapper for tp_dealloc, I propose to add a generic PyType_GetSlot function. From a stability point of view, exposing slot values is uncritical - it's just that the layout of the type object is hidden. Any objection to adding this before RC1? Regards, Martin

6 years, 8 months

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

Python-Dev February 2014