Mailman 3 November 2013 - Python-Dev

Re: [Python-Dev] [Python-checkins] peps: Update list of accepted but not yet implemented or merged peps.
by Victor Stinner 22 Nov '13

22 Nov '13

Hum, I don't think that regex module will enter Python 3.4 before this week-end, there is no PEP. For the "Introspection information for builtins", I think the PEP 436 has been accepted. The code has been merged, but the PEP status is still draft. Victor 2013/11/22 barry.warsaw <python-checkins(a)python.org>: > http://hg.python.org/peps/rev/ea82fe7aa1ff > changeset: 5311:ea82fe7aa1ff > user: Barry Warsaw <barry(a)python.org> > date: Thu Nov 21 18:21:06 2013 -0500 > summary: > Update list of accepted but not yet implemented or merged peps. > > files: > pep-0429.txt | 14 +++++++++----- > 1 files changed, 9 insertions(+), 5 deletions(-) > > > diff --git a/pep-0429.txt b/pep-0429.txt > --- a/pep-0429.txt > +++ b/pep-0429.txt > @@ -74,6 +74,15 @@ > * PEP 446, make newly created file descriptors not inherited by default > * PEP 450, basic statistics module for the standard library > > +Accepted but not yet implemented/merged: > + > +* PEP 428, the pathlib module -- object-oriented filesystem paths > +* PEP 451, a ModuleSpec Type for the Import System > +* PEP 453, pip bootstrapping/bundling with CPython > +* PEP 454, the tracemalloc module for tracing Python memory allocations > +* PEP 3154, Pickle protocol revision 4 > +* PEP 3156, improved asynchronous IO support > + > Other final large-scale changes: > > * None so far > @@ -85,12 +94,7 @@ > * PEP 441, improved Python zip application support > * PEP 447, support for __locallookup__ metaclass method > * PEP 448, additional unpacking generalizations > -* PEP 451, making custom import hooks easier to implement correctly > -* PEP 453, pip bootstrapping/bundling with CPython > -* PEP 454, PEP 445 based malloc tracing support > * PEP 455, key transforming dictionary > -* PEP 3154, Pickle protocol revision 4 > -* PEP 3156, improved asynchronous IO support > > Other proposed large-scale changes: > > > -- > Repository URL: http://hg.python.org/peps > > _______________________________________________ > Python-checkins mailing list > Python-checkins(a)python.org > https://mail.python.org/mailman/listinfo/python-checkins >

2 1

flaky tests caused by repr() sort order
by Christian Heimes 22 Nov '13

22 Nov '13

Hi, the buildbots are flaky because two repr() tests for userdict and functools.partial fail every now and then. The test cases depend on a fixed order of keyword arguments the representation of userdict and partial instances. The improved hash randomization of PEP 456 shows its power. I haven't seen the issue before because it happens rarely and mostly on 32 bit platforms. http://bugs.python.org/issue19681 http://bugs.python.org/issue19664 I'm not sure about the best approach for the issues. Either we need to change the test case and make it more resilient or the code for repr() must sort its dict keys. Christian

5 5

PEP 428 - pathlib - ready for approval
by Antoine Pitrou 21 Nov '13

21 Nov '13

Hello, Guido has told me that he was ready to approve PEP 428 (pathlib) in its latest amended form. Here is the last call for any comments or arguments against approval, before Guido marks the PEP accepted (or changes his mind :-)). Regards Antoine.

12 20

custom thread scheduler to test user synchronisation code
by Dima Tisnek 21 Nov '13

21 Nov '13

Hi, First, in case such project already exists, could someone point me towards this? It occurs to me that in some cases may be possible to run exhaustive test on user-implemented synchronisation code. let's say, in a simple case, we've got a 2 threads, some user code using threading.* primitives to synchronise these threads and a unit test that ensures the logical result of synchronisation is sane. now, cpython relies on OS to schedule these 2 threads, however, if this scheduler was under my control, I could force this test case into many different, distinct thread interleaves. Granted, the total number of interleaves is exponential and even unbounded in presence of loops. Yet for small user code and unit test, this could even be exhaustive. In a brute-force case, a new interleave should be branched off after every bytecode evaluation (and in case of extensions GIL acquisition/release). A smarter tester could possibly track primitive and data accesses by threads and only create interleave branch when there's potential of something being used by both threads. How would I go about doing this? That is how would I go about implementing own thread scheduler? Thanks, d.

1 0

[issue19494] Problems with HTTPBasicAuthHandler
by Matej Cepl 21 Nov '13

21 Nov '13

Hello, I am trying to work on fixing issue 19494 (HTTPBasicAuthHandler doesn't work with Github and other websites which require prior Authorization header in the first request). I have created this testing script calling GitHub v3 API using new “authentication handlers” (they are subclasses of HTTPSHandler, not HTTPBasicAuthHandler) and hoped that when I manage to make this script working, I could rewrite it into patch against cpython code. However, when I run this script I get this error (using python3-3.3.2-8.el7.x86_64) matej@wycliff: $ python3 test_create_1.py DEBUG:<module>:gh_url = https://api.github.com/repos/mcepl/odt2rst/issues/ DEBUG:<module>:req = <urllib.request.Request object at 0x7fee384a9fd0> DEBUG:<module>:req.type = https DEBUG:http_request:type = https Traceback (most recent call last): File "test_create_1.py", line 80, in <module> handler = opener.open(req, json.dumps(create_data).encode('utf8')) File "/usr/lib64/python3.3/urllib/request.py", line 475, in open response = meth(req, response) File "/usr/lib64/python3.3/urllib/request.py", line 587, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib64/python3.3/urllib/request.py", line 513, in error return self._call_chain(*args) File "/usr/lib64/python3.3/urllib/request.py", line 447, in _call_chain result = func(*args) File "/usr/lib64/python3.3/urllib/request.py", line 595, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 401: Unauthorized matej@wycliff: $ Could anybody suggest what I am doing wrong? I guess I need to somehow include into opener original HTTPBasicAuthHandler (for error handlers). Any ideas how to do it? Best, Matěj Cepl -- http://www.ceplovi.cz/matej/, Jabber: mcepl<at>ceplovi.cz GPG Finger: 89EF 4BC6 288A BF43 1BAB 25C3 E09F EF25 D964 84AC - Do you think of yourself as a Christian artist? - I'm an artist who is a Christian. I'm not a Christian artist. -- Johny Cash in his last interview ======================================================================= import base64 import http.client import urllib.request import json import os import logging from configparser import ConfigParser logging.basicConfig(format='%(levelname)s:%(funcName)s:%(message)s', level=logging.DEBUG) class HTTPBasicPriorAuthHandler(urllib.request.HTTPHandler): handler_order = 400 def __init__(self, *args, **kwargs): passwd_mgr = kwargs.pop('password_mgr', None) urllib.request.HTTPHandler.__init__(self, *args, **kwargs) self.passwd = passwd_mgr def http_request(self, req): logging.debug("type = {}".format(req.type)) if not req.has_header('Authorization'): user, passwd = self.passwd.find_user_password( None, req.get_full_url()) auth_str = base64.standard_b64encode( '{}:{}'.format(user, passwd).encode('ascii')) req.add_header('Authorization', 'Basic {}'.format(auth_str)) return urllib.request.AbstractHTTPHandler.do_request_(self, req) if hasattr(http.client, 'ssl'): handler_order = 400 class HTTPSBasicPriorAuthHandler(urllib.request.HTTPSHandler): def __init__(self, *args, **kwargs): passwd_mgr = kwargs.pop('password_mgr', None) urllib.request.HTTPSHandler.__init__(self, *args, **kwargs) self.passwd = passwd_mgr https_request = HTTPBasicPriorAuthHandler.http_request cp = ConfigParser() cp.read(os.path.expanduser('~/.githubrc')) # That configuration file should look something like # [github] # user=mylogin # password=myveryverysecretpassword gh_user = cp.get('github', 'user') gh_passw = cp.get('github', 'password') repo = 'odt2rst' pwd_manager = urllib.request.HTTPPasswordMgrWithDefaultRealm() pwd_manager.add_password(None, uri='https://api.github.com', user=gh_user, passwd=gh_passw) auth_prior_handler = HTTPSBasicPriorAuthHandler(password_mgr=pwd_manager) auth_prior_handler.set_http_debuglevel(2) auth_handler = urllib.request.HTTPBasicAuthHandler(password_mgr=pwd_manager) opener = urllib.request.build_opener(auth_prior_handler, auth_handler) API_URL = "https://api.github.com/repos/{0}/{1}/issues/" gh_url = API_URL.format(gh_user, repo, 1) logging.debug("gh_url = {0}".format(gh_url)) req = urllib.request.Request(gh_url) logging.debug('req = {}'.format(req)) logging.debug('req.type = {}'.format(req.type)) create_data = { 'title': 'Testing bug summary', 'body': '''This is a testing bug. I am writing here this stuff, just to have some text for body.''', 'labels': ['BEbug'] } handler = opener.open(req, json.dumps(create_data).encode('utf8')) print(handler.getcode()) print(handler)

1 0

PEP 3156 (asyncio) formally accepted
by Antoine Pitrou 20 Nov '13

20 Nov '13

Hello, I have decided to mark PEP 3156 accepted. This reflects the fact that the implementation is now in the stdlib tree, and its API has been pretty much validated during all previous discussions (mostly on the tulip mailing-list). I cannot stress enough that the main preoccupation right now should be to write and integrate module docs inside the CPython source tree :-) Congratulations to Guido and all people involved! This is a large and very significant addition to the stdlib, and as far as I can tell both design and implementation have benefitted from the contributions of many people (perhaps someone should add them to Misc/ACKS?). Regards Antoine.

2 1

Add transform() and untranform() methods
by Victor Stinner 20 Nov '13

20 Nov '13

Hi, I saw that Nick Coghlan documented codecs.encode() and codecs.decode(), and changed the exception raised when codecs like rot_13 are used on bytes.decode() and str.encode(). I don't like the functions codecs.encode() and codecs.decode() because the type of the result depends on the encoding (second parameter). We try to avoid this in Python. I would prefer to split the registry of codecs to have 3 registries: - "encoding" (a better name can found): encode str=>bytes, decode bytes=>str - bytes: encode bytes=>bytes, decode bytes=>bytes - str: encode str=>str, decode str=>str And add transform() and untransform() methods to bytes and str types. In practice, it might be same codecs registry for all codecs just with a new attribute. Examples: - utf8: encoding - zlib: bytes - rot13: str The result type of bytes.transform/untransform would be bytes, and the result type of str.transform/untransform would be str. I don't know which exception should be raised when a codec is used in the wrong method. LookupError? TypeError "codec xxx cannot be used with method xxx.xx"? Something else? codecs.encode/decode() documentation should be removed. The functions should be kept, just in case if someone uses them. Victor

15 45

Accepting PEP 456 (Secure hash algorithm)
by Nick Coghlan 20 Nov '13

20 Nov '13

Christian has indicated he now considers PEP 456, which adds an updated and configurable hash algorithm ready for pronouncement ( http://www.python.org/dev/peps/pep-0456/) I am happy the PEP and the associated implementation represent a desirable improvement to CPython, and approve of the proposal to experiment further with the small string optimisation during the beta period, before either enabling it or removing it entirely in beta 2. Accordingly, I am formally accepting the PEP for inclusion in Python 3.4 :) Regards, Nick.

3 4

Re: [Python-Dev] [RELEASED] Python 3.3.3 final
by Georg Brandl 20 Nov '13

20 Nov '13

Am 19.11.2013 17:14, schrieb Mark Lawrence: > On 19/11/2013 06:59, Georg Brandl wrote: >> >> To download Python 3.3.3 rc2 visit: >> >> http://www.python.org/download/releases/3.3.3/ >> > > Please make your mind up, final or rc2? > > Thanks everybody for your efforts, much appreciated :) It's my firm belief that every announce should have a small error to appease the gods of regression. *ahem* :) Georg

2 1

PEP 454 (tracemalloc) close to pronouncement
by Charles-François Natali 20 Nov '13

20 Nov '13

Hi, After several exchanges with Victor, PEP 454 has reached a status which I consider ready for pronuncement [1]: so if you have any last minute comment, now is the time! Cheers, cf [1] http://www.python.org/dev/peps/pep-0454/

4 6