Mailman 3 April 2014 - Python-Dev

flock in Python 3
by Jakub QB Dorňák April 13, 2014

April 13, 2014

Hi all, writing a threaded application I've been surprised that there is no object api for fcntl.flock implementing __enter__() and __exit__() methods to be used with 'with' statement. I have written one (https://pypi.python.org/pypi), but I wonder whether this could get into the Python Standard Library. Regards, QB

3 3

PEP 461: Adding % formatting to bytes and bytearray -- Final, Take 3
by Ethan Furman April 12, 2014

April 12, 2014

Okay, I included that last round of comments (from late February). Barring typos, this should be the final version. Final comments? ----------------------------------------------------------------------------- PEP: 461 Title: Adding % formatting to bytes and bytearray Version: $Revision$ Last-Modified: $Date$ Author: Ethan Furman <ethan(a)stoneleaf.us> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 2014-01-13 Python-Version: 3.5 Post-History: 2014-01-14, 2014-01-15, 2014-01-17, 2014-02-22, 2014-03-25 Resolution: Abstract ======== This PEP proposes adding % formatting operations similar to Python 2's ``str`` type to ``bytes`` and ``bytearray`` [1]_ [2]_. Rationale ========= While interpolation is usually thought of as a string operation, there are cases where interpolation on ``bytes`` or ``bytearrays`` make sense, and the work needed to make up for this missing functionality detracts from the overall readability of the code. Motivation ========== With Python 3 and the split between ``str`` and ``bytes``, one small but important area of programming became slightly more difficult, and much more painful -- wire format protocols [3]_. This area of programming is characterized by a mixture of binary data and ASCII compatible segments of text (aka ASCII-encoded text). Bringing back a restricted %-interpolation for ``bytes`` and ``bytearray`` will aid both in writing new wire format code, and in porting Python 2 wire format code. Common use-cases include ``dbf`` and ``pdf`` file formats, ``email`` formats, and ``FTP`` and ``HTTP`` communications, among many others. Proposed semantics for ``bytes`` and ``bytearray`` formatting ============================================================= %-interpolation --------------- All the numeric formatting codes (such as ``%x``, ``%o``, ``%e``, ``%f``, ``%g``, etc.) will be supported, and will work as they do for str, including the padding, justification and other related modifiers. The only difference will be that the results from these codes will be ASCII-encoded text, not unicode. In other words, for any numeric formatting code `%x`:: b"%x" % val is equivalent to ("%x" % val).encode("ascii") Examples:: >>> b'%4x' % 10 b' a' >>> b'%#4x' % 10 ' 0xa' >>> b'%04X' % 10 '000A' ``%c`` will insert a single byte, either from an ``int`` in range(256), or from a ``bytes`` argument of length 1, not from a ``str``. Examples:: >>> b'%c' % 48 b'0' >>> b'%c' % b'a' b'a' ``%s`` is included for two reasons: 1) `b` is already a format code for ``format`` numerics (binary), and 2) it will make 2/3 code easier as Python 2.x code uses ``%s``; however, it is restricted in what it will accept:: - input type supports ``Py_buffer`` [6]_? use it to collect the necessary bytes - input type is something else? use its ``__bytes__`` method [7]_ ; if there isn't one, raise a ``TypeError`` In particular, ``%s`` will not accept numbers (use a numeric format code for that), nor ``str`` (encode it to ``bytes``). Examples:: >>> b'%s' % b'abc' b'abc' >>> b'%s' % 'some string'.encode('utf8') b'some string' >>> b'%s' % 3.14 Traceback (most recent call last): ... TypeError: b'%s' does not accept numbers, use a numeric code instead >>> b'%s' % 'hello world!' Traceback (most recent call last): ... TypeError: b'%s' does not accept 'str', it must be encoded to `bytes` ``%a`` will call ``ascii()`` on the interpolated value. This is intended as a debugging aid, rather than something that should be used in production. Non-ASCII values will be encoded to either ``\xnn`` or ``\unnnn`` representation. Use cases include developing a new protocol and writing landmarks into the stream; debugging data going into an existing protocol to see if the problem is the protocol itself or bad data; a fall-back for a serialization format; or even a rudimentary serialization format when defining ``__bytes__`` would not be appropriate [8]. .. note:: If a ``str`` is passed into ``%a``, it will be surrounded by quotes. Unsupported codes ----------------- ``%r`` (which calls ``__repr__`` and returns a ``str``) is not supported. Proposed variations =================== It was suggested to let ``%s`` accept numbers, but since numbers have their own format codes this idea was discarded. It has been suggested to use ``%b`` for bytes as well as ``%s``. This was rejected as not adding any value either in clarity or simplicity. It has been proposed to automatically use ``.encode('ascii','strict')`` for ``str`` arguments to ``%s``. - Rejected as this would lead to intermittent failures. Better to have the operation always fail so the trouble-spot can be correctly fixed. It has been proposed to have ``%s`` return the ascii-encoded repr when the value is a ``str`` (b'%s' % 'abc' --> b"'abc'"). - Rejected as this would lead to hard to debug failures far from the problem site. Better to have the operation always fail so the trouble-spot can be easily fixed. Originally this PEP also proposed adding format-style formatting, but it was decided that format and its related machinery were all strictly text (aka ``str``) based, and it was dropped. Various new special methods were proposed, such as ``__ascii__``, ``__format_bytes__``, etc.; such methods are not needed at this time, but can be visited again later if real-world use shows deficiencies with this solution. Objections ========== The objections raised against this PEP were mainly variations on two themes:: - the ``bytes`` and ``bytearray`` types are for pure binary data, with no assumptions about encodings - offering %-interpolation that assumes an ASCII encoding will be an attractive nuisance and lead us back to the problems of the Python 2 ``str``/``unicode`` text model As was seen during the discussion, ``bytes`` and ``bytearray`` are also used for mixed binary data and ASCII-compatible segments: file formats such as ``dbf`` and ``pdf``, network protocols such as ``ftp`` and ``email``, etc. ``bytes`` and ``bytearray`` already have several methods which assume an ASCII compatible encoding. ``upper()``, ``isalpha()``, and ``expandtabs()`` to name just a few. %-interpolation, with its very restricted mini-language, will not be any more of a nuisance than the already existing methods. Some have objected to allowing the full range of numeric formatting codes with the claim that decimal alone would be sufficient. However, at least two formats (dbf and pdf) make use of non-decimal numbers. Footnotes ========= .. [1] http://docs.python.org/2/library/stdtypes.html#string-formatting .. [2] neither string.Template, format, nor str.format are under consideration .. [3] https://mail.python.org/pipermail/python-dev/2014-January/131518.html .. [4] to use a str object in a bytes interpolation, encode it first .. [5] %c is not an exception as neither of its possible arguments are str .. [6] http://docs.python.org/3/c-api/buffer.html examples: ``memoryview``, ``array.array``, ``bytearray``, ``bytes`` .. [7] http://docs.python.org/3/reference/datamodel.html#object.__bytes__ .. [8] https://mail.python.org/pipermail/python-dev/2014-February/132750.html Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End:

15 51

arguments policy: **kwargs.pop()
by Christian Tismer April 12, 2014

April 12, 2014

Hi guys, I tried to find advice for hours, but failed so fer, so here is my question: Whenever I think to adopt a new module that does a good job, I always can't stand the temptation to look at the coding style and certain principles. Then I rather often see things like this: class someclass(object): # note that there is no comment about argument destruction... def __init__(self, **kwargs): first_arg = kwargs.pop('option_1', somedefault) ... nth_arg = kwargs.pop('option_n', somedefault') ... That is: There are arguments "consumed" rigorously by the __init__ of a class, although it would be equivalent to use kwargs.get(..., ...). Happened to me again, today and blocked me completely, since I don't know if I saw bad programming style or if I'm mislead. I agree there are valid cases when if makes sense to filter out some arguments that a subsequently called super() might not be able to handle. Bu even then, I would probably have a copy and make the filtering more explicit. But most of the time, there is no reason for using this pattern, and there is also no comment stating that the argument dict is modified by the callee for no good reason. I always used the policy that arguments are never changed by a function, unless explicitly stated. But since I see this pattern quite frequently, I wanted to ask if I am right by thinking this way, or if the general policy is more like "arguments may be destroyed by default". What do you think? Is this bad style and should be noticed somewhere, or is the caller supposed to protect the arguments, or are my worries useless? Thanks & cheers -- Chris -- Christian Tismer :^) <mailto:tismer@stackless.com> Software Consulting : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 : *Starship* http://starship.python.net/ 14482 Potsdam : PGP key -> http://pgp.uni-mainz.de phone +49 173 24 18 776 fax +49 (30) 700143-0023 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/

6 9

Receiving email updates for the bug tracker
by Antony Lee April 11, 2014

April 11, 2014

Hi, Sorry for the slightly off-topic(?) question but I would like to know how to receive email notifications for activity on bugs I've opened on the bugs.python.org -- so far I don't receive anything even though I'm on the nosy list. Thanks, Antony

3 3

Re: [Python-Dev] Receiving email updates for the bug tracker
by Antony Lee April 11, 2014

April 11, 2014

Nope, the email is correct... 2014-04-11 12:12 GMT-07:00 Brett Cannon <brett(a)python.org>: > > > > On Fri, Apr 11, 2014 at 1:47 PM, Antony Lee <antony.lee(a)berkeley.edu>wrote: > >> Hi, >> Sorry for the slightly off-topic(?) question but I would like to know how >> to receive email notifications for activity on bugs I've opened on the >> bugs.python.org -- so far I don't receive anything even though I'm on >> the nosy list. >> > > Should be automatic. I would check your email address in your account > settings. > > -Brett > > >> Thanks, >> Antony >> >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev(a)python.org >> https://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> https://mail.python.org/mailman/options/python-dev/brett%40python.org >> >> >

2 1

Summary of Python tracker Issues
by Python tracker April 11, 2014

April 11, 2014

ACTIVITY SUMMARY (2014-04-04 - 2014-04-11) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 4567 (+17) closed 28405 (+29) total 32972 (+46) Open issues with patches: 2096 Issues opened (32) ================== #21015: support SSL_CTX_set_ecdh_auto on newer OpenSSLs http://bugs.python.org/issue21015 reopened by ned.deily #21156: Consider moving importlib.abc.InspectLoader.source_to_code() t http://bugs.python.org/issue21156 opened by brett.cannon #21157: Update imp docs for a PEP 451 world http://bugs.python.org/issue21157 opened by brett.cannon #21158: Windows installer service could not be accessed (Python bug!) http://bugs.python.org/issue21158 opened by thecheater887 #21159: configparser.InterpolationMissingOptionError is not very intui http://bugs.python.org/issue21159 opened by Claudiu.Popa #21160: incorrect comments in nturl2path.py http://bugs.python.org/issue21160 opened by Jurko.Gospodnetić #21162: code in multiprocessing.pool freeze if inside some code from s http://bugs.python.org/issue21162 opened by Ivan.K #21163: asyncio task possibly incorrectly garbage collected http://bugs.python.org/issue21163 opened by richard.kiss #21164: print unicode in Windows console http://bugs.python.org/issue21164 opened by LeslieK #21165: Optimize str.translate() for replacement with substrings and n http://bugs.python.org/issue21165 opened by haypo #21167: float('nan') returns 0.0 on Python compiled with icc http://bugs.python.org/issue21167 opened by hniksic #21169: getpass.getpass() fails with non-ASCII characters in prompt http://bugs.python.org/issue21169 opened by Arfrever #21170: Incorrect signature for unittest.TestResult.startTestRun(), .s http://bugs.python.org/issue21170 opened by Volodymyr.Sapsai #21171: Outdated usage str.encode('rot-13') in rot13 codec http://bugs.python.org/issue21171 opened by Pix #21173: WeakKeyDictionary.__len__ fragile w/ _IterationGuards http://bugs.python.org/issue21173 opened by pjenvey #21177: ValueError: byte must be in range(0, 256) http://bugs.python.org/issue21177 opened by zart #21178: doctest cause warnings in tests using generators http://bugs.python.org/issue21178 opened by oluensdorf #21180: Cannot efficiently create empty array.array of given size, inc http://bugs.python.org/issue21180 opened by pfalcon #21183: Doctest capture only AssertionError but not printed text http://bugs.python.org/issue21183 opened by osantana #21184: statistics.pvariance with known mean does not work as expected http://bugs.python.org/issue21184 opened by steven.daprano #21186: RawConfigParser __name__ option handling inconsistent http://bugs.python.org/issue21186 opened by Adam.Groszer #21188: Broken link http://bugs.python.org/issue21188 opened by tiposchi #21189: Broken link to patch http://bugs.python.org/issue21189 opened by tiposchi #21192: Idle: Print filename when running a file from editor http://bugs.python.org/issue21192 opened by terry.reedy #21193: pow(a, b, c) should not raise TypeError when b is negative and http://bugs.python.org/issue21193 opened by josh.rosenberg #21194: json.dumps with ensure_ascii=False doesn't escape control char http://bugs.python.org/issue21194 opened by weeble #21195: None float format: incomplete documentation http://bugs.python.org/issue21195 opened by lebigot #21196: Name mangling example in Python tutorial http://bugs.python.org/issue21196 opened by chandan #21197: venv does not create lib64 directory and appropriate symlinks http://bugs.python.org/issue21197 opened by BinaryKhaos #21198: Minor tarfile documentation bug http://bugs.python.org/issue21198 opened by lurchman #21199: Python on 64-bit Windows uses signed 32-bit type for read leng http://bugs.python.org/issue21199 opened by Matt.Mackall #21200: pkgutil.get_loader() fails on "__main__" http://bugs.python.org/issue21200 opened by eric.snow Most recent 15 issues with no replies (15) ========================================== #21200: pkgutil.get_loader() fails on "__main__" http://bugs.python.org/issue21200 #21198: Minor tarfile documentation bug http://bugs.python.org/issue21198 #21188: Broken link http://bugs.python.org/issue21188 #21173: WeakKeyDictionary.__len__ fragile w/ _IterationGuards http://bugs.python.org/issue21173 #21160: incorrect comments in nturl2path.py http://bugs.python.org/issue21160 #21152: Idle: timed autosave for shell (and maybe editor) window http://bugs.python.org/issue21152 #21132: Failure to import win32api http://bugs.python.org/issue21132 #21126: Integrate doctest.run_docstring_examples() with unittest http://bugs.python.org/issue21126 #21120: PyArena type is used in headers from the limited API http://bugs.python.org/issue21120 #21114: wsgiref.simple_server doesn't handle multi-line headers correc http://bugs.python.org/issue21114 #21112: 3.4 regression: unittest.expectedFailure no longer works on Te http://bugs.python.org/issue21112 #21110: Slowdown and high memory usage when adding a new module in emb http://bugs.python.org/issue21110 #21107: Add pgen.vcxproj to allow regenerating grammar files on Window http://bugs.python.org/issue21107 #21095: EmailMessage should support Header objects http://bugs.python.org/issue21095 #21091: EmailMessage.is_attachment should be a method http://bugs.python.org/issue21091 Most recent 15 issues waiting for review (15) ============================================= #21193: pow(a, b, c) should not raise TypeError when b is negative and http://bugs.python.org/issue21193 #21173: WeakKeyDictionary.__len__ fragile w/ _IterationGuards http://bugs.python.org/issue21173 #21171: Outdated usage str.encode('rot-13') in rot13 codec http://bugs.python.org/issue21171 #21170: Incorrect signature for unittest.TestResult.startTestRun(), .s http://bugs.python.org/issue21170 #21169: getpass.getpass() fails with non-ASCII characters in prompt http://bugs.python.org/issue21169 #21167: float('nan') returns 0.0 on Python compiled with icc http://bugs.python.org/issue21167 #21148: avoid memset in small tuple creation http://bugs.python.org/issue21148 #21147: sqlite3 doesn't complain if the request contains a null charac http://bugs.python.org/issue21147 #21146: update gzip usage examples in docs http://bugs.python.org/issue21146 #21141: Don't mention Perl in Windows build output http://bugs.python.org/issue21141 #21140: Idle: saving an OutputWindow should default to .txt http://bugs.python.org/issue21140 #21139: Idle: change default reformat width from 70 to 72 http://bugs.python.org/issue21139 #21137: Better repr for threading.Lock() http://bugs.python.org/issue21137 #21127: Path objects cannot be constructed from str subclasses http://bugs.python.org/issue21127 #21126: Integrate doctest.run_docstring_examples() with unittest http://bugs.python.org/issue21126 Top 10 most discussed issues (10) ================================= #21177: ValueError: byte must be in range(0, 256) http://bugs.python.org/issue21177 13 msgs #21111: PyLong_AsUnsignedLongAndOverflow does not exist http://bugs.python.org/issue21111 9 msgs #21167: float('nan') returns 0.0 on Python compiled with icc http://bugs.python.org/issue21167 9 msgs #16927: Separate built-in types from functions and group similar funct http://bugs.python.org/issue16927 8 msgs #21162: code in multiprocessing.pool freeze if inside some code from s http://bugs.python.org/issue21162 8 msgs #21101: Extend the PyDict C API to handle cases where the hash value i http://bugs.python.org/issue21101 7 msgs #1191964: asynchronous Subprocess http://bugs.python.org/issue1191964 7 msgs #20383: Add a keyword-only spec argument to types.ModuleType http://bugs.python.org/issue20383 6 msgs #21163: asyncio task possibly incorrectly garbage collected http://bugs.python.org/issue21163 6 msgs #21193: pow(a, b, c) should not raise TypeError when b is negative and http://bugs.python.org/issue21193 6 msgs Issues closed (30) ================== #12014: str.format parses replacement field incorrectly http://bugs.python.org/issue12014 closed by benjamin.peterson #16395: Documentation claims that PySequence_Fast returns a tuple, whe http://bugs.python.org/issue16395 closed by python-dev #17621: Create a lazy import loader mixin http://bugs.python.org/issue17621 closed by brett.cannon #19281: add __objclass__ to the docs http://bugs.python.org/issue19281 closed by python-dev #20334: make inspect Signature hashable http://bugs.python.org/issue20334 closed by yselivanov #20539: math.factorial may throw OverflowError http://bugs.python.org/issue20539 closed by mark.dickinson #20644: OS X installer build broken by changes to documentation build http://bugs.python.org/issue20644 closed by ned.deily #21076: Turn signal.SIG* constants into enums http://bugs.python.org/issue21076 closed by giampaolo.rodola #21117: inspect.signature: inaccuracies for partial functions http://bugs.python.org/issue21117 closed by yselivanov #21118: str.translate is absurdly slow in majority of use cases (takes http://bugs.python.org/issue21118 closed by haypo #21128: testing stdlib and compatibility with pypy http://bugs.python.org/issue21128 closed by benjamin.peterson #21133: unittest discover should allow option to run each package sepa http://bugs.python.org/issue21133 closed by michael.foord #21136: fractions.Fraction.__pow__ does unneeded renormalization http://bugs.python.org/issue21136 closed by mark.dickinson #21144: ensurepip "AssertionError: Multiple .dist-info directories" http://bugs.python.org/issue21144 closed by ned.deily #21153: bdist_rpm fails if project contains files with spaces in the n http://bugs.python.org/issue21153 closed by jason.coombs #21155: asyncio: calling _UnixSelectorEventLoop.create_unix_server(soc http://bugs.python.org/issue21155 closed by haypo #21161: list comprehensions don't see local variables in pdb in python http://bugs.python.org/issue21161 closed by georg.brandl #21166: Bus error on "AMD64 FreeBSD 9.x 3.4" buildbot http://bugs.python.org/issue21166 closed by haypo #21168: unicodeobject.c: likely type-punning may break strict-aliasing http://bugs.python.org/issue21168 closed by haypo #21172: Unexpected behaviour with logging.LogRecord "first arg is dict http://bugs.python.org/issue21172 closed by vinay.sajip #21174: A typo in the docs for "exception GeneratorExit" http://bugs.python.org/issue21174 closed by python-dev #21175: Refcounting error in str.translate fastpath http://bugs.python.org/issue21175 closed by python-dev #21176: Implement matrix multiplication operator (PEP 465) http://bugs.python.org/issue21176 closed by python-dev #21179: Rounding half to even http://bugs.python.org/issue21179 closed by mark.dickinson #21181: Inconsistent Definition of collections.namedtuple.__dict__ in http://bugs.python.org/issue21181 closed by rhettinger #21182: json.loads errors out on valid JSON http://bugs.python.org/issue21182 closed by ned.deily #21185: heapq fails to print in sorted order for certain inputs http://bugs.python.org/issue21185 closed by wchlm #21187: build-installer.py fails using Xcode 5.1 on OS X 10.9 http://bugs.python.org/issue21187 closed by ned.deily #21190: Broken download link on README for CPython docs http://bugs.python.org/issue21190 closed by orsenthil #21191: os.fdopen() may eat file descriptor and still raise exception http://bugs.python.org/issue21191 closed by benjamin.peterson

1 0

PEP 465: A dedicated infix operator for matrix multiplication
by Nathaniel Smith April 10, 2014

April 10, 2014

Hi all, I've just finished updating PEP 465 with resolutions to the various issues that were raised during the python-ideas thread about it. (The main changes since that thread are that @@ has been removed, and we now definitely propose that @ have the same precedence and associativity as *.) I believe this leaves only one open question, which is where exactly to stick the new matmul slots into PyTypeObject. This is the kind of fiddly detail that can easily be settled later if the PEP is accepted, though. So, I guess as far as I'm concerned, this is ready to go. Feedback welcome: http://legacy.python.org/dev/peps/pep-0465/ -n -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org

23 59

A Friendly IDLE
by adnanumer95＠gmail.com April 10, 2014

April 10, 2014

Greeting Everyone. First of all I want to introduce my self Adnan Umer as a student of bachelors in Information Technology. I’ve few suggestions on improving IDLE. Here are few: On windows we can open any python file from context menu because IDLE is not a application. I recommends to create a simple executable that just calls ‘idle.pyw’ module in lib\idlelib. On executing python script with IDLE we can’t determine which file is executed. I recommends to print file name before executing. I made a little try to do that and I succeed. In Python Shell Save & Save As menus are enable and using them we can save shell text as python script (.py) that never executes again on IDLE. I recommends to either disable this option or save shell text as plain text. I made a little try to disable this and succeed. There is almost no difference on displayed result of these two command >>> print (1) 1 >>> 1 1 there must be some difference as this creates a lot of confusion for beginners to understand purpose of print statement. Adnan Umer

4 3

issue with itertools leads the crash
by Kirill April 10, 2014

April 10, 2014

issue with itertools leads the crash The following code leads to system failure and crash on Ubuntu 12.04.3 with Python 2.7.6. import itertools r = range (1,10) r[1:9:2] = itertools.cycle([0]) Kirill .

10 11

Windows buildbots are red: test_idlelib
by Victor Stinner April 8, 2014

April 8, 2014

Hi, Unit tests are failing on Windows because of this issue: http://bugs.python.org/issue21059 It looks like a regression in test_idlelib introduced with this issue: http://bugs.python.org/issue15968 Zachary Ware wrote a fix: http://bugs.python.org/issue20035 Can someone please review Zachary's patch? If not, I suggest to revert changes of issue #15968 to have working Windows buildbots. Victor

2 2