Mailman 3 January 2014 - Python-Dev

Re: [Python-Dev] Argument Clinic: Bug? self converters are not preserved when cloning functions
by Larry Hastings Jan. 21, 2014

Jan. 21, 2014

Please file an issue on the tracker and add me to the nosy list. Do that next time, too; this didn't need to go to python-dev. On Jan 20, 2014 5:46 PM, Tal Einat <taleinat(a)gmail.com> wrote: > > Hi, > > I'm working on converting Objects/bytearray.c and Objects/bytes.c. > > For bytes, the strip methods need a "self converter" so that they get > a PyBytesObject* instead of PyObject*. However, having set this in > bytes.strip and "cloning" that clinic … [View More]

1 0

Argument Clinic: Bug? self converters are not preserved when cloning functions
by Tal Einat Jan. 21, 2014

Jan. 21, 2014

Hi, I'm working on converting Objects/bytearray.c and Objects/bytes.c. For bytes, the strip methods need a "self converter" so that they get a PyBytesObject* instead of PyObject*. However, having set this in bytes.strip and "cloning" that clinic definition for bytes.lstrip and bytes.rstrip, it appears that the self converter wasn't set on lstrip and rstrip. Removing the cloning and copying the argument definitions resolved the issue. Is this a bug? - Tal

1 0

PEP 461 Final?
by Ethan Furman Jan. 20, 2014

Jan. 20, 2014

Here's the text for your reading pleasure. I'll commit the PEP after I add some markup. Major change: - dropped `format` support, just using %-interpolation Coming soon: - Rationale section ;) ================================================================================ PEP: 461 Title: Adding % formatting to bytes Version: $Revision$ Last-Modified: $Date$ Author: Ethan Furman <ethan(a)stoneleaf.us> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 2014-… [View More]01-13 Python-Version: 3.5 Post-History: 2014-01-14, 2014-01-15, 2014-01-17 Resolution: Abstract ======== This PEP proposes adding % formatting operations similar to Python 2's str type to bytes [1]_ [2]_. Overriding Principles ===================== In order to avoid the problems of auto-conversion and Unicode exceptions that could plague Py2 code, all object checking will be done by duck-typing, not by values contained in a Unicode representation [3]_. Proposed semantics for bytes formatting ======================================= %-interpolation --------------- All the numeric formatting codes (such as %x, %o, %e, %f, %g, etc.) will be supported, and will work as they do for str, including the padding, justification and other related modifiers. Example:: >>> b'%4x' % 10 b' a' >>> '%#4x' % 10 ' 0xa' >>> '%04X' % 10 '000A' %c will insert a single byte, either from an int in range(256), or from a bytes argument of length 1, not from a str. Example: >>> b'%c' % 48 b'0' >>> b'%c' % b'a' b'a' %s is restricted in what it will accept:: - input type supports Py_buffer? use it to collect the necessary bytes - input type is something else? use its __bytes__ method; if there isn't one, raise a TypeError Examples: >>> b'%s' % b'abc' b'abc' >>> b'%s' % 3.14 Traceback (most recent call last): ... TypeError: 3.14 has no __bytes__ method >>> b'%s' % 'hello world!' Traceback (most recent call last): ... TypeError: 'hello world' has no __bytes__ method, perhaps you need to encode it? .. note:: Because the str type does not have a __bytes__ method, attempts to directly use 'a string' as a bytes interpolation value will raise an exception. To use 'string' values, they must be encoded or otherwise transformed into a bytes sequence:: 'a string'.encode('latin-1') Numeric Format Codes -------------------- To properly handle int and float subclasses, int(), index(), and float() will be called on the objects intended for (d, i, u), (b, o, x, X), and (e, E, f, F, g, G). Unsupported codes ----------------- %r (which calls __repr__), and %a (which calls ascii() on __repr__) are not supported. Proposed variations =================== It was suggested to let %s accept numbers, but since numbers have their own format codes this idea was discarded. It has been suggested to use %b for bytes instead of %s. - Rejected as %b does not exist in Python 2.x %-interpolation, which is why we are using %s. It has been proposed to automatically use .encode('ascii','strict') for str arguments to %s. - Rejected as this would lead to intermittent failures. Better to have the operation always fail so the trouble-spot can be correctly fixed. It has been proposed to have %s return the ascii-encoded repr when the value is a str (b'%s' % 'abc' --> b"'abc'"). - Rejected as this would lead to hard to debug failures far from the problem site. Better to have the operation always fail so the trouble-spot can be easily fixed. Originally this PEP also proposed adding format style formatting, but it was decided that format and its related machinery were all strictly text (aka str) based, and it was dropped. Various new special methods were proposed, such as __ascii__, __format_bytes___, etc.; such methods are not needed at this time, but can be visited again later if real-world use shows deficiencies with this solution. Footnotes ========= .. [1] http://docs.python.org/2/library/stdtypes.html#string-formatting .. [2] neither string.Template, format, nor str.format are under consideration. .. [3] %c is not an exception as neither of its possible arguments are unicode. Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: ================================================================================ [View Less]

12 31

Migration from Python 2.7 and bytes formatting
by Neil Schemenauer Jan. 19, 2014

Jan. 19, 2014

As I see it, there are two separate goals in adding formatting methods to bytes. One is to make it easier to write new programs that manipulate byte data. Another is to make it easier to upgrade Python 2.x programs to Python 3.x. Here is an idea to better address these separate goals. Introduce %-interpolation for bytes. Support the following format codes to aid in writing new code: %b: insert arbitrary bytes (via __bytes__ or Py_buffer) %[dox]: insert an integer, encoded as … [View More]

4 9

Re: [Python-Dev] PEP 461 updates
by Chris Barker Jan. 17, 2014

Jan. 17, 2014

I hope you didn't mean to take this off-list: On Fri, Jan 17, 2014 at 2:06 PM, Neil Schemenauer <nas(a)arctrix.com> wrote: > In gmane.comp.python.devel, you wrote: > > For the record, we've got a pretty good thread (not this good, though!) > > over on the numpy list about how to untangle the mess that has resulted > > Not sure about your definition of good. ;-) well, in the sense of "big" anyway... > Could you summarize the main points on python-dev? I'm … [View More]not feeling up to > wading through > another massive thread but I'm quite interested to hear the > challenges that numpy deals with. Well, not much new to it, really. But here's a re-cap: numpy has had an 'S' dtype for a while, which corresponded to the py2 string type (except for being fixed length). So it could auto-convert to-from python strings... all was good and happy. Enter py3: what to do? there is no py2 string type anymore. So it was decided to have the 'S' dtype correspond to the py3 bytes type. Apparently there was thought of renaming it, but the 'B' and 'b' type identifiers were already takes, so 'S' was kept. However, as we all know in this thread, the py3 bytes type is not the same thing as a py2 string (or py2 bytes, natch), and folks like to use the 'S' type for text data -- so that is kind of broken in py3. However, other folks use the 'S' type for binary data, so like (and rely on) it being mapped to the py3 bytes type. So we are stuck with that. Given the nature of numpy, and scientific data, there is talk of having a one-byte-per-char text type in numpy (there is already a unicode type, but it uses 4-bytes-per-char, as it's key to the numpy data model that all objects of a given type are the same size.) This would be analogous to the current multiple precision options for numbers. It would take up less memory, and would not be able to hold all values. It's not clear what the level of support is for this right now -- after all, you can do everything you need to do with the appropriate calls to encode() and decode(), if a bit awkward. Meanwhile, back at the ranch -- related, but separate issues have arisen with the functions that parse text files: numpy.loadtxt and numpy.genfromtxt. These functions were adapted for py3 just enough to get things to mostly work, but have some serious limitations when doing anything with unicode -- and in fact do some weird things with plain ascii text files if you ask it to create unicode objects, and that is a natural thing to do (and the "right" thing to do in the Py3 text model) if you do something like: arr = loadtxt('a_file_name', dtype=str) on py3, an str is a py3unicode string, so you get the numpy 'U' datatype but loadtxt wasn't designed to deal with that, so you can get stuff like: ["b'C:\\\\Users\\\\Documents\\\\Project\\\\mytextfile1.txt'" "b'C:\\\\Users\\\\Documents\\\\Project\\\\mytextfile2.txt'" "b'C:\\\\Users\\\\Documents\\\\Project\\\\mytextfile3.txt'"] This was (Presumably, I haven't debugged the code) due to conversion from bytes to unicode...(I'm still confused about the extra slashes) And this ascii text -- it gets worse if there is non-ascii text in there. Anyway, the truth is, this stuff is hard, but it will get at least a touch easier with PEP 461. [though to be truthful, I'm not sure why someone put a comment in the issue tracker about b'%d'%some_num being an issue ... I'm not sure how when we're going from text to numbers, not the other way around...] -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker(a)noaa.gov [View Less]

1 0

Summary of Python tracker Issues
by Python tracker Jan. 17, 2014

Jan. 17, 2014

ACTIVITY SUMMARY (2014-01-10 - 2014-01-17) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 4437 (+28) closed 27624 (+44) total 32061 (+72) Open issues with patches: 2012 Issues opened (47) ================== #14455: plistlib unable to read json and binary plist files http://bugs.python.org/issue14455 reopened by ronaldoussoren #20218: Add `pathlib.… [View More]

1 0

AC Derby and accepting None for optional positional arguments
by Ryan Smith-Roberts Jan. 17, 2014

Jan. 17, 2014

One of the downsides of converting positional-only functions to Argument Clinic is that it can result in misleading docstring signatures. Example: socket.getservbyname(servicename[, protocolname]) -> socket.getservbyname(servicename, protocolname=None) The problem with the new signature is that it indicates passing None for protocolname is the same as omitting it (the other, much larger problem is that it falsely indicates keyword compatibility, but that's a separate indoor elephant). My … [View More]

11 25

Closing the Clinic output format debate (at least for now)
by Larry Hastings Jan. 17, 2014

Jan. 17, 2014

The current tally of votes, by order of popularity: Side file: +6 Buffer: +1.5 Multiple buffers, Modified buffer, Forward buffer: +1 Original: -5 However, as stated, support for "side files" will not go in unless Guido explicitly states that it's okay with him. He has not. Therefore it's not going in. If you want this feature, take it up with our BDFL. I feel my hands are tied. Second-best is all the buffer approaches, collectively. Since there was no clear winner, I'… [View More]

4 5

python code in argument clinic annotations
by Yury Selivanov Jan. 16, 2014

Jan. 16, 2014

The whole discussion of whether clinic should write its output right in the source file (buffered or not), or in a separate sidefile, started because we currently cannot run the clinic during the build process, since it’s written in python. But what if, at some point, someone implements the Tools/clinic.py in pure C, so that integrating it directly in the build process will be possible? In this case, the question is — should we use python code in the argument clinic DSL? If we … [View More]

4 6

Re: [Python-Dev] cpython: asyncio: Fix CoroWrapper (fix my previous commit)
by Antoine Pitrou Jan. 16, 2014

Jan. 16, 2014

On Thu, 16 Jan 2014 01:55:43 +0100 (CET) victor.stinner <python-checkins(a)python.org> wrote: > http://hg.python.org/cpython/rev/f07161c4f3aa > changeset: 88494:f07161c4f3aa > user: Victor Stinner <victor.stinner(a)gmail.com> > date: Thu Jan 16 01:55:29 2014 +0100 > summary: > asyncio: Fix CoroWrapper (fix my previous commit) > > Add __name__ and __doc__ to __slots__ > > files: > Lib/asyncio/tasks.py | 4 +--- > 1 files … [View More]

3 3