Mailman 3 February 2015 - Python-Dev

Re: [Python-Dev] Emit SyntaxWarning on unrecognized backslash escapes?
by Guido van Rossum 24 Feb '15

24 Feb '15

[Adding back python-dev] Actually, I wasn't proposing to change repr() -- my sentiments are similar to Isaac Morland's. Only the error message for the most basic file open() call should be tweaked. No solution is perfect -- but it's probably common enough for someone to create a file or folder named C:\test and being stumped by "cannot open 'C:\test'." On Tue, Feb 24, 2015 at 1:25 PM, Glenn Linderman <v+python(a)g.nevcal.com> wrote: > On 2/24/2015 10:49 AM, Guido van Rossum wrote: > > I like the \x07 solution. > > > The more I think about it, the more I think this is a sufficient > solution. People that use repr to get a Python-compatible string syntax > for further program use get one. People that see them in an error message > are much less likely to be fooled into thinking it is two characters, > because they are thinking of it as two characters to start with. > > On the other hand, I have a directory full of "throw away" experimental > source files named x01, x02, x03, ... :) > > And I suppose extensive use of certain characters in repr in intermediate > or archived data could make such data grow in size. > > And while \t and \n are very commonly used escapes, maybe string repr > could have a special function called > .I_promise_not_to_report_issues_when_I_use_backslash_escapes_in_character_literals_and_get_confused() > which would switch back from \x07 to \t and \x0a to \n, etc. This would be > callable only from __main__ :) > -- --Guido van Rossum (python.org/~guido)

2 1

Fwd: str(IntEnum)
by Demian Brecht 24 Feb '15

24 Feb '15

> On Feb 20, 2015, at 8:54 AM, Brett Cannon <brett(a)python.org> wrote: > Concrete reason. The string is 'MyEnum.FOO' which is much more readable and obvious where the value came from. The fact that it can be treated as an int is the same as the reason True and False are subclasses of int; it made practical sense for compatibility with what they typically replaced, but where it made more sense to diverge and introduce new behaviour then we did so. Thanks Brett, that makes sense and was pretty much what I assumed. Reading the docs for __str__ does clarify things a bit in that the intention is to be a string representation of the object and not the value. That said, implementations seem to vary throughout the standard library and builtins: >>> str(uuid.uuid4()) 'd467d97c-fc09-4dc9-bea5-aeebdad8df8d’ >>> str(int()) ‘0' >>> str(datetime.datetime.now()) '2015-02-20 09:31:28.385539’ >>> str(IPv4Address('127.0.0.1')) '127.0.0.1' These and other implementations return a string representation of the instance’s value, not a string representation of the object itself. Whereas elsewhere in the standard library: >>> str(ProtocolError('url', 42, 'msg', '')) '<ProtocolError for url: 42 msg>’ >>> str(URLError('reason')) '<urlopen error reason>’ >>> str(Cookie(0, '', '', '4', '', '', '', '', '', '', '', 0, '', '', '', '')) '<Cookie = for :4>' The specific problem that I encountered was when swapping an IntEnum implementation for ints in http.client, which caused a change in logging output (from int.__str__ to Enum.__str__), which was a bit of a surprise, especially given the value is a builtin type. I think that a decent rule around the usage of __str__ is that it should be a string representation of the value, not of the object. Failing the ability to logically coerce the value to a string, it should simply fall back to repr(obj). Of course, I realize that the chances of this change being made to such a fundamental (and largely inconsequential) feature are likely nil, but I thought I’d share my thoughts anyways.

5 6

Request for Pronouncement: PEP 441 - Improving Python ZIP Application Support
by Paul Moore 24 Feb '15

24 Feb '15

The discussion on PEP 441 (started in thread https://mail.python.org/pipermail/python-dev/2015-February/138277.html and on the issue tracker at http://bugs.python.org/issue23491) seems to have mostly settled down. I don't think there are any outstanding tasks, so I'm happy that the PEP is ready for pronouncement. I've included the latest copy of the PEP inline below, for reference. (Can I also add a gentle reminder that PEP 486 is ready for pronouncement? It's be nice to get these into 3.5 alpha 2, which is due at the end of next week...) Paul PEP: 441 Title: Improving Python ZIP Application Support Version: $Revision$ Last-Modified: $Date$ Author: Daniel Holth <dholth(a)gmail.com>, Paul Moore <p.f.moore(a)gmail.com> Discussions-To: https://mail.python.org/pipermail/python-dev/2015-February/138277.html Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 30 March 2013 Post-History: 30 March 2013, 1 April 2013, 16 February 2015 Improving Python ZIP Application Support ======================================== Python has had the ability to execute directories or ZIP-format archives as scripts since version 2.6 [1]_. When invoked with a zip file or directory as its first argument the interpreter adds that directory to sys.path and executes the ``__main__`` module. These archives provide a great way to publish software that needs to be distributed as a single file script but is complex enough to need to be written as a collection of modules. This feature is not as popular as it should be mainly because it was not promoted as part of Python 2.6 [2]_, so that it is relatively unknown, but also because the Windows installer does not register a file extension (other than ``.py``) for this format of file, to associate with the launcher. This PEP proposes to fix these problems by re-publicising the feature, defining the ``.pyz`` and ``.pyzw`` extensions as "Python ZIP Applications" and "Windowed Python ZIP Applications", and providing some simple tooling to manage the format. A New Python ZIP Application Extension ====================================== The terminology "Python Zip Application" will be the formal term used for a zip-format archive that contains Python code in a form that can be directly executed by Python (specifically, it must have a ``__main__.py`` file in the root directory of the archive). The extension ``.pyz`` will be formally associated with such files. The Python 3.5 installer will associate ``.pyz`` and ``.pyzw`` "Python Zip Applications" with the platform launcher so they can be executed. A ``.pyz`` archive is a console application and a ``.pyzw`` archive is a windowed application, indicating whether the console should appear when running the app. On Unix, it would be ideal if the ``.pyz`` extension and the name "Python Zip Application" were registered (in the mime types database?). However, such an association is out of scope for this PEP. Python Zip applications can be prefixed with a ``#!`` line pointing to the correct Python interpreter and an optional explanation:: #!/usr/bin/env python3 # Python application packed with zipapp module (binary contents of archive) On Unix, this allows the OS to run the file with the correct interpreter, via the standard "shebang" support. On Windows, the Python launcher implements shebang support. However, it is always possible to execute a ``.pyz`` application by supplying the filename to the Python interpreter directly. As background, ZIP archives are defined with a footer containing relative offsets from the end of the file. They remain valid when concatenated to the end of any other file. This feature is completely standard and is how self-extracting ZIP archives and the bdist_wininst installer format work. Minimal Tooling: The zipapp Module ================================== This PEP also proposes including a module for working with these archives. The module will contain functions for working with Python zip application archives, and a command line interface (via ``python -m zipapp``) for their creation and manipulation. More complete tools for managing Python Zip Applications are encouraged as 3rd party applications on PyPI. Currently, pyyzer [5]_ and pex [6]_ are two tools known to exist. Module Interface ---------------- The zipapp module will provide the following functions: ``pack(target, directory, interpreter=None, main=None)`` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Writes an application archive called *target*, containing the contents of *directory*. If *interpreter* is specified, it will be written to the start of the archive as a shebang line and the file will be made executable (if no interpreter is specified, the shebang line will be omitted). If the directory contains no ``__main__.py`` file, the function will construct a ``__main__.py`` which calls the function specified in the *main* argument (which should be in the form ``'pkg.mod:fn'``). It is an error to specify *main* if the directory contains a ``__main__.py``, or to omit *main* when there is no ``__main__.py`` (as that will result in an archive which has no main function and so cannot be executed). ``get_interpreter(archive)`` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Returns the interpreter specified in the shebang line of the *archive*. If there is no shebang, the function returns ``None``. ``set_interpreter(archive, new_archive, interpreter=None)`` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Modifies the *archive*'s shebang line to contain the specified interpreter, and writes the updated archive to *new_archive*. If the *interpreter* is ``None``, removes the shebang line. Command Line Usage ------------------ The zipapp module can be run with the python ``-m`` flag. The command line interface is as follows:: python -m zipapp [options] directory Create an archive from the contents of the given directory. By default, an archive will be created with the same name as the source directory, with a .pyz extension. The following options can be specified: -o archive / --output archive The destination archive will have the specified name. -p interpreter / --python interpreter The given interpreter will be written to the shebang line of the archive. If this option is not given, the archive will have no shebang line. -m pkg.mod:fn / --main pkg.mod:fn The source directory must not have a __main__.py file. The archiver will write a __main__.py file into the target which calls fn from the module pkg.mod. The behaviour of the command line interface matches that of ``zipapp.pack()``. As noted, the archives are standard zip files, and so can be unpacked using any standard ZIP utility or Python's zipfile module. FAQ --- Are you sure a standard ZIP utility can handle ``#!`` at the beginning? Absolutely. The zipfile specification allows for arbitrary data to be prepended to a zipfile. This feature is commonly used by "self-extracting zip" programs. If your archive program can't handle this, it is a bug in your archive program. Isn't zipapp just a very thin wrapper over the zipfile module? Yes. If you prefer to build your own Python zip application archives using other tools, they will work just as well. The zipapp module is a convenience, nothing more. Why not use just use a ``.zip`` or ``.py`` extension? Users expect a ``.zip`` file to be opened with an archive tool, and expect a ``.py`` file to contain readable text. Both would be confusing for this use case. How does this compete with existing package formats? The sdist, bdist and wheel formats are designed for packaging of modules to be installed into an existing Python installation. They are not intended to be used without installing. The executable zip format is specifically designed for standalone use, without needing to be installed. They are in effect a multi-file version of a standalone Python script. Rejected Proposals ================== Convenience Values for Shebang Lines ------------------------------------ Is it worth having "convenience" forms for any of the common interpreter values? For example, ``-p 3`` meaning the same as ``-p "/usr/bin/env python3"``. It would save a lot of typing for the common cases, as well as giving cross-platform options for people who don't want or need to understand the intricacies of shebang handling on "other" platforms. Downsides are that it's not obvious how to translate the abbreviations. For example, should "3" mean "/usr/bin/env python3", "/usr/bin/python3", "python3", or something else? Also, there is no obvious short form for the key case of "/usr/bin/env python" (any available version of Python), which could easily result in scripts being written with overly-restrictive shebang lines. Overall, this seems like there are more problems than benefits, and as a result has been dropped from consideration. Registering ``.pyz`` as a Media Type -------------------------------- It was suggested [3]_ that the ``.pyz`` extension should be registered in the Unix database of extensions. While it makes sense to do this as an equivalent of the Windows installer registering the extension, the ``.py`` extension is not listed in the media types database [4]_. It doesn't seem reasonable to register ``.pyz`` without ``.py``, so this idea has been omitted from this PEP. An interested party could arrange for *both* ``.py`` and ``.pyz`` to be registered at a future date. Default Interpreter ------------------- The initial draft of this PEP proposed using ``/usr/bin/env python`` as the default interpreter. Unix users have problems with this behaviour, as the default for the python command on many distributions is Python 2, and it is felt that this PEP should prefer Python 3 by default. However, using a command of ``python3`` can result in unexpected behaviour for Windows users, where the default behaviour of the launcher for the command ``python`` is commonly customised by users, but the behaviour of ``python3`` may not be modified to match. As a result, the principle "in the face of ambiguity, refuse to guess" has been invoked, and archives have no shebang line unless explicitly requested. On Windows, the archives will still be run (with the default Python) by the launcher, and on Unix, the archives can be run by explicitly invoking the desired Python interpreter. Command Line Tool to Manage Shebang Lines ----------------------------------------- It is conceivable that users would want to modify the shebang line for an existing archive, or even just display the current shebang line. This is tricky to do so with existing tools (zip programs typically ignore prepended data totally, and text editors can have trouble editing files containing binary data). The zipapp module provides functions to handle the shebang line, but does not include a command line interface to that functionality. This is because it is not clear how to provide one without the resulting interface being over-complex and potentially confusing. Changing the shebang line is expected to be an uncommon requirement. Reference Implementation ======================== A reference implementation is at http://bugs.python.org/issue23491. References ========== .. [1] Allow interpreter to execute a zip file (http://bugs.python.org/issue1739468) .. [2] Feature is not documented (http://bugs.python.org/issue17359) .. [3] Discussion of adding a .pyz mime type on python-dev (https://mail.python.org/pipermail/python-dev/2015-February/138338.html) .. [4] Register of media types (http://www.iana.org/assignments/media-types/media-types.xhtml) .. [5] pyzzer - A tool for creating Python-executable archives (https://pypi.python.org/pypi/pyzzer) .. [6] pex - The PEX packaging toolchain (https://pypi.python.org/pypi/pex) The discussion of this PEP took place on the python-dev mailing list, in the thread starting at https://mail.python.org/pipermail/python-dev/2015-February/138277.html Copyright ========= This document has been placed into the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End:

8 24

Re: [Python-Dev] easy_install ?
by Laura Creighton 24 Feb '15

24 Feb '15

In a message of Tue, 24 Feb 2015 16:44:20 +0000, Paul Moore writes: >On 24 February 2015 at 16:30, Brett Cannon <brett(a)python.org> wrote: >> Tell people to use pip. Having ensurepip in Python 2.7 and 3.4 makes it as >> official as anything will be as the recommended tool to install projects. >> Otherwise easy_install has nothing to do directly with python-dev so I don't >> think we can comment on a group as to what people should do in terms of >> bugs, etc. > >Bug reports would go to the setuptools project, but I agree, "use pip" >is a better suggestion. And if pip won't work, it would be good to >know why. > >Paul I want to recommend that people not use easy_install. This has been my personal preferred solution for many years now. But I did not want to say such a thing if there was a known use case, one that I never have, that made easy_install better. Ok. will go back to my personal integrity which matches what python-dev recommends. Thank you for answering. Laura (answering questions in python-list and as webmaster)

1 0

Removing Python Binaries with make command
by Blaxton 23 Feb '15

23 Feb '15

Hi I have installed Python3 with running ./configure , make and then make installand now can't find an option in Makefile to remove the binaries with make command. Is there a way to cleanly remove Python3 Binaries after compiling on a system ? ThanksDB

4 3

Re: [Python-Dev] cpython: Issue #23152: Renames attribute_data_to_stat to _Py_attribute_data_to_stat
by Serhiy Storchaka 22 Feb '15

22 Feb '15

On 21.02.15 20:04, steve.dower wrote: > https://hg.python.org/cpython/rev/307713759a62 > changeset: 94720:307713759a62 > parent: 94718:4f6f4aa0d80f > user: Steve Dower <steve.dower(a)microsoft.com> > date: Sat Feb 21 10:04:10 2015 -0800 > summary: > Issue #23152: Renames attribute_data_to_stat to _Py_attribute_data_to_stat What about time_t_to_FILE_TIME?

2 5

Generate all Argument Clinic code into separate files
by Serhiy Storchaka 22 Feb '15

22 Feb '15

Currently for some files converted to use Argument Clinic the generated code is written into a separate file, for other files it is inlined. I'm going to make a number of small enhancement to Argument Clinic to generate faster code for parsing arguments (see for example issue23492). Every such step will produce large diffs for generated code and will create code churn if it is inlined and mixed up with handwritten code. It would be better when only generated files will be changed. So I suggest to move all inlined generated code in separate file. What are you think?

3 3

Enum is really serializable using json?
by Facundo Batista 22 Feb '15

22 Feb '15

We have this issue closed as resolved: http://bugs.python.org/issue18264 It's called "enum.IntEnum is not compatible with JSON serialisation", and it looks that after a long conversation they added proper support for it in Py3.4. However, this is still failing: Python 3.4.2 (default, Oct 8 2014, 13:18:07) [GCC 4.9.1] on linux >>> import enum, json >>> Foo = enum.Enum('Foo', 'bar baz') >>> json.dumps(Foo.bar) Traceback (most recent call last): ... TypeError: <Foo.bar: 1> is not JSON serializable Am I failing to understand the bug tracker saying the patch was applied for Py3.4? (thought it was more polite to ask here than to reopen the whole thread in the issue tracker) Thanks! -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ Twitter: @facundobatista

3 3

TypeError messages
by Serhiy Storchaka 22 Feb '15

22 Feb '15

Different patterns for TypeError messages are used in the stdlib: expected X, Y found expected X, found Y expected X, but Y found expected X instance, Y found X expected, not Y expect X, not Y need X, Y found X is required, not Y Z must be X, not Y Z should be X, not Y and more. What the pattern is most preferable? Some messages use the article before X or Y. Should the article be used or omitted? Some messages (only in C) truncate actual type name (%.50s, %.80s, %.200s, %.500s). Should type name be truncated at all and for how limit? Type names newer truncated in TypeError messages raised in Python code. Some messages enclose actual type name with single quotes ('%s', '%.200s'). Should type name be quoted? It is uncommon if type name contains spaces.

12 23

[Python-checkins] cpython: Issue #23152: Implement _Py_fstat() to support files larger than 2 GB on
by Ben Hoyt 21 Feb '15

21 Feb '15

When merging some changes while working on scandir, I noticed a minor issue with this commit: https://hg.python.org/cpython/rev/4f6f4aa0d80f The definition of "struct win32_stat" has been moved to fileutils.h and renamed to "struct _Py_stat_struct", which is fine -- however, the old "struct win32_stat" definition is still present (but unused) in posixmodule.c. So I think the old "struct win32_stat { ... }" definition can simply be removed from posixmodule.c now. Also, unrelated to this commit, I notice the _Py_attribute_data_to_stat function (was attribute_data_to_stat) can't fail and always returns 0, and all callers ignore its return value anyway. Can it be changed to return void? -Ben

3 2