Mailman 3 September 2017 - Python-ideas

Adding new lines to "Zen of Python"
by Thomas Güttler 11 Sep '17

11 Sep '17

I curious if there are any plans to update the "Zen of Python". What could be added to the "Zen of Python"? What do you think? Regards, Thomas Güttler -- Thomas Guettler http://www.thomas-guettler.de/ I am looking for feedback: https://github.com/guettli/programming-guidelines

7 7

PEP 561: Distributing and Packaging Type Information
by Ethan Smith 11 Sep '17

11 Sep '17

Hello, I have just published my first PEP, on packaging type information. I would appreciate comments and suggestions. The PEP can be found at https://www.python.org/dev/peps/pep-0561/ I have also duplicated the text below. Thanks! ---------------------------------------------------------- PEP: 561 Title: Distributing and Packaging Type Information Author: Ethan Smith <ethan(a)ethanhs.me> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 09-Sep-2017 Python-Version: 3.7 Post-History: Abstract ======== PEP 484 introduced type hints to Python, with goals of making typing gradual and easy to adopt. Currently, typing information must be distributed manually. This PEP provides a standardized means to package and distribute type information and an ordering for type checkers to resolve modules and collect this information for type checking using existing packaging architecture. Rationale ========= PEP 484 has a brief section on distributing typing information. In this section [1]_ the PEP recommends using ``shared/typehints/pythonX.Y/`` for shipping stub files. However, manually adding a path to stub files for each third party library does not scale. The simplest approach people have taken is to add ``site-packages`` to their ``PYTHONPATH``, but this causes type checkers to fail on packages that are highly dynamic (e.g. sqlalchemy and Django). Furthermore, package authors are wishing to distribute code that has inline type information, and there currently is no standard method to distribute packages with inline type annotations or syntax that can simultaneously be used at runtime and in type checking. Specification ============= There are several motivations and methods of supporting typing in a package. This PEP recognizes three (3) types of packages that may be created: 1. The package maintainer would like to add type information inline. 2. The package maintainer would like to add type information via stubs. 3. A third party would like to share stub files for a package, but the maintainer does not want to include them in the source of the package. This PEP aims to support these scenarios and make them simple to add to packaging and deployment. The two major parts of this specification are the packaging specifications and the resolution order for resolving module type information. This spec is meant to replace the ``shared/typehints/pythonX.Y/`` spec of PEP 484 [1]_. Packaging Type Information -------------------------- Packages must opt into supporting typing. This will be done though a distutils extension [2]_, providing a ``typed`` keyword argument to the distutils ``setup()`` command. The argument value will depend on the kind of type information the package provides. The distutils extension will be added to the ``typing`` package. Therefore a package maintainer may write :: setup( ... setup_requires=["typing"], typed="inline", ... ) Inline Typed Packages ''''''''''''''''''''' Packages that have inline type annotations simply have to pass the value ``"inline"`` to the ``typed`` argument in ``setup()``. Stub Only Packages '''''''''''''''''' For package maintainers wishing to ship stub files containing all of their type information, it is prefered that the ``*.pyi`` stubs are alongside the corresponding ``*.py`` files. However, the stubs may be put in a sub-folder of the Python sources, with the same name the ``*.py`` files are in. For example, the ``flyingcircus`` package would have its stubs in the folder ``flyingcircus/flyingcircus/``. This path is chosen so that if stubs are not found in ``flyingcircus/`` the type checker may treat the subdirectory as a normal package. The normal resolution order of checking ``*.pyi`` before ``*.py`` will be maintained. The value of the ``typed`` argument to ``setup()`` is ``"stubs"`` for this type of distribution. The author of the package is suggested to use ``package_data`` to assure the stub files are installed alongside the runtime Python code. Third Party Stub Packages ''''''''''''''''''''''''' Third parties seeking to distribute stub files are encouraged to contact the maintainer of the package about distribution alongside the package. If the maintainer does not wish to maintain or package stub files or type information inline, then a "third party stub package" should be created. The structure is similar, but slightly different from that of stub only packages. If the stubs are for the library ``flyingcircus`` then the package should be named ``flyingcircus-stubs`` and the stub files should be put in a sub-directory named ``flyingcircus``. This allows the stubs to be checked as if they were in a regular package. These packages should also pass ``"stubs"`` as the value of ``typed`` argument in ``setup()``. These packages are suggested to use ``package_data`` to package stub files. The version of the ``flyingcircus-stubs`` package should match the version of the ``flyingcircus`` package it is providing types for. Type Checker Module Resolution Order ------------------------------------ The following is the order that type checkers supporting this PEP should resolve modules containing type information: 1. User code - the files the type checker is running on. 2. Stubs or Python source in ``PYTHONPATH``. This is to allow the user complete control of which stubs to use, and patch broken stubs/inline types from packages. 3. Third party stub packages - these packages can supersede the installed untyped packages. They can be found at ``pkg-stubs`` for package ``pkg``, however it is encouraged to check their metadata to confirm that they opt into type checking. 4. Inline packages - finally, if there is nothing overriding the installed package, and it opts into type checking. 5. Typeshed (if used) - Provides the stdlib types and several third party libraries When resolving step (3) type checkers should assure the version of the stubs match the installed runtime package. Type checkers that check a different Python version than the version they run on must find the type information in the ``site-packages``/``dist-packages`` of that Python version. This can be queried e.g. ``pythonX.Y -c 'import sys; print(sys.exec_prefix)'``. It is also recommended that the type checker allow for the user to point to a particular Python binary, in case it is not in the path. To check if a package has opted into type checking, type checkers are recommended to use the ``pkg_resources`` module to query the package metadata. If the ``typed`` package metadata has ``None`` as its value, the package has not opted into type checking, and the type checker should skip that package. References ========== .. [1] PEP 484, Storing and Distributing Stub Files (https://www.python.org/dev/peps/pep-0484/#storing-and-distributing-stub-fil…) .. [2] Distutils Extensions, Adding setup() arguments (http://setuptools.readthedocs.io/en/latest/setuptools.html#adding-setup-arg…) Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End:

2 4

Re: [Python-ideas] PEP 562
by Neil Schemenauer 10 Sep '17

10 Sep '17

On 2017-09-10, Neil Schemenauer wrote: > I have something 90% working, only 90% left to go. ;-) Prototype: https://github.com/warsaw/lazyimport/blob/master/lazy_demo.py https://github.com/nascheme/cpython/tree/exec_mod Next step is to do the compiler and change importlib to do exec(code, module) rather than exec(code, module.__dict__).

1 0

PEP 560
by Ivan Levkivskyi 10 Sep '17

10 Sep '17

I have written another short PEP that proposes some minor changes to core CPython interpreter for better support of generic types. I will be grateful for comments and suggestions: https://www.python.org/dev/peps/pep-0560/ -- Ivan

1 0

factory for efficient creation of many dicts with the same keys
by Sergey Fedoseev 08 Sep '17

08 Sep '17

Hi all, Sometimes you may need to create many dicts with the same keys, but different values. For example, if you want to return data from DB as dicts. I think that special type could be added to solve this task more effectively. I created proof of concept for this and here's benchmarks: # currently the fastest way to do it AFAIK $ ./python -m timeit -s "nkeys = 5; nrows = 1000; rows = [(i,)*nkeys for i in range(nrows)]; enumerated = list(enumerate(range(nkeys)))" "for row in rows: {key: row[i] for i, key in enumerated}" 500 loops, best of 5: 645 usec per loop $ ./python -m timeit -s "nkeys = 5; nrows = 1000; rows = [(i,)*nkeys for i in range(nrows)]; factory = dict.factory(*range(nkeys)); from itertools import starmap" "for d in starmap(factory, rows): d" 5000 loops, best of 5: 81.1 usec per loop I'd like to write a patch if this idea will be accepted.

3 3

Add a more disciplined ways of registering ABCs
by אלעזר 08 Sep '17

08 Sep '17

Hi all, tl;dr: I propose adding a `register()` decorator, to be used like this: @abc.register(Abc1, Abc2) class D: ... For preexisting classes I propose adding a magic static variable `__registered__`, to be handled by ABCMeta: class Abc1(metaclass=ABCMeta): __registered__ = [D] Both forms has the effects of calling ABCMeta.register with the corresponding parameters. Explanation: It is possible to register an abstract base class B for preexisting classes A using B.register(A). This form is very flexible - probably too flexible, since the operation changes the type of A, and doing it dynamically makes it difficult for type checker (and human beings) to analyze. But what about new classes? Registering a new class is possible using B.register as a decorator, since it already its argument: @Iterable.register @Sequence.register class A: pass However, due to restrictions on the expressions allowed inside a decorator, this would not work with generic ABCs, which are common and heavily used in type-checked projects: >>> @Iterable[str].register File "<stdin>", line 1 @Iterable[str].register ^ SyntaxError: invalid syntax While it might be possible to infer the argument, it is not always the case, and adds additional unnecessary burden to type-checker and to the reader; discouraging its use. This kind of restrictions may also prevent some other conceivable forms, such as `(a)type(X).register`. Additionally, `abc.register()` is also easier to analyze as a "syntactic construct" from the point of view the type checker, similar to the way checkers handle some other constructs such as `enum()`, `cast()` and others. Finally, there's uninformative repetition of `register` for multiple ABCs, and waste of vertical screen space. Q: Why not subclass B? A: Since it forces the metaclass of A to be (a subclass of) ABCMeta, which can cause inconsistency in metaclass structure for derived classes. This issue can surface when using Enum (whose metaclass is EnumMeta): >>> class A(Iterable, Enum): pass ... Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases This causes problems in typeshed, since it uses inheritance to register `str` as a `Sequence[str]`, for example. As a result, it affects users of mypy trying to define `class A(str, Enum): ...` Q: Why not add to the typing module? A: Because registering an abstract base class has runtime meaning, such as the result of `isinstance` checks. Q: Why not add to the standard library? why not some 3rd party module? A: 1. Because dynamic usage should be discouraged, similarly to monkey-patching. Note that, like monkey-patching, this has runtime implications which cause "actions from distance". 2. Because it should be used by typeshed and mypy. The implementation of register() is straightforward: def register(*abcs): def inner(cls): for b in abcs: b.register(cls) return cls return inner Thanks, Elazar

1 0

Lazy creation of module level functions and classes
by Neil Schemenauer 08 Sep '17

08 Sep '17

This is an idea that come out of the lazy loading modules idea. Larry Hastings mentioned what a good improvement this was for PHP. I think it would help a lot of Python too. Very many functions and classes are not actually needed but are instantiated anyhow. Back of napkin idea: Write AST transformer tool, change top-level functions and classes to be like properties (can use __class__ I guess) Transform is something like: # old code def inc(x): return x + 1 # transformed code def __make_inc(code=<code object bytes>): obj = eval(code) _ModuleClass.inc = obj # only do eval once return obj inc = property(__make_inc) Totally seat of pants idea but I can't think of a reason why it shouldn't work. It seems much more powerful than lazying loading modules. In the lazy module case, you load the whole module if any part is touched. Many modules only have a small fraction of their functions and classes actually used. If this transformer idea works, the standard Python compiler could be changed to do the above stuff, no transformer needed.

1 0

Re: [Python-ideas] Extension of python/json syntax to support explicitly sets and ordered dict.
by rymg19＠gmail.com 08 Sep '17

08 Sep '17

IIRC in CPython 3.6 and PyPy dicts are ordered based on insertion anyway; although it's an implementation-specific detail, realistically it removes some of the use cases for ordered dictionary literals. -- Ryan (ライアン) Yoko Shimomura, ryo (supercell/EGOIST), Hiroyuki Sawano >> everyone elsehttp://refi64.com On Sep 7, 2017 at 6:40 AM, <Matteo Nastasi <nastasi(a)alternativeoutput.it>> wrote: Hi all, few days ago I thought about a way to rapresent sets and ordered dicts using a json compatible syntax, these are my conclusions: A set could be defined as { item1, item2, item3[...] } with {,} as an empty set An ordered dict could be defined as [ item1: value1, item2: value2 ... ] with [:] ase an empty odered dict It could be used inside python code or to serialize python structures in a json-like format (pyson maybe ?). What do you think about ? Kind regards, Matteo. -- email: nastasi(a)alternativeoutput.it, matteo.nastasi(a)gmail.com web: www.alternativeoutput.it irc: #linux-mi(a)irc.freenode.net linkedin: http://lnkd.in/SPQG87 _______________________________________________ Python-ideas mailing list Python-ideas(a)python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

2 1

Adding "View Python 3 Documentation" to all Python 2 documentation URLs
by Ryan Gonzalez 08 Sep '17

08 Sep '17

Right now, many Google searches for Python modules return the Python 2 documentation. IMO since 2 will be reaching EOL in around 3 years, it would be nice to have a giant red box at the top with a link to the Python 3 documentation. SFML already does something like this: https://www.sfml-dev.org/tutorials/2.3/ (I mean, it would be even nicer to have a "jump to latest version" for *all* not-new Python versions, though I figured just 2 would be a lot easier.) -- Ryan (ライアン) Yoko Shimomura, ryo (supercell/EGOIST), Hiroyuki Sawano >> everyone else http://refi64.com/

9 10

if <expression> as <variable>
by Denis Krienbühl 07 Sep '17

07 Sep '17

Hi! I’ve been having this idea for a few years and I thought I finally see if what others think of it. I have no experience in language design and I don’t know if this is something I’ve picked up in some other language. I also do not know what the ramifications of implementing this idea would be. I just keep thinking about it :) I quite often write code like the following in python: result = computation() if result: do_something_with_computation(result) More often than not this snippet evolves from something like this: if computation(): … That is, I use the truthiness of the value at first. As the code grows I refactor to actually do something with the result. What I would love to see is the following syntax instead, which to me is much cleaner: if computation() as result: do_something_with_result(result) Basically the result variable would be the result of the if condition’s expression and it would be available the same way it would be if we used my initial snippet (much the same way that the result of with expressions also stays around outside the with-block). Any feedback is appreciated :) Cheers, Denis

6 7

Python-ideas September 2017