Mailman 3 September 2018 - Python-ideas

Retire or reword the "Beautiful is better than ugly" Zen clause
by Samantha Quan 17 Sep '18

17 Sep '18

31 56

Re: [Python-ideas] SEC: Spectre variant 2: GCC: -mindirect-branch=thunk -mindirect-branch-register
by Wes Turner 17 Sep '18

17 Sep '18

Are all current Python builds and C extensions vulnerable to Spectre variants {1, 2, *}? There are now multiple threads: "SEC: Spectre variant 2: GCC: -mindirect-branch=thunk -mindirect-branch-register" - https://mail.python.org/mm3/archives/list/distutils-sig@python.org/thread/4… - https://mail.python.org/pipermail/python-ideas/2018-September/053473.html - https://mail.python.org/pipermail/python-dev/2018-September/155199.html Original thread (that I forwarded to security@): "[Python-ideas] Executable space protection: NX bit," https://mail.python.org/pipermail/python-ideas/2018-September/053175.html > ~ Do trampolines / nested functions in C extensions switch off the NX bit? On Sunday, September 16, 2018, Nathaniel Smith <njs(a)pobox.com> wrote: > On Wed, Sep 12, 2018, 12:29 Joni Orponen <j.orponen(a)4teamwork.ch> wrote: > >> On Wed, Sep 12, 2018 at 8:48 PM Wes Turner <wes.turner(a)gmail.com> wrote: >> >>> Should C extensions that compile all add >>> `-mindirect-branch=thunk -mindirect-branch-register` [1] to mitigate the >>> risk of Spectre variant 2 (which does indeed affect user space applications >>> as well as kernels)? >>> >> >> Are those available on GCC <= 4.2.0 as per PEP 513? >> > > Pretty sure no manylinux1 compiler is ever going to get these mitigations. > > For manylinux2010 on x86-64, we can easily use a much newer compiler: RH > maintains a recent compiler, currently gcc 7.3, or if that doesn't work for > some reason then the conda folks have be apparently figured out how to > build the equivalent from gcc upstream releases. > Are there different CFLAGS and/or gcc compatibility flags in conda builds of Python and C extensions? Where are those set in conda builds? What's the best way to set CFLAGS in Python builds and C extensions? export CFLAGS="-mindirect-branch=thunk -mindirect-branch-register" ./configure make ? Why are we supposed to use an old version of GCC that doesn't have the retpoline patches that only mitigate Spectre variant 2? > > Unfortunately, the manylinux2010 infrastructure is not quite ready... I'm > pretty sure it needs some volunteers to push it to the finish line, though > unfortunately I haven't had enough time to keep track. > "PEP 571 -- The manylinux2010 Platform Tag" https://www.python.org/dev/peps/pep-0571/ "Tracking issue for manylinux2010 rollout" https://github.com/pypa/manylinux/issues/179 Are all current Python builds and C extensions vulnerable to Spectre variants {1, 2, *}? >

1 0

SEC: Spectre variant 2: GCC: -mindirect-branch=thunk -mindirect-branch-register
by Wes Turner 16 Sep '18

16 Sep '18

Should Python builds add `-mindirect-branch=thunk -mindirect-branch-register` to CFLAGS? Where would this be to be added in the build scripts with which architectures? /QSpectre is the MSVC build flag for Spectre Variant 1: > The /Qspectre option is available in Visual Studio 2017 version 15.7 and later. https://docs.microsoft.com/en-us/cpp/build/reference/qspectre?view=vs-2017 security@ directed me to the issue tracker / lists, so I'm forwarding this to python-dev and python-ideas, as well. # Forwarded message From: *Wes Turner* <wes.turner(a)gmail.com> Date: Wednesday, September 12, 2018 Subject: SEC: Spectre variant 2: GCC: -mindirect-branch=thunk -mindirect-branch-register To: distutils-sig <distutils-sig(a)python.org> Should C extensions that compile all add `-mindirect-branch=thunk -mindirect-branch-register` [1] to mitigate the risk of Spectre variant 2 (which does indeed affect user space applications as well as kernels)? [1] https://github.com/speed47/spectre-meltdown-checker/ issues/119#issuecomment-361432244 [2] https://en.wikipedia.org/wiki/Spectre_(security_vulnerability) [3] https://en.wikipedia.org/wiki/Speculative_Store_Bypass# Speculative_execution_exploit_variants On Wednesday, September 12, 2018, Wes Turner <wes.turner(a)gmail.com> wrote: > >> On Wednesday, September 12, 2018, Joni Orponen <j.orponen(a)4teamwork.ch> >> wrote: >> >>> On Wed, Sep 12, 2018 at 8:48 PM Wes Turner <wes.turner(a)gmail.com> wrote: >>> >>>> Should C extensions that compile all add >>>> `-mindirect-branch=thunk -mindirect-branch-register` [1] to mitigate >>>> the risk of Spectre variant 2 (which does indeed affect user space >>>> applications as well as kernels)? >>>> >>> >>> Are those available on GCC <= 4.2.0 as per PEP 513? >>> >> >> AFAIU, only >> GCC 7.3 and 8 have the retpoline (indirect-branch=thunk) support enabled >> by the `-mindirect-branch=thunk -mindirect-branch-register` CFLAGS. >> > On Wednesday, September 12, 2018, Wes Turner <wes.turner(a)gmail.com> wrote: > "What is a retpoline and how does it work?" > https://stackoverflow.com/questions/48089426/what-is-a- > retpoline-and-how-does-it-work > >

1 1

+1 Pre-conditions and post-conditions by Kaufmann
by James Lu 16 Sep '18

16 Sep '18

In response to your Sat, 15 Sep 2018 22:14:43: A good and thoughtful read. I agree with all your points. +1.

1 0

Improving fn(arg=arg, name=name, wibble=wibble) code
by Jonathan Fine 14 Sep '18

14 Sep '18

I thank Steve D'Aprano for pointing me to this real-life (although perhaps extreme) code example https://github.com/Tinche/aiofiles/blob/master/aiofiles/threadpool/__init__… <code> def open(file, mode='r', buffering=-1, encoding=None, errors=None, newline=None, closefd=True, opener=None, *, loop=None, executor=None): return AiofilesContextManager(_open(file, mode=mode, buffering=buffering, encoding=encoding, errors=errors, newline=newline, closefd=closefd, opener=opener, loop=loop, executor=executor)) @asyncio.coroutine def _open(file, mode='r', buffering=-1, encoding=None, errors=None, newline=None, closefd=True, opener=None, *, loop=None, executor=None): """Open an asyncio file.""" if loop is None: loop = asyncio.get_event_loop() cb = partial(sync_open, file, mode=mode, buffering=buffering, encoding=encoding, errors=errors, newline=newline, closefd=closefd, opener=opener) f = yield from loop.run_in_executor(executor, cb) return wrap(f, loop=loop, executor=executor) </code> Anders Hovmöller has proposed a Python syntax extension to improve this code. It provides, for example return wrap(f, *, loop, executor) as a shorthand for return wrap(f, loop=loop, executor=executor) See: https://mail.python.org/pipermail/python-ideas/2018-September/053207.html I'd like us, in this thread, to discuss OTHER possible ways of improving this code. This could include refactoring, and the introduction of tools. I'm particularly interested in gathering alternatives, and at this time not much interesting in "knowing which one is best". -- Jonathan

5 30

__iter__(), keys(), and the mapping protocol
by Elias Tarhini 13 Sep '18

13 Sep '18

This has been bouncing around in my head for a while regarding the requisite keys() method on mappings: How come the ** unpacking operator, a built-in language feature, relies on a non-dunder to operate? To me, I mean to say, requiring that classes implement keys() – a method whose name is totally undistinguished – in order to conform to the mapping protocol feels like a design running counter to Python's norm of using dunders for everything "hidden". I am not sure if it feels dirty to anybody else, however. Interestingly, the docs already say <https://docs.python.org/3/reference/datamodel.html#object.__iter__> that *[f]or mappings, [__iter__()] should iterate over the keys of the container*, but it of course is not enforced in any way at present. So, then — how about enforcing it? Should __iter__(), for the reasons above, replace the current purpose of keys() in mappings? I'm not properly equipped at the moment to mess around with CPython (sorry), but I assume at a minimum this would entail either replacing all instances of PyMapping_Keys() with PyObject_GetIter() or alternatively changing PyMapping_Keys() to call the latter. Does it sound like a reasonable change overall? Eli

6 9

PTPython REPL in IDLE
by James Lu 13 Sep '18

13 Sep '18

Have y’all seen ptpython’s autocomplete and syntax highlighting features? Ptpython, usually used as a cli application, might be worth integrating into IDLE.

3 2

Re: [Python-ideas] Python dialect that compiles into python
by David Teresi 13 Sep '18

13 Sep '18

Not totally convinced about this. It would require PEP writers to fully implement their proposed feature, people can usually get a pretty good idea of how a feature is supposed to work through PEPs (most of which are extremely well written), and these "mods" wouldn't be used in production code anyway since they're not backwards compatible with the rest of the language.On Sep 10, 2018 8:15 PM, Abe Dillon <abedillon(a)gmail.com> wrote: > > [Steven D'Aprano] >> >> It would be great for non-C coders to be able to prototype proposed >> syntax changes to get a feel for what works and what doesn't. > > > I think it would be great in general for the community to be able to try out ideas and mull things over. > > If there was something like a Python Feature Index (PyFI) and you could install mods to the language, > it would allow people to try out ideas before rejecting them or incorporating them into the language > (or putting them on hold until someone suggests a better implementation). > > I could even see features that never make it into the language, but stick around PyFI and get regular > maintenance because: > A) they're controversial changes that some love and some hate > B) they make things easier in some domain but otherwise don't warrant adoption > > It would have to be made clear from the start that Python can't guarantee backward compatibility with > any mods, which should prevent excessive fragmentation (if you want your code to be portable, don't > use mod). > > On Fri, Sep 7, 2018 at 7:30 PM Steven D'Aprano <steve(a)pearwood.info> wrote: >> >> On Fri, Sep 07, 2018 at 11:57:50AM +0000, Robert Vanden Eynde wrote: >> >> > Many features on this list propose different syntax to python, >> > producing different python "dialects" that can statically be >> > transformed to python : >> >> [...] >> > Using a modified version of ast, it is relatively easy to modifiy the >> > syntax tree of a program to produce another program. So one could >> > compile the "python dialect" into regular python. The last example >> > with partially for example doesn't even need new syntax. >> >> [...] >> > Actually, I might start to write this lib, that looks fun. >> >> I encourage you to do so! It would be great for non-C coders to be able >> to prototype proposed syntax changes to get a feel for what works and >> what doesn't. >> >> There are already a few joke Python transpilers around, such as >> "Like, Python": >> >> https://jon.how/likepython/ >> >> but I think this is a promising technique that could be used more to >> keep the core Python language simple while not *entirely* closing the >> door to people using domain-specific (or project-specific) syntax. >> >> >> -- >> Steve >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas(a)python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/

1 0

Off-topic: Email with content: Cannot read property 'get' of null
by Jonathan Fine 12 Sep '18

12 Sep '18

This is off-topic, but you might enjoy reading it. I got an email today, whose entire body text was Cannot read property 'get' of null It reminded me of our discussions of None-aware operators. And also Errors should never pass silently. Unless explicitly silenced. -- Jonathan

1 0

PEP 420: implicit namespace sub-package
by Gallian Colombeau 12 Sep '18

12 Sep '18

Hello, the PEP 420 (Implicit Namespace Packages) is quite descriptive about the problem and the solution implemented back in Python 3.3 but I feel there may be a part missing (but maybe this is categorized as an implementation detail). As I understand, for a package to allow being extended in this way, it must be a namespace package and not contain a marker file. As a matter of fact, no sub-package until the top level package can have a marker file: Lib/test/namespace_pkgs project1 parent child one.py project2 parent child two.py However, what is not discussed is "implicit namespace sub-package". In Python 3.6 (I guess since the first implementation), if you have this layout: Lib/test/namespace_pkgs project1 parent # Regular package __init__.py child # Namespace package one.py you get "parent" as a regular package and "parent.child" as a namespace package and it works (although now, every package data directory became namespace packages and are importable, which may or may not be desirable). The point is, does that add any value? I wasn't able to find any discussion about this and, as far as I can see, there is actually no use case for this as there is no possible way to contribute to the "parent.child" namespace. Is that an intended behavior of PEP 420? Again, I may have missed something or misinterpreted PEP 420 but this contributes to the "Implicit package directories introduce ambiguity into file system layouts." point by Nick Coghlan that was supposed to be addressed in PEP 395. Wouldn't it be more appropriate to enforce a sub-package to be a regular package if the parent package is a regular package? Gallian -- *Gallian Colombeau * *Software engineer * hiphen-plant-logo <http://www.hiphen-plant.com> *Centre INRA PACA - UMR EMMAH* /228, route de l'aérodrome - CS 40509 84914 Avignon - Cedex 9 - France / www.hiphen-plant.com

3 2