Mailman 3 October 2017 - Python-ideas

Defining an easily installable "Recommended baseline package set"
by Nick Coghlan 02 Nov '17

02 Nov '17

Over on python-dev, the question of recommending MRAB's "regex" module over the standard library's "re" module for more advanced regular expressions recently came up again. Because of various logistical issues and backwards compatibility risks, it's highly unlikely that we'll ever be able to swap out the current _sre based re module implementation in the standard library for an implementation based on the regex module. At the same time, it would be beneficial to have a way to offer an even stronger recommendation to redistributors that we think full-featured general purpose Python scripting environments should offer the regex module as an opt-in alternative to the baseline re feature set, since that would also help with other currently difficult cases like the requests module. What I'm thinking is that we could make some relatively simple additions to the `ensurepip` and `venv` modules to help with this: 1. Add a ensurepip.RECOMMENDED_PACKAGES mapping keyed by standard library module names containing dependency specifiers for recommended third party packages for particular tasks (e.g. "regex" as an enhanced alternative to "re", "requests" as an enhanced HTTPS-centric alternative to "urllib.request") 2. Add a new `install_recommended` boolean flag to ensurepip.bootstrap 3. Add a corresponding `--install-recommended flag to the `python -m ensurepip` CLI 4. Add a corresponding `--install-recommended flag to the `python -m venv` CLI (when combined with `--without-pip`, this would run pip directly from the bundled wheel file to do the installations) We'd also need either a new informational PEP or else a section in the developer guide to say that the contents of `ensurepip.RECOMMENDED_PACKAGES` are up to the individual module maintainers (hence keying the mapping by standard library module name, rather than having a single flat list for the entire standard library). For redistributors with weak dependency support, these reference interpreter level recommendations could become redistributor level recommendations. Redistributors without weak dependency support could still make a distinction between "default" installations (which would include them) and "minimal" installations (which would exclude them). Folks writing scripts and example code for independent distribution (i.e. no explicitly declared dependencies) could then choose between relying on just the standard library (as now), or on the standard library plus independently versioned recommended packages. Cheers, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

13 39

Re: [Python-ideas] Add single() to itertools
by Greg Ewing 31 Oct '17

31 Oct '17

2 1

Re: [Python-ideas] Python-ideas Digest, Vol 131, Issue 106
by Ivan Pozdeev 31 Oct '17

31 Oct '17

On 31.10.2017 8:37, python-ideas-request(a)python.org wrote: > On Tue, Oct 31, 2017 at 3:50 PM, Ivan Pozdeev via Python-ideas > <python-ideas(a)python.org> wrote: >> On 30.10.2017 17:32, Guido van Rossum wrote: >>> This is a key example of a case where code speaks. Can you write an >>> implementation of how you would want single() to work in Python code? >>> >>> On Mon, Oct 30, 2017 at 2:49 AM, Ivan Pozdeev via Python-ideas >>> <python-ideas(a)python.org <mailto:python-ideas@python.org>> wrote: >>> >>> The initial post on the above link summarizes the suggested >>> implementation pretty well. >>> >> |defsingle(i): try: ||v =i.next() >> |||exceptStopIteration:||||raiseException('No values')|||try: ||i.next() >> ||exceptStopIteration: ||returnv||else: ||raiseException('Too many values')| >> ||printsingle(name forname in('bob','fred')ifname=='bob')||| | >> >> || > raise WhitespaceDamagedException from None > > ChrisA Thunderbird jerked on me big time. It never did anything like this before! Switched off Digest mode, individual messages aren't so complicated. def single(i): try: v =i.next() except StopIteration: raise ValueError('No items') try: i.next() except StopIteration: return v else: raise ValueError('More than one item') print single(name for name in('bob','fred') if name=='bob') -- Regards, Ivan

6 8

Add processor generation to wheel metadata
by Ivan Pozdeev 31 Oct '17

31 Oct '17

Generally, packages are compiled for the same processor generation as the corresponding Python. But not always -- e.g. NumPy opted for SSE2 even for Py2 to work around some compiler bug (https://github.com/numpy/numpy/issues/6428). I was bitten by that at an old machine once and found out that there is no way for `pip' to have checked for that. Besides, performance-oriented packages like the one mentioned could probably benefit from newer instructions. Regarding identifiers: gcc, cl and clang all have their private directories of generation identifiers: https://gcc.gnu.org/onlinedocs/gcc-4.7.1/gcc/i386-and-x86_002d64-Options.ht… https://msdn.microsoft.com/en-us/library/7t5yh4fd.aspx https://clang.llvm.org/doxygen/Driver_2ToolChains_2Arch_2X86_8cpp_source.ht… Linux packages typically use gcc's ones. Clang generally follows in gcc's footsteps and accepts cl's IDs, too, as aliases. So, using the IDs of whatever compiler is used to build the package (i.e. most likely the canonical compiler for CPython for that platform) looks like the simple&stupid(r) way - we can just take the value of the "march" argument. The tricky part is mapping the system's processor to an ID when checking compatibility: the logic will have to keep a directory (that's the job of `wheel' package maintainers though, I guess). I also guess that there are cases where there's no such thing as _the_ system's processor. -- Regards, Ivan

2 1

IMAP4.__exit__ counterintuitive for with blocks
by Drew 30 Oct '17

30 Oct '17

IMAP4.close closes the selected inbox and commits changes such as deletions. It is not called on IMAP4.__exit__ (only logout is, which doesn't call close in its call stack) however, so: with imaplib.IMAP4_SSL(...) as i: ... would fail to commit those changes. close must be explicitly invoked i.e. with imaplib.IMAP4_SSL(...) as i: ... i.close() This is counterintuitive however, as the with statement is meant to automatically clean up. Another programmer might come along and delete i.close() because it seems unnecessary. Now changes aren't being committed and the programmer doesn't realize it's because of this weird Python idiosyncracy. Python IO such as open commits changes automatically, so I'm not sure why IMAP4 doesn't and only logs out.

3 3

Thread.__init__ should call super()
by Ilya Kulakov 30 Oct '17

30 Oct '17

Since one of the legit use-cases of using the Thread class is subclassing, I think it's __init__ should call super() to support cooperative inheritance. Or perhaps there is a good reason for not doing so? Best Regards, Ilya Kulakov

5 9

Add single() to itertools
by Ivan Pozdeev 30 Oct '17

30 Oct '17

The eponymous C#'s LINQ method, I found very useful in the following, quite recurring use-case: I need to get a specific element from a data structure that only supports search semantics (i.e. returns a sequence/iterator of results). For that, I specify very precise search criteria, so only that one item is supposed to be found. But I'd rather verify that just in case. "A data structure that only supports search semantics" is a recurring phenomenon due to this: I make a special-purpose data structure (usually for some domain-specific data like task specification or data directory) using a combination of existing and/or new containers. Now, these types do not enforce all the integrity constraints of my data. And I don't wish to construct a special-purpose class, complete with validation procedures, and pass records into it one by one etc -- when I can just write an initializer, loading all the data at once in a nicely readable construct, and call it a day. So, when querying this structure, I "know" that there should only be one item satisfying a certain criteria - if there's more, or less, something is wrong with the data. https://stackoverflow.com/questions/46009985/get-contentcontrol-by-title-or… is the most recent occasion where I had this use case (that is not Python but the concept is language-agnostic). -- Regards, Ivan

3 2

Composition over Inheritance
by Soni L. 30 Oct '17

30 Oct '17

As recent threads indicate, composition may sometimes be better than inheritance. And so I'd like to propose composition as a built-in feature. My idea is syntax of the form o.[c].m(), where o is an object, c is a component, m is a method. I am not sure how you'd set components, or test for components, but I don't think it makes sense to be able to do o.[c][x] or x=o.[c], because those break the idea of automatically passing the object as an argument (unless we create whole wrapper objects every time the syntax is used, and that's a bit ew. also how would you handle o.[c1].[c2] ?). Thus I think o.[c].m() should be syntax sugar for o[c].m(o), with o being evaluated only once, as that solves a lot of current issues relating to inheritance while introducing very few issues relating to python's "everything is separate" (e.g. __get__ vs __getattr__) policy.This also makes setting components and testing for components fairly trivial, and completely avoids the issues mentioned above by making their syntax illegal. (Disclaimer: This was inspired by my own programming language, Cratera[1], so I'm a bit biased here. Cratera was, in turn, inspired by Rust[2] traits. Note however that the original plans for Cratera were far more flexible, including allowing the "problematic" o.[c1].[c2].m() and o.[c][x].m(). I can go into more detail on how those should work, if wanted, but implementation-wise it's not looking good.) [1] https://bitbucket.org/TeamSoni/cratera [2] https://www.rust-lang.org/

9 22

Re: [Python-ideas] Composition over Inheritance
by Koos Zevenhoven 29 Oct '17

29 Oct '17

(looks like you forgot to reply to the list) On Sun, Oct 29, 2017 at 7:47 AM, Greg Ewing <greg.ewing(a)canterbury.ac.nz> wrote: > Koos Zevenhoven wrote: > >> It looks like you can just as easily drive the car without having the key >> > > That's been found to be a problem in real life, too. > More than one Python-programming car thief has been > caught with this function in their personal library: > > def hotwire(car): > car.engine.start() Yes, but my point was still about what the public API is. Starting the car with the key should be easier than starting it without the key. Surely you should be able to start the engine without the key if you look under the hood and figure it out. And maybe you'd need a tool for that to be kept in your garage for debugging, but not as a button attached to the outside of the car or on the dashboard :-). The problem becomes even worse if you instead make Car inherit from its parts (including Engine), because the relationship of a car and and engine is not well described by inheritance. A car is not a kind of engine, nor should it pretend to be an implementation of an engine. A car *comprises* (or contains) an engine. And who knows, maybe it has two engines! -- Koos -- + Koos Zevenhoven + http://twitter.com/k7hoven +

1 0

Dollar operator suggestion
by Yan Pas 29 Oct '17

29 Oct '17

I've looked up this feature in haskell. Dollar sign operator is used to avoid parentheses. Rationalle: Python tends to use functions instead of methods ( e.g. len([1,2,3]) instead of [1,2,3].len() ). Sometimes the expression inside parentheses may become big and using a lot of parentheses may tend to bad readability. I suggest the following syntax: len $ [1,2,3] Functions map be also chained: len $ list $ map(...) This operator may be used for function composition too: foo = len $ set $ in the same as foo = lambda *as,**kas : len(set(*as, **kas)) in current syntax Regards, Yan

10 9