Python-ideas November 2017

python-ideas@python.org

95 participants
60 discussions

Re: [Python-ideas] How assignment should work with generators?

by Alon Snir

On Wed, Nov 29, 2017 at 5:46 AM, Alon Snir <AlonSnir(a)hotmail.com> wrote: > I would like to mention that the issue of assignment to a target list, is also relevant to the case of elementwise assignment to a mutable sequence (e.g. lists and arrays). Here as well, the rhs of the assignment statement states the number of elements to be assigned. Consequently, the following example will get into infinite loop: > >>>> from itertools import count >>>> A = [] >>>> A[:] = count() > > Writing "A[:2] = count()" will cause the same result. > Here as well, it is currently safer to use islice if the rhs is a generator or an iterator. e.g.: > >>>> it = count() >>>> A[:] = islice(it,2) > > In my opinion, it is be better to devise a solution that could be applied in both cases. Maybe a new kind of assignment operator that will be dedicated to this kind of assignment. i.e. elementwise assignment with restriction on the number of elements to be assigned, based on the length of the lhs object (or the number of targets in the target list). > ChrisA wrote: Hmm. The trouble is that slice assignment doesn't have a fixed number of targets. If you say "x, y = spam", there's a clear indication that 'spam' needs to provide exactly two values; but "A[:] = spam" could have any number of values, and it'll expand or shrink the list accordingly. Rhodri James wrote: Flatly, no. It is better not to ask for things you don't want in the first place, in this case the infinite sequence. Still, don't let me discourage you from working on this. If you can define how such an assignment would work, or even the length of A[:] as an assignment target, I'm not going to dismiss it out of hand. My answer: The idea is to define an alternative assignment rule, that is to assign exactly as many elements as the current length of the lhs object (without expanding or shrinking it). Suppose "?=" is the operator for the alternative assignment rule; A=[None]*2; and "iterator" is any iterator (finite or infinite). In this case, the following code: >>> A ?= iterator would behave like this: >>> A[:] = islice(iterator, 2) # where 2 is the length of A And as suggested earlier for the case of assignment to a target list, the following code: >>> x, y ?= iterator would behave like this: >>> x, y = islice(iterator, 2) # where 2 is the number of targets Regarding the length issue: Is there any difficulty in finding the length of a sliced sequence? After all, the range object has a len method. Therefore, the length of A[s] (where s is a slice object) could be evaluated as follows: >>> len(range(*s.indices(len(A))))

3 years, 10 months

Stub class for Generic to further improve PEP 560

by Ilya Kulakov

As explained in PEP 560, creation of Generic subclasses is much slower (~7x on my machine). My guess is that a limited number of applications actually need Generic classes at "production" runtime (i.e. when not under analysis of mypy or IDE). I propose a special class that will alias typing.Generic whenever typing.TYPE_CHECKING == True and a no-op stub otherwise. Best Regards, Ilya Kulakov

3 years, 10 months

Re: [Python-ideas] [Python-Dev] What's the status of PEP 505: None-aware operators?

by Stephan Houben

What about more English-like syntax: X or else Y E.g. cache.get(foo) or else expensive_call(foo) Stephan Op 29 nov. 2017 12:54 schreef "Serhiy Storchaka" <storchaka(a)gmail.com>: 29.11.17 11:45, Steven D'Aprano пише: On Wed, Nov 29, 2017 at 09:14:12AM +0200, Serhiy Storchaka wrote: > >> 29.11.17 08:08, Steven D'Aprano пише: >> >>> Perl is hardly the only language with null-coalescing operators -- we >>> might better describe ?? as being familiar to C#, PHP, Swift and Dart. >>> That's two mature, well-known languages and two up-and-coming languages. >>> >> >> What is the syntax of the ternary operator in these languages? >> > > All four use: > > condition ? first : second > > for the ternary if operator. > If all four use ?, it is natural that in operators which are shortcuts of the ternary operator they use ?. But in Python the bar of introducing ? is higher. _______________________________________________ Python-ideas mailing list Python-ideas(a)python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

3 years, 10 months

Re: [Python-ideas] [Python-Dev] What's the status of PEP 505: None-aware operators?

by David Mertz

I like much of the thinking in Random's approach. But I still think None isn't quite special enough to warrant it's own syntax. However, his '(or None: name.strip()[4:].upper())' makes me realize that what is being asked in all the '?(', '?.', '?[' syntax ideas is a kind of ternary expression. Except the ternary isn't based on whether a predicate holds, but rather on whether an exception occurs (AttributeError, KeyError, TypeError). And the fallback in the ternary is always None rather than being general. I think we could generalize this to get something both more Pythonic and more flexible. E.g.: val = name.strip()[4:].upper() except None This would just be catching all errors, which is perhaps too broad. But it *would* allow a fallback other than None: val = name.strip()[4:].upper() except -1 I think some syntax could be possible to only "catch" some exceptions and let others propagate. Maybe: val = name.strip()[4:].upper() except (AttributeError, KeyError): -1 I don't really like throwing a colon in an expression though. Perhaps some other word or symbol could work instead. How does this read: val = name.strip()[4:].upper() except -1 in (AttributeError, KeyError) Where the 'in' clause at the end would be optional, and default to 'Exception'. I'll note that what this idea DOES NOT get us is: val = timeout ?? local_timeout ?? global_timeout Those values that are "possibly None" don't raise exceptions, so they wouldn't apply to this syntax. Yours, David... On Wed, Nov 29, 2017 at 9:03 AM, Random832 <random832(a)fastmail.com> wrote: > On Tue, Nov 28, 2017, at 15:31, Raymond Hettinger wrote: > > > > > I also cc python-dev to see if anybody here is strongly in favor or > against this inclusion. > > > > Put me down for a strong -1. The proposal would occasionally save a few > > keystokes but comes at the expense of giving Python a more Perlish look > > and a more arcane feel. > > > > One of the things I like about Python is that I can walk non-programmers > > through the code and explain what it does. The examples in PEP 505 look > > like a step in the wrong direction. They don't "look like Python" and > > make me feel like I have to decrypt the code to figure-out what it does. > > > > timeout ?? local_timeout ?? global_timeout > > 'foo' in (None ?? ['foo', 'bar']) > > requested_quantity ?? default_quantity * price > > name?.strip()[4:].upper() > > user?.first_name.upper() > > Since we're looking at different syntax for the ?? operator, I have a > suggestion for the ?. operator - and related ?[] and ?() that appeared > in some of the proposals. How about this approach? > > Something like (or None: ...) as a syntax block in which any operation > [lexically within the expression, not within e.g. called functions, so > it's different from simply catching AttributeError etc, even if that > could be limited to only catching when the operand is None] on None that > is not valid for None will yield None instead. > > This isn't *entirely* equivalent, but offers finer control. > > v = name?.strip()[4:].upper() under the old proposal would be more or > less equivalent to: > > v = name.strip()[4:].upper() if name is not None else None > > Whereas, you could get the same result with: > (or None: name.strip()[4:].upper()) > > Though that would technically be equivalent to these steps: > v = name.strip if name is not None else None > v = v() if v """"" > v = v[4:] """"""" > v = v.upper """"""" > v = v() """"""" > > The compiler could optimize this case since it knows none of the > operations are valid on None. This has the advantage of being explicit > about what scope the modified rules apply to, rather than simply > implicitly being "to the end of the chain of dot/bracket/call operators" > > It could also be extended to apply, without any additional syntax, to > binary operators (result is None if either operand is None) (or None: a > + b), for example, could return None if either a or b is none. > > [I think I proposed this before with the syntax ?(...), the (or None: > ...) is just an idea to make it look more like Python.] > _______________________________________________ > Python-Dev mailing list > Python-Dev(a)python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > mertz%40gnosis.cx > -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.

3 years, 10 months

Immutable dictionaries

by Asen Bozhilov

This is my first post here. I have strong experience with JavaScript and I'm lucky that I could move forward to Python. What I miss in Python are immutable dictionaries. They are especially useful for configurations and call functions which expect dictionary as argument. In my opinion they would let a place for underlying optimizations. I'd like to propose also literaling syntax for immutable dictionaries. immutable_dict = ( 'key1' : 'value1', 'key2' : 'value2' ) This syntax is not ambiguous with expression statements and tuple literals, but it requires a bit lookahed during the parsing. I would appreciate your opinions on the topic. Most interesting for me is why they are not already part of the language? Kind regards, Asen Bozhilov

3 years, 10 months

Re: [Python-ideas] [Python-Dev] What's the status of PEP 505: None-aware operators?

by David Mertz

True enough. I remember the prior discussion, but didn't look up the PEP number. I know it's hard to revisit these ideas, but occasionally it works, especially if a nicer spelling is found (Barry proposed some a bit different than my ideas) On Nov 29, 2017 9:55 AM, "Eric V. Smith" <eric(a)trueblade.com> wrote: On Nov 29, 2017, at 12:40 PM, David Mertz <mertz(a)gnosis.cx> wrote: I like much of the thinking in Random's approach. But I still think None isn't quite special enough to warrant it's own syntax. However, his '(or None: name.strip()[4:].upper())' makes me realize that what is being asked in all the '?(', '?.', '?[' syntax ideas is a kind of ternary expression. Except the ternary isn't based on whether a predicate holds, but rather on whether an exception occurs (AttributeError, KeyError, TypeError). And the fallback in the ternary is always None rather than being general. I think we could generalize this to get something both more Pythonic and more flexible. E.g.: val = name.strip()[4:].upper() except None This would just be catching all errors, which is perhaps too broad. But it *would* allow a fallback other than None: val = name.strip()[4:].upper() except -1 I think some syntax could be possible to only "catch" some exceptions and let others propagate. Maybe: val = name.strip()[4:].upper() except (AttributeError, KeyError): -1 I don't really like throwing a colon in an expression though. Perhaps some other word or symbol could work instead. How does this read: val = name.strip()[4:].upper() except -1 in (AttributeError, KeyError) Where the 'in' clause at the end would be optional, and default to 'Exception'. I'll note that what this idea DOES NOT get us is: val = timeout ?? local_timeout ?? global_timeout Those values that are "possibly None" don't raise exceptions, so they wouldn't apply to this syntax. See the rejected PEP 463 for Exception catching expressions. Eric. Yours, David... On Wed, Nov 29, 2017 at 9:03 AM, Random832 <random832(a)fastmail.com> wrote: > On Tue, Nov 28, 2017, at 15:31, Raymond Hettinger wrote: > > > > > I also cc python-dev to see if anybody here is strongly in favor or > against this inclusion. > > > > Put me down for a strong -1. The proposal would occasionally save a few > > keystokes but comes at the expense of giving Python a more Perlish look > > and a more arcane feel. > > > > One of the things I like about Python is that I can walk non-programmers > > through the code and explain what it does. The examples in PEP 505 look > > like a step in the wrong direction. They don't "look like Python" and > > make me feel like I have to decrypt the code to figure-out what it does. > > > > timeout ?? local_timeout ?? global_timeout > > 'foo' in (None ?? ['foo', 'bar']) > > requested_quantity ?? default_quantity * price > > name?.strip()[4:].upper() > > user?.first_name.upper() > > Since we're looking at different syntax for the ?? operator, I have a > suggestion for the ?. operator - and related ?[] and ?() that appeared > in some of the proposals. How about this approach? > > Something like (or None: ...) as a syntax block in which any operation > [lexically within the expression, not within e.g. called functions, so > it's different from simply catching AttributeError etc, even if that > could be limited to only catching when the operand is None] on None that > is not valid for None will yield None instead. > > This isn't *entirely* equivalent, but offers finer control. > > v = name?.strip()[4:].upper() under the old proposal would be more or > less equivalent to: > > v = name.strip()[4:].upper() if name is not None else None > > Whereas, you could get the same result with: > (or None: name.strip()[4:].upper()) > > Though that would technically be equivalent to these steps: > v = name.strip if name is not None else None > v = v() if v """"" > v = v[4:] """"""" > v = v.upper """"""" > v = v() """"""" > > The compiler could optimize this case since it knows none of the > operations are valid on None. This has the advantage of being explicit > about what scope the modified rules apply to, rather than simply > implicitly being "to the end of the chain of dot/bracket/call operators" > > It could also be extended to apply, without any additional syntax, to > binary operators (result is None if either operand is None) (or None: a > + b), for example, could return None if either a or b is none. > > [I think I proposed this before with the syntax ?(...), the (or None: > ...) is just an idea to make it look more like Python.] > _______________________________________________ > Python-Dev mailing list > Python-Dev(a)python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/mertz% > 40gnosis.cx > -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th. _______________________________________________ Python-Dev mailing list Python-Dev(a)python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/ eric%2Ba-python-dev%40trueblade.com

3 years, 10 months

generator vs iterator etc. (was: How assignment should work with generators?)

by Koos Zevenhoven

On Mon, Nov 27, 2017 at 3:55 PM, Steven D'Aprano <steve(a)pearwood.info> wrote: > On Mon, Nov 27, 2017 at 12:17:31PM +0300, Kirill Balunov wrote: > > > > 2. Should this work only for generators or for any iterators? > > I don't understand why you are even considering singling out *only* > generators. A generator is a particular implementation of an iterator. I > can write: > > def gen(): > yield 1; yield 2; yield 3 > > it = gen() > > or I can write: > > it = iter([1, 2, 3]) > > and the behaviour of `it` should be identical. > > > I can see where this is coming from. The thing is that "iterator" and "generator" are mostly synonymous, except two things: (1) Generators are iterators that are produced by a generator function (2) Generator functions are sometimes referred to as just "generators" The concept of "generator" thus overlaps with both "iterator" and "generator function". Then there's also "iterator" and "iterable", which are two different things: (3) If `obj` is an *iterable*, then `it = iter(obj)` is an *iterator* (over the contents of `obj`) ( 4) Iterators yield values, for example on explicit calls to next(it). Personally I have leaned towards keeping a clear distinction between "generator function" and "generator", which leads to the situation that "generator" and "iterator" are mostly synonymous for me. Sometimes, for convenience, I use the term "generator" to refer to "iterators" more generally. This further seems to have a minor benefit that "generators" and "iterables" are less easily confused with each other than "iterators" and "iterables". I thought about this issue some time ago for the `views` package, which has a separation between sequences (seq) and other iterables (gen): https://github.com/k7hoven/views The functionality provided by `views.gen` is not that interesting—it's essentially a subset of itertools functionality, but with an API that parallels `views.seq` which works with sequences (iterable, sliceable, chainable, etc.). I used the name `gen`, because iterator/iterable variants of the functionality can be implemented with generator functions (although also with other kinds of iterators/iterables). Calling the thing `iter` would have conflicted with the builtin `iter`. HOWEVER, this naming can be confusing for those that lean more towards using "generator" to also mean "generator function", and for those that are comfortable with the term "iterator" despite its resemblance to "iterable". Now I'm actually seriously considering to consider renaming `views.gen` to ` views.iter` when I have time. After all, there's already `views.range` which "conflicts" with the builtin range. Anyway, the point is that the naming is suboptimal. SOLUTION: Maybe (a) all iterators should be called iterators or (b) all iterators should be called generators, regardless of whether they are somehow a result of a generator function having been called in the past. (I'm not going into the distinction between things that can receive values via `send` or any other possible distinctions between different types of iterators and iterables.) —Koos (discussion originated from python-ideas, but cross-posted to python-dev in case there's more interest there) -- + Koos Zevenhoven + http://twitter.com/k7hoven +

3 years, 10 months

Logging Levels

by Mike Miller

Hi, I use the logging module extensively for even the simplest scripts, one of the reasons there's been less difficulty moving to Python 3 at work. One of the "nano features" I've often added to its config is the addition of a custom log level. Never mentioned it before because of its triviality and minor use cases, but just realized that I've been doing it five years now and happy with it, so why not? NOTE (~35) It is called the "note" level and used when one needs to express something important, yet positive, and have it be output by default. The standard levels don't support this currently, you often have to potentially scare the end user with a warning or higher to have a message emitted. Typically I use it to return important information that was asked for specifically and retrieved successfully, e.g.: log.note('Your token is: %s', token) log.note(f'⏵ {item.id} {item.name}') There are other examples. Sphinx, has the concept of note admonitions for docs in addition to warning and danger. Bootstrap has note banners for web/apps. There is something important to express or highlight, but nothing to worry about. FATAL (alias of CRITICAL) Can't find it now, but believe the docs in the past stated that CRITICAL was meant for *unrecoverable* errors. I've never had a project where I didn't shut down immediately after such an occurrence. Therefore I find "FATAL" a more accurate description of what happened. Log4j and other industry loggers use this level name as well. There is also an aesthetic reason to prefer "fatal". It is shorter and so aligns better with other level names for readability, e.g.: console_format = ' %(levelname)-7.7s %(message)s' Tried but never found a good abbreviation for critical, unfortunately. The other option is to add length to align the field. Most messages use the shorter level names (debug, info) so extra length results in wasted space that is very rarely needed. Hopefully someone else finds these useful. Neither depends on the other. -Mike

3 years, 10 months

Proposal: allow length_hint to specify infinite iterators

by Steven D'Aprano

PEP 424 allows iterators to optionally offer a hint as to how long they will be: https://www.python.org/dev/peps/pep-0424/ Unfortunately, there's no good way for an iterator to report that it is infinitely long. Consequently, even those which are known to be infinite report finite lengths: py> from itertools import count py> from operator import length_hint py> infinite = count() py> length_hint(infinite) 0 This wastes the opportunity to fail fast on operations which cannot possibly succeed, e.g. list(count()) must eventually fail with MemoryError. Or worse: if the OS starts thrashing trying to meet the memory requests, you can lock up the computer. I propose that we: (1) extend the __length_hint__ protocol to allow iterators to report that they are infinite; (2) and recommend that consumers of iterators (such as list) that require finite input should fail fast in the event of an infinite iterator. Four possible ways that __length_hint__ and operator.length_hint might signal an infinite iterator: (a) return a negative value such as -1 (this is currently an error); (b) return some special sentinel value; (c) take the convention that returning sys.maxint means infinity; (d) raise an exception. The advantage of (d) is that consumers of check __length_hint__ don't need to do anything special to fail fast on infinite iterators: py> class Thing: ... def __length_hint__(self): ... raise ValueError('infinite') ... def __iter__(self): ... return count() ... py> x = Thing() py> list(x) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 3, in __length_hint__ ValueError: infinite but if they can cope with such, they can explicitly catch the exception. Thoughts? -- Steve

3 years, 10 months

Looking for input to help with the pip situation

by Michel Desmoulin

Hello, Today I'm going to give a training in Python again. And again it will go the same way. On Mac I will have to make people install python, then tell them to use pip3. On Windows, I will have to warn them about checking the "add python executable to system path" (that one of them will ALWAYS miss anyway). Then tell them to use py -3.x -m pip because some of them will have several versions of Python installed. Then on linux, I will tell them to install python-pip and python-venv and use python3 -m pip. I'll talk about --user, but commands won't be usable on some machine where the Scripts or bin dir is not in the system path. Then I will make them create a virtualenv so that they can avoid messing with their system python and finally can just use "pip install" like in most tutorials on the Web. And eventually I'll talk about pipenv and conda. The first one so they don't have to think about activating the virtualenv everytime, or pip freeze, or create the venv, or add it to gitignore, etc. The second because anaconda is very popular on windows. There is no way a beginner is going to get any that by themselves without a lot of time and pain. They will read some tutorial on the web and struggle to make sens of what pip is and why "pip install" doesn't work and why "python sucks". I think Python is offering an incredible experience for first timer. However, the whole "where is walpip" shenanigans is not one of them. I really want some people from this list to discuss here so we can find a way to either unify a bit the way we install and use pip, or find a way to express a tutorial that always works for people on the most popular platforms and spread the word so that any doc uses it. Michel

3 years, 10 months

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

Python-ideas November 2017