Mailman 3 November 2017 - Python-ideas

Re: [Python-ideas] Should Python have user-defined constants?
by Saeed Baig Nov. 22, 2017

Nov. 22, 2017

Hi. Just to respond to some peoples’ points… Bernardo asked 'This would be "constant" as in Java constants right? Referential constants, so that if an object were mutable you would still be able to change its internal state.’ Um, I’m not entirely sure what you mean, since I’ve never used Java. But if it means what I think it means, I would be INCLINED to say no. If, for example, I saw code like this: x = [9, 5, 7, 1] let y = x y[0] = 42 print(y) … I would expect that code to raise an error, not print “[42, 5, 7, 1]”, since you’re trying to modify the “constant” y in place, which… well, would kind of go against the idea of y being a constant, I thought (unless I’m misunderstanding what you meant by “object", since everything in Python’s an object). However, I might change my opinion if I heard a good reason why Java-like constants would be better. Serhiy asked, in relation to constants, “To do what? What problem do you need to solve?”. No problem in particular, to be honest. I just thought they’d be nice since they’d increase confidence that my variables-intended-to-be-constants wouldn’t get reassigned, and just to increase readability/explicitness to other programmers. > On 21 Nov 2017, at 20:08, python-ideas-request(a)python.org wrote: > > Send Python-ideas mailing list submissions to > python-ideas(a)python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/python-ideas > or, via email, send a message with subject or body 'help' to > python-ideas-request(a)python.org > > You can reach the person managing the list at > python-ideas-owner(a)python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Python-ideas digest..." > > > Today's Topics: > > 1. Re: Should Python have user-defined constants? (Steven D'Aprano) > 2. Re: Should Python have user-defined constants? (St?fane Fermigier) > 3. Re: Should Python have user-defined constants? (Bernardo Sulzbach) > 4. Star assignment in iterator way? (Kirill Balunov) > 5. Re: Star assignment in iterator way? (Serhiy Storchaka) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Tue, 21 Nov 2017 19:13:42 +1100 > From: Steven D'Aprano <steve(a)pearwood.info> > To: python-ideas(a)python.org > Subject: Re: [Python-ideas] Should Python have user-defined constants? > Message-ID: <20171121081342.GK19802(a)ando.pearwood.info> > Content-Type: text/plain; charset=us-ascii > > On Tue, Nov 21, 2017 at 02:38:45AM -0500, Joseph Jevnik wrote: > >> How is that different from "pi = 3.14"? > > pi = 3.14 > pi = 5 > print(pi) > # prints 5 > > let pi = 3.14 > pi = 5 > # raises an exception > > > > -- > Steve > > > ------------------------------ > > Message: 2 > Date: Tue, 21 Nov 2017 09:26:17 +0100 > From: St?fane Fermigier <sf(a)fermigier.com> > To: Python-Ideas <python-ideas(a)python.org> > Subject: Re: [Python-ideas] Should Python have user-defined constants? > Message-ID: > <CABuJJj5-ZhuWTBns8rm8jVWPaOGq08MM+X0ryQp1z7ogQFsHvw(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > Javascript (ES6) has 'let' and 'const' for mutable and constant variables, > respectively. > > When programming in JS, I find myself aggressively aiming for as little > 'let' and as much 'const' as reasonably possible, since reasoning about > constants is much easier than about variables. In this context, 'const' is > used as a marker, a runtime checker, and, with proper tooling (IDE or > linter), a coding-time reminder to differentiate between these two > fundamental behaviours. > > I'm not +1 on introducing this idea to Python yet, but not -1 either - this > deserves some discussion, if this has not been discussed already. > > Cheers, > > S. > > On Tue, Nov 21, 2017 at 9:13 AM, Steven D'Aprano <steve(a)pearwood.info> > wrote: > >> On Tue, Nov 21, 2017 at 02:38:45AM -0500, Joseph Jevnik wrote: >> >>> How is that different from "pi = 3.14"? >> >> pi = 3.14 >> pi = 5 >> print(pi) >> # prints 5 >> >> let pi = 3.14 >> pi = 5 >> # raises an exception >> >> >> >> -- >> Steve >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas(a)python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ >> > > > > -- > Stefane Fermigier - http://fermigier.com/ - http://twitter.com/sfermigier - > http://linkedin.com/in/sfermigier > Founder & CEO, Abilian - Enterprise Social Software - > http://www.abilian.com/ > Chairman, Free&OSS Group / Systematic Cluster - > http://www.gt-logiciel-libre.org/ > Co-Chairman, National Council for Free & Open Source Software (CNLL) - > http://cnll.fr/ > Founder & Organiser, PyData Paris - http://pydata.fr/ > --- > ?You never change things by ?ghting the existing reality. To change > something, build a new model that makes the existing model obsolete.? ? R. > Buckminster Fuller >

11 16

Make bytes __repr__ and __str__ representation different?
by Kirill Balunov Nov. 22, 2017

Nov. 22, 2017

Currently, __repr__ and __str__ representation of bytes is the same. Perhaps it is worth making them different, this will make it easier to visually perceive them as a container of integers from 0 to 255, instead of a mixture of printable and non-printable ascii characters. It is proposed: a) __str__ - leave unchanged b) __repr__ - represent as sequence of escaped hex >>> a = bytes([42,43,44,45,46]) >>> a # Current b'*+-./' >>> a # Proposed b'\x2a\x2b\x2d\x2e\x2f' As you can see, the second example is more easily perceived as a sequence, in which '\' is also perceived as ',' in list or tuple. In addition, 2020 is close, it allows the new Pythonistas not to take them as an ascii mixture strings. With kind regards, -gdg

6 6

Re: [Python-ideas] Make bytes __repr__ and __str__ representation different?
by Steven D'Aprano Nov. 21, 2017

Nov. 21, 2017

On Tue, Nov 21, 2017 at 01:35:41PM -0200, Joao S. O. Bueno wrote: > On 21 November 2017 at 13:16, Steven D'Aprano <steve(a)pearwood.info> wrote: > > I'd rather leave __str__ and __repr__ alone. Changing them will have > > huge backwards compatibility implications. I'd rather give bytes a > > hexdump() method that returns a string: > > > > '2a 2b 2d 2e 2f' > > Do you mean, like bytes.hex? Guido's Time Machine strikes again! -- Steve

1 0

Re: [Python-ideas] Should Python have user-defined constants?
by Hugh Fisher Nov. 21, 2017

Nov. 21, 2017

> Message: 4 > Date: Tue, 21 Nov 2017 18:05:17 +0000 > From: ????? <elazarg(a)gmail.com> > To: Chris Barker <chris.barker(a)noaa.gov> > Cc: Python-Ideas <python-ideas(a)python.org> > Subject: Re: [Python-ideas] Should Python have user-defined constants? [munch] > > It was mentioned that there are related conventions in Python. Good point. > But they are not as strong as the "convention" that tuples and strings > cannot change, and this is useful in reasoning about code - in > understanding the program, essentially. > > When I read "final int x = 5;" in Java, I don't have to think about it > anymore - it's 5. When I read "X = 5" in Python, it might be a constant, > but it might also be a misnomer, or something that used to be a constant, > or a class reassigned. I still have to watch whether it will be changed, > intentionally or accidentally, by this or other module. It does not even > communicate intention in an unambiguous way. Here I strongly disagree. The uppercase convention for things you really should not be assigning to has been in Python for ages and well understood. math.pi is the usual example brought up by people who want const/final, perhaps as a glaring exception to the rule. (Can math.PI be added to the standard library?) I think that adding const/let/final to Python would decrease readability and the ability to reason about the code for me, because now I would be reading foo += 1 and I could not easily decide whether this was wrong or not, because it might have been declared as immutable. Textual conventions are popular in many programming languages, not just Python, because they are easier to remember than individual type/attribute declarations. > In my opinion, the inability to control mutation in Python is the biggest > readability issue of the language (thank god monkey-patching is > discouraged). I think this is a separate issue to const/let/final. (And have you read PEP 351?) I don't miss simple scalar const/#define values in Python. In C/Java/Ada/... they are necessary to define array bounds, but everything in Python is dynamic and you can ask an array or whatever how big it is. In C/Ada/C#/... const protects again accidentally passing a value by reference/address and modifying it, but Python doesn't have pass by reference. No pointer aliasing in Python... As for monkey patching, or even just math.pi = 3.0 that is being unable to make dictionaries read-only. I agree, I get frustrated when I've accidentally misspelled an attribute name and Python just creates a new instance value instead of reporting an error. But this kind of dynamic object creation is so fundamental to how Python works, and also the source of so much power and flexibility, that I think it is worth living with the occasional glitch. I would not want to see Python allow module or class writers to declare "my code is perfect and not to be changed". But I would not mind if I could write code "I should not be changing this module/class/object , raise an exception if I do". -- cheers, Hugh Fisher

1 0

Star assignment in iterator way?
by Kirill Balunov Nov. 21, 2017

Nov. 21, 2017

Hello. Currently during star assignement the new list is created. What was the idea to produce new list instead of returning an iterator? It seems to me that returning an iterator more suited to the spirit of Python 3. There are three cases: 1. a,b,c,*d = something_iterable 2. *a,b,c,d = something_iterable 3. a,*b,c,d = something_iterable The first one is obvious. For the rest two we always need to iterate through entire iterable to achieve values for b,c,d (or c,d) binding. But this can be done more memory effiecient than currently (may be I'm wrong). And we can iterate in space of last three (or two) variables. Some rough (simplified) Python code: from itertools import islice, chain from collections import deque def good_star_exp(signature, seq): if signature.count('*') > 1: raise SyntaxError('two starred expressions in assignment') vrs = signature.split(',') idx_max = len(vrs) - 1 star_pos, = (i for i,v in enumerate(vrs) if '*' in v) #First case if star_pos == idx_max: head = islice(seq, idx_max) tail = islice(seq, idx_max, None) return chain(head, (tail,)) #Second case elif star_pos == 0: tail = deque(maxlen=idx_max) for seq_idx_max, v in enumerate(seq): tail.append(v) head = islice(seq, 0, seq_idx_max-(idx_max-1)) return chain([head], tail) #Third case else: head = islice(seq, star_pos) tail = deque(maxlen=(idx_max-star_pos)) for seq_idx_max, v in enumerate(seq): tail.append(v) mid = islice(seq, star_pos, seq_idx_max-(idx_max-2)) return chain(head, [mid], tail) ls = range(100000) a,b,c,d = good_star_exp('a,b,c,*d', ls) a,b,c,d = good_star_exp('*a,b,c,d', ls) a,b,c,d = good_star_exp('a,*b,c,d', ls) Of course this version has drawbacks (the first that come to mind): 1. Will *b see change if rhs is some muttable sequence? 2. Will *b one way iterator or somethong like range? But still it seems to me that the "iterator way" has more useful applications. With best regards, -gdg

5 8

Should Python have user-defined constants?
by Saeed Baig Nov. 21, 2017

Nov. 21, 2017

Hey guys I am thinking of perhaps writing a PEP to introduce user-defined constants to Python. Something along the lines of Swift’s “let” syntax (e.g. “let pi = 3.14”). Do you guys think it would be a good idea? Why or why not? Do you think there’s a better way to do it? I’d like to know what others think about this idea before making any formal submission (I’ve already posted this same question on python-list, but I just wanted to gauge opinion here too).

13 16

Rewriting the "roundrobin" recipe in the itertools documentation
by bunslow Nov. 21, 2017

Nov. 21, 2017

For taking values alternately from a series of iterables, there's two primary functions: builtin.zip itertools.zip_longest zip of course stops when the shortest iterable ends. zip_longest is generally a useful substitute for when you don't want the zip behavior, but it fills extra values in the blanks rather than just ignoring a finished iterator and moving on with the rest. This latter most use case is at least somewhat common, according to this[1] StackOverflow question (and other duplicates), in addition to the existence of the `roundrobin` recipe[2] in the itertools docs. The recipe satisfies this use case, and its code is repeated in the StackOverflow answer. However, it is remarkably unpythonic, in my opinion, which is one thing when such is necessary to achieve a goal, but for this functionality, such is most definitely *not* necessary. I'll paste the code here for quick reference: def roundrobin(*iterables): "roundrobin('ABC', 'D', 'EF') --> A D E B F C" pending = len(iterables) nexts = cycle(iter(it).__next__ for it in iterables) while pending: try: for next in nexts: yield next() except StopIteration: pending -= 1 nexts = cycle(islice(nexts, pending)) Things that strike me as unpythonic: 1) requiring the total number of input iterables 2) making gratuitous use of `next`, 3) using a while loop in code dealing with iterables, 4) combining loops, exceptions, and composed itertools functions in non-obvious ways that make control flow difficult to determine Now, I get it, looking at the "roughly equivalent to" code for zip_longest in the docs, there doesn't seem to be much way around it for generally similar goals, and as I said above, unpythonic is fine when necessary (practicality beats purity), but in this case, for being a "recipe" in the itertools docs, it should *make use* of the zip_longest which already does all the unpythonic stuff for you (though honestly I'm not convinced either that the zip_longest code in the docs is the most possible pythonic-ness). Instead, the following recipe (which I also submitted to the StackOverflow question, and which is generally similar to several other later answers, all remarking that they believe it's more pythonic) is much cleaner and more suited to demonstrating the power of itertools to new developers than the mess of a "recipe" pasted above. def roundrobin(*iters): "roundrobin('ABC', 'D', 'EF') --> A D E B F C" # Perhaps "flat_zip_nofill" is a better name, or something similar sentinel = object() for tup in it.zip_longest(*iters, fillvalue=sentinel): yield from (x for x in tup if x is not sentinel) In particular, this is just an extremely thin wrapper around zip_longest, whose primary purpose is to eliminate the otherwise-mandatory "fillvalues" that zip_longest requires to produce uniform-length tuples. It's also an excellent example of how to make best pythonic use of iterables in general, and itertools in particular, and as such a much better implementation to be demonstrated in documentation. I would thus advocate that the former recipe is replaced with the latter recipe, being much more pythonic, understandable, and useful for helping new developers acquire the style of python. (Using the common linguistics analogy: a dictionary and grammar for a spoken language may be enough to communicate, but we rely on a large body of literature -- fiction, research, poetry, etc -- as children to get that special flavor and most expressive taste to the language. The stdlib is no Shakespeare, but it and its docs still form an important part of the formative literature of the Python language.) I realize at the end of the day this is a pretty trivial and ultimately meaningless nit to pick, but I've never contributed before and have a variety of similar minor pain points in the docs/stdlib, and I'm trying to gauge 1) how well this sort of minor QoL improvement is wanted, and 2) even if it is wanted, am I going about it the right way. If the answers to both of these questions are positive regarding this particular case, then I'll look into making a BPO issue and pull request on GitHub, which IIUC is the standard path for contributions. Thank you for your consideration. ~~~~ [1]: https://stackoverflow.com/questions/3678869/ pythonic-way-to-combine-two-lists-in-an-alternating-fashion/ [2]: https://docs.python.org/3/library/itertools.html#itertools-recipes

9 13

Ignorable whitespaces in the re.VERBOSE mode
by Serhiy Storchaka Nov. 21, 2017

Nov. 21, 2017

Currently the re module ignores only 6 ASCII whitespaces in the re.VERBOSE mode: U+0009 CHARACTER TABULATION U+000A LINE FEED U+000B LINE TABULATION U+000C FORM FEED U+000D CARRIAGE RETURN U+0020 SPACE Perl ignores characters that Unicode calls "Pattern White Space" in the /x mode. It ignores additional 5 non-ASCII characters. U+0085 NEXT LINE U+200E LEFT-TO-RIGHT MARK U+200F RIGHT-TO-LEFT MARK U+2028 LINE SEPARATOR U+2029 PARAGRAPH SEPARATOR The regex module just ignores characters for which str.isspace() returns True. It ignores additional 20 non-ASCII whitespace characters, including characters U+001C..001F whose classification as whitespaces is questionable, but doesn't ignore LEFT-TO-RIGHT MARK and RIGHT-TO-LEFT MARK. U+001C [FILE SEPARATOR] U+001D [GROUP SEPARATOR] U+001E [RECORD SEPARATOR] U+001F [UNIT SEPARATOR] U+00A0 NO-BREAK SPACE U+1680 OGHAM SPACE MARK U+2000 EN QUAD U+2001 EM QUAD U+2002 EN SPACE U+2003 EM SPACE U+2004 THREE-PER-EM SPACE U+2005 FOUR-PER-EM SPACE U+2006 SIX-PER-EM SPACE U+2007 FIGURE SPACE U+2008 PUNCTUATION SPACE U+2009 THIN SPACE U+200A HAIR SPACE U+202F NARROW NO-BREAK SPACE U+205F MEDIUM MATHEMATICAL SPACE U+3000 IDEOGRAPHIC SPACE Is it worth to extend the set of ignored whitespaces to "Pattern Whitespaces"? Would it add any benefit? Or add confusion? Should this depend on the re.ASCII mode? Should the byte b'\x85' be ignorable in verbose bytes patterns? And there is a similar question about the Python parser. If Python uses Unicode definition for identifier, shouldn't it accept non-ASCII "Pattern Whitespaces" as whitespaces? There will be technical problems with supporting this, but are there any benefits? https://perldoc.perl.org/perlre.html https://www.unicode.org/reports/tr31/tr31-4.html#Pattern_Syntax https://unicode.org/L2/L2005/05012r-pattern.html

8 13

Adding a new function "zip_flat" to itertools (Re: Rewriting the "roundrobin" recipe in the itertools documentation)
by bunslow Nov. 21, 2017

Nov. 21, 2017

Dear all: thank you for your replies and thoughts, most especially Steve and Terry. I am more-or-less new to contributing to Python, so I wasn't sure that the bug tracker was the best way to start -- I was looking for a sanity check and received exactly what I wanted :) Thanks to the feedback here, the bug tracker issue will be much cleaner from the start. Regarding the meta discussion about my OP: yes it was long winded, detailed, pedantic and (to a certain extent) bombastic, but I was imaging the many possible responses to the suggestion (and suggested replacement) the I felt I should explain where I was coming from. Even though there was a lot of disagreement about how I described the current recipe as not very Pythonic, I'm very glad for all the perspectives and lessons I got from reading the ensuing discussions. Now, having had some time to think, I've come up with further thoughts on the topic. In particular, I was going to create a new bug, and wrote several paragraphs on the topic summarizing this thread, and my thoughts. I'll paste those paragraphs here: """ To summarize, I found the current implementation of the "roundrobin" function difficult to understand, and proposed a much simpler solution that, while producing correct results, isn't quite "correct" in the sense that it glosses over a detail before "manually" correcting the detail at the end, and as such is prone to severe inefficiency in extreme cases. There were a smattering of comments from several people indicating that they liked the simpler recipe better, despite its performance drawbacks. Terry Reedy proposed a slightly rewritten version of the current recipe, which implements the correct algorithm (without glossing over and manually correcting the details). Although I have since changed my perceptions of the original, now understanding how it works, the rewritten version from Terry was clearer enough that I was able to understand *it* where I could not previously understand the original. (My newfound understanding of the original is largely derived from making sense of the rewritten version, which properly clued me in to what cycle and islice actually do. Either way, the current recipe can certainly be improved. Although I now find the original and its rewrite to be algorithmically clean and correct (even if the code and inline comments can be improved), the StackOverflow question ( https://stackoverflow.com/questions/3678869/pythonic-way-to-combine-two-lis…) that originally got me thinking about the problem demonstrates that the algorithmically clean way is *not* obvious at all to people who aren't very familiar with the itertools module -- which is the large majority of people who use Python (even if that's a very small fraction of the people reading this bug). The second from top answer is the answer which references the recipe in the docs, but as my own first post to python-ideas demonstrates, the (large?) majority of python users aren't familiar enough with the itertools module to be able to understand that recipe, and I also believe many people were looking for one or two liners to use in their own code without a function call. Further confusion on the overall topic is the lack of a clear name -- "roundrobin", "alternate", "interleave", "merge", and variations and others. """ Having completed those, I found a roughly duplicate StackOverflow question to the one from my OP: https://stackoverflow.com/questions/243865/how-do-i-merge-two-python-iterat… Besides emphasizing my points about not having even a clear name for the topic, a desire for one liners, mass confusion around the issue (especially regarding flattening zip [which terminates on the shortest input, a hidden gotcha], zip_longest [someone else found the same solution as me and others in this op), and all around failure to generate anything even resembling a consensus on the topic, I also found this answer: https://stackoverflow.com/questions/243865/how-do-i-merge-two-python-iterat… which proposes a solution that is both more correct and efficient than the zip_longest-with-sentinels, and also noticeably more readable than either the original doc recipe or even Terry's cleaned up replacement of it. Given this variety of problems with the issue, I now think that -- while updating the itertools recipe is certainly better than nothing -- the better thing to do might be to just add a new function to itertools called "zip_flat" which solves this problem. In addition to answering the stack overflow questions with ongoing debate about efficiency, correctness, and pythonicity (pythonicness?), it would also help to greatly clarify the naming issue as well. (Sidenote: whoever came up with "zip" as the name for the builtin was quite creative. It's a remarkably short and descriptive.) What are the sentiments of readers here? If positive, I'll create an issue on BPO about zip_flat (rather than just improving the docs recipe). (Sorry Steve for bringing this back to -ideas! At least this time I'm proposing an addition to the language itself! :) Thanks for your consideration, Bill On Thu, Nov 16, 2017 at 5:06 PM, Steven D'Aprano <steve(a)pearwood.info> wrote: > On Thu, Nov 16, 2017 at 02:56:29PM -0500, Terry Reedy wrote: > > > >3) using a while loop in code dealing with iterables, > > > > I agree that this is not necessary, and give a replacement below. > > The OP isn't just saying that its unnecessary in this case, but that its > unPythonic to ever use a while loop in code dealing with iterables. I > disagree with that stronger statement. > > > > >def roundrobin(*iters): > > > "roundrobin('ABC', 'D', 'EF') --> A D E B F C" > > > # Perhaps "flat_zip_nofill" is a better name, or something similar > > > sentinel = object() > > > for tup in it.zip_longest(*iters, fillvalue=sentinel): > > > yield from (x for x in tup if x is not sentinel) > > > > This adds and then deletes grouping and fill values that are not wanted. > > To me, this is an 'algorithm smell'. One of the principles of > > algorithm design is to avoid unnecessary calculations. For an edge case > > such as roundrobin(1000000*'a', ''), the above mostly does unnecessary > work. > > Its a recipe, not a tuned and optimized piece of production code. > > And if you're going to criticise code on the basis of efficiency, then I > would hope you've actually profiled the code first. Because it isn't > clear to me at all that what you call "unnecessary work" is more > expensive than re-writing the recipe using a more complex algorithm with > calls to cycle and islice. > > But I'm not here to nit-pick your recipe over the earlier ones. > > > [...] > > Since I have a competing 'improvement', I would hold off on a PR until > > Raymond Hettinger, the itertools author, comments. > > Raise a doc issue on the tracker, and take the discussion there. I think > that this is too minor an issue to need long argument on the list. > Besides, it's not really on-topic as such -- it isn't about a change to > the language. Its about an implementation change to a recipe in the > docs. > > > > -- > Steve > _______________________________________________ > Python-ideas mailing list > Python-ideas(a)python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ >

5 8

Consider (one day) adding an inheritance order class precedence mechanism
by Neil Girdhar Nov. 21, 2017

Nov. 21, 2017

Sometimes I get MRO failures when defining classes. For example, if R < E, C B < S, E S < C Z < B, R Then Z cannot be instantiated because C precedes E in B and E precedes C in R. The problem is that when the inheritance graph was topologically-sorted in B, the order was S, C, E. It could just as easily have been S, E, C. So, one solution is to add C explicitly to B's base class list, but this is annoying because B might not care about C (it's an implementation detail of S). It also means that if the hierarchy changes, a lot of these added extra base classes need to be fixed. I propose adding a magic member to classes: __precedes__ that is a list of classes. So, the fix for the above problem would be to define E as follows: class E: from whatever import C __precedes__ = [C] Then, when topologically-sorting (so-called linearizing) the ancestor classes, Python can try to ensure that E precedes C when defining B. Best, Neil

7 21