Mailman 3 September 2018 - Python-ideas

Re: [Python-ideas] "old" values in postconditions
by James Lu 24 Sep '18

24 Sep '18

You could disassemble (import dis) the lambda to biew the names of the lambdas. @before(lambda self, key, _, length, get: self.length(), self.get(key)) Perhaps you could disassemble the function code and look at all operations or accesses that are done to “old.” and evaluate those expressions before the function runs. Then you could “replace” the expression. @post(lambda self, key, old: old.get is None and old.length + 1 == self.length()) Either the system would grab old.get and old.length or be greedy and grab old.get is None and old.length + 1. It would then replace the old.get and old.length with injects that only respond to is None and +1. Or, a syntax like this @post(lambda self, key, old: [old.get(old.key)] is None and [old.self.length() + 1] == self.length()) Where the stuff inside the brackets is evaluated before the decorated function runs. It would be useful for networking functions or functions that do something ephemeral, where data related to the value being accessed needed for the expression no longer exists after the function. This does conflict with list syntax forever, so maybe either force people to do list((expr,)) or use an alternate syntax like one item set syntax { } or double set syntax {{ }} or double list syntax [[ ]]. Ditto with having to avoid the literals for the normal meaning. You could modify Python to accept any expression for the lambda function and propose that as a PEP. (Right now it’s hardcoded as a dotted name and optionally a single argument list surrounded by parentheses.) I suggest that instead of “@before” it’s “@snapshot” and instead of “old” it’s “snapshot”. Python does have unary plus/minus syntax as well as stream operators (<<, >>) and list slicing syntax and the @ operator and operators & and | if you want to play with syntax. There’s also the line continuation character for crazy lambdas. Personally I prefer @post(lambda self, key, old: {{old.self.get(old.key)}} and {{old.self.length() + 1}} == self.length()) because it’s explicit about what it does (evaluate the expressions within {{ }} before the function runs. I also find it elegant. Alternatively, inside the {{ }} could be a special scope where locals() is all the arguments @pre could’ve received as a dictionary. For either option you can remove the old parameter from the lambda. Example: @post(lambda self, key: {{self.get(key)}} and {{self.length() + 1}} == self.length()) Perhaps the convention should be to write {{ expr }} (with the spaces in between). You’d probably have to use the ast module to inspect it instead of the dis modul. Then find some way to reconstruct the expressions inside the double brackets- perhaps by reconstructing the AST and compiling it to a code object, or perhaps by finding the part of the string the expression is located. dis can give you the code as a string and you can run a carefully crafted regex on it.

1 0

Asynchronous exception handling around with/try statement borders
by Erik Bray 24 Sep '18

24 Sep '18

Hi folks, I normally wouldn't bring something like this up here, except I think that there is possibility of something to be done--a language documentation clarification if nothing else, though possibly an actual code change as well. I've been having an argument with a colleague over the last couple days over the proper way order of statements when setting up a try/finally to perform cleanup of some action. On some level we're both being stubborn I think, and I'm not looking for resolution as to who's right/wrong or I wouldn't bring it to this list in the first place. The original argument was over setting and later restoring os.environ, but we ended up arguing over threading.Lock.acquire/release which I think is a more interesting example of the problem, and he did raise a good point that I do want to bring up. </prologue> My colleague's contention is that given lock = threading.Lock() this is simply *wrong*: lock.acquire() try: do_something() finally: lock.release() whereas this is okay: with lock: do_something() Ignoring other details of how threading.Lock is actually implemented, assuming that Lock.__enter__ calls acquire() and Lock.__exit__ calls release() then as far as I've known ever since Python 2.5 first came out these two examples are semantically *equivalent*, and I can't find any way of reading PEP 343 or the Python language reference that would suggest otherwise. However, there *is* a difference, and has to do with how signals are handled, particularly w.r.t. context managers implemented in C (hence we are talking CPython specifically): If Lock.__enter__ is a pure Python method (even if it maybe calls some C methods), and a SIGINT is handled during execution of that method, then in almost all cases a KeyboardInterrupt exception will be raised from within Lock.__enter__--this means the suite under the with: statement is never evaluated, and Lock.__exit__ is never called. You can be fairly sure the KeyboardInterrupt will be raised from somewhere within a pure Python Lock.__enter__ because there will usually be at least one remaining opcode to be evaluated, such as RETURN_VALUE. Because of how delayed execution of signal handlers is implemented in the pyeval main loop, this means the signal handler for SIGINT will be called *before* RETURN_VALUE, resulting in the KeyboardInterrupt exception being raised. Standard stuff. However, if Lock.__enter__ is a PyCFunction things are quite different. If you look at how the SETUP_WITH opcode is implemented, it first calls the __enter__ method with _PyObjet_CallNoArg. If this returns NULL (i.e. an exception occurred in __enter__) then "goto error" is executed and the exception is raised. However if it returns non-NULL the finally block is set up with PyFrame_BlockSetup and execution proceeds to the next opcode. At this point a potentially waiting SIGINT is handled, resulting in KeyboardInterrupt being raised while inside the with statement's suite, and finally block, and hence Lock.__exit__ are entered. Long story short, because Lock.__enter__ is a C function, assuming that it succeeds normally then with lock: do_something() always guarantees that Lock.__exit__ will be called if a SIGINT was handled inside Lock.__enter__, whereas with lock.acquire() try: ... finally: lock.release() there is at last a small possibility that the SIGINT handler is called after the CALL_FUNCTION op but before the try/finally block is entered (e.g. before executing POP_TOP or SETUP_FINALLY). So the end result is that the lock is held and never released after the KeyboardInterrupt (whether or not it's handled somehow). Whereas, again, if Lock.__enter__ is a pure Python function there's less likely to be any difference (though I don't think the possibility can be ruled out entirely). At the very least I think this quirk of CPython should be mentioned somewhere (since in all other cases the semantic meaning of the "with:" statement is clear). However, I think it might be possible to gain more consistency between these cases if pending signals are checked/handled after any direct call to PyCFunction from within the ceval loop. Sorry for the tl;dr; any thoughts?

7 15

JS’ governance model is worth inspecting
by James Lu 24 Sep '18

24 Sep '18

JS’ decisions are made by a body known as TC39, a fairly/very small group of JS implementers. First, JS has an easy and widely supported way to modify the language for yourself: Babel. Babel transpires your JS to older JS, which is then run. You can publish your language modification on the JS package manager, npm. When a feature is being considered for inclusion in mainline JS, the proposal must first gain a champion (represented by 🚀)that is a member of TC-39. The guidelines say that the proposal’s features should already have found use in the community. Then it moves through three stages, and the champion must think the proposal is ready for the next stage before it can move on. I’m hazy on what the criterion for each of the three stages is. The fourth stage is approved. I believe the global TC39 committee meets regularly in person, and at those meetings, proposals can advance stages- these meetings are frequent enough for the process to be fast and slow enough that people can have the time to try out a feature before it becomes main line JS. Meeting notes are made public. The language and its future features are discussed on ESDiscuss.org, which is surprisingly filled with quality and respectful discussion, largely from experts in the JavaScript language. I’m fairly hazy on the details, this is just the summary off the top of my head. — I’m not saying this should be Python’s governance model, just to keep JS’ in mind.

5 5

Re: [Python-ideas] JS’ governance model is worth inspecting
by James Lu 24 Sep '18

24 Sep '18

> To my mind, there is one very big reason we should be cautious about > adopting JS language-design policies, namely, that they have led to a > very, very poorly designed language. No doubt a good deal of that is > baggage from early stages in which JS had a poor to nonexistent language > design governance model. Nonetheless, the failure of JS to fix its > numerous fundamental flaws, and especially the rapid feature churn in > recent years, suggests to me that their model should be viewed with > skepticism. I disagree. The language is often very flexible and effective in its domains. I don’t know what you mean by “rapid feature churn”, churn usually means existing features are superseded by newer ones- this isn’t the case. JS is much more nuanced than it appears on the surface. It’s understandable that those with only a glossing of JS look down on it, because JS really was a primitive language a few years ago. You can learn about JS in depth with the poorly-named “You don’t know JS” free online book.

3 2

Proposal for an inplace else (?=) operator
by Lee Braiden 22 Sep '18

22 Sep '18

Could I get some feedback on this? I'd like to know if anyone thinks it might make it through the pep process before spending too much (more) time on it. That said, it seems valuable to me, and I'm willing to put in the time, of course, IF it has a chance. --------------------------- Problem: Due to (biggest python WTF) (Problem 1), which prevents actual default argument values being set in a function signature, many functions look like: > def teleport(from, to, passenger, hitchhiker=None, food_accessory=None, comfort_accessory=None): > if hitchhiker is None: > hitchhiker = Fly() > > if food_accessory is None: > food_accessory = Cheeseburger() > > if comfort_accessory is None: > comfort_accessory = Towel() > > ... This None checking and setting is unwieldy (Problem 2) boilerplate, which is responsible for many extra lines of code in python (Problem 3), and tends to distract from the real code (Problem 4) in a function. To reduce boilerplate, a ternary or binary expression can be used: > def teleport(from, to, passenger, hitchhiker=None, accessories=[]): > hitchhiker = hitchhiker if hitchhiker or Fly() # Existing Solution A > def teleport(from, to, passenger, hitchhiker=None, accessories=[]): > hitchhiker = hitchhiker or Fly() # Existing Solution B These help, but are still quite repetitive: * Existing Solution A is often avoided simply because many Pythonistas dislike tenery expressions (perhaps due to readability or hidden code branch concerns), and can quickly become unwieldy when the new value (Fly()) is a more complex expression, such as a list comprehension: > def teleport(from, to, passenger, hitchhiker=None, accessories=[]): > hitchhiker = hitchhiker if hitchhiker or filter(lambda h: not h.already_hitching(), available_hitchhikers)[0] * Existing Solution B is less unwieldy (solving Problem 2), yet still suffers from repetition (Problems 2, 3, and 4). In a similar scenario, when we want to populate an empty list (say, accesories), we could write: > accessories |= [Cheeseburger(), Towel()] # Almost-Solution C However, this is not actually a solution, because: * The inplace-or (|=) operator is not provided for None in python: > food_accessor = None > food_accessory |= Cheeseburger() Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: unsupported operand type(s) for |=: 'NoneType' and 'Cheeseburger' This could be added, but would not solve the issue, because: * If an non-default (non-None) argument value WERE provided, we be would modifying the argument unintentionally: > class Larry: > def __ior__(self, other): > print("{} modified".format(self)) > > teleport(..., hitchhiker=Larry(), ...) <__main__.Larry object at 0x7f1ad9828be0> modified And so Problems 1,2,3, and 4 are compounded. Proposal: The addition of a ?= operator could provide an elegant solution: > def teleport(from, to, hitchiker=None, food_accessory=None, comfort_accessory=None): > hitchhiker ?= Fly() > food_accessory ?= Cheeseburger() > comfort_accessory ?= Towel() In these examples, > a = None > a ?= b > c = [1, 2] > c ?= d Would be equivalent to (assuming ?= was called __ielse__): > class ExtendedNone(NoneType) > def __ielse__(self, other): > return other > > class ielse_list(list): > def __ielse__(self, other): > return self > > None = ExtendedNone() > > a = None > a = a.__ielse__(b) > > c = iff_list([1, 2]) > c = a.__ielse__(d) Although explicitly provided for list above, this ielse operator could be defined (again, as `return other` for NoneType only, but defaulted to `return self` for all other values of `a`, requiring very little implementation effort or intrusion into other types. Possible interaction with typing It may be also possible to define a ? suffix on function arguments, so that: > def func(a?): > a ?= [1,2] greatly shorting the current verbose form: > from typing import Optional > > def func(a = None : Optional[Any])): > a = [1,2] if a is None else a and equivalent to: > from typing import Optional > > def func(a: Optional[Any]): > a = a.__ielse__([1,2]) Possible alternatives: * The __ielse__ name is highly debatable. Alternatives such as __iif__ (inplace if), __imaybe__, __isomeor__ (borrowing from Rusts's Some/None terminology, and reflecting the intended use as in relation to None types, DB nulls, and similar void-like values). * The ? function argument suffix is not necessary to implement the core ?= proposal. * If the ? function argument suffix is implemented, implementation via typing.Optional is not necessary; it could also be simply implemented so that: > def f(a?): > ... is equivalent to: > def f(a=None): > ... --------------------------- Feedback would be much appreciated. -- Lee

6 5

Pre-conditions and post-conditions
by Marko Ristin-Kaufmann 22 Sep '18

22 Sep '18

Hi, I would be very interested to bring design-by-contract into python 3. I find design-by-contract particularly interesting and indispensable for larger projects and automatic generation of unit tests. I looked at some of the packages found on pypi and also we rolled our own solution (https://github.com/Parquery/icontract/). I also looked into https://www.python.org/dev/peps/pep-0316/. However, all the current solutions seem quite clunky to me. The decorators involve an unnecessary computational overhead and the implementation of icontract became quite tricky once we wanted to get the default values of the decorated function. Could somebody update me on the state of the discussion on this matter? I'm very grateful for any feedback on this!

19 53

Re: [Python-ideas] Moving to another forum system where
by James Lu 22 Sep '18

22 Sep '18

One of the reasons Guido left was the insane volume of emails he had to read on Python-ideas. > A tiny bit of discussion is still better than none at all. > And even if there's no discussion, there's a name attached > to the message, which makes it more personal and meaningful > than a "+1" counter getting incremented somewhere. > > Counting only makes sense if the counts are going to be > treated as votes, and we don't do that. I agree. I think this is good evidence in favor of using GitHub pull requests or GitHub issues- you can see exactly who +1’d a topic. GitHub also has moderation tools and the ability to delete comments that are irrelevant, and edit comments that are disrespectful. > I hope that, if any such change is made, a forum system is > chosen that allows full participation via either email or news. > Otherwise it will probably mean the end of my participation, > because I don't have time to chase down and wrestle with > multiple web forums every day. +1, everyone should be accommodated. I believe GitHub has direct email capability. If you watch the repository and have email notifications on, you can reply directly to an email and it will be sent as a reply. — To solve the problem of tons of email for controversial decisions like :=, I don’t think GitHub issues would actually be the solution. The best solution would to have admins receive all the email, and broadcast a subset of the email sent, only broadcasting new arguments and new opinions. Admins can do this “summary duty” every 12 hours on a rotating basis, where each admin takes turns doing summary duty. This solution would mean a slower iteration time for the conversation, but it would significantly lessen the deluge of email, and I think that would make it more bearable for people participating in the conversation. After all, once a proposal has been fleshed out, what kind of conversation needs more than say 30 rounds of constructive discussion- in that case, if people reply every 25 hours, the discussion would be done in a month. For transparency purposes, all of the email can be made received for approval can be published online.

5 5

Re: [Python-ideas] Retire or reword the "Beautiful is better than ugly" Zen clause
by Davide Rizzo 21 Sep '18

21 Sep '18

Regardless of whether the original posting is a trolling attempt or not, the argument does have value that I want to corroborate with an anecdote. At one Python conference in Italy, participants were given an elegantly designed and well-crafted t-shirt with a writing in large characters that read "Beautiful is better than ugly" in reference to the Zen of Python. Back home, a close person who is not a Python programmer nor familiar with PEP 20 saw the t-shirt. They were horrified by the out-of-context sentence for reasons similar to what has been already stated in support of this argument. It prompted them of lookism and judgmentality, and found the message to be disturbing in its suggestion to compare by some standard of beauty and to discriminate. Let me add some context: this person is socially and politically active (maybe what someone would call a "SJW"; definitely not what anyone would call "politically correct"), and is specially sensitive to issues of discrimination and sexism. This was enough, though, for me to wonder what kind of message I would be projecting by wearing that writing on me. I've been since then discouraged to ever wear the t-shirt in any public context. This story might have limited value because it's one anecdote, and because the central point is the impact of the clause when taken out of its original context. I don't want to construct this as an argument in favor of removal of the clause, but I want to mention this as evidence that it does carry emotionally (negatively) charged content. If this content can be avoided without compromising the purpose and function of the message, than by all means I would welcome and support the change. It's meaningful, as a community, to show willingness to respond to discomfort of any kind. In this case, I even see the potential to convey the original message in a more powerful way than the current formulation does. I'm not a good candidate for this, as the chosen language for this community is English, which is not my native language nor a language I feel very good at. I appreciate the poetic style of the original, and I think that Tim Peters has done an outstanding job at capturing these ideas into easy and humor-rich language. The opportunity would be to express the idea of aesthetic appeal of code in some way beyond the simplistic judgmental labelling of "beautiful" vs "ugly". To be fair, in my experience this has been a source of confusion to many Python newcomers, as the notion of "beauty", as with any other value judgment, is highly relative to the subject evaluating it. I've seen people think of the Python community as conceited because they would think they possess some absolute metric of beauty. One way out of the impasse is to draw upon the feeling behind the adjective. We call "beautiful" something that appeals to us, makes us comfortable, or inspires us awe. Ugly is something that makes us uncomfortable, repels us, disconcerts us. "Let awe and disconcert drive you"? "Attraction and repulsion are important"? "If it disturbs you, it's probably wrong"? I know these are terrible and will all fail the spirit and the style of the original, but I'm dropping suggestions with the hope to stimulate some constructive thought on the matter. I'm fine with PEP 20 being unchanged; and my goal is not to find a replacement or urge for a change, but rather to be willing to think about it. Cheers, Davide

22 35

Moving to another forum system where moderation is possible
by James Lu 20 Sep '18

20 Sep '18

I agree completely. I propose Python register a trial of Stack Overflow Teams. Stack Overflow Teams is essentially your own private Stack Overflow. (I will address the private part later.) Proposals would be questions and additions or criticism would be answers. You can express your support or dissent of a proposal using the voting. Flags and reviews can be used to moderate. Stack Overflow Chat can be used for quick and casual discussion, and also to move irregular or irrelevant discussions away from the main site. Although the Stack Overflow platform is typically used for technical Q&A, there is precedent for using it as a way to discuss proposals: this is precisely what Meta Stack Overflow dies and it’s seen decent success. Anyone can register a @python.org email. Stack Overflow Teams can be configured to allow anyone with a @python.org email join the python-ideas team. I’m sure Stack Overflow Inc. is willing to grant Stack Overflow Teams to the PSF pro bono after the trial period expires. You can configure stack overflow to get email notifications as well. > On Sep 17, 2018, at 1:16 PM, Anders Hovmöller <boxed(a)killingar.net> wrote: > > >> It’s been almost a week since this “discussion” first started. Can we please stop this in the name of productive work on python-ideas? > > A better use of time might be to discuss moving to a better forum system where moderation is easier/possible. Email somehow has a shape that makes those things 100% probable and you can’t easily silence discussions that are uninteresting. > > / Anders

25 46

Re: [Python-ideas] Moving to another forum system where moderation is possible
by James Lu 20 Sep '18

20 Sep '18

> Frankly, I think the bigger issue is all too human -- we get sucked in and > participate when we really know we shouldn't (or maybe that's just me). > That may be why some people misbehave, but we have no way of discouraging that misbehavior. > And I'm having a hard time figuring out how moderation would actually > result in the "good" discussion we really want in an example like the > "beautiful is better than ugly" issue, without someone trusted individual > approving every single post -- I don't imagine anyone wants to do that. In a forum, the beautiful is better than ugly issue would be locked. No more posts can be added. If someone wants to discuss another proposal branching off of the original discussion, they can start a new thread. If they just want to lampoon, we can courteously ask them to 1) take it elsewhere or 2) move the post to a “malarkey” section of the forum where people don’t get notified.

5 6