Python-ideas September 2018

python-ideas@python.org

108 participants
94 discussions

Asynchronous exception handling around with/try statement borders

by Erik Bray

Hi folks, I normally wouldn't bring something like this up here, except I think that there is possibility of something to be done--a language documentation clarification if nothing else, though possibly an actual code change as well. I've been having an argument with a colleague over the last couple days over the proper way order of statements when setting up a try/finally to perform cleanup of some action. On some level we're both being stubborn I think, and I'm not looking for resolution as to who's right/wrong or I wouldn't bring it to this list in the first place. The original argument was over setting and later restoring os.environ, but we ended up arguing over threading.Lock.acquire/release which I think is a more interesting example of the problem, and he did raise a good point that I do want to bring up. </prologue> My colleague's contention is that given lock = threading.Lock() this is simply *wrong*: lock.acquire() try: do_something() finally: lock.release() whereas this is okay: with lock: do_something() Ignoring other details of how threading.Lock is actually implemented, assuming that Lock.__enter__ calls acquire() and Lock.__exit__ calls release() then as far as I've known ever since Python 2.5 first came out these two examples are semantically *equivalent*, and I can't find any way of reading PEP 343 or the Python language reference that would suggest otherwise. However, there *is* a difference, and has to do with how signals are handled, particularly w.r.t. context managers implemented in C (hence we are talking CPython specifically): If Lock.__enter__ is a pure Python method (even if it maybe calls some C methods), and a SIGINT is handled during execution of that method, then in almost all cases a KeyboardInterrupt exception will be raised from within Lock.__enter__--this means the suite under the with: statement is never evaluated, and Lock.__exit__ is never called. You can be fairly sure the KeyboardInterrupt will be raised from somewhere within a pure Python Lock.__enter__ because there will usually be at least one remaining opcode to be evaluated, such as RETURN_VALUE. Because of how delayed execution of signal handlers is implemented in the pyeval main loop, this means the signal handler for SIGINT will be called *before* RETURN_VALUE, resulting in the KeyboardInterrupt exception being raised. Standard stuff. However, if Lock.__enter__ is a PyCFunction things are quite different. If you look at how the SETUP_WITH opcode is implemented, it first calls the __enter__ method with _PyObjet_CallNoArg. If this returns NULL (i.e. an exception occurred in __enter__) then "goto error" is executed and the exception is raised. However if it returns non-NULL the finally block is set up with PyFrame_BlockSetup and execution proceeds to the next opcode. At this point a potentially waiting SIGINT is handled, resulting in KeyboardInterrupt being raised while inside the with statement's suite, and finally block, and hence Lock.__exit__ are entered. Long story short, because Lock.__enter__ is a C function, assuming that it succeeds normally then with lock: do_something() always guarantees that Lock.__exit__ will be called if a SIGINT was handled inside Lock.__enter__, whereas with lock.acquire() try: ... finally: lock.release() there is at last a small possibility that the SIGINT handler is called after the CALL_FUNCTION op but before the try/finally block is entered (e.g. before executing POP_TOP or SETUP_FINALLY). So the end result is that the lock is held and never released after the KeyboardInterrupt (whether or not it's handled somehow). Whereas, again, if Lock.__enter__ is a pure Python function there's less likely to be any difference (though I don't think the possibility can be ruled out entirely). At the very least I think this quirk of CPython should be mentioned somewhere (since in all other cases the semantic meaning of the "with:" statement is clear). However, I think it might be possible to gain more consistency between these cases if pending signals are checked/handled after any direct call to PyCFunction from within the ceval loop. Sorry for the tl;dr; any thoughts?

2 years, 6 months

JS’ governance model is worth inspecting

by James Lu

JS’ decisions are made by a body known as TC39, a fairly/very small group of JS implementers. First, JS has an easy and widely supported way to modify the language for yourself: Babel. Babel transpires your JS to older JS, which is then run. You can publish your language modification on the JS package manager, npm. When a feature is being considered for inclusion in mainline JS, the proposal must first gain a champion (represented by 🚀)that is a member of TC-39. The guidelines say that the proposal’s features should already have found use in the community. Then it moves through three stages, and the champion must think the proposal is ready for the next stage before it can move on. I’m hazy on what the criterion for each of the three stages is. The fourth stage is approved. I believe the global TC39 committee meets regularly in person, and at those meetings, proposals can advance stages- these meetings are frequent enough for the process to be fast and slow enough that people can have the time to try out a feature before it becomes main line JS. Meeting notes are made public. The language and its future features are discussed on ESDiscuss.org, which is surprisingly filled with quality and respectful discussion, largely from experts in the JavaScript language. I’m fairly hazy on the details, this is just the summary off the top of my head. — I’m not saying this should be Python’s governance model, just to keep JS’ in mind.

2 years, 6 months

Re: [Python-ideas] JS’ governance model is worth inspecting

by James Lu

> To my mind, there is one very big reason we should be cautious about > adopting JS language-design policies, namely, that they have led to a > very, very poorly designed language. No doubt a good deal of that is > baggage from early stages in which JS had a poor to nonexistent language > design governance model. Nonetheless, the failure of JS to fix its > numerous fundamental flaws, and especially the rapid feature churn in > recent years, suggests to me that their model should be viewed with > skepticism. I disagree. The language is often very flexible and effective in its domains. I don’t know what you mean by “rapid feature churn”, churn usually means existing features are superseded by newer ones- this isn’t the case. JS is much more nuanced than it appears on the surface. It’s understandable that those with only a glossing of JS look down on it, because JS really was a primitive language a few years ago. You can learn about JS in depth with the poorly-named “You don’t know JS” free online book.

2 years, 6 months

Proposal for an inplace else (?=) operator

by Lee Braiden

Could I get some feedback on this? I'd like to know if anyone thinks it might make it through the pep process before spending too much (more) time on it. That said, it seems valuable to me, and I'm willing to put in the time, of course, IF it has a chance. --------------------------- Problem: Due to (biggest python WTF) (Problem 1), which prevents actual default argument values being set in a function signature, many functions look like: > def teleport(from, to, passenger, hitchhiker=None, food_accessory=None, comfort_accessory=None): > if hitchhiker is None: > hitchhiker = Fly() > > if food_accessory is None: > food_accessory = Cheeseburger() > > if comfort_accessory is None: > comfort_accessory = Towel() > > ... This None checking and setting is unwieldy (Problem 2) boilerplate, which is responsible for many extra lines of code in python (Problem 3), and tends to distract from the real code (Problem 4) in a function. To reduce boilerplate, a ternary or binary expression can be used: > def teleport(from, to, passenger, hitchhiker=None, accessories=[]): > hitchhiker = hitchhiker if hitchhiker or Fly() # Existing Solution A > def teleport(from, to, passenger, hitchhiker=None, accessories=[]): > hitchhiker = hitchhiker or Fly() # Existing Solution B These help, but are still quite repetitive: * Existing Solution A is often avoided simply because many Pythonistas dislike tenery expressions (perhaps due to readability or hidden code branch concerns), and can quickly become unwieldy when the new value (Fly()) is a more complex expression, such as a list comprehension: > def teleport(from, to, passenger, hitchhiker=None, accessories=[]): > hitchhiker = hitchhiker if hitchhiker or filter(lambda h: not h.already_hitching(), available_hitchhikers)[0] * Existing Solution B is less unwieldy (solving Problem 2), yet still suffers from repetition (Problems 2, 3, and 4). In a similar scenario, when we want to populate an empty list (say, accesories), we could write: > accessories |= [Cheeseburger(), Towel()] # Almost-Solution C However, this is not actually a solution, because: * The inplace-or (|=) operator is not provided for None in python: > food_accessor = None > food_accessory |= Cheeseburger() Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: unsupported operand type(s) for |=: 'NoneType' and 'Cheeseburger' This could be added, but would not solve the issue, because: * If an non-default (non-None) argument value WERE provided, we be would modifying the argument unintentionally: > class Larry: > def __ior__(self, other): > print("{} modified".format(self)) > > teleport(..., hitchhiker=Larry(), ...) <__main__.Larry object at 0x7f1ad9828be0> modified And so Problems 1,2,3, and 4 are compounded. Proposal: The addition of a ?= operator could provide an elegant solution: > def teleport(from, to, hitchiker=None, food_accessory=None, comfort_accessory=None): > hitchhiker ?= Fly() > food_accessory ?= Cheeseburger() > comfort_accessory ?= Towel() In these examples, > a = None > a ?= b > c = [1, 2] > c ?= d Would be equivalent to (assuming ?= was called __ielse__): > class ExtendedNone(NoneType) > def __ielse__(self, other): > return other > > class ielse_list(list): > def __ielse__(self, other): > return self > > None = ExtendedNone() > > a = None > a = a.__ielse__(b) > > c = iff_list([1, 2]) > c = a.__ielse__(d) Although explicitly provided for list above, this ielse operator could be defined (again, as `return other` for NoneType only, but defaulted to `return self` for all other values of `a`, requiring very little implementation effort or intrusion into other types. Possible interaction with typing It may be also possible to define a ? suffix on function arguments, so that: > def func(a?): > a ?= [1,2] greatly shorting the current verbose form: > from typing import Optional > > def func(a = None : Optional[Any])): > a = [1,2] if a is None else a and equivalent to: > from typing import Optional > > def func(a: Optional[Any]): > a = a.__ielse__([1,2]) Possible alternatives: * The __ielse__ name is highly debatable. Alternatives such as __iif__ (inplace if), __imaybe__, __isomeor__ (borrowing from Rusts's Some/None terminology, and reflecting the intended use as in relation to None types, DB nulls, and similar void-like values). * The ? function argument suffix is not necessary to implement the core ?= proposal. * If the ? function argument suffix is implemented, implementation via typing.Optional is not necessary; it could also be simply implemented so that: > def f(a?): > ... is equivalent to: > def f(a=None): > ... --------------------------- Feedback would be much appreciated. -- Lee

2 years, 6 months

Pre-conditions and post-conditions

by Marko Ristin-Kaufmann

Hi, I would be very interested to bring design-by-contract into python 3. I find design-by-contract particularly interesting and indispensable for larger projects and automatic generation of unit tests. I looked at some of the packages found on pypi and also we rolled our own solution (https://github.com/Parquery/icontract/). I also looked into https://www.python.org/dev/peps/pep-0316/. However, all the current solutions seem quite clunky to me. The decorators involve an unnecessary computational overhead and the implementation of icontract became quite tricky once we wanted to get the default values of the decorated function. Could somebody update me on the state of the discussion on this matter? I'm very grateful for any feedback on this!

2 years, 7 months

Re: [Python-ideas] Moving to another forum system where

by James Lu

One of the reasons Guido left was the insane volume of emails he had to read on Python-ideas. > A tiny bit of discussion is still better than none at all. > And even if there's no discussion, there's a name attached > to the message, which makes it more personal and meaningful > than a "+1" counter getting incremented somewhere. > > Counting only makes sense if the counts are going to be > treated as votes, and we don't do that. I agree. I think this is good evidence in favor of using GitHub pull requests or GitHub issues- you can see exactly who +1’d a topic. GitHub also has moderation tools and the ability to delete comments that are irrelevant, and edit comments that are disrespectful. > I hope that, if any such change is made, a forum system is > chosen that allows full participation via either email or news. > Otherwise it will probably mean the end of my participation, > because I don't have time to chase down and wrestle with > multiple web forums every day. +1, everyone should be accommodated. I believe GitHub has direct email capability. If you watch the repository and have email notifications on, you can reply directly to an email and it will be sent as a reply. — To solve the problem of tons of email for controversial decisions like :=, I don’t think GitHub issues would actually be the solution. The best solution would to have admins receive all the email, and broadcast a subset of the email sent, only broadcasting new arguments and new opinions. Admins can do this “summary duty” every 12 hours on a rotating basis, where each admin takes turns doing summary duty. This solution would mean a slower iteration time for the conversation, but it would significantly lessen the deluge of email, and I think that would make it more bearable for people participating in the conversation. After all, once a proposal has been fleshed out, what kind of conversation needs more than say 30 rounds of constructive discussion- in that case, if people reply every 25 hours, the discussion would be done in a month. For transparency purposes, all of the email can be made received for approval can be published online.

2 years, 7 months

Re: [Python-ideas] Retire or reword the "Beautiful is better than ugly" Zen clause

by Davide Rizzo

Regardless of whether the original posting is a trolling attempt or not, the argument does have value that I want to corroborate with an anecdote. At one Python conference in Italy, participants were given an elegantly designed and well-crafted t-shirt with a writing in large characters that read "Beautiful is better than ugly" in reference to the Zen of Python. Back home, a close person who is not a Python programmer nor familiar with PEP 20 saw the t-shirt. They were horrified by the out-of-context sentence for reasons similar to what has been already stated in support of this argument. It prompted them of lookism and judgmentality, and found the message to be disturbing in its suggestion to compare by some standard of beauty and to discriminate. Let me add some context: this person is socially and politically active (maybe what someone would call a "SJW"; definitely not what anyone would call "politically correct"), and is specially sensitive to issues of discrimination and sexism. This was enough, though, for me to wonder what kind of message I would be projecting by wearing that writing on me. I've been since then discouraged to ever wear the t-shirt in any public context. This story might have limited value because it's one anecdote, and because the central point is the impact of the clause when taken out of its original context. I don't want to construct this as an argument in favor of removal of the clause, but I want to mention this as evidence that it does carry emotionally (negatively) charged content. If this content can be avoided without compromising the purpose and function of the message, than by all means I would welcome and support the change. It's meaningful, as a community, to show willingness to respond to discomfort of any kind. In this case, I even see the potential to convey the original message in a more powerful way than the current formulation does. I'm not a good candidate for this, as the chosen language for this community is English, which is not my native language nor a language I feel very good at. I appreciate the poetic style of the original, and I think that Tim Peters has done an outstanding job at capturing these ideas into easy and humor-rich language. The opportunity would be to express the idea of aesthetic appeal of code in some way beyond the simplistic judgmental labelling of "beautiful" vs "ugly". To be fair, in my experience this has been a source of confusion to many Python newcomers, as the notion of "beauty", as with any other value judgment, is highly relative to the subject evaluating it. I've seen people think of the Python community as conceited because they would think they possess some absolute metric of beauty. One way out of the impasse is to draw upon the feeling behind the adjective. We call "beautiful" something that appeals to us, makes us comfortable, or inspires us awe. Ugly is something that makes us uncomfortable, repels us, disconcerts us. "Let awe and disconcert drive you"? "Attraction and repulsion are important"? "If it disturbs you, it's probably wrong"? I know these are terrible and will all fail the spirit and the style of the original, but I'm dropping suggestions with the hope to stimulate some constructive thought on the matter. I'm fine with PEP 20 being unchanged; and my goal is not to find a replacement or urge for a change, but rather to be willing to think about it. Cheers, Davide

2 years, 7 months

Moving to another forum system where moderation is possible

by James Lu

I agree completely. I propose Python register a trial of Stack Overflow Teams. Stack Overflow Teams is essentially your own private Stack Overflow. (I will address the private part later.) Proposals would be questions and additions or criticism would be answers. You can express your support or dissent of a proposal using the voting. Flags and reviews can be used to moderate. Stack Overflow Chat can be used for quick and casual discussion, and also to move irregular or irrelevant discussions away from the main site. Although the Stack Overflow platform is typically used for technical Q&A, there is precedent for using it as a way to discuss proposals: this is precisely what Meta Stack Overflow dies and it’s seen decent success. Anyone can register a @python.org email. Stack Overflow Teams can be configured to allow anyone with a @python.org email join the python-ideas team. I’m sure Stack Overflow Inc. is willing to grant Stack Overflow Teams to the PSF pro bono after the trial period expires. You can configure stack overflow to get email notifications as well. > On Sep 17, 2018, at 1:16 PM, Anders Hovmöller <boxed(a)killingar.net> wrote: > > >> It’s been almost a week since this “discussion” first started. Can we please stop this in the name of productive work on python-ideas? > > A better use of time might be to discuss moving to a better forum system where moderation is easier/possible. Email somehow has a shape that makes those things 100% probable and you can’t easily silence discussions that are uninteresting. > > / Anders

2 years, 7 months

Re: [Python-ideas] Moving to another forum system where moderation is possible

by James Lu

> Frankly, I think the bigger issue is all too human -- we get sucked in and > participate when we really know we shouldn't (or maybe that's just me). > That may be why some people misbehave, but we have no way of discouraging that misbehavior. > And I'm having a hard time figuring out how moderation would actually > result in the "good" discussion we really want in an example like the > "beautiful is better than ugly" issue, without someone trusted individual > approving every single post -- I don't imagine anyone wants to do that. In a forum, the beautiful is better than ugly issue would be locked. No more posts can be added. If someone wants to discuss another proposal branching off of the original discussion, they can start a new thread. If they just want to lampoon, we can courteously ask them to 1) take it elsewhere or 2) move the post to a “malarkey” section of the forum where people don’t get notified.

2 years, 7 months

Re: [Python-ideas] Moving to another forum system where moderation is possible

by James Lu

> It's absence is a big advantage. We're not a social network with > "likes". We don't need a bunch of argumentless "voting". Up/ down voting indicates how much consensus we have among the entire community- an expert might agree with another expert’s arguments but not have anything else to add, and an outsider might agree with the scenario an expert presents without having much more to add. Granular up/down votes are useful. > Believe it or not, I like the fact that you can't just edit posts. I've > lost count of the number of forum threads I've been on where comments to > the initial post make *no sense* because that initial post is nothing > like it was to start with. There is version history. Not all of us have the time to read through every single post beforehand to get the current state of discussion. Hmm, what if we used GitHub as a discussion forum? You’d make a pull request with an informal proposal to a repository. Then people can comment on lines in the diff and reply to each other there. The OP can update their branch to change their proposal- expired/stale comments on old diffs are automatically hidden. You can also create a competing proposal by forming from the OP’s branch and sending a new PR. > Just editing your post and expecting people to notice > is not going to cut it. You would ping someone after editing the post. > Approximately none of this has anything to do with the medium. If the > mailing list is obscure (and personally I don't think it is), it just > needs better advertising. A poorly advertised forum is equally > undiscoverable. It does have to do with the medium. First, people aren’t used to mailing lists- but that’s not what’s important here. If the PSF advertised for people to sign up over say twitter, then we’d get even more email. More +1 and more -1. Most of us don’t want more mailing list volume. The fact that you can’t easily find an overview people will post arguments that have already been made if they don’t have the extreme patience to read all that has been said before. For the rest of your comments, I advise you to read the earlier discussion that other people had in response to my email. > That message was rather bad in my not so humble opinion -- it was > just "I want my +1 button" without any argument. Your message is much > better as it have arguments. See, the absence of the button work! > > We're proposing and *discussing* things here not "likes" each other. > Write your arguments or be silent. Please respond to the actual arguments in both of the two emails that have arguments in support of +1/-1. +1/-1 reflects which usage scenarios people find valuable, since Python features sometimes do benefit one group at the detriment to another. Or use syntax/behavior for one thing that could be used for another thing, and some programming styles of python use cases would prefer one kind of that syntax/behavior.

2 years, 7 months

← Newer
1
2
3
4
5
6
7
8
9
10
Older →

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

Python-ideas September 2018