Mailman 3 September 2018 - Python-ideas

Re: [Python-ideas] Keyword only argument on function call
by Jeroen Demeyer Sept. 11, 2018

Sept. 11, 2018

On 2018-09-11 14:20, Steven D'Aprano wrote: > On Tue, Sep 11, 2018 at 08:53:55PM +1000, Steven D'Aprano wrote: > [...] >> Or perhaps we could have an officially blessed way to give tools a hint >> as to what the real signature is. >> >> class Child(Parent): >> @signature_hint(Parent.method) >> def method(self, **kwargs): >> pass I was planning to submit a PEP to support a __signature__ attribute (without the _hint) in inspect.… [View More]

1 0

On the Python function call interface
by Jonathan Fine Sept. 11, 2018

Sept. 11, 2018

I hope that in this thread we will share and develop our understanding of how Python handles the interface between defining a function and calling a function. In this message, I describe a tension between the caller and implementer of a function. I intend in further messages to cover Elias Tarhini's post on __iter__(), keys(), and the mapping protocol [This affects the behaviour of fn(**kwargs), perhaps to our disadvantage.] https://mail.python.org/pipermail/python-ideas/2018-… [View More]September/053320.html Marko Ristin-Kaufmann post on Pre-conditions and post-conditions [This is, in part, about wrapping a function, so it can be monitored.] https://mail.python.org/pipermail/python-ideas/2018-August/052781.html and probably some other topics. My main concern is that we know what choices are already available, and the human forces that good design decisions will balance. The signature of a function has at least three purposes. First, to provide the CALLER of the function with guidance as to how the function should be used. Second, to provide the IMPLEMENTER of the function with already initialised variables, local to the function body. Third, to provide both caller and implementer with visible default values. When there are many arguments, these two purposes can be opposed. Here's an example. The IMPLEMENTER might want to write (not tested) def method(self, **kwargs): # Do something with kwargs. # Possibly mutate kwargs. # Change values, remove items, add items. # And now pass the method up. super().method(**kwargs) In some case, the implementer might prefer def method(self, aaa, bbb, **kwargs): # Do something with aaa and bbb. super().method(**kwargs) However, the CALLER might wish that the implementer has a signature def method(self, aaa, bbb, ccc, ddd, eee, fff, ggg, hhh): and this encourages the implementer to write super().method(aaa=aaa, ...). However, there is an alternative: def method(self, aaa, bbb, ccc, ddd, eee, fff, ggg, hhh): lcls = dict(locals()) lcls.pop('aaa') lcls.pop('bbb') # Do something with aaa and bbb. super().method(**lcls) This implementation bring benefits to both the user and the implementer. But it's decidedly odd that the local variables ccc through to hhh are initialised, but not explicitly used. I think it would help to dig deeper into this, via some well-chosen examples, taken for real code. By the way, Steve Barnes has suggested Python be extended to provide "access to a method, or dictionary, called __params__ which would give access, as a dictionary, to the parameters as supplied in the call, (or filled in by the defaults)." [See: https://mail.python.org/pipermail/python-ideas/2018-September/053322.html] Such a thing already exists, and I've just showed how it might be used. https://docs.python.org/3/library/functions.html#locals >>> def fn(a, b, c, d=4, e=5): return locals() >>> fn(1, 2, 3, e=6) {'e': 6, 'd': 4, 'c': 3, 'a': 1, 'b': 2} I think, once we understand Python function and code objects better, we can make progress without extending Python, and know better what extensions are needed. Much of what we need to know is contained in the inspect module: https://docs.python.org/3/library/inspect.html -- Jonathan [View Less]

2 1

Re: [Python-ideas] Python dialect that compiles into python
by James Lu Sept. 11, 2018

Sept. 11, 2018

I wholly support this proposal.

1 0

Python dialect that compiles into python
by Robert Vanden Eynde Sept. 11, 2018

Sept. 11, 2018

Many features on this list propose different syntax to python, producing different python "dialects" that can statically be transformed to python : - a,b += f(x) → _t = f(x); a += _t; b += _t; (augmented assignement unpacking) - a = 2x + 1 → a = 2*x + 1 (juxtaposition is product) - f(*, x, y) → f(x=x, y=y) (simplekwargs) - DSL specific language - all def become @partially def - etc... Using a modified version of ast, it is relatively easy to modifiy the syntax tree of a program to produce … [View More]

7 6

Re: [Python-ideas] Keyword only argument on function call
by Chris Barker Sept. 11, 2018

Sept. 11, 2018

Did you mean to take this off-list? I hope not, as I'm bringing it back on. On Tue, Sep 11, 2018 at 8:09 AM, Anders Hovmöller <boxed(a)killingar.net> wrote: > I've spent this whole thread thinking: "who in the world is writing code > with a lot of spam=spam arguments? If you are transferring that much state > in a function call, maybe you should have a class that holds that state? Or > pass in a **kwargs dict? > > > Kwargs isn’t good because it breaks static … [View More]analysis which we really want. > well, Python isn't a static language, and I personally have my doubts about trying to make it more so -- but that makes it sound like we need some solution for type annotations, rather than executable code. But see my other note -- I do agree that a well specified function signature is a good thing. > for .format() -- that's why we now have f-strings -- done. > > > Not really no. F-strings can’t be used for many strings if you need to > localize your app. If you don’t have that need then yes f-strings are great > and I’m fortunate to work on an app where we don’t need to but we shouldn’t > dismiss people who have this need. > Darn -- I hope this was brought up in the original f-string conversation. > for templates -- are you really passing all that data in from a bunch of > variables?? as opposed to, say, a dict? That strikes me as getting code and > data confused (which is sometimes hard not to do...) > > Yes. For example we have decorators on our views that fetch up objects > from our DB based on url fragments or query parameters and then pass to the > function. This means we have all access control in our decorators and you > can’t forget to do this because a view without access control decorators > are stopped by a middleware. So several of variables are there before we > even start the actual view function. > hmm -- this is a bit of what I mean by mixing data and code -- in my mind a database record is more data than code, so better to use a dict than a class with attributes matching teh fields. However, I also see the advantage of mixing the code -- providing additional logic, and pre and post processing, like it sounds like you are doing. But python is a nifty dynamic language -- could your views have an "as_dict" property that provided just the fields in the existing instance? If you were in Stockholm I’d offer you to come by our office and I could > show some code behind closed doors :) > As it happens, I am much closer than usual -- in the Netherlands, but still not Stockholm :-) - CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker(a)noaa.gov [View Less]

1 0

Positional-only parameters
by Victor Stinner Sept. 11, 2018

Sept. 11, 2018

Hi, For technical reasons, many functions of the Python standard libraries implemented in C have positional-only parameters. Example: ------- $ ./python Python 3.7.0a0 (default, Feb 25 2017, 04:30:32) >>> help(str.replace) replace(self, old, new, count=-1, /) # <== notice "/" at the end ... >>> "a".replace("x", "y") # ok 'a' >>> "a".replace(old="x", new="y") # ERR! TypeError: replace() takes at least 2 arguments (0 given) ------- When converting the … [View More]

19 27

Re: [Python-ideas] Keyword only argument on function call
by Steven D'Aprano Sept. 10, 2018

Sept. 10, 2018

On Sat, Sep 08, 2018 at 09:54:59PM +0200, Anders Hovmöller wrote: [...] [Steven (me)] > > If we believe that this consistency is desirable then maybe this would > > be a good thing. Linters could warn when you use "name=spam" instead of > > "*, name"; style guides can demand that code always uses this idiom > > whenever practical, tutorials and blog posts will encourage it, and the > > peer pressure to rename variables to match the called function's > … [View More]> parameters would be a good thing too. > > That’s the same straw man argument as before as far as I can tell. [...] Enough with the false accusations of straw-manning. I am stating my prediction of the consequence of your proposal. Just because you dislike the conclusion I draw doesn't make it a straw-man, and your repeated false accusations (whether intentional or not) amount to Poisoning The Well. "Steven's arguments are straw-men, so we don't need to address them or pay attention to him." Except it isn't a straw-man. You might disagree, which is your right, you might even just dismiss them as "well that's your opinion" (it *is* my opinion, but I hope a reasoned, logical one), you might even think that the consequences I state are a good thing. But stop trying to poison my reputation by labelling me a straw-manner. > > But if consistency for consistency's sake is not generally a good thing, > > then we ought not to add such syntax just for conciseness. > > But conciseness for conciseness sake is just as not-good and we do > have special syntax for that: positional arguments. I’m proposing to > level the playing field. For short argument lists, there's little or nothing wrong with positional arguments. In fact, I would say that positional arguments are, in some circumstances, far better: len(obj) versus len(target=obj) mylist.append(item) versus mylist.append(obj=item) My argument isn't about making absolute judgements of what is good or bad in all cases. It is about weighing up the benefit in some cases (which I have acknowledged) against the disadvantage in other cases. [...] > > Of course I understand that with this proposal, there's nothing > > *forcing* people to use it. But it shifts the *preferred* idiom from > > explicit "name=spam" to implicit "*, name" and puts the onus on people > > to justify why they aren't naming their local variables the same as the > > function parameter, instead of treating "the same name" as just another > > name. > > Did you just argue that my proposed syntax is so great it’s going to > become the preferred idiom? :) No. I argued that your proposed syntax could become the perferred idiom because it is short and concise and people mistake that for "great". I'm not blind to the advantage. If I had to make many function calls that looked like this: result = function(spam=spam, eggs=eggs, cheese=cheese, foo=foo, bar=bar, baz=baz, fe=fe, fi=fi, fo=fo, fum=fum) I'd be sick of it too and want something better. But we have to balance many competing and often contradictory interests, and I don't think you have made your case that the pain of the above is worse than the (alleged) benefit of being able to write it in a more terse, implicit way: result = function(*, spam, eggs, cheese, foo, bar, baz, fe, fi, fo, fum) is shorter, but is it really better? I don't think you've made that case. But many people do seem to think shorter is better as a general rule. > > but that active preference for long parameter lists seems to be a very > > rare, more common is the view that *at best* long parameter lists is a > > necessary evil that needs mitigation. > > I think that’s overly doom and gloomy. Just look at all the arguments > for open(). Yes, open() is a good example of how long parameter lists are not necessarily a bad thing. With sensible defaults, probably 95% of calls to open need no more than one or two arguments. But can you remember the Bad Old Days before programming languages supported default values? I do. Now imagine making a call to open() with eight required arguments. > In any case I’m proposing a way to mitigate the pain in a subset of > cases (probably not the case of open()!). Yes, I acknowledge that. Just because I sympathise with your pain doesn't mean I want to see your syntax added to the language. > > certainly wouldn't want to put a hard limit on the number of > > parameters allowed. But in general, I think it is unquestionable that > > long parameter lists are a code-smell. > > Or a domain smell. Some domains are smelly. Indeed. > > It is also relevant in this sense. Large, complex function calls are > > undoubtably painful. We have mitigated that pain somewhat by various > > means, probably the best of which are named keyword arguments, and > > sensible default values. The unintended consequence of this is that it > > has reduced the pressure on developers to redesign their code to avoid > > long function signatures, leading to more technical debt in the long > > run. > > So... you’re arguing Python would have been better off without keyword > arguments and default values? Not at all. As I already stated, keyword arguments are great, and I'm not arguing for hair-shirt programming where we intentionally make things as painful as possible. I'll make an analogy here. Pain, real physical pain, is bad, and it is both kind and medically advantagous to reduce it when necessary. But *necessary* is the key word, because sometimes pain is an important biological signal that says Don't Do That. People with no pain receptors tend to die young because they repeatedly injure themselves and don't even know it. Good doctors make careful judgement about the minimum amount of painkillers needed, and don't hand out morphine for stubbed toes because the consequences will be worse than the benefit gained. The analogy in programming follows: reducing immediate pain can, sometimes, cause long-term pain in the form of technical debt, which is worse. It takes a careful balancing act to decide when it is appropriate to take on technical debt in order to avoid short-term annoyance. We cannot hope to make such a sensible decision if we focus only on the immediate relief and not on the long term consequences. > > Your suggestion would also reduce the pain of functions that require > > many arguments. That is certainly good news if the long argument list is > > *truly necessary* but it does nothing to reduce the amount of complexity > > or technical debt. > > The unintended consequence is likewise that it > > reduces the pressure on developers to avoid designing such functions in > > the first place. > > > > This might sound like I am a proponent of hair-shirt programming where > > everything is made as painful as possible so as to force people to > > program the One True Way. That's not my intention at all. I love my > > syntactic sugar as much as the next guy. But I'd rather deal with the > > trap of technical debt and excessive complexity by avoiding it in the > > first place, not by making it easier to fall into. > > But you’ve only proposed or implied ways to avoid it by force and > pain. I don't think the pain is all that great. Its a minor annoyance to write keyword arguments like parameter=parameter, and a good IDE or editor can help there. But I acknowledge there is some pain, and the more you need to do this, the more you feel it. But in any case, you made a proposal. I don't have to come up with a better proposal before I am permitted to argue against yours. "We must do something. This is something, therefore we must do it." is never a valid argument. > And you’re arguing my suggestion is bad because it lowers pain. > So I think you’re contradicting yourself here. That’s fine but you > should admit it and call it a trade off at least. I have done nothing but call it a trade off! I've repeatedly argued that in my opinion the benefit (which I have repeatedly acknowledged!) doesn't outweigh the (predicted) costs as I see them. [...] > >> There are actual APIs that have lots of arguments. GUI toolkits are a great > >> example. Another great example is to send a context dict to a template > >> engine. > > > > Indeed. And I'm sympathetic that some tasks are inherently complex and > > require many arguments. Its a matter of finding a balance between being > > able to use them, without encouraging them. > > What does that mean? We shouldn’t encourage GUI toolkits for Python? > Or we shouldn’t encourage working with complex domains? You probably > meant to say something else but it came across weirdly. Sorry for being unclear. I meant that we shouldn't encourage complex, multi-argument functions, while still accepting that sometimes they are unavoidable (and even occasionally a good thing). > > You claimed the benefit of "conciseness", but that doesn't actually > > exist unless your arguments are already local variables named the same > > as the parameters of the function you are calling. > > Which they often already are. Again: I have numbers. You do not. I don't dispute the numbers you get from your code base. You should be careful about generalising from your code to other people's, especially if you haven't seen their code. > > Pointing out that weakness in your argument is not a straw man. > > You did notice that not just me called it a straw man on this list right? Yes I did. Many people call the world flat too. What's your point? -- Steve [View Less]

3 3

Add Unicode-aware str.reverse() function?
by Paddy3118 Sept. 10, 2018

Sept. 10, 2018

I wrote a blog post <http://paddy3118.blogspot.com/2009/07/case-of-disappearing-over-bar.html>nearly a decade ago on extending a Rosetta Code task example <http://rosettacode.org/wiki/Reverse_a_string>to handle the correct reversal of strings with combining characters. On checking my blog statistics today I found that it still had a readership and revisited the code <http://rosettacode.org/wiki/Reverse_a_string#Python:_Unicode_reversal> (and updated it to Python3.6).. … [View More]

5 6

Re: [Python-ideas] Improving fn(arg=arg, name=name, wibble=wibble) code
by Ben Lewis Sept. 8, 2018

Sept. 8, 2018

> > It’s obvious but there is one easy way to shorten the code: using > **kwargs. It’s way shorter but the down sides are: - the “real” function signature gets hidden so IDEs for example won’t pick > it up > - the error when you make a mistake when calling is not in your code > anymore but one level down. This is confusing. One could imagine solving this specific case by having a > type annotation of “this function has the types of that function”. Maybe: > def _open(*… [View More]

1 0

Re: [Python-ideas] Keyword only argument on function call
by David Mertz Sept. 8, 2018

Sept. 8, 2018

> > def foo(): > a, b, c = 1, 2, 3 > function(a=77, **use('b d')) > > foo() > You get the output “77 None 33 None”. So basically it doesn’t work at all. > For the reason I wrote clearly in my last mail. You have to dig through the > stack frames to make use() work. > OK, you are right. Improved implementation. Still not very hard. In any case, I'm concerned with the API to *use* the `use()` function, not how it's implemented. The point is really just … [View More]

1 0