On Sun, Apr 19, 2020 at 5:59 PM David Mertz <mertz@gnosis.cx> wrote:
On Sun, Apr 19, 2020 at 7:00 AM Chris Angelico <rosuav@gmail.com> wrote:
Yes, exactly. One strike doesn't mean it's out, and with a proposal
like this, it's a matter of looking at a whole lot of imperfect
alternatives and seeing which tradeoffs we want to go with. (And
status quo is one such alternative, with tradeoffs of "names need to
be duplicated".)

I think you've missed on alternative.  Well, it's a variation on "status quo":  Use a silly magic helper function like my Q() or Alex' dict_of() in his sorcery library.  (Albeit, I tried his library, and it seemed to have a bug I filed an issue on, which might be fixed yesterday; I haven't tried again).

But if anyone really wants abbreviated calls with parameters now, they can use:

    process_record(**Q(email, firstname, lastname, cellphone))

That gets skipping `email=email, ...` if you really want.  And it's within a character or two of the length of other proposals.

Understand that I'm not really advocating for a magic function like Q().  I think it's important to keep in mind that anyone COULD create such a thing for 20 years, and very few people bothered to.  If repeating names was actually such a big pain point, a decent solution has been available for a very long time, but it didn't itch enough for many people to write a few lines with some slight magic to scratch it.

I think I'm uniquely qualified to say with certainty that this is 100% not true. A basic version like Q("email firstname lastname") like you wrote is indeed easy to do correctly, but I've said why I wouldn't want to use it and I think others would agree. The real thing, `Q(email, firstname, lastname, cellphone)`, is very hard to do correctly.

Q was essentially requested in this question: https://stackoverflow.com/questions/18425225/getting-the-name-of-a-variable-as-a-string

  1. The question is very popular. People have wanted this feature for a long time. I'd say that's a strong point in favour of these proposals in general.
  2. The question was asked in 2013, and went a long time without a satisfying answer.
  3. Most answers either say it's not possible or give naive implementations that easily fail.
  4. The currently accepted answer (which only appeared in 2019) looks legitimate and some people probably trust it, but it's very shaky. [It uses a regex](https://github.com/pwwang/python-varname/blob/25785b6aab2cdffc71095642e4d48a52548bbdf1/varname.py#L61) to parse the Python, in much the same way that one shouldn't parse HTML. [I just told the author to use my library instead](https://github.com/pwwang/python-varname/issues/3), and he quickly agreed that was much better and made me a collaborator.
Writing sorcery was hard. The first version was already hard enough, and [had several limitations](https://github.com/alexmojaki/sorcery/tree/7c85e5d802de26a435e4d190e02ca9326b6e7e57#rules-for-casting-spells). For example, you couldn't  use dict_of (i.e. Q()) twice in the same line.

Some time later I discovered [icecream](https://github.com/gruns/icecream). It seemed to have solved this problem - it could detect the correct call when there were several on the same line. It had about a thousand stars, 150 commits, and claimed to be well tested. I thought it was the real deal, and clearly other people did too. It had a complicated implementation that analysed the bytecode and did lots of manual parsing.

When I tried to incorporate this implementation into my own code, I realised that [it was in fact deeply broken and failed in many different simple cases](https://github.com/gruns/icecream/pull/33). People hadn't actually noticed because they rarely used it inside an expression, [they usually just wrote `ic(x)` on its own line](https://github.com/gruns/icecream/issues/39#issuecomment-552611756).

But I used the idea of analysing bytecode to write my own implementation, [executing](https://github.com/alexmojaki/executing). It was extremely hard. At several points I thought it was unsolvable or that I'd fool myself into thinking I'd solved it when actually it was quietly broken. At one point I was going crazy trying to figure out why the peephole optimizer was behaving inconsistently; I think it was because I hadn't correctly copied locations between AST nodes. Ultimately it's one of my proudest achievements. I'm pretty sure no one else has gotten this far. I think some others (e.g. icecream, varname, and amusingly, [q](https://github.com/zestyping/q)) got quite far, with significant effort, but still had a long way to go, and I'm not sure how much they're aware of that, let alone their users. I see you've tried while I wrote this, and it's pretty clear it's very far from robust.

And despite all that, as you've seen, sorcery still has limitations. It clashes with magic like pytest, IPython, and birdseye, and the only way to deal with that is with special cases where I think they're worth it. It can't work without access to the source code, e.g. in the standard interactive shell, or in .pyc files. And it relies on the implementation details of CPython and PyPy.

Nevertheless, in normal cases, I'm 99.9% sure that it would work correctly. And yet I still never use it, and I probably still wouldn't if I was 100% sure. If it became part of the standard library and was blessed by the Python community, I'd use it. Or if this proposal went through, I'd use the new syntax with glee.

Note that the use case of icecream and q has now been fulfilled by the new debugging syntax `f'{foo=}'`. Your argument could have equally be made against that syntax, but if it was, it failed, and rightly so because there was no simple function that could correctly replicate that behaviour. If we had nice syntax for keyword arguments though, it'd be trivial to write a function which could be used like `magic_print(**, foo)` and which could easily be customised to format and write its output in all sorts of ways. Having that earlier would have significantly reduced the case for the f-string syntax.

So no, this is not a problem that can be solved by a function in current Python.

On the other hand, we can change Python to make writing functions like that safe and easy. If code objects held a mapping from bytecode positions to AST nodes (which in turn would ideally know their source code, but that's not essential, e.g. it's not needed for the keyword argument stuff) then writing Q() and other neat magical functions (see the rest of sorcery for examples) would be relatively easy and reliable. And it would have other great applications, like making [tracebacks more detailed](https://github.com/ipython/ipython/pull/12150). I'd be really excited about that, and I've thought about proposing it here before. But it might be opening Pandora's box, and I expect there'd be plenty of (probably valid) objections.