I have a working implementation for a new syntax which would make using keyword arguments a lot nicer. Wouldn't it be awesome if instead of:
foo(a=a, b=b, c=c, d=3, e=e)
we could just write:
foo(*, a, b, c, d=3, e)
and it would mean the exact same thing? This would not just be shorter but would create an incentive for consistent naming across the code base.
So the idea is to generalize the * keyword only marker from function to also have the same meaning at the call site: everything after * is a kwarg. With this feature we can now simplify keyword arguments making them more readable and concise. (This syntax does not conflict with existing Python code.)
The full PEP-style suggestion is here: https://gist.github.com/boxed/f72221e7e77370be3e5703087c1ba54d
I have also written an analysis tool you can use on your code base to see what kind of impact this suggestion might have. It's available at https://gist.github.com/boxed/610b2ba73066c96e9781aed7c0c0b25c . The results for django and twisted are posted as comments to the gist.
We've run this on our two big code bases at work (both around 250kloc excluding comments and blank lines). The results show that ~30% of all arguments would benefit from this syntax.
Me and my colleague Johan Lübcke have also written an implementation that is available at: https://github.com/boxed/cpython
/ Anders Hovmöller
> Which features of the TC39 committee's ECMAscript (ES) language governance
> model would be helpful to incorporate into the Python language governance
> model?
Having “beta” or “alpha” editions of features, special versions of the interpreter people can test out to see if they prefer the version with the new feature.
To prevent splintering, the main releases would only support the main feature set. In a worst case scenario, people can compile incompatible code to .pyc before running it.
Perhaps it’s because fewer Python functions involve transitioning between states. Web development and statistics don’t involve many state transition. State transitions are where I think I would find it useful to write contracts out explicitly.
You could disassemble (import dis) the lambda to biew the names of the lambdas.
@before(lambda self, key, _, length, get: self.length(), self.get(key))
Perhaps you could disassemble the function code and look at all operations or accesses that are done to “old.” and evaluate those expressions before the function runs. Then you could “replace” the expression.
@post(lambda self, key, old: old.get is None and old.length + 1 ==
self.length())
Either the system would grab old.get and old.length or be greedy and grab old.get is None and old.length + 1. It would then replace the old.get and old.length with injects that only respond to is None and +1.
Or, a syntax like this
@post(lambda self, key, old: [old.get(old.key)] is None and [old.self.length() + 1] ==
self.length())
Where the stuff inside the brackets is evaluated before the decorated function runs. It would be useful for networking functions or functions that do something ephemeral, where data related to the value being accessed needed for the expression no longer exists after the function.
This does conflict with list syntax forever, so maybe either force people to do list((expr,)) or use an alternate syntax like one item set syntax { } or double set syntax {{ }} or double list syntax [[ ]]. Ditto with having to avoid the literals for the normal meaning.
You could modify Python to accept any expression for the lambda function and propose that as a PEP. (Right now it’s hardcoded as a dotted name and optionally a single argument list surrounded by parentheses.)
I suggest that instead of “@before” it’s “@snapshot” and instead of “old” it’s “snapshot”.
Python does have unary plus/minus syntax as well as stream operators (<<, >>) and list slicing syntax and the @ operator and operators & and | if you want to play with syntax. There’s also the line continuation character for crazy lambdas.
Personally I prefer
@post(lambda self, key, old: {{old.self.get(old.key)}} and {{old.self.length() + 1}} ==
self.length())
because it’s explicit about what it does (evaluate the expressions within {{ }} before the function runs. I also find it elegant.
Alternatively, inside the {{ }} could be a special scope where locals() is all the arguments @pre could’ve received as a dictionary. For either option you can remove the old parameter from the lambda. Example:
@post(lambda self, key: {{self.get(key)}} and {{self.length() + 1}} ==
self.length())
Perhaps the convention should be to write {{ expr }} (with the spaces in between).
You’d probably have to use the ast module to inspect it instead of the dis modul. Then find some way to reconstruct the expressions inside the double brackets- perhaps by reconstructing the AST and compiling it to a code object, or perhaps by finding the part of the string the expression is located. dis can give you the code as a string and you can run a carefully crafted regex on it.
Hi folks,
I normally wouldn't bring something like this up here, except I think
that there is possibility of something to be done--a language
documentation clarification if nothing else, though possibly an actual
code change as well.
I've been having an argument with a colleague over the last couple
days over the proper way order of statements when setting up a
try/finally to perform cleanup of some action. On some level we're
both being stubborn I think, and I'm not looking for resolution as to
who's right/wrong or I wouldn't bring it to this list in the first
place. The original argument was over setting and later restoring
os.environ, but we ended up arguing over
threading.Lock.acquire/release which I think is a more interesting
example of the problem, and he did raise a good point that I do want
to bring up.
</prologue>
My colleague's contention is that given
lock = threading.Lock()
this is simply *wrong*:
lock.acquire()
try:
do_something()
finally:
lock.release()
whereas this is okay:
with lock:
do_something()
Ignoring other details of how threading.Lock is actually implemented,
assuming that Lock.__enter__ calls acquire() and Lock.__exit__ calls
release() then as far as I've known ever since Python 2.5 first came
out these two examples are semantically *equivalent*, and I can't find
any way of reading PEP 343 or the Python language reference that would
suggest otherwise.
However, there *is* a difference, and has to do with how signals are
handled, particularly w.r.t. context managers implemented in C (hence
we are talking CPython specifically):
If Lock.__enter__ is a pure Python method (even if it maybe calls some
C methods), and a SIGINT is handled during execution of that method,
then in almost all cases a KeyboardInterrupt exception will be raised
from within Lock.__enter__--this means the suite under the with:
statement is never evaluated, and Lock.__exit__ is never called. You
can be fairly sure the KeyboardInterrupt will be raised from somewhere
within a pure Python Lock.__enter__ because there will usually be at
least one remaining opcode to be evaluated, such as RETURN_VALUE.
Because of how delayed execution of signal handlers is implemented in
the pyeval main loop, this means the signal handler for SIGINT will be
called *before* RETURN_VALUE, resulting in the KeyboardInterrupt
exception being raised. Standard stuff.
However, if Lock.__enter__ is a PyCFunction things are quite
different. If you look at how the SETUP_WITH opcode is implemented,
it first calls the __enter__ method with _PyObjet_CallNoArg. If this
returns NULL (i.e. an exception occurred in __enter__) then "goto
error" is executed and the exception is raised. However if it returns
non-NULL the finally block is set up with PyFrame_BlockSetup and
execution proceeds to the next opcode. At this point a potentially
waiting SIGINT is handled, resulting in KeyboardInterrupt being raised
while inside the with statement's suite, and finally block, and hence
Lock.__exit__ are entered.
Long story short, because Lock.__enter__ is a C function, assuming
that it succeeds normally then
with lock:
do_something()
always guarantees that Lock.__exit__ will be called if a SIGINT was
handled inside Lock.__enter__, whereas with
lock.acquire()
try:
...
finally:
lock.release()
there is at last a small possibility that the SIGINT handler is called
after the CALL_FUNCTION op but before the try/finally block is entered
(e.g. before executing POP_TOP or SETUP_FINALLY). So the end result
is that the lock is held and never released after the
KeyboardInterrupt (whether or not it's handled somehow).
Whereas, again, if Lock.__enter__ is a pure Python function there's
less likely to be any difference (though I don't think the possibility
can be ruled out entirely).
At the very least I think this quirk of CPython should be mentioned
somewhere (since in all other cases the semantic meaning of the
"with:" statement is clear). However, I think it might be possible to
gain more consistency between these cases if pending signals are
checked/handled after any direct call to PyCFunction from within the
ceval loop.
Sorry for the tl;dr; any thoughts?
JS’ decisions are made by a body known as TC39, a fairly/very small group of JS implementers.
First, JS has an easy and widely supported way to modify the language for yourself: Babel. Babel transpires your JS to older JS, which is then run.
You can publish your language modification on the JS package manager, npm.
When a feature is being considered for inclusion in mainline JS, the proposal must first gain a champion (represented by 🚀)that is a member of TC-39. The guidelines say that the proposal’s features should already have found use in the community. Then it moves through three stages, and the champion must think the proposal is ready for the next stage before it can move on. I’m hazy on what the criterion for each of the three stages is. The fourth stage is approved.
I believe the global TC39 committee meets regularly in person, and at those meetings, proposals can advance stages- these meetings are frequent enough for the process to be fast and slow enough that people can have the time to try out a feature before it becomes main line JS. Meeting notes are made public.
The language and its future features are discussed on ESDiscuss.org, which is surprisingly filled with quality and respectful discussion, largely from experts in the JavaScript language.
I’m fairly hazy on the details, this is just the summary off the top of my head.
—
I’m not saying this should be Python’s governance model, just to keep JS’ in mind.
> To my mind, there is one very big reason we should be cautious about
> adopting JS language-design policies, namely, that they have led to a
> very, very poorly designed language. No doubt a good deal of that is
> baggage from early stages in which JS had a poor to nonexistent language
> design governance model. Nonetheless, the failure of JS to fix its
> numerous fundamental flaws, and especially the rapid feature churn in
> recent years, suggests to me that their model should be viewed with
> skepticism.
I disagree. The language is often very flexible and effective in its domains. I don’t know what you mean by “rapid feature churn”, churn usually means existing features are superseded by newer ones- this isn’t the case.
JS is much more nuanced than it appears on the surface. It’s understandable that those with only a glossing of JS look down on it, because JS really was a primitive language a few years ago.
You can learn about JS in depth with the poorly-named “You don’t know JS” free online book.
Could I get some feedback on this? I'd like to know if anyone thinks it
might make it through the pep process before spending too much (more) time
on it. That said, it seems valuable to me, and I'm willing to put in the
time, of course, IF it has a chance.
---------------------------
Problem:
Due to (biggest python WTF) (Problem 1), which prevents actual default
argument values being set in a function signature, many
functions look like:
> def teleport(from, to, passenger, hitchhiker=None,
food_accessory=None, comfort_accessory=None):
> if hitchhiker is None:
> hitchhiker = Fly()
>
> if food_accessory is None:
> food_accessory = Cheeseburger()
>
> if comfort_accessory is None:
> comfort_accessory = Towel()
>
> ...
This None checking and setting is unwieldy (Problem 2) boilerplate,
which is responsible for many extra lines of code in python
(Problem 3), and tends to distract from the real code (Problem 4) in a
function.
To reduce boilerplate, a ternary or binary expression can be used:
> def teleport(from, to, passenger, hitchhiker=None,
accessories=[]):
> hitchhiker = hitchhiker if hitchhiker or Fly()
# Existing Solution A
> def teleport(from, to, passenger, hitchhiker=None,
accessories=[]):
> hitchhiker = hitchhiker or Fly()
# Existing Solution B
These help, but are still quite repetitive:
* Existing Solution A is often avoided simply because many Pythonistas
dislike tenery expressions (perhaps due to
readability or hidden code branch concerns), and can quickly become
unwieldy when the new value (Fly()) is a more
complex expression, such as a list comprehension:
> def teleport(from, to, passenger, hitchhiker=None,
accessories=[]):
> hitchhiker = hitchhiker if hitchhiker or filter(lambda h: not
h.already_hitching(), available_hitchhikers)[0]
* Existing Solution B is less unwieldy (solving Problem 2), yet still
suffers from repetition (Problems 2, 3, and 4).
In a similar scenario, when we want to populate an empty list (say,
accesories), we could write:
> accessories |= [Cheeseburger(), Towel()]
# Almost-Solution C
However, this is not actually a solution, because:
* The inplace-or (|=) operator is not provided for None in python:
> food_accessor = None
> food_accessory |= Cheeseburger()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for |=: 'NoneType' and
'Cheeseburger'
This could be added, but would not solve the issue, because:
* If an non-default (non-None) argument value WERE provided, we be
would modifying
the argument unintentionally:
> class Larry:
> def __ior__(self, other):
> print("{} modified".format(self))
>
> teleport(..., hitchhiker=Larry(), ...)
<__main__.Larry object at 0x7f1ad9828be0> modified
And so Problems 1,2,3, and 4 are compounded.
Proposal:
The addition of a ?= operator could provide an elegant solution:
> def teleport(from, to, hitchiker=None, food_accessory=None,
comfort_accessory=None):
> hitchhiker ?= Fly()
> food_accessory ?= Cheeseburger()
> comfort_accessory ?= Towel()
In these examples,
> a = None
> a ?= b
> c = [1, 2]
> c ?= d
Would be equivalent to (assuming ?= was called __ielse__):
> class ExtendedNone(NoneType)
> def __ielse__(self, other):
> return other
>
> class ielse_list(list):
> def __ielse__(self, other):
> return self
>
> None = ExtendedNone()
>
> a = None
> a = a.__ielse__(b)
>
> c = iff_list([1, 2])
> c = a.__ielse__(d)
Although explicitly provided for list above, this ielse operator could
be defined
(again, as `return other` for NoneType only, but defaulted to `return
self` for
all other values of `a`, requiring very little implementation effort or
intrusion
into other types.
Possible interaction with typing
It may be also possible to define a ? suffix on function arguments, so
that:
> def func(a?):
> a ?= [1,2]
greatly shorting the current verbose form:
> from typing import Optional
>
> def func(a = None : Optional[Any])):
> a = [1,2] if a is None else a
and equivalent to:
> from typing import Optional
>
> def func(a: Optional[Any]):
> a = a.__ielse__([1,2])
Possible alternatives:
* The __ielse__ name is highly debatable. Alternatives such as __iif__
(inplace if), __imaybe__, __isomeor__ (borrowing from Rusts's Some/None
terminology, and reflecting the intended use as in relation to None types,
DB nulls, and similar void-like values).
* The ? function argument suffix is not necessary to implement the core
?= proposal.
* If the ? function argument suffix is implemented, implementation via
typing.Optional is not necessary; it could also be simply implemented so
that:
> def f(a?):
> ...
is equivalent to:
> def f(a=None):
> ...
---------------------------
Feedback would be much appreciated.
--
Lee
Hi,
I would be very interested to bring design-by-contract into python 3. I
find design-by-contract particularly interesting and indispensable for
larger projects and automatic generation of unit tests.
I looked at some of the packages found on pypi and also we rolled our own
solution (https://github.com/Parquery/icontract/). I also looked into
https://www.python.org/dev/peps/pep-0316/.
However, all the current solutions seem quite clunky to me. The decorators
involve an unnecessary computational overhead and the implementation of
icontract became quite tricky once we wanted to get the default values of
the decorated function.
Could somebody update me on the state of the discussion on this matter?
I'm very grateful for any feedback on this!
One of the reasons Guido left was the insane volume of emails he had to read on Python-ideas.
> A tiny bit of discussion is still better than none at all.
> And even if there's no discussion, there's a name attached
> to the message, which makes it more personal and meaningful
> than a "+1" counter getting incremented somewhere.
>
> Counting only makes sense if the counts are going to be
> treated as votes, and we don't do that.
I agree. I think this is good evidence in favor of using GitHub pull requests or GitHub issues- you can see exactly who +1’d a topic.
GitHub also has moderation tools and the ability to delete comments that are irrelevant, and edit comments that are disrespectful.
> I hope that, if any such change is made, a forum system is
> chosen that allows full participation via either email or news.
> Otherwise it will probably mean the end of my participation,
> because I don't have time to chase down and wrestle with
> multiple web forums every day.
+1, everyone should be accommodated. I believe GitHub has direct email capability. If you watch the repository and have email notifications on, you can reply directly to an email and it will be sent as a reply.
—
To solve the problem of tons of email for controversial decisions like :=, I don’t think GitHub issues would actually be the solution. The best solution would to have admins receive all the email, and broadcast a subset of the email sent, only broadcasting new arguments and new opinions.
Admins can do this “summary duty” every 12 hours on a rotating basis, where each admin takes turns doing summary duty.
This solution would mean a slower iteration time for the conversation, but it would significantly lessen the deluge of email, and I think that would make it more bearable for people participating in the conversation. After all, once a proposal has been fleshed out, what kind of conversation needs more than say 30 rounds of constructive discussion- in that case, if people reply every 25 hours, the discussion would be done in a month.
For transparency purposes, all of the email can be made received for approval can be published online.
