Hi,
For technical reasons, many functions of the Python standard libraries
implemented in C have positional-only parameters. Example:
-------
$ ./python
Python 3.7.0a0 (default, Feb 25 2017, 04:30:32)
>>> help(str.replace)
replace(self, old, new, count=-1, /) # <== notice "/" at the end
...
>>> "a".replace("x", "y") # ok
'a'
>>> "a".replace(old="x", new="y") # ERR!
TypeError: replace() takes at least 2 arguments (0 given)
-------
When converting the methods of the builtin str type to the internal
"Argument Clinic" tool (tool to generate the function signature,
function docstring and the code to parse arguments in C), I asked if
we should add support for keyword arguments in str.replace(). The
answer was quick: no! It's a deliberate design choice.
Quote of Yury Selivanov's message:
"""
I think Guido explicitly stated that he doesn't like the idea to
always allow keyword arguments for all methods. I.e. `str.find('aaa')`
just reads better than `str.find(needle='aaa')`. Essentially, the idea
is that for most of the builtins that accept one or two arguments,
positional-only parameters are better.
"""
http://bugs.python.org/issue29286#msg285578
I just noticed a module on PyPI to implement this behaviour on Python functions:
https://pypi.python.org/pypi/positional
My question is: would it make sense to implement this feature in
Python directly? If yes, what should be the syntax? Use "/" marker?
Use the @positional() decorator?
Do you see concrete cases where it's a deliberate choice to deny
passing arguments as keywords?
Don't you like writing int(x="123") instead of int("123")? :-) (I know
that Serhiy Storshake hates the name of the "x" parameter of the int
constructor ;-))
By the way, I read that "/" marker is unknown by almost all Python
developers, and [...] syntax should be preferred, but
inspect.signature() doesn't support this syntax. Maybe we should fix
signature() and use [...] format instead?
Replace "replace(self, old, new, count=-1, /)" with "replace(self,
old, new[, count=-1])" (or maybe even not document the default
value?).
Python 3.5 help (docstring) uses "S.replace(old, new[, count])".
Victor
I just spent a few minutes staring at a bug caused by a missing comma
-- I got a mysterious argument count error because instead of foo('a',
'b') I had written foo('a' 'b').
This is a fairly common mistake, and IIRC at Google we even had a lint
rule against this (there was also a Python dialect used for some
specific purpose where this was explicitly forbidden).
Now, with modern compiler technology, we can (and in fact do) evaluate
compile-time string literal concatenation with the '+' operator, so
there's really no reason to support 'a' 'b' any more. (The reason was
always rather flimsy; I copied it from C but the reason why it's
needed there doesn't really apply to Python, as it is mostly useful
inside macros.)
Would it be reasonable to start deprecating this and eventually remove
it from the language?
--
--Guido van Rossum (python.org/~guido)
Hi All,
I find importing defaultdict from collections to be clunky and it seems
like having a default should just be an optional keyword to dict. Thus,
something like,
d = dict(default=int)
would be the same as
from collections import defaultdict
d = defaultdict(int)
Any thoughts?
Thanks,
++Steve
Hi. I suspect that this may have been discussed to death at some point
in the past, but I've done some searching and I didn't come up with
much. Apologies if I'm rehashing an old argument ;)
I often find myself writing __init__ methods of the form:
def __init__(self, foo, bar, baz, spam, ham):
self.foo = foo
self.bar = bar
self.baz = baz
self.spam = spam
self.ham = ham
This seems a little wordy and uses a lot of vertical space on the
screen. Occasionally, I have considered something like:
def __init__(self, foo, bar, baz, spam, ham):
self.foo, self.bar, self.baz, self.spam, self.ham = \
foo, bar, baz, spam, ham
... just to make it a bit more compact - though in practice, I'd
probably not do that with a list quite that long ... two or three items
at most:
def __init__(self, foo, bar, baz):
self.foo, self.bar, self.baz = foo, bar, baz
When I do that I'm torn because I know it has a runtime impact to create
and unpack the implicit tuples and I'm also introducing a style
asymmetry in my code just because of the number of parameters a method
happens to have.
So why not have an augmented assignment operator for object attributes?
It addresses one of the same broad issues that the other augmented
assignment operators were introduced for (that of repeatedly spelling
names).
The suggestion therefore is:
def __init__(self, foo, bar, baz, spam, ham):
self .= foo, bar, baz, spam, ham
This is purely syntactic sugar for the original example:
def __init__(self, foo, bar, baz, spam, ham):
self.foo = foo
self.bar = bar
self.baz = baz
self.spam = spam
self.ham = ham
... so if any of the attributes have setters, then they are called as
usual. It's purely a syntactic shorthand. Any token which is not
suitable on the RHS of the dot in a standard "obj.attr =" assignment is
a syntax error (no "self .= 1").
The comma-separators in the example are not creating a tuple object,
they would work at the same level in the parser as the import
statement's comma-separated lists - in the same way that "from pkg
import a, b, c" is the same as saying:
import pkg
a = pkg.a
b = pkg.b
c = pkg.c
... "self .= a, b, c" is the same as writing:
self.a = a
self.b = b
self.c = c
E.
There's a PR to the peps proposal here:
https://github.com/python/peps/pull/242
The full text of the current proposal is below. The motivation for this is
that for complex decorators, even if the type checker can figure out what's
going on (by taking the signature of the decorator into account), it's
sometimes helpful to the human reader of the code to be reminded of the
type after applying the decorators (or a stack thereof). Much discussion
can be found in the PR. Note that we ended up having `Callable` in the type
because there's no rule that says a decorator returns a function type (e.g.
`property` doesn't).
This is a small thing but I'd like to run it by a larger audience than the
core mypy devs who have commented so far.
Here's the proposed text (wordsmithing suggestions in the PR please):
+Decorators
+----------
+
+Decorators can modify the types of the functions or classes they
+decorate. Use the ``decorated_type`` decorator to declare the type of
+the resulting item after all other decorators have been applied::
+
+ from typing import ContextManager, Iterator, decorated_type
+ from contextlib import contextmanager
+
+ class DatabaseSession: ...
+
+ @decorated_type(Callable[[str], ContextManager[DatabaseSession]])
+ @contextmanager
+ def session(url: str) -> Iterator[DatabaseSession]:
+ s = DatabaseSession(url)
+ try:
+ yield s
+ finally:
+ s.close()
+
+The argument of ``decorated_type`` is a type annotation on the name
+being declared (``session``, in the example above). If you have
+multiple decorators, ``decorated_type`` must be topmost. The
+``decorated_type`` decorator is invalid on a function declaration that
+is also decorated with ``overload``, but you can annotate the
+implementation of the overload series with ``decorated_type``.
+
--
--Guido van Rossum (python.org/~guido <http://python.org/%7Eguido>)
I'm gonna take a shot at elaborating this point.
We Python programmers often tout Python as a high-level, high-productivity
language where complex and useful things are easy to do. However, vanilla
Python classes are anything but high level; they're basically glorified
dictionaries that require an awful lot of boilerplate for basic features.
__init__ verbosity is just one of the symptoms of this.
I'm going to posit we need declarative classes. (This is what a library
like attrs provides, basically.) For a class to be declarative, it needs to
be possible to inspect the class for its attributes and more.
I'm putting forward three examples. These examples are based on attrs since
that's what I consider to be the best way of having declarative classes in
Python today.
First example. Here's a slide from a recent talk I've given to a local
Python meetup: http://tinche.github.io/talks/attrs/#/8/5 This is a class
decorator that, when applied to any attrs class, will generate a simple
shallow copy() method for that class. This copy method will be faster than
copy.copy() and IMO less surprising since it will actually invoke __init__
(and all logic inside). This isn't production code, just a proof of concept
for the talk.
Second example: I'm working on a library that can turn dictionaries into
attrs classes (i.e. turn unstructured data into structured data). Example:
http://cattrs.readthedocs.io/en/latest/structuring.html#attrs-classes. When
dealing with nested classes the library not only needs to know exactly
which fields a class has but also the types of those fields, recursively
(for which I'm planning to use attrs metadata). This functionality is
useful when preparing data to be stored or sent elsewhere, since
serialization libraries and clients in other languages expect json/msgpack
etc.
Third example: I work at a mobile games company. The backends are Python,
the games are written in other languages. The backends and the games share
data structures - requests, responses, the save game, game data. I want to
have a single source of truth for these structures, so I want to generate
classes/structs in other languages from the backend Python classes.
All of these examples require declarative classes to do properly. Other
approaches are, I suppose, possible in some cases - like inspecting
__init__ and whatnot. But I claim the underlying issue is that we should be
declaring classes differently, both to enable these new, high-level use
cases and because declarative classes are much more readable and contain
less boilerplate than what we have now, and this is (for me) Python is
about.
Message: 1
> Date: Fri, 28 Apr 2017 09:28:55 +1000
> From: Steven D'Aprano <steve(a)pearwood.info>
> To: python-ideas(a)python.org
> Subject: Re: [Python-ideas] Augmented assignment syntax for objects.
> Message-ID: <20170427232853.GC22525(a)ando.pearwood.info>
> Content-Type: text/plain; charset=iso-8859-1
>
> On Wed, Apr 26, 2017 at 08:52:46PM -0400, Juancarlo A?ez wrote:
>
> > In my experience, what Python is lacking is a way to declare attributes
> > outside of the constructor. Take a look at how it's done in C#, Swisft,
> or
> > Go.
>
> Since you apparently already know how they do it, how about telling us
> (1) how they do it, and (2) *why* they do it?
>
>
> > Object attributes outside of the constructor would solve things more
> > relevant than the vertical space used when assigning constructor
> parameters
> > to attributes.
>
> Solve which things?
>
>
> > For example, multiple inheritance is well designed in
> > Python, except that it often requires constructors with no parameters,
> > which leads to objects with no default attributes.
>
> Can you elaborate?
>
>
>
> --
> Steve
>
>
Notice that I said *discourage* rather than *deprecate*.
Quoting the documentation:
The operator module exports a set of efficient functions
corresponding to the intrinsic operators of Python. For
example, operator.add(x, y) is equivalent to the expression
x+y. The function names are those used for special class
methods; variants without leading and trailing __ are also
provided for convenience.
https://docs.python.org/3/library/operator.html
I propose four simple documentation changes, and no code change:
(1) The quoted paragraph is factually wrong to say:
"The function names are those used for special class methods"
We can fix that by changing it to:
"Some function names are those used for special class methods".
(2) Replace the word "convenience" in the quoted paragraph by
"backward compatibility";
(3) Add a note close to the top of the page that the non-dunder
names are preferred for new code.
(4) And a note stating that existing dunder functions will
remain, but no new ones will be added.
The existing dunder names will remain aliases to the non-dunder names;
they will not be deprecated (maybe in Python 5000 *wink*). Those who
really want to use them can continue to do so.
Regarding point (1) above:
- Not all operator functions have a dunder alias.
- The dunder functions don't always correspond to a dunder method. For
example, there is operator.__concat__ for sequence concatenation, but no
str.__concat__ or list.__concat__ methods.
- Even when the dunder function corresponds by name to the dunder
method, they don't mean the same thing. For example, operator.__add__ is
*not* the same as just calling the first argument's __add__ method.
- And finally, I fail to see how having to type an extra four characters
is a "convenience".
Long ago, when the operator module was first introduced, there was a
much stronger correspondence between the operator.__dunder__ functions
and dunder methods. But I think that correspondence is now so weak that
we should simply treat it as a historical artifact.
--
Steve
I recently filed this as a bug, and was asked to repost to python-dev or
python-ideas for greater visibility:
http://bugs.python.org/issue30140
Without further ado, here is my original report:
---------------------------
We are writing a system for overloading NumPy operations (see PR [1] and
design doc [2]) that is designed to copy and extend Python's system for
overloading binary operators.
The reference documentation on binary arithmetic [3] states:
> Note: If the right operand's type is a subclass of the left operand’s
type and that subclass provides the reflected method for the operation,
this method will be called before the left operand’s non-reflected method.
This behavior allows subclasses to override their ancestors’ operations.
However, this isn't actually done if the right operand merely inherits from
the left operand's type. In practice, CPython requires that the right
operand defines a different method before it defers to it. Note that the
behavior is different for comparisons, which defer to subclasses regardless
of whether they implement a new method [4].
I think this behavior is a mistake and should be corrected. It is just as
useful to write generic binary arithmetic methods that are well defined on
subclasses as generic comparison operations. In fact, this is exactly the
design pattern we propose for objects implementing special operators like
NumPy arrays (see NDArrayOperatorsMixin in [1] and [2]).
Here is a simple example, of a well-behaved type that implements addition
by wrapping its value and that returns NotImplemented when the other
operand has the wrong type:
class A:
def __init__(self, value):
self.value = value
def __add__(self, other):
if not isinstance(other, A):
return NotImplemented
return type(self)(self.value + other.value)
__radd__ = __add__
def __repr__(self):
return f'{type(self).__name__}({self.value!r})'
class B(A):
pass
class C(A):
def __add__(self, other):
if not isinstance(other, A):
return NotImplemented
return type(self)(self.value + other.value)
__radd__ = __add__
A does not defer to subclass B:
>>> A(1) + B(1)
A(2)
But it does defer to subclass C, which defines new methods (literally
copied/pasted) for __add__/__radd__:
>>> A(1) + C(1)
C(2)
With the current behavior, special operator implementations need to
explicitly account for the possibility that they are being called from a
subclass by returning NotImplemented. My guess is that this is rarely done,
which means that most of these methods are broken when used with
subclasses, or subclasses needlessly reimplement these methods.
Can we fix this logic for Python 3.7?
[1] https://github.com/numpy/numpy/pull/8247
[2]
https://github.com/charris/numpy/blob/406bbc652424fff332f49b0d2f2e5aedd8191…
[3] https://docs.python.org/3/reference/datamodel.html#object.__ror__
[4] http://bugs.python.org/issue22052
-----------------------------
Mark Dickinson's response:
This is probably worth bringing up on the python-dev or python-ideas
mailing lists for greater visibility. I can't think of any plausible
non-historical reason why it makes sense for comparisons to behave one way
and arithmetic operators another. Altering this might be a PEP-level
change, though.
The "Coercion rules" section[1] of the Python 2.7 docs is a bit more
explicit about the intent:
"""
Exception to the previous item: if the left operand is an instance of a
built-in type or a new-style class, and the right operand is an instance of
a proper subclass of that type or class and overrides the base’s __rop__()
method, the right operand’s __rop__() method is tried before the left
operand’s __op__() method.
"""
so the check for an override was clearly intentional, rather than an
implementation convenience or accident. (It's also clearly intentional in
the source and comments.) The 3.x docs don't have the "and overrides"
language; I haven't figured out why and when that language changed.
[1]
https://docs.python.org/release/2.7.6/reference/datamodel.html#coercion-rul…
You might want to check out attrs (http://attrs.readthedocs.io/en/stable/).
It can generate the __init__ for you, and much much more.
Date: Tue, 25 Apr 2017 14:33:49 +0200
> From: George Fischhof <george(a)fischhof.hu>
> To: Paul Moore <p.f.moore(a)gmail.com>
> Cc: Python-Ideas <python-ideas(a)python.org>, "Steven D'Aprano"
> <steve(a)pearwood.info>
> Subject: Re: [Python-ideas] Augmented assignment syntax for objects.
> Message-ID:
> <
> CAFwcP0gjg1BXeQ2pmv637bXG6+vcXiVu1QNEkdetuHjmoT_yCw(a)mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> 2017. ?pr. 25. de. 10:04 ezt ?rta ("Paul Moore" <p.f.moore(a)gmail.com>):
>
> On 25 April 2017 at 03:53, Steven D'Aprano <steve(a)pearwood.info> wrote:
> > On Tue, Apr 25, 2017 at 02:08:05AM +0100, Erik wrote:
> >
> >> I often find myself writing __init__ methods of the form:
> >>
> >> def __init__(self, foo, bar, baz, spam, ham):
> >> self.foo = foo
> >> self.bar = bar
> >> self.baz = baz
> >> self.spam = spam
> >> self.ham = ham
> >>
> >> This seems a little wordy and uses a lot of vertical space on the
> >> screen.
> >
> > It does, and while both are annoying, in the grand scheme of things
> > they're a very minor annoyance. After all, this is typically only an
> > issue once per class, and not even every class, and vertical space is
> > quite cheap. In general, the barrier for accepting new syntax is quite
> > high, and "minor annoyance" generally doesn't reach it.
>
> I suspect that with a suitably creative use of inspect.signature() you
> could write a decorator for this:
>
> @auto_attrs
> def __init__(self, a, b, c):
> # Remaining init code, called with self.a, self.b and self.c set
>
> I don't have time to experiment right now, but will try to find time
> later. If nothing else, such a decorator would be a good prototype for
> the proposed functionality, and may well be sufficient for the likely
> use cases without needing a syntax change.
>
> Paul
>
>
[redirecting back to the list]
On 04/19/2017 11:15 PM, Stephan Hoyer wrote:
> Ethan and Steven,
>
> Thanks for your feedback on this one. I agree that it probably doesn't make sense for the standard library.
>
> I'm still not really happy with any of the standard approaches for choosing a function based on an enum value -- they
> all seem pretty verbose/ugly -- but clearly I'm bothered by this more than most, and the standard library is not a good
> place for novel solutions.
I'm curious, what did you find ugly with:
class TestEnum(CallableEnum):
@enum
def hello(text):
"a pleasant greeting"
print('hello,', text)
@enum
def goodbye(text):
print('goodbye,', text)
--
~Ethan~
