Hi all,
In the operator module, both operator.concat and
operator.add are described as performing a + b.
When defining a type in pure python, it is not possible to give these different meanings.
However, when defining a C extension type, it is possible to fill both
tp_as_number->nb_add and tp_as_sequence->sq_concat with
different implementations to obtain different behaviors.
Is the fact that operator.concat (PySequence_Concat) and
operator.add (PyNumber_Add) can have different behaviors an
implementation detail of CPython, or is it intended behavior in the language itself?
I ask because a recent PR for numpy proposes implementing both of these slots, and discussion turned to whether it was intended to be possible to do this in the first place, and if so whether a patch will be needed to PyPy.
It seems that there was at least some vague intent for this to be possible - In bpo-29139, Serhiy said
Third-party classes (maybe NumPy arrays, I don't
know) can have different implementations of sq_concat and
nb_add.
It seems to me there are at least three stances that could be taken here:
- Specifying both is considered invalid: python should consider emitting a warning in
Type_READYif both are filled. - Specifying both is considered an implementation detail specific to CPython: the C API docs for the type slots should indicate this
- Specifying both is explicitly allowed and considered a language feature.
__concat__should be added as a slot_wrapper aroundsq_concatto allow the language feature to be accessed without writing C extensions.
Eric Wieser
Apologies if this is not the right list - this didn't feel right for python-ideas or bpo. I'm more than happy to repost this elsewhere if asked.
But how would you invoke them, other than using operator.add and operator.concat? A better UI would be to define new methods with more discoverable names.
On Fri, Jun 12, 2020 at 7:29 AM Eric Wieser wieser.eric+numpy@gmail.com wrote:
Hi all,
In the operator module, both operator.concat and
operator.add are
described as performing a + b.
When defining a type in pure python, it is not possible to give these
different meanings.
However, when defining a C extension type, it is possible to fill both
tp_as_number->nb_add and tp_as_sequence->sq_concat with
different
implementations to obtain different behaviors.
Is the fact that operator.concat (PySequence_Concat) and
operator.add (PyNumber_Add) can have different behaviors an
implementation detail of CPython, or is it intended behavior in the
language itself?
I ask because a recent PR for numpy proposes implementing both of these slots, and discussion turned to whether it was intended to be possible to do this in the first place, and if so whether a patch will be needed to PyPy.
It seems that there was at least some vague intent for this to be possible
- In bpo-29139, Serhiy said
Third-party classes (maybe NumPy arrays, I don't
know) can have
different implementations of sq_concat and nb_add.
It seems to me there are at least three stances that could be taken here:
- Specifying both is considered invalid: python should consider emitting a
warning in
Type_READYif both are filled. - Specifying both is considered an implementation detail specific to CPython: the C API docs for the type slots should indicate this
- Specifying both is explicitly allowed and considered a language feature.
__concat__should be added as a slot_wrapper aroundsq_concatto allow the language feature to be accessed without writing C extensions.
Eric Wieser
Apologies if this is not the right list - this didn't feel right for python-ideas or bpo. I'm more than happy to repost this elsewhere if asked.
Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/HCXMI7LP... Code of Conduct: http://python.org/psf/codeofconduct/
-- --Guido van Rossum (python.org/~guido) Pronouns: he/him **(why is my pronoun here?) http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...
But how would you invoke them, other than using operator.add and operator.concat?
The case I was thinking of was just a direct call to
operator.concat.
The motivation here is simply to make operator.concat do
something non-surprising on numpy arrays (be that np.concatenate(a, b),
or simply raise TypeError).
I doubt numpy would actually advocate using it, but we can at least remove
the foot-gun.
Eric
On Fri, 12 Jun 2020 at 17:18, Guido van Rossum guido@python.org wrote:
But how would you invoke them, other than using operator.add and operator.concat? A better UI would be to define new methods with more discoverable names.
On Fri, Jun 12, 2020 at 7:29 AM Eric Wieser wieser.eric+numpy@gmail.com wrote:
Hi all,
In the operator module, both operator.concat and
operator.add are
described as performing a + b.
When defining a type in pure python, it is not possible to give these
different meanings.
However, when defining a C extension type, it is possible to fill both
tp_as_number->nb_add and tp_as_sequence->sq_concat with
different
implementations to obtain different behaviors.
Is the fact that operator.concat (PySequence_Concat) and
operator.add (PyNumber_Add) can have different behaviors an
implementation detail of CPython, or is it intended behavior in the
language itself?
I ask because a recent PR for numpy proposes implementing both of these slots, and discussion turned to whether it was intended to be possible to do this in the first place, and if so whether a patch will be needed to PyPy.
It seems that there was at least some vague intent for this to be possible - In bpo-29139, Serhiy said
Third-party classes (maybe NumPy arrays, I don't
know) can have
different implementations of sq_concat and nb_add.
It seems to me there are at least three stances that could be taken here:
- Specifying both is considered invalid: python should consider emitting
a warning in
Type_READYif both are filled. - Specifying both is considered an implementation detail specific to CPython: the C API docs for the type slots should indicate this
- Specifying both is explicitly allowed and considered a language
feature.
__concat__should be added as a slot_wrapper aroundsq_concatto allow the language feature to be accessed without writing C extensions.
Eric Wieser
Apologies if this is not the right list - this didn't feel right for python-ideas or bpo. I'm more than happy to repost this elsewhere if asked.
Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/HCXMI7LP... Code of Conduct: http://python.org/psf/codeofconduct/
-- --Guido van Rossum (python.org/~guido) Pronouns: he/him **(why is my pronoun here?) http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...
On Fri, Jun 12, 2020 at 12:56 PM Eric Wieser wieser.eric+numpy@gmail.com wrote:
But how would you invoke them, other than using operator.add and operator.concat?
The case I was thinking of was just a direct call to
operator.concat.
The motivation here is simply to make operator.concat do
something non-surprising on numpy arrays (be that np.concatenate(a, b),
or simply raise TypeError).
I doubt numpy would actually advocate using it, but we can at least remove
the foot-gun.
Oh, that makes sense, as a CPython specific behavior. What does it do currently? Is the problem that it does the same thing as add? I'd vote for raising -- you shouldn't encourage or enable code that uses this pattern. (In general I am a big fan of just never using the operator module.)
-- --Guido van Rossum (python.org/~guido) Pronouns: he/him **(why is my pronoun here?) http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...
On Jun 12, 2020, at 04:21, Eric Wieser wieser.eric+numpy@gmail.com wrote:
It seems to me there are at least three stances that could be taken here:
- Specifying both is considered invalid: python should consider emitting a warning in
Type_READYif both are filled. - Specifying both is considered an implementation detail specific to CPython: the [C API docs for the type slots][2] should indicate this
- Specifying both is explicitly allowed and considered a language feature.
__concat__should be added as a slot_wrapper aroundsq_concatto allow the language feature to be accessed without writing C extensions.
If you can define the behavior at the C layer and not at the Python layer, then I think almost by definition it’s an implementation detail of CPython. Whether that’s intentional or not would help determine whether this should be demoted to a bug with an emitted warning, or promoted to a full-on feature supported in the language.
Cheers, -Barry
On Sat., 13 Jun. 2020, 2:51 am Barry Warsaw, barry@python.org wrote:
On Jun 12, 2020, at 04:21, Eric Wieser wieser.eric+numpy@gmail.com wrote:
It seems to me there are at least three stances that could be taken here:
- Specifying both is considered invalid: python should consider emitting
a warning in
Type_READYif both are filled. - Specifying both is considered an implementation detail specific to CPython: the [C API docs for the type slots][2] should indicate this
- Specifying both is explicitly allowed and considered a language
feature.
__concat__should be added as a slot_wrapper aroundsq_concatto allow the language feature to be accessed without writing C extensions.
If you can define the behavior at the C layer and not at the Python layer, then I think almost by definition it’s an implementation detail of CPython. Whether that’s intentional or not would help determine whether this should be demoted to a bug with an emitted warning, or promoted to a full-on feature supported in the language.
The one part that has already sort of leaked through to the language level is that only "nb_add" fully supports operand coercion.
https://bugs.python.org/issue11477 covers some of the oddities that arise when implementing only sq_concat.
I've mostly given up on ever fixing that, as I'm not sure we can do it without breaking the workarounds that people have in place for the oddities in the status quo (there were enough other projects relying on the existing semantics that the PyPy devs found it necessary to mimic CPython's implementation quirks in this regard).
That said, adding an sq_concat that raises a "don't do that" exception to an existing type that already implements nb_add sounds fine to me - from the Python level everything except operator.concat would encounter the nb_add slot first, so the exception should only trip up genuinely problematic code.
Cheers, Nick.
Cheers, -Barry
Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/DO5J63K3... Code of Conduct: http://python.org/psf/codeofconduct/
-
Barry Warsaw -
Eric Wieser -
Guido van Rossum -
Nick Coghlan