Deprecate dunder functions from operator module
(For the avoidance of all doubt, the following has nothing to do with dunder methods like int.__add__ etc., but only the functions in the operator module.) The operator module contains various function versions of operators, e.g. operator.add for + , operator.sub for - , etc. There are also dunder versions of each function, and according to the documentation: The function names are those used for special class methods; variants without leading and trailing __ are also provided for convenience. https://docs.python.org/3/library/operator.html (except that's not quite right, because there are no functions __radd__ etc. matching the special class methods). The existence of redundant dunder functions came as a complete surprise to me when I recently discovered it, and I daresay it will come as a surprise to many people. Having two versions in the operator module has been known to lead to confusion, e.g.: https://mail.python.org/pipermail/tutor/2014-October/102877.html All the third party documentation I've seen for the operator module refers to the dunderless versions. I doubt that there is anyone who prefers to write operator.__add__ instead of operator.add, and Terry Reedy has kindly determined that (aside from tests) the standard library doesn't use any of the dunder versions. https://mail.python.org/pipermail/python-list/2014-October/679226.html I propose a few things: * institute a policy that, in the event of a new function being added to the operator module, only the dunderless version will be added; * change the documentation to make it clear that the dunderless versions should be used, rather than merely being "convenience" functions; * add a prominent note that the dunder versions exist for backwards compatibility only and should not be used in new code. At this point, I do NOT propose having the dunder versions of the functions raise a depreciation warning. I would be content for them to merely be documented as discouraged, and defer actual depreciation until Python 5000 or so. (But if there is a strong preference for actual deprecation, I won't argue against it.) Thoughts? -- Steven
On 10/29/2014 05:53 PM, Steven D'Aprano wrote:
I propose a few things:
* institute a policy that, in the event of a new function being added to the operator module, only the dunderless version will be added;
* change the documentation to make it clear that the dunderless versions should be used, rather than merely being "convenience" functions;
* add a prominent note that the dunder versions exist for backwards compatibility only and should not be used in new code.
+1 -- ~Ethan~
On 30 October 2014 11:38, Ethan Furman <ethan@stoneleaf.us> wrote:
On 10/29/2014 05:53 PM, Steven D'Aprano wrote:
I propose a few things:
* institute a policy that, in the event of a new function being added to the operator module, only the dunderless version will be added;
* change the documentation to make it clear that the dunderless versions should be used, rather than merely being "convenience" functions;
* add a prominent note that the dunder versions exist for backwards compatibility only and should not be used in new code.
+1
+0 from me (I agree the dunder versions are likely pointless, but I also think they're pretty harmless). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 10/29/2014 06:38 PM, Ethan Furman wrote:
On 10/29/2014 05:53 PM, Steven D'Aprano wrote:
I propose a few things:
* institute a policy that, in the event of a new function being added to the operator module, only the dunderless version will be added;
* change the documentation to make it clear that the dunderless versions should be used, rather than merely being "convenience" functions;
* add a prominent note that the dunder versions exist for backwards compatibility only and should not be used in new code.
+1
Actually, make that -1. I'm just crafting some tests to explore how NotImplemented impacts various classes, and the dunder versions make the whole thing much nicer. -- ~Ethan~
On Sat, Nov 01, 2014 at 01:02:19PM -0700, Ethan Furman wrote:
On 10/29/2014 06:38 PM, Ethan Furman wrote:
On 10/29/2014 05:53 PM, Steven D'Aprano wrote:
I propose a few things:
* institute a policy that, in the event of a new function being added to the operator module, only the dunderless version will be added;
* change the documentation to make it clear that the dunderless versions should be used, rather than merely being "convenience" functions;
* add a prominent note that the dunder versions exist for backwards compatibility only and should not be used in new code.
+1
Actually, make that -1.
I'm just crafting some tests to explore how NotImplemented impacts various classes, and the dunder versions make the whole thing much nicer.
I'm surprised. I can't imagine dunder names making anything look nicer. It sounds like you are writing "operator.__add__" instead of "operator.add", which doesn't look nicer to me. Can you show me your code? Note that the dunder versions won't be going away soon, if at all, so if your code is relying on them, they will still be there. They just won't be prominently advertised. -- Steven
On 11/03/2014 07:57 PM, Steven D'Aprano wrote:
On Sat, Nov 01, 2014 at 01:02:19PM -0700, Ethan Furman wrote:
On 10/29/2014 06:38 PM, Ethan Furman wrote:
On 10/29/2014 05:53 PM, Steven D'Aprano wrote:
I propose a few things:
* institute a policy that, in the event of a new function being added to the operator module, only the dunderless version will be added;
* change the documentation to make it clear that the dunderless versions should be used, rather than merely being "convenience" functions;
* add a prominent note that the dunder versions exist for backwards compatibility only and should not be used in new code.
+1
Actually, make that -1.
I'm just crafting some tests to explore how NotImplemented impacts various classes, and the dunder versions make the whole thing much nicer.
I'm surprised. I can't imagine dunder names making anything look nicer. It sounds like you are writing "operator.__add__" instead of "operator.add", which doesn't look nicer to me.
The dunder versions are being used to check if a particular method is in a type's dictionary, to see how the subclass should be tailored for a test. Without the dunder versions I would have had to create my own mapping between dunder and dunderless names.
Can you show me your code?
Check out the "Role of NotImplemented" over on pydev. The script is the last half of the first post. -- ~Ethan~
On Mon, Nov 03, 2014 at 08:04:10PM -0800, Ethan Furman wrote:
On 11/03/2014 07:57 PM, Steven D'Aprano wrote:
I'm just crafting some tests to explore how NotImplemented impacts various classes, and the dunder versions make the whole thing much nicer.
I'm surprised. I can't imagine dunder names making anything look nicer. It sounds like you are writing "operator.__add__" instead of "operator.add", which doesn't look nicer to me.
The dunder versions are being used to check if a particular method is in a type's dictionary, to see how the subclass should be tailored for a test. Without the dunder versions I would have had to create my own mapping between dunder and dunderless names.
The dunder *methods* aren't going away. The dunder *functions* in operator won't be going away either. But even if they did, it is trivial to change: getattr(operator, op) # op looks like '__add__', etc. to getattr(operator, op.strip('_')) except of course for full generality you might need a mapping to cover cases where the operator function isn't the same as the dunder method. E.g. operator.truth, operator.contains. (I don't think they are relevant to your case though.) In any case, unless I'm missing something, I don't think that even full-blown removal of the dunder functions would affect your code in any significant way, and I'm not arguing for removal.
Can you show me your code?
Check out the "Role of NotImplemented" over on pydev. The script is the last half of the first post.
https://mail.python.org/pipermail/python-dev/2014-November/136875.html -- Steven
On 10/29/2014 8:53 PM, Steven D'Aprano wrote:
(For the avoidance of all doubt, the following has nothing to do with dunder methods like int.__add__ etc., but only the functions in the operator module.)
The operator module contains various function versions of operators, e.g. operator.add for + , operator.sub for - , etc.
There are also dunder versions of each function, and according to the documentation:
The function names are those used for special class methods; variants without leading and trailing __ are also provided for convenience.
https://docs.python.org/3/library/operator.html
(except that's not quite right, because there are no functions __radd__ etc. matching the special class methods).
The existence of redundant dunder functions came as a complete surprise to me when I recently discovered it, and I daresay it will come as a surprise to many people. Having two versions in the operator module has been known to lead to confusion, e.g.:
https://mail.python.org/pipermail/tutor/2014-October/102877.html
All the third party documentation I've seen for the operator module refers to the dunderless versions.
I doubt that there is anyone who prefers to write operator.__add__ instead of operator.add, and Terry Reedy has kindly determined that (aside from tests) the standard library doesn't use any of the dunder versions.
I only determined before that 'operator.__' is never used except for 6 tests. I just grepped Lib/*.py for 'from operator import' and none of the 28 line imports a dunder version.
https://mail.python.org/pipermail/python-list/2014-October/679226.html
I propose a few things:
* institute a policy that, in the event of a new function being added to the operator module, only the dunderless version will be added;
* change the documentation to make it clear that the dunderless versions should be used, rather than merely being "convenience" functions;
* add a prominent note that the dunder versions exist for backwards compatibility only and should not be used in new code.
At this point, I do NOT propose having the dunder versions of the functions raise a depreciation warning. I would be content for them to merely be documented as discouraged, and defer actual depreciation until Python 5000 or so. (But if there is a strong preference for actual deprecation, I won't argue against it.)
Thoughts?
I think the dunder versions should at least be deprecated in the doc, and possibly in the functions also. Could we also augment 2to3. I went to https://code.openhub.net/search and searched for "from operator import" and in the first 160 Python hits, there were no dunders. (Anyone else could continue checking from page 17.) Not counting 'attrgetter' and 'itemgetter' hits might leave 100 non-dunder imports. "from operator import mul" has 150 hits, while same with "__mul__" has one hit in a 'pythran' test/test_operator.py file. For 'add' and '__add__', the numbers are 330 and 8. For operator.add and operator.__add__, dunder use is more common: 2079 versus 187, but still under 10% -- Terry Jan Reedy
participants (4)
-
Ethan Furman -
Nick Coghlan -
Steven D'Aprano -
Terry Reedy