It is really necessary to check that a object has a Py_TPFLAGS_HEAPTYPE flag?
Hi everywhere. I decided to figure out why it is impossible to change methods of built-in types and found ``` if (!(type->tp_flags & Py_TPFLAGS_HEAPTYPE)) { PyErr_Format( PyExc_TypeError, "can't set attributes of built-in/extension type '%s'", type->tp_name); return -1; } ``` in function type_setattro. If comment this line and try to patch method __eq__ on the list for example the code will work as expected Test: ``` with patch("builtins.list.__eq__", return_value=False): assert [] == [] ``` Result: `AssertionError` as expected. Is there a real need to disallow patching of built-in types or is it just legacy?
The last time I noticed this question (probably around python 2.4?), it was considered a deliberate decision. There are languages with more open classes, such as (IIRC) Ruby and Smalltalk, but Python chose to remain somewhat closed, because monkey-patching is undesirable, and can be a problem for security audits. Whether it would really open additional attack vectors or crash possibilities ... wasn't judged worth the time to analyze or the risk of being wrong. Maybe those tradeoffs have changed, but if you don't get any other answers, that (and the bias for status quo) is the likely explanation. -jJ
On Tue, Feb 9, 2021 at 4:53 AM Kazantcev Andrey <heckad@yandex.ru> wrote:
I decided to figure out why it is impossible to change methods of built-in types and found ``` if (!(type->tp_flags & Py_TPFLAGS_HEAPTYPE)) { PyErr_Format( PyExc_TypeError, "can't set attributes of built-in/extension type '%s'", type->tp_name); return -1; } ``` in function type_setattro. If comment this line and try to patch method __eq__ on the list for example the code will work as expected
I removed this "if" and tested: $ ./python Python 3.10.0a5+ (heads/master-dirty:ab2d481639, Feb 16 2021, 00:55:58)
"a" == "b" False str.__eq__=lambda x,y: True "a" == "b" True
It seems technically possible to override attributes/methods of built-in types, but the question is more if it's desirable? IMO it goes against the specification of the Python language itself. For example, in Python, it's not allowed to add methods to built-in types. Example (with the removed "if")
str.strip_spaces=lambda s: s.strip(" \t") " a ".strip_spaces() 'a'
The problem is that built-in types are shared by everything, and adding non-standard methods means that code using it becomes non-standard (cannot be used on an "unmodified" Python). Do you want to see projects popping up on PyPI which rely on a special (non-standard) str methods like strip_spaces()? I didn't use Ruby on Rails, but the feedback that I heard is that the ability to extend built-in types is appealing, but it is problematic in practice. Victor -- Night gathers, and now my watch begins. It shall not end until my death.
On Mon, Feb 15, 2021 at 4:11 PM Victor Stinner <vstinner@python.org> wrote:
I didn't use Ruby on Rails, but the feedback that I heard is that the ability to extend built-in types is appealing, but it is problematic in practice.
Monkeypatching's such a part of the Ruby (and Javascript) culture that there isn't much pushback over it. The biggest problem is that it's such a part of the culture that it's not uncommon to have multiple modules redefining the same name in incompatible ways, which is as painful to track down as it sounds. Ruby also has refinements, which only modify within a specific scope (up to a single file), so forgetting whether it's enabled can get you even inside the same project. -Em
Victor Stinner wrote:
It seems technically possible to override attributes/methods of built-in types, but the question is more if it's desirable?
The problem is that you cannot override the method not only in built-in types but also, for example, in `lxml.etree` classes. I wrote a module that changes the `type_setattro` method to mine, which does not have this check. And I'm wondering if there are any problems in this solution (in addition to philosophical ones) or everything will work as it should (and not as an inheritance from built-in types). Thank you for participating in the discussion.
participants (4)
-
Emily Bowman -
Jim J. Jewett -
Kazantcev Andrey -
Victor Stinner