Re: Enhancing variable scope control
This is very, very deliberately a design goal of Python.
Most large Ruby projects have adopted code standards to explicitly
I suspect we're likely not talking about this same "this." Or if we are, I'd love to hear a justification -- any justification -- against what I'm talking about, because to date, I've never run into one. Many have tried, too. :) I'm talking about adding new functionality. NOT modifying existing functions. NOT replacing existing functions. NOT modifying existing, standard class internal variables or states. So If I add TestDottedQuad() to str, and once there it can't be replaced in the running instance, and you were to, unknowingly, end up using my extended string class, tell me where the harm is. No existing str functionality is modified. Anything you ever used will continue to work just as you expect, because it has not been modified in any way. If you don't know that TestDottedQuad() is there, it's essentially non-existent for you. If you do, and you use it... well, that's what it's there for, isn't it? In any case, I wanted that ability because I was doing a lot of interesting things to strings, and str was doing a stellar job of making that more difficult. I made it work WITHOUT monkey patching in any way, and also in a completely invisible fashion to anyone who has to parse my actual .py files, so I'm good. Preprocessors are very useful for such things. I get the extended syntax; everyone else gets standard (much more clumsy) python. prohibit the behavior you want, because the dangers greatly outweigh the benefits... Again, explain a danger to a nominal class user that arises due to adding NEW, immutable in the current instance, functions/functionality to a class that do not alter the existing base functionality. ANY danger. I'm very interested. In the context of Python, please. If you start talking Ruby, I'm unable to parse. --Ben
On Thu, Dec 01, 2022 at 03:19:38PM -0700, Anony Mous wrote:
What is the `str` interface? Perhaps not the best example, because strings have so many methods, but it will do. Under Python's design, we know what methods strings have. We absolutely categorically know that if `type(obj) is str` then we can rely on a specific interface, namely 47 methods and 33 dunders. What is the `str` interface with monkey-patching allowed? It's impossible to predict. The string interface depends on which methods have been monkey-patched in. It gets worse if we allow monkey-patching of individual instances as well as the entire class. Now you don't necessarily know that two strings support the same operations, even if their type is the same. If you have a string, you don't know whether or not it will support the TestDottedQuad method. These are points of friction that can make the language harder to use. How is a beginner supposed to know which methods are native to strings, and which are monkey-patched in? Where is `str.TestDottedQuad` implemented and documented? If you have a class and method you are not familiar with, say `Widget.frob()`, in a large application you don't know well, how do you find where `frob` was added to the class? It could have come from anywhere. What happens if you go to monkey-patch string with TestDottedQuad and *some completed unrelated library* has beat you to it and already done so? Monkey-patching is safe so long as you are the only one doing it. As soon as libraries get in the act, things go down hill very quickly. These are not insurmountable problems. Python supports powerful intraspection tools. Most classes that we write in pure Python, using the `class` keyword, support monkey-patching not just the class but individual instances as well, and it is considered a feature that most classes are extendable in that way. We mostly deal with that feature by *not using it*. The Ruby communittee learned the same lesson: the best way to use monkey-patching is to not use monkey-patching. https://avdi.codes/why-monkeypatching-is-destroying-ruby/ So the ability to monkey-patch classes and instances is considered to be feature of marginal usefulness. Sure, sometimes its handy, but mostly it just adds complexity. When it comes to builtins, the deciding factor is that the builtins are programmed in C (in CPython, other interpreters may do differently), and for speed and efficiency, and immutability, they usually don't include a `__dict__` that you can monkey-patch into. The class itself may have a read-only mapping proxy rather than a dict you can add items into. So in principle Python supports monkey-patching, but in practice CPython at least typically removes that support from most builtin types for efficiency reasons. And because monkey-patching is frowned upon, we don't miss it.
I find that implausible. You can do whatever interesting things you like with strings, you just can't use method syntax. You can't use `mystring.TestDottedQuad()` but you can use `TestDottedQuad(mystring)`, which is just a change in order (and one character fewer to type). I suppose the one thing you can't easily do with function syntax is give each individual instance its own distinct method. So if you have three instances, spam, eggs, cheese, with monkey-patching you can give all three instances their own frob() method, each method doing something different. But that way leads to chaos.
As I said above, that's fine when you are the only one doing it. But as soon as monkey-patching becomes popular, and everyone starts using it, then you have to deal with conflicts. Library A and library B both want to monkey-patch strings with the same method. Now you can only use one or the other. If they just used functions, they would be fine, because A.TestDottedQuad and B.TestDottedQuad live in separate namespaces, but with monkey-patching they both try to install into the same namespace, causing a clash. -- Steve
I entirely agree with all of Steven's points. I was going to reply with similar comments, but his are precisely spot-on, and probably more detailed than I would have provided. In particular, the Avdi Grimm essay that Steven linked too is one of the main references I would have provided as well. Redefining methods at the instance level is somewhat dangerous. But redefining the classes themselves after definition—and especially doing so for built-in classes—is where the greatest danger arises. What happens is that your `.TestDottedQuad()` becomes part of some package that is generally useful, and some *other* package starts to build it in as a dependency. Who doesn't want a string to be able to test whether it's a dotted quad, after all? But then more packages build on that other package as well, and a whole tower of dependencies lead back to your custom string method. When you realize that your method fails to handle some edge case correctly—or you change your mind about what "correctly" means for this method—that whole tower comes tumbling down because of the changed behavior. Maybe downstream pins to an old version of MousLib, but eventually that version becomes unmaintained or incompatible with a later version of Python or some other library. Maybe the IANA or IETF updates a standard that factually redefines what "dotted quad" means. Maybe the method needs to grow an optional parameter. Or maybe you turn out to be the author of "left-pad" ( https://qz.com/646467/how-one-programmer-broke-the-internet-by-deleting-a-ti... ). Writing a function `def test_dotted_quad(s str): ...` is perfectly straightforward (and snake-cased to by more Pythonic). But writing a class that inherits from `str` is also perfectly plausible, and you can add whatever methods you like. Those instances of `MousString` are still perfectly able to do all the string-ish things you'd like them to. But we nicely encapsulate "being able to do other things" within that class. This danger isn't *absent* from the standard methods already built into basic types. The Unicode Consortium could make changes that makes `mystring.title()` no longer behave the way it SHOULD under some edge cases (maybe scholars decide the handling of illuminated capitals in Asomtavruli is wrong in the Unicode 15.0 standard). But if that occurs, there will be PEPs, or at least substantial and careful discussion among the core developers. Yours, David... On Thu, Dec 1, 2022 at 8:02 PM Steven D'Aprano <steve@pearwood.info> wrote:
-- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.
On Thu, Dec 01, 2022 at 03:19:38PM -0700, Anony Mous wrote:
What is the `str` interface? Perhaps not the best example, because strings have so many methods, but it will do. Under Python's design, we know what methods strings have. We absolutely categorically know that if `type(obj) is str` then we can rely on a specific interface, namely 47 methods and 33 dunders. What is the `str` interface with monkey-patching allowed? It's impossible to predict. The string interface depends on which methods have been monkey-patched in. It gets worse if we allow monkey-patching of individual instances as well as the entire class. Now you don't necessarily know that two strings support the same operations, even if their type is the same. If you have a string, you don't know whether or not it will support the TestDottedQuad method. These are points of friction that can make the language harder to use. How is a beginner supposed to know which methods are native to strings, and which are monkey-patched in? Where is `str.TestDottedQuad` implemented and documented? If you have a class and method you are not familiar with, say `Widget.frob()`, in a large application you don't know well, how do you find where `frob` was added to the class? It could have come from anywhere. What happens if you go to monkey-patch string with TestDottedQuad and *some completed unrelated library* has beat you to it and already done so? Monkey-patching is safe so long as you are the only one doing it. As soon as libraries get in the act, things go down hill very quickly. These are not insurmountable problems. Python supports powerful intraspection tools. Most classes that we write in pure Python, using the `class` keyword, support monkey-patching not just the class but individual instances as well, and it is considered a feature that most classes are extendable in that way. We mostly deal with that feature by *not using it*. The Ruby communittee learned the same lesson: the best way to use monkey-patching is to not use monkey-patching. https://avdi.codes/why-monkeypatching-is-destroying-ruby/ So the ability to monkey-patch classes and instances is considered to be feature of marginal usefulness. Sure, sometimes its handy, but mostly it just adds complexity. When it comes to builtins, the deciding factor is that the builtins are programmed in C (in CPython, other interpreters may do differently), and for speed and efficiency, and immutability, they usually don't include a `__dict__` that you can monkey-patch into. The class itself may have a read-only mapping proxy rather than a dict you can add items into. So in principle Python supports monkey-patching, but in practice CPython at least typically removes that support from most builtin types for efficiency reasons. And because monkey-patching is frowned upon, we don't miss it.
I find that implausible. You can do whatever interesting things you like with strings, you just can't use method syntax. You can't use `mystring.TestDottedQuad()` but you can use `TestDottedQuad(mystring)`, which is just a change in order (and one character fewer to type). I suppose the one thing you can't easily do with function syntax is give each individual instance its own distinct method. So if you have three instances, spam, eggs, cheese, with monkey-patching you can give all three instances their own frob() method, each method doing something different. But that way leads to chaos.
As I said above, that's fine when you are the only one doing it. But as soon as monkey-patching becomes popular, and everyone starts using it, then you have to deal with conflicts. Library A and library B both want to monkey-patch strings with the same method. Now you can only use one or the other. If they just used functions, they would be fine, because A.TestDottedQuad and B.TestDottedQuad live in separate namespaces, but with monkey-patching they both try to install into the same namespace, causing a clash. -- Steve
I entirely agree with all of Steven's points. I was going to reply with similar comments, but his are precisely spot-on, and probably more detailed than I would have provided. In particular, the Avdi Grimm essay that Steven linked too is one of the main references I would have provided as well. Redefining methods at the instance level is somewhat dangerous. But redefining the classes themselves after definition—and especially doing so for built-in classes—is where the greatest danger arises. What happens is that your `.TestDottedQuad()` becomes part of some package that is generally useful, and some *other* package starts to build it in as a dependency. Who doesn't want a string to be able to test whether it's a dotted quad, after all? But then more packages build on that other package as well, and a whole tower of dependencies lead back to your custom string method. When you realize that your method fails to handle some edge case correctly—or you change your mind about what "correctly" means for this method—that whole tower comes tumbling down because of the changed behavior. Maybe downstream pins to an old version of MousLib, but eventually that version becomes unmaintained or incompatible with a later version of Python or some other library. Maybe the IANA or IETF updates a standard that factually redefines what "dotted quad" means. Maybe the method needs to grow an optional parameter. Or maybe you turn out to be the author of "left-pad" ( https://qz.com/646467/how-one-programmer-broke-the-internet-by-deleting-a-ti... ). Writing a function `def test_dotted_quad(s str): ...` is perfectly straightforward (and snake-cased to by more Pythonic). But writing a class that inherits from `str` is also perfectly plausible, and you can add whatever methods you like. Those instances of `MousString` are still perfectly able to do all the string-ish things you'd like them to. But we nicely encapsulate "being able to do other things" within that class. This danger isn't *absent* from the standard methods already built into basic types. The Unicode Consortium could make changes that makes `mystring.title()` no longer behave the way it SHOULD under some edge cases (maybe scholars decide the handling of illuminated capitals in Asomtavruli is wrong in the Unicode 15.0 standard). But if that occurs, there will be PEPs, or at least substantial and careful discussion among the core developers. Yours, David... On Thu, Dec 1, 2022 at 8:02 PM Steven D'Aprano <steve@pearwood.info> wrote:
-- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.
participants (3)
-
Anony Mous -
David Mertz, Ph.D. -
Steven D'Aprano