I was prompted to post this after seeing comments on "Proposal to change List Sequence Repetition (*) so it is not useless for Mutable Objects" that you cannot in general tell whether an object is mutable or not. Well, why shouldn't you be able to tell? Python is superb at introspection, but ISTM there's a gap here. Here's an idea for Python N (N>=4): Every object has a boolean __mutable__ attribute. It would e.g. be False for ints and strings, True by default for objects that can be mutable, such as lists and dicts. It could be an optional extra argument to mutable object constructors (so you have the option of making them immutable): L = list(somesequence, mutable=False) There could also be a syntax for list/set literals and dictionary displays to indicate that they should be immutable (just as we preface literal strings with 'r' to indicate raw strings). I'm not sure what; I thought of f[1,2,3] for a literal immutable ("frozen") list, but that's already valid syntax. You can "freeze" a mutable object by changing its __mutable__ attribute from True to False. You are not allowed to change it from False to True, nor to mutate an object that has it False. I am sure there must be advantages in being able to tell if an object is mutable or not. (Perhaps "hashable" equals "not mutable" ?) Now: You don't need the frozenset class. You just have a set with __mutable__ = False. And you can freeze dicts (classes? modules?) etc., to make sure users of your code don't mess with them. And (fasten your safety belts): You don't need tuples. You just have a list with __mutable__ = False. And you don't have to explain to beginners one of the most FAQ "Why do we need lists and tuples?" Best wishes (and sorry I can't seem to get my e-mail out of double vertical spacing mode), Rob Cliffe
On Wed, Jun 1, 2016, at 18:26, Rob Cliffe wrote:
Er... so does setting __mutable__ = False cause/require contained objects to be transitively mutable? Because if it does, you still need tuples to have an "immutable list of mutable objects". If it doesn't, then this doesn't solve the other thread's problem.
On Wed, Jun 01, 2016 at 11:26:07PM +0100, Rob Cliffe wrote:
I hate to break it to you, but merely setting a flag on an object doesn't make it mutable or immutable *wink* That means that every class (possibly including builtins) needs to be re-written so that every mutator method checks this flag and decides whether or not to allow the mutation. For example, you suggest: "You don't need tuples. You just have a list with __mutable__ = False." Now the code for list append must be: def append(self, item): # only written in C, because its a built-in if self.__mutable__: ... else: raise SomeError Now multiply that by *every* mutator class and method. This will be especially burdensome for people writing classes intended to be mutable, since they have to support immutability whether they want it or not. (Since the caller might change obj.__mutable__ to False, which is allowed.) This will break type-checking and duck-typing and make feature detection a pain: # currently this works if hasattr(obj, 'append'): handle_things_that_can_append(obj) else: handle_things_that_cant_append(obj) # with your scheme if hasattr(obj, 'append'): if obj.__mutable__: handle_things_that_can_append(obj) else: handle_things_that_cant_append(obj) else: handle_things_that_cant_append(obj)
This goes against the "no constant bool arguments" design guideline.
We already have syntax for a built-in immutable list-like sequence object (a "frozen list" if you will): (1, 2, 3)
No it does not. (1, 2, [3]) is both immutable and unhashable.
And we get this functionality for free, right? *wink* Somebody has to write this code. I'm not sure they will appreciate having to throw away all the perfectly good frozenset code and tests, and re-writing set to handle both cases. Not to mention the breaking of backwards compatibility to get rid of frozenset.
(1) I don't remember ever seeing a beginner confused by this difference, but I see plenty of experienced programmers who are sure that they must be. (2) Have you actually read the FAQ? Because if you have, you should realise that having an immutable list is not the same as having a tuple. Tuples are intended for "record-like" structures, where the items' positions are meaningful, while lists are intended for "array-like" structures where all the items have the same type and their position is not meaningful. While it's not compulsory to treat tuples and lists this way, it is very useful to do so. -- Steve
On 02.06.2016 15:05, Rob Cliffe wrote:
I don't think that's an overly stupid idea. In such regard, Python wouldn't be the first project dealing with mutability in this manner. PostgreSQL already does. One question I would have: would it be enforced? Or can objects designer choose between enforced immutability and informative immutability? Or would it be enforced inconsistently at some places where at other places it does not matter. To me the 100% enforced variant would be the least troublesome. One definite advantage of broader use of immutable objects would be optimization: using the same memory parts, re-using the same object etc. It's not my field in computer science but other people will be able to imagine a lot of other possibilities here. Best, Sven
Just remembering that one is free to implement whatever "switchable" containers one wants in the language as it is now - no need to force (full or slightly) incompatible changes that imply on performance degradation onto every single Python user for a feature that was not needed up to this day. Implementing this in sequence-abc, and mapping-abc classes is trivial. A reasonable request for the language could be exactly to allow a "__mutable__" property on the sequence, container and mapping protocols, and a "mutable" built-in callable, that in the absence of "__mutable__" could check for the methods signature (if it does not have "__mutable__" but has "__setitem__" , then mutable(obj) returns True, for example). On 2 June 2016 at 10:55, Sven R. Kunze <srkunze@mail.de> wrote:
On 6 June 2016 at 19:05, Sven R. Kunze <srkunze@mail.de> wrote:
No. Trivially a method on an extension object can be storing state in a C struct. Let trivially: Class IsThisMutable: def __init__(self): self._cache = {} def random(self): self._cache[len(self._cache)] = random.random() return self._cache[len(self._cache)-1] def size(self): return len(self._cache) this has interior mutability - it uses __setitem__ on self._cache to track that state. You could remove __setitem__ from IsThisMutable instances post construction and it would still be getting mutated. -Rob
On 06.06.2016 09:55, Robert Collins wrote:
That's what I thought as well. So, I see a huge benefit for enforced immutability when it comes to reducing errors and not re-inventing the wheel. Thinking this further, __init__ would be the only function to change the state of an immutable object. Once created, it will never change. Immutable also implies hashability IMHO. Moreover, immutable object would not be allowed to query data from global/external variables as those can change and would change the observable state of the object without the object noticing. So, the allowed way of creating a state for an immutable object would be using a new container as you did (by defining self._cache) and store immutable objects only there. Would this make sense? Sven
On Mon, Jun 6, 2016, at 06:44, Sven R. Kunze wrote:
You're continuing to fail to address the fact that some aspects of this proposal implicitly claim that immutable objects shall not hold references to mutable objects (a tuple containing a reference to a list is not hashable*, does this mean tuples are not immutable?) while other claims (OP's claim that an "immutable list" could substitute for a tuple) imply that they can.
How are you gonna stop them? *Java seems to work fine with mutable lists being hashable and trusting users to responsibly manage the life cycles of dictionaries using them as keys, but this is an argument that we don't really need to start up again here and now.
On 06.06.2016 15:34, Random832 wrote:
(a tuple containing a reference to a list is not hashable*, does this mean tuples are not immutable?)
Your question in brackets is actually the most important question here. I would say it depends on who one defines it but I don't see a contradiction to have both variants of tuples in the same languages. Whatever fits your needs more in a certain situation.
With a hammer? I don't know how you want me to answer that. It's an implementation detail to me as other languages can handle it properly. There is a way of stopping them by preventing all ways of manipulating state variables of an object. What those ways are is more a topic for long-served devs of CPython 4 or 5 ;-) Sven
On Mon, Jun 06, 2016 at 12:44:10PM +0200, Sven R. Kunze wrote:
Python classes have supported "cooperative immutability" for 20+ years. For example, the pure Python Decimal and Fraction classes are immutable only by convention, if you modify the _private attributes you can change them. And yet they work fine. I'm not saying that "real immutability" wouldn't be nice to have, but I think that describing it as huge benefit is possible overstating it.
Thinking this further, __init__ would be the only function to change the state of an immutable object. Once created, it will never change.
How would you enforce that?
Immutable also implies hashability IMHO.
That's incorrect. We all agree that tuples are immutable, I trust. py> t = (1, 2, {}, 3) py> hash(t) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: unhashable type: 'dict' Immutable collections are only hashable if all their components are hashable.
How would you enforce that?
I don't understand this paragraph, sorry. -- Steve
On 06.06.2016 16:40, Steven D'Aprano wrote:
Nobody says it does not work fine. But sometimes, people need an immutability class blueprint. I saw several implementation attempts for such immutable-appearing objects. They tend to be hard to maintain.
I'm not saying that "real immutability" wouldn't be nice to have, but I think that describing it as huge benefit is possible overstating it.
Since Python became a Turing-complete language, most PEPs are just that: preventing the re-invention of the wheel. And that's always a huge benefit; especially viewed economically.
That's an implementation detail.
So what? I don't see your point in quoting current behavior. Especially, because this is a theoretical discussion about a hypothetical behavior which might differ from the current one. Furthermore, it depends on how you define what a tuple is. Is it the mere container to n (possibly mutable) objects? Or does it also comprise its components? In the former case, it's the current behavior IIRC. In the later case, its components need to be immutable as well. That's basically what I was trying to say below.
Don't give access to them in the first place. That again seems to me like another implementation detail.
(this might be because you split it up into several pieces? ;) ) I might refer here to your tuple example above. The tuple itself might look immutable but its components are not. What immutability means instead for a domain-specific object is that it needs to look and to feel immutable. Usually, ones does not inspect such objects but use it via its public API. So, the output of its API definitely needs to be constant through out the life-time of the object. Internally, the object could need to store more than a single number to represent its state. Maybe, even a container of containers could be necessary. But then all those containers and the containers' contents need to be immutable as well to assist the designer of the object's class. That's at least the way I understand it how immutability would need to work in order to prevent errors and simplify developers' lives. Sven
On 06/06/2016 11:44, Sven R. Kunze wrote:
That would be one viable design. But aren't there use cases for initially creating an object as mutable, then at some point "freezing" it by changing the mutable flag from True to False? (Obviously the converse would not be allowed.) Regards, Rob Cliffe
On 08.06.2016 16:33, Eric V. Smith wrote:
See the rejected PEP-351 for one version of the freeze protocol.
Wow, still 11 years later equal thoughts arise again. Reading through the rejection notes, I can see the concerns are as abstract as the use-cases (" good design", "net harmful effect", "Liskov violation", ...). I think the reservations are similar to those to other dynamic-increasing features like duck-typing, lambdas, closures: changing/defining an object right there where you need it to be changed/defined with the intend to convert it to a traditional (static) style later. I have to admit I don't like this style much but I catch myself doing it recently quite often. So, just for the sake of completeness, the concrete use-cases from my point of view: 1) debugging (like assert which theoretically is not necessary in production) 2) providing a correct/drop-in blue-print for immutable objects (the "intentional, essential part" from Raymond's post) 3) benefits of immutable objects (like faster development, error-reduction, etc.) <- not sure if that qualifies as a use-case This, said. I for one would not be averse to a immutability mixin/freezing protocol although I don't see a point in producing freezed copies recursively as PEP-351 did. Best, Sven
On Wed, Jun 1, 2016 at 9:35 PM Steven D'Aprano <steve@pearwood.info> wrote:
That was a recent discussion and Guido suggested it was unlikely anyone would need an optional boolean flag to be dynamic in the code. It would be much more likely that the call site would always pick one or the other and that flag could be constant. If it's a constant, then it'd be better to have two different well-named constructors, rather than a boolean parameter. Thus to evaluate this idea, we should ask: Would any call site need to set that flag dynamically? If it's simply a tool for introspection, rather than construction, how often do you find yourself checking for is-mutable? immutables = (str, bytes, int, float, complex, bool, tuple, frozenset) if isinstance(obj, immutables): ... I don't think I've ever made that check. I've checked for is-basic-type, but that's different.
On Wed, Jun 1, 2016, at 18:26, Rob Cliffe wrote:
Er... so does setting __mutable__ = False cause/require contained objects to be transitively mutable? Because if it does, you still need tuples to have an "immutable list of mutable objects". If it doesn't, then this doesn't solve the other thread's problem.
On Wed, Jun 01, 2016 at 11:26:07PM +0100, Rob Cliffe wrote:
I hate to break it to you, but merely setting a flag on an object doesn't make it mutable or immutable *wink* That means that every class (possibly including builtins) needs to be re-written so that every mutator method checks this flag and decides whether or not to allow the mutation. For example, you suggest: "You don't need tuples. You just have a list with __mutable__ = False." Now the code for list append must be: def append(self, item): # only written in C, because its a built-in if self.__mutable__: ... else: raise SomeError Now multiply that by *every* mutator class and method. This will be especially burdensome for people writing classes intended to be mutable, since they have to support immutability whether they want it or not. (Since the caller might change obj.__mutable__ to False, which is allowed.) This will break type-checking and duck-typing and make feature detection a pain: # currently this works if hasattr(obj, 'append'): handle_things_that_can_append(obj) else: handle_things_that_cant_append(obj) # with your scheme if hasattr(obj, 'append'): if obj.__mutable__: handle_things_that_can_append(obj) else: handle_things_that_cant_append(obj) else: handle_things_that_cant_append(obj)
This goes against the "no constant bool arguments" design guideline.
We already have syntax for a built-in immutable list-like sequence object (a "frozen list" if you will): (1, 2, 3)
No it does not. (1, 2, [3]) is both immutable and unhashable.
And we get this functionality for free, right? *wink* Somebody has to write this code. I'm not sure they will appreciate having to throw away all the perfectly good frozenset code and tests, and re-writing set to handle both cases. Not to mention the breaking of backwards compatibility to get rid of frozenset.
(1) I don't remember ever seeing a beginner confused by this difference, but I see plenty of experienced programmers who are sure that they must be. (2) Have you actually read the FAQ? Because if you have, you should realise that having an immutable list is not the same as having a tuple. Tuples are intended for "record-like" structures, where the items' positions are meaningful, while lists are intended for "array-like" structures where all the items have the same type and their position is not meaningful. While it's not compulsory to treat tuples and lists this way, it is very useful to do so. -- Steve
On 02.06.2016 15:05, Rob Cliffe wrote:
I don't think that's an overly stupid idea. In such regard, Python wouldn't be the first project dealing with mutability in this manner. PostgreSQL already does. One question I would have: would it be enforced? Or can objects designer choose between enforced immutability and informative immutability? Or would it be enforced inconsistently at some places where at other places it does not matter. To me the 100% enforced variant would be the least troublesome. One definite advantage of broader use of immutable objects would be optimization: using the same memory parts, re-using the same object etc. It's not my field in computer science but other people will be able to imagine a lot of other possibilities here. Best, Sven
Just remembering that one is free to implement whatever "switchable" containers one wants in the language as it is now - no need to force (full or slightly) incompatible changes that imply on performance degradation onto every single Python user for a feature that was not needed up to this day. Implementing this in sequence-abc, and mapping-abc classes is trivial. A reasonable request for the language could be exactly to allow a "__mutable__" property on the sequence, container and mapping protocols, and a "mutable" built-in callable, that in the absence of "__mutable__" could check for the methods signature (if it does not have "__mutable__" but has "__setitem__" , then mutable(obj) returns True, for example). On 2 June 2016 at 10:55, Sven R. Kunze <srkunze@mail.de> wrote:
On 6 June 2016 at 19:05, Sven R. Kunze <srkunze@mail.de> wrote:
No. Trivially a method on an extension object can be storing state in a C struct. Let trivially: Class IsThisMutable: def __init__(self): self._cache = {} def random(self): self._cache[len(self._cache)] = random.random() return self._cache[len(self._cache)-1] def size(self): return len(self._cache) this has interior mutability - it uses __setitem__ on self._cache to track that state. You could remove __setitem__ from IsThisMutable instances post construction and it would still be getting mutated. -Rob
On 06.06.2016 09:55, Robert Collins wrote:
That's what I thought as well. So, I see a huge benefit for enforced immutability when it comes to reducing errors and not re-inventing the wheel. Thinking this further, __init__ would be the only function to change the state of an immutable object. Once created, it will never change. Immutable also implies hashability IMHO. Moreover, immutable object would not be allowed to query data from global/external variables as those can change and would change the observable state of the object without the object noticing. So, the allowed way of creating a state for an immutable object would be using a new container as you did (by defining self._cache) and store immutable objects only there. Would this make sense? Sven
On Mon, Jun 6, 2016, at 06:44, Sven R. Kunze wrote:
You're continuing to fail to address the fact that some aspects of this proposal implicitly claim that immutable objects shall not hold references to mutable objects (a tuple containing a reference to a list is not hashable*, does this mean tuples are not immutable?) while other claims (OP's claim that an "immutable list" could substitute for a tuple) imply that they can.
How are you gonna stop them? *Java seems to work fine with mutable lists being hashable and trusting users to responsibly manage the life cycles of dictionaries using them as keys, but this is an argument that we don't really need to start up again here and now.
On 06.06.2016 15:34, Random832 wrote:
(a tuple containing a reference to a list is not hashable*, does this mean tuples are not immutable?)
Your question in brackets is actually the most important question here. I would say it depends on who one defines it but I don't see a contradiction to have both variants of tuples in the same languages. Whatever fits your needs more in a certain situation.
With a hammer? I don't know how you want me to answer that. It's an implementation detail to me as other languages can handle it properly. There is a way of stopping them by preventing all ways of manipulating state variables of an object. What those ways are is more a topic for long-served devs of CPython 4 or 5 ;-) Sven
On Mon, Jun 06, 2016 at 12:44:10PM +0200, Sven R. Kunze wrote:
Python classes have supported "cooperative immutability" for 20+ years. For example, the pure Python Decimal and Fraction classes are immutable only by convention, if you modify the _private attributes you can change them. And yet they work fine. I'm not saying that "real immutability" wouldn't be nice to have, but I think that describing it as huge benefit is possible overstating it.
Thinking this further, __init__ would be the only function to change the state of an immutable object. Once created, it will never change.
How would you enforce that?
Immutable also implies hashability IMHO.
That's incorrect. We all agree that tuples are immutable, I trust. py> t = (1, 2, {}, 3) py> hash(t) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: unhashable type: 'dict' Immutable collections are only hashable if all their components are hashable.
How would you enforce that?
I don't understand this paragraph, sorry. -- Steve
On 06.06.2016 16:40, Steven D'Aprano wrote:
Nobody says it does not work fine. But sometimes, people need an immutability class blueprint. I saw several implementation attempts for such immutable-appearing objects. They tend to be hard to maintain.
I'm not saying that "real immutability" wouldn't be nice to have, but I think that describing it as huge benefit is possible overstating it.
Since Python became a Turing-complete language, most PEPs are just that: preventing the re-invention of the wheel. And that's always a huge benefit; especially viewed economically.
That's an implementation detail.
So what? I don't see your point in quoting current behavior. Especially, because this is a theoretical discussion about a hypothetical behavior which might differ from the current one. Furthermore, it depends on how you define what a tuple is. Is it the mere container to n (possibly mutable) objects? Or does it also comprise its components? In the former case, it's the current behavior IIRC. In the later case, its components need to be immutable as well. That's basically what I was trying to say below.
Don't give access to them in the first place. That again seems to me like another implementation detail.
(this might be because you split it up into several pieces? ;) ) I might refer here to your tuple example above. The tuple itself might look immutable but its components are not. What immutability means instead for a domain-specific object is that it needs to look and to feel immutable. Usually, ones does not inspect such objects but use it via its public API. So, the output of its API definitely needs to be constant through out the life-time of the object. Internally, the object could need to store more than a single number to represent its state. Maybe, even a container of containers could be necessary. But then all those containers and the containers' contents need to be immutable as well to assist the designer of the object's class. That's at least the way I understand it how immutability would need to work in order to prevent errors and simplify developers' lives. Sven
On 06/06/2016 11:44, Sven R. Kunze wrote:
That would be one viable design. But aren't there use cases for initially creating an object as mutable, then at some point "freezing" it by changing the mutable flag from True to False? (Obviously the converse would not be allowed.) Regards, Rob Cliffe
On 08.06.2016 16:33, Eric V. Smith wrote:
See the rejected PEP-351 for one version of the freeze protocol.
Wow, still 11 years later equal thoughts arise again. Reading through the rejection notes, I can see the concerns are as abstract as the use-cases (" good design", "net harmful effect", "Liskov violation", ...). I think the reservations are similar to those to other dynamic-increasing features like duck-typing, lambdas, closures: changing/defining an object right there where you need it to be changed/defined with the intend to convert it to a traditional (static) style later. I have to admit I don't like this style much but I catch myself doing it recently quite often. So, just for the sake of completeness, the concrete use-cases from my point of view: 1) debugging (like assert which theoretically is not necessary in production) 2) providing a correct/drop-in blue-print for immutable objects (the "intentional, essential part" from Raymond's post) 3) benefits of immutable objects (like faster development, error-reduction, etc.) <- not sure if that qualifies as a use-case This, said. I for one would not be averse to a immutability mixin/freezing protocol although I don't see a point in producing freezed copies recursively as PEP-351 did. Best, Sven
On Wed, Jun 1, 2016 at 9:35 PM Steven D'Aprano <steve@pearwood.info> wrote:
That was a recent discussion and Guido suggested it was unlikely anyone would need an optional boolean flag to be dynamic in the code. It would be much more likely that the call site would always pick one or the other and that flag could be constant. If it's a constant, then it'd be better to have two different well-named constructors, rather than a boolean parameter. Thus to evaluate this idea, we should ask: Would any call site need to set that flag dynamically? If it's simply a tool for introspection, rather than construction, how often do you find yourself checking for is-mutable? immutables = (str, bytes, int, float, complex, bool, tuple, frozenset) if isinstance(obj, immutables): ... I don't think I've ever made that check. I've checked for is-basic-type, but that's different.
participants (8)
-
Eric V. Smith -
Joao S. O. Bueno -
Michael Selik -
Random832 -
Rob Cliffe -
Robert Collins -
Steven D'Aprano -
Sven R. Kunze