PyTypeObject type names in Modules/
Hello and happy 2013, Something I noticed earlier today is that some C versions of stdlib modules define their name similarly to the Python version in their PyTypeObject. Some examples: Decimal, xml.etree's Element. Others prepend an understore, like _pickle.Pickler and many others. What are the tradeoffs involved in this choice? Is there a "right" thing for types that are supposed to be compatible (i.e. the C extension, where available, replaces the Python implementation seamlessly)? I can think of some meanings for pickling. Unpickling looks at the class name to figure out how to unpickle a user-defined object, so this can affect the pickle/unpickle compatibility between the C and Python versions. What else? Eli
2013/1/1 Eli Bendersky
Hello and happy 2013,
Something I noticed earlier today is that some C versions of stdlib modules define their name similarly to the Python version in their PyTypeObject. Some examples: Decimal, xml.etree's Element. Others prepend an understore, like _pickle.Pickler and many others.
What are the tradeoffs involved in this choice? Is there a "right" thing for types that are supposed to be compatible (i.e. the C extension, where available, replaces the Python implementation seamlessly)?
I can think of some meanings for pickling. Unpickling looks at the class name to figure out how to unpickle a user-defined object, so this can affect the pickle/unpickle compatibility between the C and Python versions. What else?
I don't it's terribly important except if the object from the C module is directly exposed through the API it's nicer if it's __name__ doesn't have a leading underscore. -- Regards, Benjamin
On Tue, Jan 1, 2013 at 8:15 AM, Benjamin Peterson
2013/1/1 Eli Bendersky
: Hello and happy 2013,
Something I noticed earlier today is that some C versions of stdlib modules define their name similarly to the Python version in their PyTypeObject. Some examples: Decimal, xml.etree's Element. Others prepend an understore, like _pickle.Pickler and many others.
What are the tradeoffs involved in this choice? Is there a "right" thing for types that are supposed to be compatible (i.e. the C extension, where available, replaces the Python implementation seamlessly)?
I can think of some meanings for pickling. Unpickling looks at the class name to figure out how to unpickle a user-defined object, so this can affect the pickle/unpickle compatibility between the C and Python versions. What else?
I don't it's terribly important except if the object from the C module is directly exposed through the API it's nicer if it's __name__ doesn't have a leading underscore.
Hi Benjamin, Can you elaborate - what you mean by "is directly exposed through the API"? For example, Pickler in 3.x:
import pickle pickle.Pickler.__name__ 'Pickler' pickle.Pickler.__module__ '_pickle'
2013/1/1 Eli Bendersky
On Tue, Jan 1, 2013 at 8:15 AM, Benjamin Peterson
wrote: 2013/1/1 Eli Bendersky
: Hello and happy 2013,
Something I noticed earlier today is that some C versions of stdlib modules define their name similarly to the Python version in their PyTypeObject. Some examples: Decimal, xml.etree's Element. Others prepend an understore, like _pickle.Pickler and many others.
What are the tradeoffs involved in this choice? Is there a "right" thing for types that are supposed to be compatible (i.e. the C extension, where available, replaces the Python implementation seamlessly)?
I can think of some meanings for pickling. Unpickling looks at the class name to figure out how to unpickle a user-defined object, so this can affect the pickle/unpickle compatibility between the C and Python versions. What else?
I don't it's terribly important except if the object from the C module is directly exposed through the API it's nicer if it's __name__ doesn't have a leading underscore.
Hi Benjamin,
Can you elaborate - what you mean by "is directly exposed through the API"?
For example, Pickler in 3.x:
import pickle pickle.Pickler.__name__ 'Pickler' pickle.Pickler.__module__ '_pickle'
That's an example of what I meant. -- Regards, Benjamin
2013/1/1 Eli Bendersky
: Hello and happy 2013,
Something I noticed earlier today is that some C versions of stdlib modules define their name similarly to the Python version in their
PyTypeObject.
Some examples: Decimal, xml.etree's Element. Others prepend an understore, like _pickle.Pickler and many others.
I don't it's terribly important except if the object from the C module is directly exposed through the API it's nicer if it's __name__ doesn't have a leading underscore.
As a followup question: would it be considered a compatibility-breaking change to rename PyTypeObject names? As a concrete example, in Modules/_elementtree.c the name of Element_Type (essentially the xml.etree.ElementTree.Element replacement in C) is "Element". For the purpose of pickling/unpickling it should be named "_elementtree.Element" or "xml.etree.ElementTree.Element" or some such thing. Can such a change be made between 3.3 and 3.4? Between 3.3 and 3.3.1? Eli
2013/1/3 Eli Bendersky
2013/1/1 Eli Bendersky
: Hello and happy 2013,
Something I noticed earlier today is that some C versions of stdlib modules define their name similarly to the Python version in their PyTypeObject. Some examples: Decimal, xml.etree's Element. Others prepend an understore, like _pickle.Pickler and many others.
I don't it's terribly important except if the object from the C module is directly exposed through the API it's nicer if it's __name__ doesn't have a leading underscore.
As a followup question: would it be considered a compatibility-breaking change to rename PyTypeObject names? As a concrete example, in Modules/_elementtree.c the name of Element_Type (essentially the xml.etree.ElementTree.Element replacement in C) is "Element". For the purpose of pickling/unpickling it should be named "_elementtree.Element" or "xml.etree.ElementTree.Element" or some such thing. Can such a change be made between 3.3 and 3.4? Between 3.3 and 3.3.1?
I don't know much about etree. Can you pickle Element? -- Regards, Benjamin
On Thu, Jan 3, 2013 at 6:43 AM, Benjamin Peterson
2013/1/3 Eli Bendersky
: 2013/1/1 Eli Bendersky
: Hello and happy 2013,
Something I noticed earlier today is that some C versions of stdlib modules define their name similarly to the Python version in their PyTypeObject. Some examples: Decimal, xml.etree's Element. Others prepend an understore, like _pickle.Pickler and many others.
I don't it's terribly important except if the object from the C
module
is directly exposed through the API it's nicer if it's __name__ doesn't have a leading underscore.
As a followup question: would it be considered a compatibility-breaking change to rename PyTypeObject names? As a concrete example, in Modules/_elementtree.c the name of Element_Type (essentially the xml.etree.ElementTree.Element replacement in C) is "Element". For the purpose of pickling/unpickling it should be named "_elementtree.Element" or "xml.etree.ElementTree.Element" or some such thing. Can such a change be made between 3.3 and 3.4? Between 3.3 and 3.3.1?
I don't know much about etree. Can you pickle Element?
etree has a C accelerator that was improved and extended in 3.3 and was made the default when importing etree. But a regression (issue #16076) occurs because _elementree.Element has no pickling support, while the Python version does by default (being a plain Python class). In the aforementioned issue we're trying to resolve the strategy for supporting pickling for _elementtree.Element. I hope my understanding of unpickling is correct - it does need a class name to find the module that can unpickle the object, right? If this is correct, than such a change (name of the type) may be required to solve the regression between 3.3 and 3.3.1 Eli
2013/1/3 Eli Bendersky
etree has a C accelerator that was improved and extended in 3.3 and was made the default when importing etree. But a regression (issue #16076) occurs because _elementree.Element has no pickling support, while the Python version does by default (being a plain Python class). In the aforementioned issue we're trying to resolve the strategy for supporting pickling for _elementtree.Element. I hope my understanding of unpickling is correct - it does need a class name to find the module that can unpickle the object, right? If this is correct, than such a change (name of the type) may be required to solve the regression between 3.3 and 3.3.1
Yes, but you're probably going to have to do more than change the class name in order for pickling to work for C types. -- Regards, Benjamin
On Thu, Jan 3, 2013 at 6:50 AM, Benjamin Peterson
2013/1/3 Eli Bendersky
: etree has a C accelerator that was improved and extended in 3.3 and was made the default when importing etree. But a regression (issue #16076) occurs because _elementree.Element has no pickling support, while the Python version does by default (being a plain Python class). In the aforementioned issue we're trying to resolve the strategy for supporting pickling for _elementtree.Element. I hope my understanding of unpickling is correct - it does need a class name to find the module that can unpickle the object, right? If this is correct, than such a change (name of the type) may be required to solve the regression between 3.3 and 3.3.1
Yes, but you're probably going to have to do more than change the class name in order for pickling to work for C types.
Yes, of course. All of that is already implemented in patches Daniel Shahaf has submitted to the issue. The "controversial" question here is whether it's valid to change the __module__ of the type between 3.3 and 3.3.1 ? Eli
Eli Bendersky
Yes, of course. All of that is already implemented in patches Daniel Shahaf has submitted to the issue. The "controversial" question here is whether it's valid to change the __module__ of the type between 3.3 and 3.3.1 ?
In 3.2 and in the Python version of 3.3 we have this:
Element.__module__ 'xml.etree.ElementTree'
So IMHO changing the C version to do the same in 3.3.1 is a compatibility fix. Stefan Krah
2013/1/3 Eli Bendersky
On Thu, Jan 3, 2013 at 6:50 AM, Benjamin Peterson
wrote: 2013/1/3 Eli Bendersky
: etree has a C accelerator that was improved and extended in 3.3 and was made the default when importing etree. But a regression (issue #16076) occurs because _elementree.Element has no pickling support, while the Python version does by default (being a plain Python class). In the aforementioned issue we're trying to resolve the strategy for supporting pickling for _elementtree.Element. I hope my understanding of unpickling is correct - it does need a class name to find the module that can unpickle the object, right? If this is correct, than such a change (name of the type) may be required to solve the regression between 3.3 and 3.3.1
Yes, but you're probably going to have to do more than change the class name in order for pickling to work for C types.
Yes, of course. All of that is already implemented in patches Daniel Shahaf has submitted to the issue. The "controversial" question here is whether it's valid to change the __module__ of the type between 3.3 and 3.3.1 ?
Failure to pickle suddenly is more of a compatibility issue IMO. -- Regards, Benjamin
participants (3)
-
Benjamin Peterson
-
Eli Bendersky
-
Stefan Krah