-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
So I had planned to do a bunch of work last night looking at the release blocker issues, but nature intervened. A bunch of severe thunderstorms knock out my 'net access until this morning.
I'll try to find some time during the day to look at the RB issues. Hopefully we can get Guido to look at them too and Pronounce on some of them. Guido please start with:
http://bugs.python.org/issue643841
My plan is to begin building the betas tonight, at around 9 or 10pm EDT (0100 to 0200 UTC Thursday). If a showstopper comes up before then, I'll email the list. If you think we really aren't ready for beta, then I would still like to get a release out today. In that case, we'll call it alpha and delay the betas.
- -Barry
On 2008-06-11 13:35, Barry Warsaw wrote:
So I had planned to do a bunch of work last night looking at the release blocker issues, but nature intervened. A bunch of severe thunderstorms knock out my 'net access until this morning.
I'll try to find some time during the day to look at the RB issues. Hopefully we can get Guido to look at them too and Pronounce on some of them. Guido please start with:
http://bugs.python.org/issue643841
My plan is to begin building the betas tonight, at around 9 or 10pm EDT (0100 to 0200 UTC Thursday). If a showstopper comes up before then, I'll email the list. If you think we really aren't ready for beta, then I would still like to get a release out today. In that case, we'll call it alpha and delay the betas.
There are two things I'd like to get in to 3.0:
* .transform()/.untransform() methods (this is mostly done, just need to add the methods to PyUnicode, PyBytes and PyByteArray)
* cleanup of the PyUnicode_AsString() and PyUnicode_AsStringAndSize() C APIs (these APIs don't fit into the naming scheme used in the Unicode API and have a few other issues as well, see issue 2799; at the very least they should be made interpreter internal, ie. rename them to _PyUnicode_AsString() and _PyUnicode_AsStringAndSize() to prevent their use in extensions)
I did not have time in the last few days to work on these and won't in the next few days either. Next week looks much better.
If it's ok to make the above changes after the release (whatever you call it ;-), that would be great.
Thanks,
M.-A. Lemburg wrote:
On 2008-06-11 13:35, Barry Warsaw wrote:
So I had planned to do a bunch of work last night looking at the release blocker issues, but nature intervened. A bunch of severe thunderstorms knock out my 'net access until this morning.
I'll try to find some time during the day to look at the RB issues. Hopefully we can get Guido to look at them too and Pronounce on some of them. Guido please start with:
http://bugs.python.org/issue643841
My plan is to begin building the betas tonight, at around 9 or 10pm EDT (0100 to 0200 UTC Thursday). If a showstopper comes up before then, I'll email the list. If you think we really aren't ready for beta, then I would still like to get a release out today. In that case, we'll call it alpha and delay the betas.
There are two things I'd like to get in to 3.0:
- .transform()/.untransform() methods (this is mostly done, just need to add the methods to PyUnicode, PyBytes and PyByteArray)
What would these methods do? Use the codec machinery without any type checks?
I think for transformations we don't need the full codec machinery:
We probably don't need extensible error handling.
There are transformation that are not invertible, so it doesn't make sense to have both operations in one object. If the operation *is* invertible, two tranformers can be used.
Do we really need a registry that maps function named to functions?
A simple API might look like this:
class TransformInfo: # stateless transformer def transform(self, input):
# return stateful incremental transformer def incrementaltransformer(self):
# wrap stream in a transforming stream def streamtransformer(self, stream):
incrementaltransformer() would return an object that has one method:
def transform(self, input, final=False);
[...]
Servus, Walter
On 2008-06-11 17:15, Walter Dörwald wrote:
M.-A. Lemburg wrote:
On 2008-06-11 13:35, Barry Warsaw wrote:
So I had planned to do a bunch of work last night looking at the release blocker issues, but nature intervened. A bunch of severe thunderstorms knock out my 'net access until this morning.
I'll try to find some time during the day to look at the RB issues. Hopefully we can get Guido to look at them too and Pronounce on some of them. Guido please start with:
http://bugs.python.org/issue643841
My plan is to begin building the betas tonight, at around 9 or 10pm EDT (0100 to 0200 UTC Thursday). If a showstopper comes up before then, I'll email the list. If you think we really aren't ready for beta, then I would still like to get a release out today. In that case, we'll call it alpha and delay the betas.
There are two things I'd like to get in to 3.0:
- .transform()/.untransform() methods (this is mostly done, just need to add the methods to PyUnicode, PyBytes and PyByteArray)
What would these methods do? Use the codec machinery without any type checks?
As discussed in another thread some weeks ago:
.transform() and .untransform() use the codecs to apply same-type conversions. They do apply type checks to make sure that the codec does indeed return the same type.
E.g. text.transform('xml-escape') or data.transform('base64').
I think for transformations we don't need the full codec machinery: ...
No need to invent another wheel :-) The codecs already exist for Py2.x and can be used by the .encode()/.decode() methods in Py2.x (where no type checks occur).
In Py3.x, .encode()/.decode() only allow conversions of the type unicode <-> bytes. .transform()/.untransform() add conversions of the type unicode <-> unicode or bytes <-> bytes.
All other conversions in Py3.x have to go through codecs.encode() and codecs.decode() which are the generic codec access functions from the codec registry.
M.-A. Lemburg wrote:
On 2008-06-11 17:15, Walter Dörwald wrote:
M.-A. Lemburg wrote:
On 2008-06-11 13:35, Barry Warsaw wrote:
So I had planned to do a bunch of work last night looking at the release blocker issues, but nature intervened. A bunch of severe thunderstorms knock out my 'net access until this morning.
I'll try to find some time during the day to look at the RB issues. Hopefully we can get Guido to look at them too and Pronounce on some of them. Guido please start with:
http://bugs.python.org/issue643841
My plan is to begin building the betas tonight, at around 9 or 10pm EDT (0100 to 0200 UTC Thursday). If a showstopper comes up before then, I'll email the list. If you think we really aren't ready for beta, then I would still like to get a release out today. In that case, we'll call it alpha and delay the betas.
There are two things I'd like to get in to 3.0:
- .transform()/.untransform() methods (this is mostly done, just need to add the methods to PyUnicode, PyBytes and PyByteArray)
What would these methods do? Use the codec machinery without any type checks?
As discussed in another thread some weeks ago:
.transform() and .untransform() use the codecs to apply same-type conversions. They do apply type checks to make sure that the codec does indeed return the same type.
E.g. text.transform('xml-escape') or data.transform('base64').
So what would a base64 codec do with the errors argument?
I think for transformations we don't need the full codec machinery: ...
No need to invent another wheel :-) The codecs already exist for Py2.x and can be used by the .encode()/.decode() methods in Py2.x (where no type checks occur).
By using a new API we could get rid of old warts. For example: Why does the stateless encoder/decoder return how many input characters/bytes it has consumed? It must consume *all* bytes anyway!
In Py3.x, .encode()/.decode() only allow conversions of the type unicode <-> bytes. .transform()/.untransform() add conversions of the type unicode <-> unicode or bytes <-> bytes.
All other conversions in Py3.x have to go through codecs.encode() and codecs.decode() which are the generic codec access functions from the codec registry.
Servus, Walter
On 2008-06-12 16:59, Walter Dörwald wrote:
M.-A. Lemburg wrote:
.transform() and .untransform() use the codecs to apply same-type conversions. They do apply type checks to make sure that the codec does indeed return the same type.
E.g. text.transform('xml-escape') or data.transform('base64').
So what would a base64 codec do with the errors argument?
It could use it to e.g. try to recover as much data as possible from broken input data.
Currently (in Py2.x), it raises an exception if you pass in anything but "strict".
I think for transformations we don't need the full codec machinery: ...
No need to invent another wheel :-) The codecs already exist for Py2.x and can be used by the .encode()/.decode() methods in Py2.x (where no type checks occur).
By using a new API we could get rid of old warts. For example: Why does the stateless encoder/decoder return how many input characters/bytes it has consumed? It must consume *all* bytes anyway!
No, it doesn't and that's the point in having those return values :-)
Even though the encoder/decoders are stateless, that doesn't mean they have to consume all input data. The caller is responsible to make sure that all input data was in fact consumed.
You could for example have a decoder that stops decoding after having seen a block end indicator, e.g. a base64 line end or XML closing element.
Just because all codecs that ship with Python always try to decode the complete input doesn't mean that the feature isn't being used. The interface was designed to allow for the above situations.
M.-A. Lemburg wrote:
On 2008-06-12 16:59, Walter Dörwald wrote:
M.-A. Lemburg wrote:
.transform() and .untransform() use the codecs to apply same-type conversions. They do apply type checks to make sure that the codec does indeed return the same type.
E.g. text.transform('xml-escape') or data.transform('base64').
So what would a base64 codec do with the errors argument?
It could use it to e.g. try to recover as much data as possible from broken input data.
Currently (in Py2.x), it raises an exception if you pass in anything but "strict".
I think for transformations we don't need the full codec machinery: ...
No need to invent another wheel :-) The codecs already exist for Py2.x and can be used by the .encode()/.decode() methods in Py2.x (where no type checks occur).
By using a new API we could get rid of old warts. For example: Why does the stateless encoder/decoder return how many input characters/bytes it has consumed? It must consume *all* bytes anyway!
No, it doesn't and that's the point in having those return values :-)
Even though the encoder/decoders are stateless, that doesn't mean they have to consume all input data. The caller is responsible to make sure that all input data was in fact consumed.
You could for example have a decoder that stops decoding after having seen a block end indicator, e.g. a base64 line end or XML closing element.
So how should the UTF-8 decoder know that it has to stop at a closing XML element?
Just because all codecs that ship with Python always try to decode the complete input doesn't mean that the feature isn't being used.
I know of no other code that does. Do you have an example for this use.
The interface was designed to allow for the above situations.
Then could we at least have a new codec method that does:
def statelesencode(self, input): (output, consumed) = self.encode(input) assert len(input) == consumed return output
Servus, Walter
On 2008-06-13 11:32, Walter Dörwald wrote:
M.-A. Lemburg wrote:
On 2008-06-12 16:59, Walter Dörwald wrote:
M.-A. Lemburg wrote:
.transform() and .untransform() use the codecs to apply same-type conversions. They do apply type checks to make sure that the codec does indeed return the same type.
E.g. text.transform('xml-escape') or data.transform('base64').
So what would a base64 codec do with the errors argument?
It could use it to e.g. try to recover as much data as possible from broken input data.
Currently (in Py2.x), it raises an exception if you pass in anything but "strict".
I think for transformations we don't need the full codec machinery: ...
No need to invent another wheel :-) The codecs already exist for Py2.x and can be used by the .encode()/.decode() methods in Py2.x (where no type checks occur).
By using a new API we could get rid of old warts. For example: Why does the stateless encoder/decoder return how many input characters/bytes it has consumed? It must consume *all* bytes anyway!
No, it doesn't and that's the point in having those return values :-)
Even though the encoder/decoders are stateless, that doesn't mean they have to consume all input data. The caller is responsible to make sure that all input data was in fact consumed.
You could for example have a decoder that stops decoding after having seen a block end indicator, e.g. a base64 line end or XML closing element.
So how should the UTF-8 decoder know that it has to stop at a closing XML element?
The UTF-8 decoder doesn't support this, but you could write a codec that applies this kind of detection, e.g. to not try to decode partial UTF-8 byte sequences at the end of input, which would then result in error.
Just because all codecs that ship with Python always try to decode the complete input doesn't mean that the feature isn't being used.
I know of no other code that does. Do you have an example for this use.
I already gave you a few examples.
The interface was designed to allow for the above situations.
Then could we at least have a new codec method that does:
def statelesencode(self, input): (output, consumed) = self.encode(input) assert len(input) == consumed return output
You mean as method to the Codec class ?
Sure, we could do that, but please use a different name, e.g. .encodeall() and .decodeall() - .encode() and .decode() are already stateles (and so would the new methods be), so "stateless" isn't all that meaningful in this context.
We could also add such a check to the PyCodec_Encode() and _Decode() functions. They currently do not apply the above check.
In Python, those two functions are exposed as codecs.encode() and codecs.decode().
M.-A. Lemburg wrote:
On 2008-06-13 11:32, Walter Dörwald wrote:
M.-A. Lemburg wrote:
On 2008-06-12 16:59, Walter Dörwald wrote:
M.-A. Lemburg wrote:
.transform() and .untransform() use the codecs to apply same-type conversions. They do apply type checks to make sure that the codec does indeed return the same type.
E.g. text.transform('xml-escape') or data.transform('base64').
So what would a base64 codec do with the errors argument?
It could use it to e.g. try to recover as much data as possible from broken input data.
Currently (in Py2.x), it raises an exception if you pass in anything but "strict".
I think for transformations we don't need the full codec machinery: ...
No need to invent another wheel :-) The codecs already exist for Py2.x and can be used by the .encode()/.decode() methods in Py2.x (where no type checks occur).
By using a new API we could get rid of old warts. For example: Why does the stateless encoder/decoder return how many input characters/bytes it has consumed? It must consume *all* bytes anyway!
No, it doesn't and that's the point in having those return values :-)
Even though the encoder/decoders are stateless, that doesn't mean they have to consume all input data. The caller is responsible to make sure that all input data was in fact consumed.
You could for example have a decoder that stops decoding after having seen a block end indicator, e.g. a base64 line end or XML closing element.
So how should the UTF-8 decoder know that it has to stop at a closing XML element?
The UTF-8 decoder doesn't support this, but you could write a codec that applies this kind of detection, e.g. to not try to decode partial UTF-8 byte sequences at the end of input, which would then result in error.
Just because all codecs that ship with Python always try to decode the complete input doesn't mean that the feature isn't being used.
I know of no other code that does. Do you have an example for this use.
I already gave you a few examples.
Maybe I was unclear, I meant real world examples, not hypothetical ones.
The interface was designed to allow for the above situations.
Then could we at least have a new codec method that does:
def statelesencode(self, input): (output, consumed) = self.encode(input) assert len(input) == consumed return output
You mean as method to the Codec class ?
No, I meant as a method for the CodecInfo clas.
Sure, we could do that, but please use a different name, e.g. .encodeall() and .decodeall() - .encode() and .decode() are already stateles (and so would the new methods be), so "stateless" isn't all that meaningful in this context.
I like the names encodeall/decodeall!
We could also add such a check to the PyCodec_Encode() and _Decode() functions. They currently do not apply the above check.
In Python, those two functions are exposed as codecs.encode() and codecs.decode().
This change will probably have to wait for the 2.7 cycle.
Servus, Walter
Walter Dörwald wrote:
[...]
Sure, we could do that, but please use a different name, e.g. .encodeall() and .decodeall() - .encode() and .decode() are already stateles (and so would the new methods be), so "stateless" isn't all that meaningful in this context.
I like the names encodeall/decodeall!
We could also add such a check to the PyCodec_Encode() and _Decode() functions. They currently do not apply the above check.
In Python, those two functions are exposed as codecs.encode() and codecs.decode().
This change will probably have to wait for the 2.7 cycle.
BTW, what I noticed is that the unicode-internal codec seems to be broken:
import codecs codecs.getencoder("unicode-internal")(u"abc")
('a\x00b\x00c\x00', 6)
I would have expected it to return:
import codecs codecs.getencoder("unicode-internal")(u"abc")
('a\x00b\x00c\x00', 3)
Servus, Walter
M.-A. Lemburg schrieb:
On 2008-06-11 13:35, Barry Warsaw wrote:
So I had planned to do a bunch of work last night looking at the release blocker issues, but nature intervened. A bunch of severe thunderstorms knock out my 'net access until this morning.
I'll try to find some time during the day to look at the RB issues. Hopefully we can get Guido to look at them too and Pronounce on some of them. Guido please start with:
http://bugs.python.org/issue643841
My plan is to begin building the betas tonight, at around 9 or 10pm EDT (0100 to 0200 UTC Thursday). If a showstopper comes up before then, I'll email the list. If you think we really aren't ready for beta, then I would still like to get a release out today. In that case, we'll call it alpha and delay the betas.
There are two things I'd like to get in to 3.0:
Another thing that should get in is the PEP 3138 implementation, that is, the ascii() builtin and repr() changes.
Georg
On Wed, Jun 11, 2008 at 7:32 AM, M.-A. Lemburg mal@egenix.com wrote:
There are two things I'd like to get in to 3.0:
- .transform()/.untransform() methods (this is mostly done, just need
to add the methods to PyUnicode, PyBytes and PyByteArray)
I'm +0 on this. It is very minor syntactic sugar that typically saves you only one import. I expect the argument will nearly always be a literal, e.g. data.transform('gzip') rather than a variable like data.transform(compression_method). But it *is* convenient and can make code more readable, e.g.
if compressed: data = data.untransform('gzip')
Nobody will have to guess what that does. (IMO the confusion about which direction the transformation goes is theoretical. except perhaps in the case of rot13. :-)
- cleanup of the PyUnicode_AsString() and PyUnicode_AsStringAndSize()
C APIs (these APIs don't fit into the naming scheme used in the Unicode API and have a few other issues as well, see issue 2799; at the very least they should be made interpreter internal, ie. rename them to _PyUnicode_AsString() and _PyUnicode_AsStringAndSize() to prevent their use in extensions)
I'm okay with this too.
I did not have time in the last few days to work on these and won't in the next few days either. Next week looks much better.
If it's ok to make the above changes after the release (whatever you call it ;-), that would be great.
That's up to the release manager, but IMO we could have a small list of *specific* things that are not yet implemented in beta 1 but that we know will be in beta 2. This is IMO better than just calling it another alpha, because that keeps the floodgates for more feature change requests open for another month.
--Guido
Thanks,
Marc-Andre Lemburg eGenix.com
Professional Python Services directly from the Source (#1, Jun 11 2008)
Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
2008-07-07: EuroPython 2008, Vilnius, Lithuania 25 days to go
:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 _______________________________________________ Python-3000 mailing list Python-3000@python.org http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org
On Wed, Jun 11, 2008 at 4:35 AM, Barry Warsaw barry@python.org wrote:
So I had planned to do a bunch of work last night looking at the release blocker issues, but nature intervened. A bunch of severe thunderstorms knock out my 'net access until this morning.
I'll try to find some time during the day to look at the RB issues. Hopefully we can get Guido to look at them too and Pronounce on some of them. Guido please start with:
I've added a comment. Let me know if anything I said is unclear.
My plan is to begin building the betas tonight, at around 9 or 10pm EDT (0100 to 0200 UTC Thursday). If a showstopper comes up before then, I'll email the list. If you think we really aren't ready for beta, then I would still like to get a release out today. In that case, we'll call it alpha and delay the betas.
I'd rather call it beta even if certain things are still known to change in the next release. Beta means we'll have a much easier time pushing back on random other feature change proposals.
- -Barry
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Jun 11, 2008, at 1:02 PM, Guido van Rossum wrote:
On Wed, Jun 11, 2008 at 4:35 AM, Barry Warsaw barry@python.org wrote:
So I had planned to do a bunch of work last night looking at the release blocker issues, but nature intervened. A bunch of severe thunderstorms knock out my 'net access until this morning.
I'll try to find some time during the day to look at the RB issues. Hopefully we can get Guido to look at them too and Pronounce on some of them. Guido please start with:
I've added a comment. Let me know if anything I said is unclear.
Nope, it was perfect, thanks.
My plan is to begin building the betas tonight, at around 9 or 10pm EDT (0100 to 0200 UTC Thursday). If a showstopper comes up before then, I'll email the list. If you think we really aren't ready for beta, then I would still like to get a release out today. In that case, we'll call it alpha and delay the betas.
I'd rather call it beta even if certain things are still known to change in the next release. Beta means we'll have a much easier time pushing back on random other feature change proposals.
Sounds good. I'm going to go through the other release critical issues and will follow up on this thread if there are any other questions.
- -Barry
Guido van Rossum wrote:
On Wed, Jun 11, 2008 at 4:35 AM, Barry Warsaw barry@python.org wrote:
So I had planned to do a bunch of work last night looking at the release blocker issues, but nature intervened. A bunch of severe thunderstorms knock out my 'net access until this morning.
I'll try to find some time during the day to look at the RB issues. Hopefully we can get Guido to look at them too and Pronounce on some of them. Guido please start with:
I've added a comment. Let me know if anything I said is unclear.
The bugtracker seems to be offline atm - I'll reply there once I can get to it again (as well as switching this issue back to being a documentation one).
I don't think we're going to see a major clamor for a value-based delegation mixin in the standard library until people using classic classes for value-based delegation start making serious attempts to port to Py3k (where that approach is no longer available). At the moment, such classes only need to care about the methods they want to fiddle with, leaving everything else to __getattr__ based delegation.
I've pushed as hard as I'm personally willing to for this without convincing anyone else that it's worth doing, so I'll start working on a documentation patch for the language reference instead which explicitly splits the special methods into the following categories:
1. Method lookup MUST bypass __getattribute__, shadowing the attribute in the instance dictionary MUST NOT have any ill effects. (All tp_* C-level slots and slots looked up via _PyType_Lookup will fit into this category)
2. Method lookup MAY bypass __getattribute__, shadowing the attribute in the instance dictionary MAY have ill effects. (slots such as __enter__ and __exit__ that are looked up via normal attribute lookup in CPython will fit into this category)
3. Technically a subcategory of group 1, these are special methods which can affect the interpreter's behaviour by their mere definition on a type. (The __get__, __set__ and __delete__ descriptor protocol methods fall into this category)
Cheers, Nick.
[Barry]
[Guido]
I've added a comment. Let me know if anything I said is unclear.
On Thu, Jun 12, 2008 at 3:35 AM, Nick Coghlan ncoghlan@gmail.com wrote:
The bugtracker seems to be offline atm - I'll reply there once I can get to it again (as well as switching this issue back to being a documentation one).
I don't think we're going to see a major clamor for a value-based delegation mixin in the standard library until people using classic classes for value-based delegation start making serious attempts to port to Py3k (where that approach is no longer available). At the moment, such classes only need to care about the methods they want to fiddle with, leaving everything else to __getattr__ based delegation.
Whether they'll care about this issue of course depends on whether they need overloaded operators and other special delegations to be delegated transparently. We'll have to see how important this is. New-style classes have been around and recommended for a long time -- why haven't people pushed for a proxy class before?
I've pushed as hard as I'm personally willing to for this without convincing anyone else that it's worth doing,
What does *it* refer to? Changing the behavior, or adding a proxy class to the stdlib? I'm -1000 on the former, but only -0 on the latter -- as I wrote in the tracker, I just don't want to see an unproven proxy class (and I don't like the module name).
so I'll start working on a documentation patch for the language reference instead which explicitly splits the special methods into the following categories:
Thanks for doing this, it is needed regardless!
- Method lookup MUST bypass __getattribute__, shadowing the attribute in
the instance dictionary MUST NOT have any ill effects. (All tp_* C-level slots and slots looked up via _PyType_Lookup will fit into this category)
Watch out: I think the term "method lookup" may be confusing here. Certainly when the user writes x.__foo__(), the instance dict *is* consulted. It is only on *implied* lookups (e.g. x[y] or x+y) where the instance dict is bypassed.
- Method lookup MAY bypass __getattribute__, shadowing the attribute in the
instance dictionary MAY have ill effects. (slots such as __enter__ and __exit__ that are looked up via normal attribute lookup in CPython will fit into this category)
Why even have a MAY category? Are you expecting these to become tp_ slots at some point?
- Technically a subcategory of group 1, these are special methods which can
affect the interpreter's behaviour by their mere definition on a type. (The __get__, __set__ and __delete__ descriptor protocol methods fall into this category)
I don't follow why this is relevant. This is a different, AFAIK orthogonal issue, used in many places: *if* an object used in a certain context has a specific attribute, *then* that attribute is used, *otherwise* a default action is taken. Applies to __repr__ just as much. These belong in category 1 if and only if the lookup bypasses the instance dict.
Guido van Rossum wrote:
[Barry]
[Guido]
I've added a comment. Let me know if anything I said is unclear.
On Thu, Jun 12, 2008 at 3:35 AM, Nick Coghlan ncoghlan@gmail.com wrote:
The bugtracker seems to be offline atm - I'll reply there once I can get to it again (as well as switching this issue back to being a documentation one).
I don't think we're going to see a major clamor for a value-based delegation mixin in the standard library until people using classic classes for value-based delegation start making serious attempts to port to Py3k (where that approach is no longer available). At the moment, such classes only need to care about the methods they want to fiddle with, leaving everything else to __getattr__ based delegation.
Whether they'll care about this issue of course depends on whether they need overloaded operators and other special delegations to be delegated transparently. We'll have to see how important this is. New-style classes have been around and recommended for a long time -- why haven't people pushed for a proxy class before?
It's only in Python 3 that old style classes are going away fully, so up until now you could at least use a classic class to do the proxying.
I've written my own proxy classes before that look very similar to this, and there are other proxy classes around that do the same (I thought one was by Phillip J Eby but can't find a reference easily). The last one I wrote was to proxy CPython objects from IronPython via Python.NET...
I would prefer it if the proxy class wrapped the return values of inplace operations.
Michael Foord
I've pushed as hard as I'm personally willing to for this without convincing anyone else that it's worth doing,
What does *it* refer to? Changing the behavior, or adding a proxy class to the stdlib? I'm -1000 on the former, but only -0 on the latter -- as I wrote in the tracker, I just don't want to see an unproven proxy class (and I don't like the module name).
so I'll start working on a documentation patch for the language reference instead which explicitly splits the special methods into the following categories:
Thanks for doing this, it is needed regardless!
- Method lookup MUST bypass __getattribute__, shadowing the attribute in
the instance dictionary MUST NOT have any ill effects. (All tp_* C-level slots and slots looked up via _PyType_Lookup will fit into this category)
Watch out: I think the term "method lookup" may be confusing here. Certainly when the user writes x.__foo__(), the instance dict *is* consulted. It is only on *implied* lookups (e.g. x[y] or x+y) where the instance dict is bypassed.
- Method lookup MAY bypass __getattribute__, shadowing the attribute in the
instance dictionary MAY have ill effects. (slots such as __enter__ and __exit__ that are looked up via normal attribute lookup in CPython will fit into this category)
Why even have a MAY category? Are you expecting these to become tp_ slots at some point?
- Technically a subcategory of group 1, these are special methods which can
affect the interpreter's behaviour by their mere definition on a type. (The __get__, __set__ and __delete__ descriptor protocol methods fall into this category)
I don't follow why this is relevant. This is a different, AFAIK orthogonal issue, used in many places: *if* an object used in a certain context has a specific attribute, *then* that attribute is used, *otherwise* a default action is taken. Applies to __repr__ just as much. These belong in category 1 if and only if the lookup bypasses the instance dict.
Michael Foord wrote:
Guido van Rossum wrote:
Whether they'll care about this issue of course depends on whether they need overloaded operators and other special delegations to be delegated transparently. We'll have to see how important this is. New-style classes have been around and recommended for a long time -- why haven't people pushed for a proxy class before?
It's only in Python 3 that old style classes are going away fully, so up until now you could at least use a classic class to do the proxying.
I've written my own proxy classes before that look very similar to this, and there are other proxy classes around that do the same (I thought one was by Phillip J Eby but can't find a reference easily). The last one I wrote was to proxy CPython objects from IronPython via Python.NET...
I would prefer it if the proxy class wrapped the return values of inplace operations.
Yeah, the latest version on the issue tracker does that, and allows subclasses to define a return_inplace() method to alter the behaviour (e.g. a weakref.proxy equivalent wouldn't want to wrap the return values so that it can ensure there is always at least one strong reference to the result of the operation).
Since you can also replace the .target attribute with a property to affect how the target object is stored and accessed, it's a reasonably flexible approach.
Cheers, Nick.
Guido van Rossum wrote:
[Barry]
[Guido]
I've added a comment. Let me know if anything I said is unclear.
On Thu, Jun 12, 2008 at 3:35 AM, Nick Coghlan ncoghlan@gmail.com wrote:
The bugtracker seems to be offline atm - I'll reply there once I can get to it again (as well as switching this issue back to being a documentation one).
I don't think we're going to see a major clamor for a value-based delegation mixin in the standard library until people using classic classes for value-based delegation start making serious attempts to port to Py3k (where that approach is no longer available). At the moment, such classes only need to care about the methods they want to fiddle with, leaving everything else to __getattr__ based delegation.
Whether they'll care about this issue of course depends on whether they need overloaded operators and other special delegations to be delegated transparently. We'll have to see how important this is. New-style classes have been around and recommended for a long time -- why haven't people pushed for a proxy class before?
There was an easier way to do it in the form of classic classes - the 2,x interpreter is riddled with special cases that ensure that __getattr__ is always consulted when looking for special methods on a classic class. The tracker issue regarding the fact that things aren't so simple with new style classes was actually raised way back in 2002 when someone tried to port such a class to new-style and discovered that overriding __getattribute__ was no longer enough.
I've pushed as hard as I'm personally willing to for this without convincing anyone else that it's worth doing,
What does *it* refer to? Changing the behavior, or adding a proxy class to the stdlib? I'm -1000 on the former, but only -0 on the latter -- as I wrote in the tracker, I just don't want to see an unproven proxy class (and I don't like the module name).
"It" referred to adding the proxy class - I'm personally ambivalent on adding it at this point, because the complexity of it reduces my confidence that I got it right, but it also makes it seem unfair to users of this feature of classic classes to take it away in 3.0 without giving them some kind of functional replacement.
As for as the module name goes, I don't particularly like it either - dropping something in the types module instead would be an alternative option.
so I'll start working on a documentation patch for the language reference instead which explicitly splits the special methods into the following categories:
Thanks for doing this, it is needed regardless!
- Method lookup MUST bypass __getattribute__, shadowing the attribute in
the instance dictionary MUST NOT have any ill effects. (All tp_* C-level slots and slots looked up via _PyType_Lookup will fit into this category)
Watch out: I think the term "method lookup" may be confusing here. Certainly when the user writes x.__foo__(), the instance dict *is* consulted. It is only on *implied* lookups (e.g. x[y] or x+y) where the instance dict is bypassed.
Ah good point, I'll make sure to be careful with that.
- Method lookup MAY bypass __getattribute__, shadowing the attribute in the
instance dictionary MAY have ill effects. (slots such as __enter__ and __exit__ that are looked up via normal attribute lookup in CPython will fit into this category)
Why even have a MAY category? Are you expecting these to become tp_ slots at some point?
Either tp_* slots, or just having the invocation bypass the instance attributes and only look at the object's type.
I think it would actually be desirable for this category to be empty from a purity point of view (with all the special methods in category 1), but given that CPython itself currently doesn't conform to such a language spec, this seems to be the next best option (i.e. allowing other implementations or later versions of CPython to put these special methods in category 1 along with the rest of the special methods)
- Technically a subcategory of group 1, these are special methods which can
affect the interpreter's behaviour by their mere definition on a type. (The __get__, __set__ and __delete__ descriptor protocol methods fall into this category)
I don't follow why this is relevant. This is a different, AFAIK orthogonal issue, used in many places: *if* an object used in a certain context has a specific attribute, *then* that attribute is used, *otherwise* a default action is taken. Applies to __repr__ just as much. These belong in category 1 if and only if the lookup bypasses the instance dict.
Actual hasattr() checks aren't a problem - those hit __getattribute__ and a delegating class can correctly check them against the target object. Methods like __str__ or __repr__ also aren't a major issue - those are easy to delegate in a way that reproduces the same behaviour as if the delegating class wasn't there (just reinvoke the appropriate builtin on your target object).
This category is specifically for method checks in category 1 which bypass __getattribute__ *and* have significant effects on the way an object gets handled that can't be readily dealt with by a value-based delegation class - the most significant methods I've actually found in that category so far are the descriptor protocol methods (that's why my ProxyMixin class skipped delegating them).
As long as the callable() builtin is around, __call__ actually lands in this category as well (since defining it will affect the answer returned by the callable() builtin). Being able to return different proxy classes with and without __callable__ defined is actually the reason weakref.proxy is a factory function rather than a type in its own right.
Cheers, Nick.
Guido van Rossum wrote:
[Barry]
[Guido]
I've added a comment. Let me know if anything I said is unclear.
On Thu, Jun 12, 2008 at 3:35 AM, Nick Coghlan ncoghlan@gmail.com wrote:
The bugtracker seems to be offline atm - I'll reply there once I can get to it again (as well as switching this issue back to being a documentation one).
I don't think we're going to see a major clamor for a value-based delegation mixin in the standard library until people using classic classes for value-based delegation start making serious attempts to port to Py3k (where that approach is no longer available). At the moment, such classes only need to care about the methods they want to fiddle with, leaving everything else to __getattr__ based delegation.
Whether they'll care about this issue of course depends on whether they need overloaded operators and other special delegations to be delegated transparently. We'll have to see how important this is. New-style classes have been around and recommended for a long time -- why haven't people pushed for a proxy class before?
It's only in Python 3 that old style classes are going away fully, so up until now you could at least use a classic class to do the proxying.
I've written my own proxy classes before that look very similar to this, and there are other proxy classes around that do the same (I thought one was by Phillip J Eby but can't find a reference easily). The last one I wrote was to proxy CPython objects from IronPython via Python.NET...
I would prefer it if the proxy class wrapped the return values of inplace operations.
Michael Foord
I've pushed as hard as I'm personally willing to for this without convincing anyone else that it's worth doing,
What does *it* refer to? Changing the behavior, or adding a proxy class to the stdlib? I'm -1000 on the former, but only -0 on the latter -- as I wrote in the tracker, I just don't want to see an unproven proxy class (and I don't like the module name).
so I'll start working on a documentation patch for the language reference instead which explicitly splits the special methods into the following categories:
Thanks for doing this, it is needed regardless!
- Method lookup MUST bypass __getattribute__, shadowing the attribute in
the instance dictionary MUST NOT have any ill effects. (All tp_* C-level slots and slots looked up via _PyType_Lookup will fit into this category)
Watch out: I think the term "method lookup" may be confusing here. Certainly when the user writes x.__foo__(), the instance dict *is* consulted. It is only on *implied* lookups (e.g. x[y] or x+y) where the instance dict is bypassed.
- Method lookup MAY bypass __getattribute__, shadowing the attribute in the
instance dictionary MAY have ill effects. (slots such as __enter__ and __exit__ that are looked up via normal attribute lookup in CPython will fit into this category)
Why even have a MAY category? Are you expecting these to become tp_ slots at some point?
- Technically a subcategory of group 1, these are special methods which can
affect the interpreter's behaviour by their mere definition on a type. (The __get__, __set__ and __delete__ descriptor protocol methods fall into this category)
I don't follow why this is relevant. This is a different, AFAIK orthogonal issue, used in many places: *if* an object used in a certain context has a specific attribute, *then* that attribute is used, *otherwise* a default action is taken. Applies to __repr__ just as much. These belong in category 1 if and only if the lookup bypasses the instance dict.
On 6/12/08, Nick Coghlan ncoghlan@gmail.com wrote:
documentation patch for the language reference ... following categories:
...
- Method lookup MAY bypass __getattribute__, shadowing the attribute in
the instance dictionary MAY have ill effects. (slots such as __enter__ and __exit__ that are looked up via normal attribute lookup in CPython will fit into this category)
Should this category really be enumerated?
I thought that was the default meaning of __name__, so the real clarification is:
(1) Requiring that the specific names in category 1 MUST be treated this way.
(2) Mentioning __*__ and listing any known exceptions. (Can "next" be treated this way despite the lack of __*__? Is it forbidden to treat __context__ this way?)
-jJ
On Wed, Jun 11, 2008 at 7:35 AM, Barry Warsaw barry@python.org wrote:
My plan is to begin building the betas tonight, at around 9 or 10pm EDT (0100 to 0200 UTC Thursday). If a showstopper comes up before then, I'll email the list. If you think we really aren't ready for beta, then I would still like to get a release out today. In that case, we'll call it alpha and delay the betas.
I have two release blockers pending review:
http://bugs.python.org/issue2918 http://bugs.python.org/issue2917
I believe both patches are ready to be committed to the py3k branch. However, I would certainly like that someone would review the patches (or at least test them).
Right now, I am currently looking at fixing issue 2919 (http://bugs.python.org/issue2919). The profile and the cProfile module differ much more than I originally expected. So, I won't be able to get these two for the beta.
I have also been looking at http://bugs.python.org/issue2874, in which Benjamin Peterson proposed an simple solution to fix it. Although I haven't tried his approach, I think I could get this one done for today.
Finally, I would like to commit the patch in http://bugs.python.org/issue2523 which fix the quadratic behavior in BufferedReader.read(). It would also be nice to have someone else experienced with the io module to review the patch.
Cheers, -- Alexandre
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Jun 11, 2008, at 4:54 PM, Alexandre Vassalotti wrote:
On Wed, Jun 11, 2008 at 7:35 AM, Barry Warsaw barry@python.org wrote:
My plan is to begin building the betas tonight, at around 9 or 10pm EDT (0100 to 0200 UTC Thursday). If a showstopper comes up before then, I'll email the list. If you think we really aren't ready for beta, then I would still like to get a release out today. In that case, we'll call it alpha and delay the betas.
I have two release blockers pending review:
http://bugs.python.org/issue2918 http://bugs.python.org/issue2917
I believe both patches are ready to be committed to the py3k branch. However, I would certainly like that someone would review the patches (or at least test them).
In IRC, Benjamin agreed to review these and commit them if they look okay. I'm still 3-4 hours away from making the betas, so there's time to do this and still see how the buildbots react.
- -Barry
On Wed, Jun 11, 2008 at 2:42 PM, Barry Warsaw barry@python.org wrote:
On Jun 11, 2008, at 4:54 PM, Alexandre Vassalotti wrote:
I have two release blockers pending review:
http://bugs.python.org/issue2918 http://bugs.python.org/issue2917
I believe both patches are ready to be committed to the py3k branch. However, I would certainly like that someone would review the patches (or at least test them).
In IRC, Benjamin agreed to review these and commit them if they look okay. I'm still 3-4 hours away from making the betas, so there's time to do this and still see how the buildbots react.
That's great, and I see these were committed and the bugs closed. Next time, can we also have some kind of OK from the reviewer (Benjamin, I presume) in the issue tracker? IRC does not leave a usable trail of decisions.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Jun 11, 2008, at 10:16 PM, Guido van Rossum wrote:
On Wed, Jun 11, 2008 at 2:42 PM, Barry Warsaw barry@python.org wrote:
On Jun 11, 2008, at 4:54 PM, Alexandre Vassalotti wrote:
I have two release blockers pending review:
http://bugs.python.org/issue2918 http://bugs.python.org/issue2917
I believe both patches are ready to be committed to the py3k branch. However, I would certainly like that someone would review the patches (or at least test them).
In IRC, Benjamin agreed to review these and commit them if they look okay. I'm still 3-4 hours away from making the betas, so there's time to do this and still see how the buildbots react.
That's great, and I see these were committed and the bugs closed. Next time, can we also have some kind of OK from the reviewer (Benjamin, I presume) in the issue tracker? IRC does not leave a usable trail of decisions.
Good point. - -Barry
participants (9)
-
Alexandre Vassalotti -
Barry Warsaw -
Georg Brandl -
Guido van Rossum -
Jim Jewett -
M.-A. Lemburg -
Michael Foord -
Nick Coghlan -
Walter Dörwald