Maybe I overstate the subject line, but there are serious limitations on the .unwrap() concept. The limitation stems from the "blackbox" concept for object spaces. In general, we cannot be sure that unwrap will give a valid object. It may work for a direct representation of Python objects (such as trivial and standard object spaces), but when we start to use object space representations that do not exactly mirror Python semantics, unwrap isn't guarranteed to work. For example, if an annotation-like object space represents obects as type & bool (for truth tracking, for example), upon calling wrap(), it discards all information about value. So there is no way we can say that s.unwrap(s.wrap(x)) == x. Even if an object space keeps some kind of info on the wrapped object such that it can be unwrapped, there is no guarrantee that something valid will remain after any sort of operations on it, as a different object space may induce a different or multiple path(s) through the code. This is probably tractable within the object space itself, as any sort of vagarities in the wrapping can be compensated for on a space by space basis. However, if we wish to have a general code evaluation framework for multiple object spaces, the code evaluation code has to be general. At the moment there are a number of places in the code evaluation framework which call unwrap(). - I think it is mostly for unwrapping code objects. The reason I bring this up is I think that this is the reason the general builtin-module import code I posted earlier crashes for the annotation object space. The code attemped to unwrap attributes of the sys module, assuming that normal Python sematics are valid. This doesn't hold for annotation object space ... hence the crash. So if we wish to be general in the execution framework, we need to limit the use of unwrap() to uses which are guarranteed to be semantically valid by the obect space interface. (At least for the subset of object spaces we wish to use for a particular code execution framework.) A small but significant correllary to this is that the semantics of the operations within a general object space should not be assumed to be Python-like. That is, operations on object space obects should be performed for object space effects, not for execution level effects. This poses significant problems for the application space helpers to compute execution level relevant results. I'm not sure how to address this yet. -Rocco P.S. I appologize if this has been discused before on IRC. Has any progress been made in creating an archive of IRC logs? I get the impression a fair number of significant points have been made on IRC, and it would be best to archive them such that others not present can benefit from them as well. __________________________________________________________________ McAfee VirusScan Online from the Netscape Network. Comprehensive protection for your entire computer. Get your free trial today! http://channels.netscape.com/ns/computing/mcafee/index.jsp?promo=393397 Get AOL Instant Messenger 5.1 free of charge. Download Now! http://aim.aol.com/aimnew/Aim/register.adp?promo=380455
Hello Rocco, On Sat, Aug 23, 2003 at 09:00:58PM -0400, Rocco Moretti wrote:
Maybe I overstate the subject line, but there are serious limitations on the .unwrap() concept.
Yes, I agree with your analysis. I actually think that we are using unwrap() for different purposes at different places, and we should clearly mark these differences...
all information about value. So there is no way we can say that s.unwrap(s.wrap(x)) == x.
I am now coming closer to the idea that wrap() should be redefined as well, because it is used not only on simple types but also on "internal interpreter structures", like code objects plus others more internals than that. I now tend to think about wrap(x) as 'I want you, object space, to build something that acts as reference to my object x'. The unwrap() operation would then only be defined on objects that have been wrapped, and the object space should somehow guarantee that "unwrap(wrap(x)) is x". Now this is not exactly the way wrap() and unwrap() are used in the current code base. We also use wrap(x) when x is a simple immutable object, to build simple blackboxed objects (ints, strs...). And we also use unwrap(x) when we expect 'x' to be such an object, to get a plain int or str out of a blackbox. Moreover, in the light of the current refactoring on branch 'builtinrefactor', I think it would make sense to say that the object spaces (all of them, whatever their semantics are) have to support a particular kind of interpreter-controlled semantics on complex objects created by wrap(). In other words, the interpreter would control what occurs when object space operations involve such an object. So, for example, space.getattr() on a wrapped code object would force the object space to call back the interpreter, which would perform the necessary attribute-reading operation. (I think it is reasonable to say that the object space should not be responsible for that because it is essentially an interpreter-controlled object. The same would go for, say, wrapped frames, tracebacks, whatever, possibly module objects too. There is no need to code logic for these in each and every object space, as we are now doing in std and ann). In summary we'd have two usages for wrap()/unwrap() -- let's try to figure out some better names: * wrap(x) -> create a blackboxed reference for the interpreter object x * unwrap(w_x,ExpectedType) -> inverse of the previous operation * newint(i), newstr(s)... -> create simple object space objects * unwrap(w_x,int), unwrap(w_x,str) -> inverse of the previous operation In this idea, unwrap() would be a bit like PyArg_ParseTuple in that it would check what type of interpreter-level object it is supposed to return, possibly raising a TypeError. Still this doesn't directly address the whole issue of application-level helpers. I tend to think now that we should be very careful before using app-level helpers. All the troubles we had with the argument-parsing one were solved when Holger finally convinced me to just try to write the damn thing directly at interpreter-level, which I did without too much trouble after all. I'd currently advice to reserve app-level helpers to tasks that could really be thought of as part of a user program. For example I think it is ok to keep at app-level the logic that decodes what a 'raise' statement really meant, e.g. 'raise X, Y' -> 'raise X(Y)', because this is the kind of thing that a user could write in a small general-purpose function of his program if it wasn't built in the interpreter. A bient�t, Armin.
Armin Rigo <arigo@tunes.org> writes:
Hello Rocco,
Oops, I've been meaning to reply to this too...
Yup.
[snippety]
(note this is like a bit like Py_BuildValue in the C API) I'd *much* rather write space.build(1) than space.newint(1). This would be a convenience method that takes a fixed set of types (maybe just integers and strings...).
* unwrap(w_x,int), unwrap(w_x,str) -> inverse of the previous operation
unbuild? If we have different calls to unwrap doing radically different things, I think they should have different names...
It seems that way. Cheers, mwh -- Good? Bad? Strap him into the IETF-approved witch-dunking apparatus immediately! -- NTK now, 21/07/2000
Hello Michael, On Tue, Sep 02, 2003 at 03:10:06PM +0100, Michael Hudson wrote:
Ok. Maybe the whole issue should be sorted out in the general context of how to declare "gateways" between interpreter- and app-level. For example, given an interpreter-level class class X: def __init__(self, frame): self.w_stuff = space.newdict([]) self.n = 5 self.frame = frame def dosomething(self, w_x, i): return self.n how could we cleanly specify that we want 'n' and 'dosomething' be app-level-visible as, respectively, an integer object and a method taking two arguments the second of which must be an integer? In other words the whole Py_BuildValue / Py_BuildTuple / PyArg_ParseTuple business, plus structmember.c. The point of giving the same name to wrap/unwrap and to the proposed build/unbuild is to have a uniform way of exposing attributes of different kinds, as shown on the above example: * w_stuff is a wrapped app-level object all along, no conversion required. * n is an integer, must use build/unbuild. * frame is another interp-level internal object, must use wrap/unwrap. This is the reason why we might want a single function pair for all three cases. Alternatively we could use 'type descriptor' objects: * t_wrapped.wrap(space, w_stuff) -> w_stuff * t_int.wrap(space, 5) -> w_5 * t_interplevel.wrap(space, frame) -> wrapped frame * t_wrapped.unwrap(space, w_stuff) -> w_stuff * t_int.unwrap(space, w_5) -> 5 * t_interplevel.unwrap(space, w_frame) -> frame The advantage is that we can declare function signatures explicitely, say appmethod([t_wrapped, t_int], t_int) for the above dosomething() method, and it is clear that the wrap/unwrap methods of the specified type descriptors will be used for the conversion at appropriate times. (Type descriptors are also a good place to store extra information and functionality if needed, e.g. the corresponding C type. The idea comes from Thomas Heller's ctypes module.) The t_xxx objects above could also be classes instead, with wrap/unwrap class or static methods. This would let us use the interpreter-level class hierarchy directly instead of the t_interplevel trick: Frame.unwrap(space, w_frame) -> frame This is more explicit because it can check that we actually have, not just an interpreter-level object, but a Frame instance. We could also put the space argument at the end, so that we can write Frame.wrap(frame, space) -> w_frame or frame.wrap(space) -> w_frame and thus wrap (but not unwrap) could actually be a regular method. As a final note, there would then be a lot of different wrap/unwrap methods defined all around which handle different types -- certainly a good thing for flexibility and also for the annotation object space, which would not get confused by the fact that a single build() or unwrap() could accept a whole lot of different types. A bient�t, Armin.
Hello Rocco, On Sat, Aug 23, 2003 at 09:00:58PM -0400, Rocco Moretti wrote:
Maybe I overstate the subject line, but there are serious limitations on the .unwrap() concept.
Yes, I agree with your analysis. I actually think that we are using unwrap() for different purposes at different places, and we should clearly mark these differences...
all information about value. So there is no way we can say that s.unwrap(s.wrap(x)) == x.
I am now coming closer to the idea that wrap() should be redefined as well, because it is used not only on simple types but also on "internal interpreter structures", like code objects plus others more internals than that. I now tend to think about wrap(x) as 'I want you, object space, to build something that acts as reference to my object x'. The unwrap() operation would then only be defined on objects that have been wrapped, and the object space should somehow guarantee that "unwrap(wrap(x)) is x". Now this is not exactly the way wrap() and unwrap() are used in the current code base. We also use wrap(x) when x is a simple immutable object, to build simple blackboxed objects (ints, strs...). And we also use unwrap(x) when we expect 'x' to be such an object, to get a plain int or str out of a blackbox. Moreover, in the light of the current refactoring on branch 'builtinrefactor', I think it would make sense to say that the object spaces (all of them, whatever their semantics are) have to support a particular kind of interpreter-controlled semantics on complex objects created by wrap(). In other words, the interpreter would control what occurs when object space operations involve such an object. So, for example, space.getattr() on a wrapped code object would force the object space to call back the interpreter, which would perform the necessary attribute-reading operation. (I think it is reasonable to say that the object space should not be responsible for that because it is essentially an interpreter-controlled object. The same would go for, say, wrapped frames, tracebacks, whatever, possibly module objects too. There is no need to code logic for these in each and every object space, as we are now doing in std and ann). In summary we'd have two usages for wrap()/unwrap() -- let's try to figure out some better names: * wrap(x) -> create a blackboxed reference for the interpreter object x * unwrap(w_x,ExpectedType) -> inverse of the previous operation * newint(i), newstr(s)... -> create simple object space objects * unwrap(w_x,int), unwrap(w_x,str) -> inverse of the previous operation In this idea, unwrap() would be a bit like PyArg_ParseTuple in that it would check what type of interpreter-level object it is supposed to return, possibly raising a TypeError. Still this doesn't directly address the whole issue of application-level helpers. I tend to think now that we should be very careful before using app-level helpers. All the troubles we had with the argument-parsing one were solved when Holger finally convinced me to just try to write the damn thing directly at interpreter-level, which I did without too much trouble after all. I'd currently advice to reserve app-level helpers to tasks that could really be thought of as part of a user program. For example I think it is ok to keep at app-level the logic that decodes what a 'raise' statement really meant, e.g. 'raise X, Y' -> 'raise X(Y)', because this is the kind of thing that a user could write in a small general-purpose function of his program if it wasn't built in the interpreter. A bient�t, Armin.
Armin Rigo <arigo@tunes.org> writes:
Hello Rocco,
Oops, I've been meaning to reply to this too...
Yup.
[snippety]
(note this is like a bit like Py_BuildValue in the C API) I'd *much* rather write space.build(1) than space.newint(1). This would be a convenience method that takes a fixed set of types (maybe just integers and strings...).
* unwrap(w_x,int), unwrap(w_x,str) -> inverse of the previous operation
unbuild? If we have different calls to unwrap doing radically different things, I think they should have different names...
It seems that way. Cheers, mwh -- Good? Bad? Strap him into the IETF-approved witch-dunking apparatus immediately! -- NTK now, 21/07/2000
Hello Michael, On Tue, Sep 02, 2003 at 03:10:06PM +0100, Michael Hudson wrote:
Ok. Maybe the whole issue should be sorted out in the general context of how to declare "gateways" between interpreter- and app-level. For example, given an interpreter-level class class X: def __init__(self, frame): self.w_stuff = space.newdict([]) self.n = 5 self.frame = frame def dosomething(self, w_x, i): return self.n how could we cleanly specify that we want 'n' and 'dosomething' be app-level-visible as, respectively, an integer object and a method taking two arguments the second of which must be an integer? In other words the whole Py_BuildValue / Py_BuildTuple / PyArg_ParseTuple business, plus structmember.c. The point of giving the same name to wrap/unwrap and to the proposed build/unbuild is to have a uniform way of exposing attributes of different kinds, as shown on the above example: * w_stuff is a wrapped app-level object all along, no conversion required. * n is an integer, must use build/unbuild. * frame is another interp-level internal object, must use wrap/unwrap. This is the reason why we might want a single function pair for all three cases. Alternatively we could use 'type descriptor' objects: * t_wrapped.wrap(space, w_stuff) -> w_stuff * t_int.wrap(space, 5) -> w_5 * t_interplevel.wrap(space, frame) -> wrapped frame * t_wrapped.unwrap(space, w_stuff) -> w_stuff * t_int.unwrap(space, w_5) -> 5 * t_interplevel.unwrap(space, w_frame) -> frame The advantage is that we can declare function signatures explicitely, say appmethod([t_wrapped, t_int], t_int) for the above dosomething() method, and it is clear that the wrap/unwrap methods of the specified type descriptors will be used for the conversion at appropriate times. (Type descriptors are also a good place to store extra information and functionality if needed, e.g. the corresponding C type. The idea comes from Thomas Heller's ctypes module.) The t_xxx objects above could also be classes instead, with wrap/unwrap class or static methods. This would let us use the interpreter-level class hierarchy directly instead of the t_interplevel trick: Frame.unwrap(space, w_frame) -> frame This is more explicit because it can check that we actually have, not just an interpreter-level object, but a Frame instance. We could also put the space argument at the end, so that we can write Frame.wrap(frame, space) -> w_frame or frame.wrap(space) -> w_frame and thus wrap (but not unwrap) could actually be a regular method. As a final note, there would then be a lot of different wrap/unwrap methods defined all around which handle different types -- certainly a good thing for flexibility and also for the annotation object space, which would not get confused by the fact that a single build() or unwrap() could accept a whole lot of different types. A bient�t, Armin.
participants (3)
-
Armin Rigo -
Michael Hudson -
roccomoretti＠netscape.net