
I'd like to do a 2.3b1 release someday. Maybe at the end of next week, that would be Friday April 25. If anyone has something that needs to be done before this release go out, please let me know! Assigning a SF bug or patch to me and setting the priority to 7 is a good way to get my attention. --Guido van Rossum (home page: http://www.python.org/~guido/)

Right. Fix away.
How about new tests?
Feel free to add new unit tests, as long as the whole unit test suite passes when you commit. --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum <guido@python.org> writes:
I would still like to work on http://www.python.org/sf/595026 support for masks in getargs.c. Jack requested that this change should be implement shortly after the release of 2.3a2, but this is too late now as it seems ;-) What to do? Implement it now and commit it after 2.3b1 is released, or delay this until 2.3 final is released. I have to admit that I'm sure I can implement it for 32-bit Windows, but it would have to be tested (and maybe completed) on other, especially 64-bit platforms as well. And it introduces incompatibilities. BTW: Since you want to release a beta version, what's the state of the FutureWarning about hex/oct constants: will this stay the way it is? Thomas

Great!
Do it ASAP.
If you can get something rough into 2.3b1, it can be improved while 2.3b2 is cooking.
And it introduces incompatibilities.
What kind? I thought it would be a new format code?
BTW: Since you want to release a beta version, what's the state of the FutureWarning about hex/oct constants: will this stay the way it is?
Probably, unless you hve a better idea. :-( --Guido van Rossum (home page: http://www.python.org/~guido/)

Ok, working on it.
Two new format codes ('k' and 'K'), and changes to existing format codes - per your request: | How about the following counterproposal. This also changes some of the | other format codes to be a little more regular. | | Code C type Range check | | b unsigned char 0..UCHAR_MAX | B unsigned char none ** | h unsigned short 0..USHRT_MAX | H unsigned short none ** | i int INT_MIN..INT_MAX | I * unsigned int 0..UINT_MAX | l long LONG_MIN..LONG_MAX | k * unsigned long none | L long long LLONG_MIN..LLONG_MAX | K * unsigned long long none | | Notes: | | * New format codes. | | ** Changed from previous "range-and-a-half" to "none"; the | range-and-a-half checking wasn't particularly useful.
I haven't used warnings very much, but is there a possibility to disable them per module? You get a lot of them if you 'import win32con' for example. Thomas

Oh of course. None to worry about IMO.
Yes, you can suppress warnings per module. Please read the docs. --Guido van Rossum (home page: http://www.python.org/~guido/)

On woensdag, apr 16, 2003, at 20:11 Europe/Amsterdam, Thomas Heller wrote:
Do I understand correctly that there is no format code that works on both 2.2 and 2.3 that converts 32 bit quantities without complaining (B and H will work for 8 and 16 bit quantities)? That may be a serious problem for PyObjC.... -- - Jack Jansen <Jack.Jansen@oratrix.com> http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman -

I have a couple of small patches and bugs to review. Should be no problem getting these in this weekend. Am working on a more cache friendly dict lookup strategy. If it is not ready for prime time in the next few days, it will have to wait for Py2.4. A couple of bytecode optimizations may also have to wait for Py2.4. For some reason, Basicblock(nop, jump_if_true) is not always directly substitutable for Basicblock(unary_not, jump_if_false). I suspect the three-way return value for PyObject_IsTrue() but it could be something else. Raymond Hettinger

On Wed, Apr 16, 2003 at 11:52:10AM -0400, Guido van Rossum wrote:
Well, there is the CALL_ATTR patch (http://www.python.org/sf/709744) that Brett and I worked on at the PyCon sprints. It's finished (barring tests) for classic classes, and writing tests is not very inspiring because all functionality is already tested in the standard test suite. However, it doesn't do anything with newstyle classes at all, yet. I've had suprisingly little time since PyCon (it's amazing how not being at the office for two weeks makes people shove work your way -- I guess they realized I couldn't object :) but I'm in the process of grokking newstyle classes. So far, I've been alternating from 'Wow! to 'Au!', and I'll send another email after this one for clarification of a few issues :) Anyway, if anyone has straightforward ideas about how CALL_ATTR should deal with newstyle classes (if at all), please inform me (preferably via SF) or just grab the patch and run with it. I'm still confused about descrgets and where they come from. -- Thomas Wouters <thomas@xs4all.net> Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

On Thu, Apr 17, 2003 at 05:27:22PM +0200, Thomas Wouters wrote:
So far, I've been alternating from 'Wow! to 'Au!', and I'll send another email after this one for clarification of a few issues :)
Nevermind that. A "D'oh" slipped into the stream, and I think I get it now. At least the stuff that wasn't working is working now. I wouldn't mind if someone pointed me to a xxtype.c (newstyle class in C) like we have xxobject and xxsubclass though... So far, it's been so simple I fear I'm missing something. -- Thomas Wouters <thomas@xs4all.net> Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

[Thomas]
And I want the new-style classes version!
Even without so much that problem here, I was buried in email for a week after returning from Python UK. :-)
Yes, please. Here's a quick explanation of descriptors: A descriptor is something that lives in a class' __dict__, and primarily affects instance attribute lookup. A descriptor has a __get__ method (in C this is the tp_descrget function in its type object) and the instance attribute lookup calls this to "bind" the descriptor to a specific instance. This is what turns a function into a bound method object in Python 2.2. In earlier versions, functions were special-cased by the instance getattr code; the special case has been subsumed by looking for a __get__ method. Yes, this means that a plain Python function object is a descriptor! Because the instance getattr code returns whatever __get__ returns as the result of the attribute lookup, this is also how properties work: they have a __get__ method that calls the property-get" function. A descriptor's __get__ method is also called for class attribute lookup (with the instance argument set to NULL or None). And a descsriptor's __set__ method is called for instance attribute assignment; but not for class attribute assignment. Hope this helps! --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum wrote:
Could you put such short overviews somewhere on the Python Wiki ? They sure help in understanding what is going on behind the scenes without having to grep through tons of source code :-) Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Software directly from the Source (#1, Apr 17 2003)
Python/Zope Products & Consulting ... http://www.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
EuroPython 2003, Charleroi, Belgium: 68 days left

Could you put such short overviews somewhere on the Python Wiki ?
I don't have the time for that. When I want to publish stuff like this somewhere, I need to spend time to make it all correct, complete etc.
They sure help in understanding what is going on behind the scenes without having to grep through tons of source code :-)
You should start by reading http://www.python.org/2.2.2/descrintro.html If you still have questions about descriptors after reading that, grepping the source is an option. :-) --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum <guido@python.org> writes:
Besides which, it's already in the docs. Correct, complete, and all that ;-) http://www.python.org/dev/doc/devel/ref/descriptors.html -- Dave Abrahams Boost Consulting www.boost-consulting.com

On Thu, Apr 17, 2003 at 11:53:01AM -0400, Guido van Rossum wrote:
Yes, please. Here's a quick explanation of descriptors:
[ the descriptor describes descriptors ]
Hope this helps!
Well, yes, in that it reminded me to stop looking for how functions get turned into methods. That part is the same for old-style classes, though, and not quite what I'm confused about. What the call_attr patch does is shortcut the instance_getattr functions in a new function, to do just that what is necessary (and no more.) _Py_instance_getmethod() returns NULL for anything that isn't a method, too, letting the slow case handle it. When it does find a would-be method, it returns the unwrapped function. The call_attr function basically does a PyInstance_Check() and a _Py_instance_getmethod(), and calls the returned function. The problem I have with newstyle classes is where to shortcut what. I understand now how to detect a would-be method, but I'm not sure how to get unwrapped attributes. As far as I understand, types can provide their own getattr function with complete control over descriptors, so there isn't much to shortcut. Unless I should make the shortcut depend on the actual value of tp_getattro, as in shortcut only if it actually is PyObject_GenericGetAttr ? In that case, I'm somewhat sceptical about the speed benefit's cost in maintenance, as it would require a near copy of PyObject_GenericGetAttr (which is already a near-copy of a few other functions :) It's also very hard to control any nested getattrs (possible, I think, because the process goes over all bases' dicts and the instance dict.) Or can we reduce the number of steps PyObject_GenericGetAttr goes through if we know we are just looking for a method ? I don't believe so, but I'm not sure. (Looking at PyObject_GenericGetAttr with that in mind, I wonder if there isn't a possible crash there. In the first MRO lookup, looking for descr's, if a non-data-descr is found, it is kept around but not INCREF'd until later, after the instance-dict is searched. Am I wrong in believing the PyDict_GetItem of the instance dict can call Python code ? There isn't even as much as an assert(PyDict_Check(dict)) there.) -- Thomas Wouters <thomas@xs4all.net> Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

On Thu, Apr 17, 2003 at 10:59:56PM +0200, Thomas Wouters wrote:
Well, I went ahead and did that, and uploaded the new patch to SF. The result is somewhat annoying, but explainable: The patch is now 3% _slower_ than an unmodified Python, whereas the patch without support for newstyle classes was a good 5% _faster_ than unmodified. This is both according to PyBench (which doesn't use newstyle classes) and according to 'time timeit.py pass' (which does use newstyle classes.) Timing just 'x.foo()' where 'x' is a newstyle class instance is about 20% faster, against 25-30% for oldstyle classes. The overall slowdown is caused by the fact that the patch only treats PyFunctions (functions written in Python) specially, and not PyMethodDescrs (PyCFunctions wrapped in PyMethodDefs wrapped in a descriptor.) This is because it would still need to instantiate a PyCFunctionObject (a PyObject wrapper for a PyCFunction, which is just a C function-pointer) OR it would need to do all interpretation of METH_* arguments and a bunch of argument-preparing itself. Another possible cause for the slowdown (but almost certainly not as substantial as the type-with-C-function one) is calling an almost-method on a newstyle class; a callable object that is an attribute of a type (or instance of the type) but is not a PyFunction or PyMethodDescr. The way the current mechanisms works, it would have to traverse the MRO and (possibly) check the instance dict twice; first to determine that it's not a PyFunction in _PyObject_Generic_getmethod() and then again in the regular run though PyObject_GenericGetAttr(). Examples of this case would be staticmethods, classmethods, and other callable objects as attributes. I do not believe this is a substantial party though. The slowdown can be fixed in two ways: handing PyMethodDescrs as well, in _PyObject_Generic_getmethod(), or removing the double lookups. Hm, wait, handling PyMethodDescrs may not be as tricky as I thought... hrm... I'll look at it tomorrow, it's time for bed. -- Thomas Wouters <thomas@xs4all.net> Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

On Fri, Apr 18, 2003 at 02:06:50AM +0200, Thomas Wouters wrote:
Hm, wait, handling PyMethodDescrs may not be as tricky as I thought... hrm... I'll look at it tomorrow, it's time for bed.
I did a quick hack to the same effect, and it still came out a 1% loss (so about 6% against the no-newstyle patch) in PyBench and a few timeit tests. Sigh. I guess the non-method overhead is just too large, or there are more almost-methods than I figured. I'll start work on a more lookup-saving _PyObject_Generic_getmethod tomorrow or this weekend (and will probably do _Py_instance_getmethod that way too, while I'm at it.) -- Thomas Wouters <thomas@xs4all.net> Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

On Fri, Apr 18, 2003 at 02:34:31AM +0200, Thomas Wouters wrote:
On Fri, Apr 18, 2003 at 02:06:50AM +0200, Thomas Wouters wrote:
Hm, wait, handling PyMethodDescrs may not be as tricky as I thought... hrm... I'll look at it tomorrow, it's time for bed.
Okay, for those who care about this but aren't on Patches, I just uploaded a new CALL_ATTR patch, version 4. It's actually two separate versions (3 and 4): maintainable, and fast. See the SF patch comment for more details :) However, I spent most of tonight trying to clock the patch, only to come to the conclusion that benchmarks suck. Which I already knew :) PyBench did a reasonable job pointing me towards slowness, but the main slowdowns I see with PyBench I cannot reproduce with timeit.py. I think I stopped trusting PyBench when it reported the patch was 2% slower, but did so 5% faster -- consistently. So, if anyone has any *real* programs they can test the patch with, I would be much obliged. Otherwise we'll have to check it in claiming it gives a 20% performance boost on ... methods of newstyle classes ,,, and 30% on ... methods of old-style classes. :) The patch is here: http://www.python.org/sf/709744 Where-...-reads-"empty-no-argument"-and-,,,-reads-"that-use- -PyObject_GenericGetAttr"-in-very-very-small-letters'ly y'rs, -- Thomas Wouters <thomas@xs4all.net> Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

So, if anyone has any *real* programs they can test the patch
I tried it on some of my apps which moderately exercise both new and old style classes. None of the apps improved and one a 1% worse. Both pybench and pystone were worse by 1%. Also, line 767 in classobject.c has an unreferenced variable, f. Raymond Hettinger

It can, if there's a key whose type has a custom __eq__ or __cmp__. So indeed, if this custom __eq__ is evil enough to delete the corresponding key from the class dict, it could cause descr to point to freed memory. I won't try to construct a case, but it's not impossible. :-( Fixing this would make the code even hairier though... :-(
There isn't even as much as an assert(PyDict_Check(dict)) there.)
All over the place it is assumed and ensured that a types tp_dict and an instance's __dict__ are always real dicts. The only way this could be violated would be by C code defining a type that violates this. --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum <guido@python.org> writes:
Indeed, there are several examples of this sort of thing already in Lib/test/test_mutants.py. Cheers, M. -- If comp.lang.lisp *is* what vendors are relying on to make or break Lisp sales, that's more likely the problem than is the effect of any one of us on such a flimsy marketing strategy... -- Kent M Pitman, comp.lang.lisp

On Friday, Apr 18, 2003, at 01:22 Europe/London, Guido van Rossum wrote: Note that this is quite some time ago :-) I've been reading through some messages that have been ticked for rather too long in my python-dev mail folder...
Indeed not! Here's mine: import gc class Evil(object): def __hash__(self): return hash('attr') def __eq__(self, other): global foo foo = gc.get_referrers(C.attr) del C.attr return 0 class Descr(object): def __get__(self, ob, type=None): return 1 class C(object): attr = Descr() c = C() c.__dict__[Evil()] = 0 c.attr `foo` It's always a bit hard to be sure that stuff like this will actually crash the interpreter, which is what the gimmicks with gc.get_referrers & `foo` are in aid of. This crashes every Python I have lying around on my iBook -- the 2.2[.0] in /usr/bin, heads of release22-maint, release23-maint, the CVS trunk, a random build of 2.2.3c1, Jack's 2.3 build. Etc ;-) I'll check on linux when I get into work.
Fixing this would make the code even hairier though... :-(
Pff, it's not so bad, just a little refcount code jiggling. I'll make a real patch including the above test case soon (once I'm reasonably confident that I'm not leaking references). And then I find out running test_descr in a loop leaks 166(!) references *anyway* (run the attached blah.py in a debug build, and some print test or other that I don't understand fails when run like this, so I fiddled it) Should I have been expecting this, or is it time for some ref-leak hunting? Actually, the process seems to grow -- slowly -- when you loop, so it's probably a real problem. I gather it's best to ask about this sort of thing when Tim is off sick from work :-) Cheers, mwh

[Michael Hudson]
Only if you want help <wink>. Since Zope has worse leak problems than Python, Zope testing has more tools to dig into it: http://cvs.zope.org/Zope3/test.py That's the Zope3 unittest test driver. The TrackRefs class is a handy little beast, which uses a debug build's sys.getobjects() to keep track of how many objects of each *type* exist across test runs, and how many refcounts total they account for. After the first few iterations, TrackRefs doesn't itself distort the results it prints. It will tell you the types of the objects that are leaking, and also whether objects are actually leaking, or that merely refcounts to existing objects are leaking (e.g., a common kind of "leak" is to forget to decref Py_None, but memory doesn't actually grow then). Do use 2.3 if you try this. The "list of all objects" before 2.3 turned out to be missing many objects that tend to leak (None, booleans, any number of type objects, essentially all statically allocated objects), and far fewer are missing in 2.3. It's typically easy to find out which types of objects are leaking this way. Then it's typical to add more code to TraceRefs.update() to get more details. In Zope use, it's surprising how often that still left us scratching our heads! But Zope has lots of extension types coded in C, massive interconnections among them, and none of them originally participated in cyclic gc. Python leaks are usually easier to plug (for example, in Python, they're almost always in new code).

"Tim Peters" <tim.one@comcast.net> writes:
Actually, I found the worst culprit this morning. Some dunderhead had got the decrefs wrong on error return from type_set_bases... I'll check my fix in momentarily.
Cool! I'd blundered my way to something like an ad hoc version of this, next time I'll use something that someone's actually thought about :-)
Do use 2.3 if you try this.
Well, yeah. Turned out the worst leaks weren't in 2.2 code anyway.
Bingo! I did all these tests incorporating my safety fixes to PyObject_GenericGetAttr so I think they're probably solid, too. Cheers, mwh -- The PROPER way to handle HTML postings is to cancel the article, then hire a hitman to kill the poster, his wife and kids, and fuck his dog and smash his computer into little bits. Anything more is just extremism. -- Paul Tomblin, asr

On Thursday, Aug 7, 2003, at 14:08 Europe/London, Michael Hudson wrote:
[I'll observe that trying to wrap running a test in a refcount neutral way is entertaining...]
OK, so I've now blundered my way to the attached, which when run produces this: ints seems to leak 1 references... <type 'long'> 1 4 multi seems to leak 9 references... <type 'tuple'> 1 5 <type 'classobj'> 1 4 <type 'dict'> 1 4 <type 'int'> 1 3 <type 'str'> 0 4 <type 'NoneType'> 0 1 slots seems to leak 10 references... <type 'tuple'> 5 20 <type 'type'> 0 5 errors seems to leak 9 references... <type 'tuple'> 4 16 <type 'type'> 0 5 subtype_resurrection seems to leak 2 references... (the names are all names of functions in test_descr). I might have a look at these over the next few days (and also might ponder other bits of the test suite). Cheers, mwh

Michael Hudson <mwh@python.net> writes:
http://www.python.org/sf/784825 Cheers, mwh -- ... with these conditions cam the realisation that ... nothing turned a perfectly normal healthy individual into a great political or military leader better than irreversible brain damage. -- The Hitch-Hikers Guide to the Galaxy, Episode 11

Thomas Wouters <thomas@xs4all.net>:
The problem I have with newstyle classes is where to shortcut what.
It sounds to me like descriptor objects will need to have a __callattr__ slot added. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+

While we're on the topic -- Guid, how would you feel about the idea of giving built-in function objects the same instance- binding behaviour as interpreted functions? This would help Pyrex considerably, because currently I have to resort to a kludge to make Pyrex-defined functions work as methods. It mostly works, but it has some side effects, such as breaking the most common idiomatic usage of staticmethod() and classmethod(). If built-in functions were more like interpreted functions in this regard, all these problems would go away. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+

There are two ways to "bind" a built-in function to an object. One would be to do what happens for Python functions, which is in effect a currying: f.__get__(obj) yields a function g that when called as g(arg1, ...) calls f(obj, arg1, ...). (In fact, I've recently checked in a change that makes instancemethod a general currying function on the first argument. :-) But the other interpretation, which might be more appropriate for C functions, is that the bound instance is passed to the first argument at the *C* level, usually called self: PyObject * my_c_function(PyObject *self, PyObject *args) { ... } Which one would you like? I think we could do each rather easily (perhaps the first more easily because the type needed to represent the bound method already exist; for the second I think we'd have to introduce a new helper object type). --Guido van Rossum (home page: http://www.python.org/~guido/)

That's the one I'm talking about. I forgot to explain that the problem occurs when I'm creating a *Python* class object and populating it with functions that are supposed to be methods. Currently I have to manually wrap each function in an unbound method object before putting it in the class's __dict__. If that happened automatically on access, I would be able to create Python classes that behave more like the real thing. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+

OK, are you up for submitting a patch? --Guido van Rossum (home page: http://www.python.org/~guido/)

On woensdag, apr 16, 2003, at 17:52 Europe/Amsterdam, Guido van Rossum wrote:
The getargs mods got checked in just this morning, even though I explicitly and rather strongly asked that if these mods be made they be checked in *long* before a release was due:-( This means that all the Mac modules are now 100% dead. The same is probably true for PyObjC. And PyObjC has the added problem that it needs to be compatible with both 2.3b1 and 2.2 (notice that that is "2.2", not "2.2.X": PyObjC has to work with /usr/bin/python that Apple ships, which is 2.2 at the moment). I assume there are format codes that will convert 16 bit and 32 bit integer quantities without any checks on both 2.2 and 2.3, but I haven't investigated yet. And there may be problems with other wrapper packages (win32, wxPython, PyOpenGL) too. I will start fixing things, but there are only 4 real days left before April 25, given easter, so I would strongly urge for postponing the release date for another two weeks or so. -- - Jack Jansen <Jack.Jansen@oratrix.com> http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman -

Sorry, I forgot. Did you make a note of that on the SF patch?
Maybe we should retract the changes to existing format codes that make them more restrictive? That should revive any code that's curerntly dead, right?
That would endanger then entire release schedule to the point of pushing the 2.3 release past the OSCON conference (July 7-11). --Guido van Rossum (home page: http://www.python.org/~guido/)

On donderdag, apr 17, 2003, at 22:17 Europe/Amsterdam, Guido van Rossum wrote:
Yes, I'm pretty sure I did. Thomas also seems to refer to it...
That would be much better. if "l" (lower case ell) would continue to accept anything I wouldn't have to change anything. Of course I've been busy all night fixing code, but apart from a couple of hand-crafted modules I haven't checked anything in yet. I will check it in on a branch later tonight, and then I'll either forget about the branch or merge it, depending on the resolution of this. -- - Jack Jansen <Jack.Jansen@oratrix.com> http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman -

Jack Jansen <Jack.Jansen@oratrix.com> writes:
He did, and I also mentioned it yesterday. OTOH, I had sitting a first version of the patch on SF for a rather long time (shortly after the alpha2 release), asking for feedback, but didn't get any.
Guido has also suggested to keep another code without changes, I cannot remember which one it was, but there is a comment on SF. I have the impression that the new test_getargs2.py test makes it easy to change the behaviour and verify it to anything we want. In case it is too much trouble, why not backout all this again (although someone else would have to do it, I'm basically offline until tuesday), and check in after the b1 release. Sorry, Thomas

That was my fault -- I was too busy. :-(
That was 'h'.
I'll back out the change to 'h', which is the only incompatible change I can see (unless you consider accepting *more* than before an error). Thomas made no changes to 'l', so I'm not sure what that is about -- maybe the problem is with unsigned hex constants? --Guido van Rossum (home page: http://www.python.org/~guido/)

On vrijdag, apr 18, 2003, at 01:14 Europe/Amsterdam, Guido van Rossum wrote:
Right, 'h' turns out to be the problem. I changed a lot of 'l's to 'k's, but it seems this one is the real killer. -- - Jack Jansen <Jack.Jansen@oratrix.com> http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman -

Right, 'h' turns out to be the problem. I changed a lot of 'l's to 'k's, but it seems this one is the real killer.
So now that I rolled back 'h', is there any reason not to keep the rest of these changes? --Guido van Rossum (home page: http://www.python.org/~guido/)

On vrijdag, apr 18, 2003, at 15:25 Europe/Amsterdam, Guido van Rossum wrote:
No, everything is fine as it is now. I'm happy again! -- - Jack Jansen <Jack.Jansen@oratrix.com> http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman -

On vrijdag, apr 18, 2003, at 01:14 Europe/Amsterdam, Guido van Rossum wrote:
Okay, great!! Is this a temporary measure, i.e. is the semantic change to 'h' going to come back after 2.3 is out? -- - Jack Jansen <Jack.Jansen@oratrix.com> http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman -

I don't see why -- it always was a signed short, let it stay that way. --Guido van Rossum (home page: http://www.python.org/~guido/)

On Wed, 16 Apr 2003, Guido van Rossum wrote:
The UnixWare build is way dead right now. (today's CVS) cc -c -K pentium,host,inline,loop_unroll,alloca -DNDEBUG -O -I. -I/opt/src/utils/python/python/dist/src/Include -DPy_BUILD_CORE -o Modules/python.o /opt/src/utils/python/python/dist/src/Modules/python.c UX:acomp: ERROR: "/usr/include/sys/select.h", line 45: identifier redeclared: fd_set UX:acomp: ERROR: "/usr/include/sys/select.h", line 72: identifier redeclared: select gmake: *** [Modules/python.o] Error 1
-- Tim Rice Multitalents (707) 887-1469 tim@multitalents.net

On 22 Apr 2003 at 7:28, Martin v. Löwis wrote:
I'm sorry I'm not in a position to fix it, but I do have an un-opened Unixware Advanced Server 2.01 box set (docs and media) if anyone wants them. Personally, I think Unixware is dead. Novell dropped it ages ago. -- Brad Clements, bkc@murkworks.com (315)268-1000 http://www.murkworks.com (315)268-9812 Fax http://www.wecanstopspam.org/ AOL-IM: BKClements

That doesn't look like a *new* problem to me; if sys/select.h is being included twice, that probably was so for a long time. You may be the only person with access to this platform. Can you find the problem? Was this present in 2.3a2? --Guido van Rossum (home page: http://www.python.org/~guido/)

On Tue, 22 Apr 2003, Guido van Rossum wrote:
I think it was in 2.3a1 and probably before. It looks like the problem is having both sys/time.h and sys/select.h included when both _XOPEN_SOURCE and _XOPEN_SOURCE_EXTENDED are defined. SYS_SELECT_WITH_SYS_TIME is not defined in pyconfig.h so configure is detecting the problem. It's just that SYS_SELECT_WITH_SYS_TIME is not user anywhere in the code. Something like this will get things a lot farther. ------------------------ --- pyport.h.old 2003-04-17 13:17:24.000000000 -0700 +++ pyport.h 2003-04-22 08:51:43.230240009 -0700 @@ -115,7 +115,9 @@ #ifdef HAVE_SYS_SELECT_H +#ifdef SYS_SELECT_WITH_SYS_TIME #include <sys/select.h> +#endif #endif /* !HAVE_SYS_SELECT_H */ ------------------------ -- Tim Rice Multitalents (707) 887-1469 tim@multitalents.net

On Tue, 22 Apr 2003, Tim Rice wrote:
Well after patching pyport.h for the sys/select problem, I had errors because of missing u_int and u_long data types. Patch configure.in, pyconfig.h.in, pyport.h. Now u_char, and u_short. Patch configure.in, pyconfig.h.in, pyport.h. some more. Now missing defines of NI_MAXHOST, NI_NUMERICHOST, & NI_MAXSERV. At that point I said to myself "This is nuts, 2.2.2 worked fine". So I backed out all my other patches and added this one. -------------------------- --- configure.in.old 2003-04-17 13:16:42.000000000 -0700 +++ configure.in 2003-04-22 15:26:13.450080095 -0700 @@ -124,6 +124,8 @@ # of union __?sigval. Reported by Stuart Bishop. SunOS/5.6) define_xopen_source=no;; + OpenUNIX/8.* | UnixWare/7.*) + define_xopen_source=no;; esac if test $define_xopen_source = yes -------------------------- Builds fine now. -- Tim Rice Multitalents (707) 887-1469 tim@multitalents.net

Tim Rice <tim@multitalents.net> writes:
Well after patching pyport.h for the sys/select problem, I had errors because of missing u_int and u_long data types.
In this form, I consider the patch unacceptable. Setting define_xopen_source should be the last resort, to be used only if the operating system is broken in the sense of not working at all as an X/Open system, for compiling software. If this is indeed the case that OpenUnix cannot work with _XOPEN_SOURCE defined, giving one instance of an unsolvable problem in a comment that explains why it should be disabled. See the comments for other systems as to how to explain such problems. Saying there are "errors" is too unspecific; saying that u_int is not defined but needed for the signature of the foo_bar function would be ok. Please post your updated patch to SF. Regards, Martin

Right. Fix away.
How about new tests?
Feel free to add new unit tests, as long as the whole unit test suite passes when you commit. --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum <guido@python.org> writes:
I would still like to work on http://www.python.org/sf/595026 support for masks in getargs.c. Jack requested that this change should be implement shortly after the release of 2.3a2, but this is too late now as it seems ;-) What to do? Implement it now and commit it after 2.3b1 is released, or delay this until 2.3 final is released. I have to admit that I'm sure I can implement it for 32-bit Windows, but it would have to be tested (and maybe completed) on other, especially 64-bit platforms as well. And it introduces incompatibilities. BTW: Since you want to release a beta version, what's the state of the FutureWarning about hex/oct constants: will this stay the way it is? Thomas

Great!
Do it ASAP.
If you can get something rough into 2.3b1, it can be improved while 2.3b2 is cooking.
And it introduces incompatibilities.
What kind? I thought it would be a new format code?
BTW: Since you want to release a beta version, what's the state of the FutureWarning about hex/oct constants: will this stay the way it is?
Probably, unless you hve a better idea. :-( --Guido van Rossum (home page: http://www.python.org/~guido/)

Ok, working on it.
Two new format codes ('k' and 'K'), and changes to existing format codes - per your request: | How about the following counterproposal. This also changes some of the | other format codes to be a little more regular. | | Code C type Range check | | b unsigned char 0..UCHAR_MAX | B unsigned char none ** | h unsigned short 0..USHRT_MAX | H unsigned short none ** | i int INT_MIN..INT_MAX | I * unsigned int 0..UINT_MAX | l long LONG_MIN..LONG_MAX | k * unsigned long none | L long long LLONG_MIN..LLONG_MAX | K * unsigned long long none | | Notes: | | * New format codes. | | ** Changed from previous "range-and-a-half" to "none"; the | range-and-a-half checking wasn't particularly useful.
I haven't used warnings very much, but is there a possibility to disable them per module? You get a lot of them if you 'import win32con' for example. Thomas

Oh of course. None to worry about IMO.
Yes, you can suppress warnings per module. Please read the docs. --Guido van Rossum (home page: http://www.python.org/~guido/)

On woensdag, apr 16, 2003, at 20:11 Europe/Amsterdam, Thomas Heller wrote:
Do I understand correctly that there is no format code that works on both 2.2 and 2.3 that converts 32 bit quantities without complaining (B and H will work for 8 and 16 bit quantities)? That may be a serious problem for PyObjC.... -- - Jack Jansen <Jack.Jansen@oratrix.com> http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman -

I have a couple of small patches and bugs to review. Should be no problem getting these in this weekend. Am working on a more cache friendly dict lookup strategy. If it is not ready for prime time in the next few days, it will have to wait for Py2.4. A couple of bytecode optimizations may also have to wait for Py2.4. For some reason, Basicblock(nop, jump_if_true) is not always directly substitutable for Basicblock(unary_not, jump_if_false). I suspect the three-way return value for PyObject_IsTrue() but it could be something else. Raymond Hettinger

On Wed, Apr 16, 2003 at 11:52:10AM -0400, Guido van Rossum wrote:
Well, there is the CALL_ATTR patch (http://www.python.org/sf/709744) that Brett and I worked on at the PyCon sprints. It's finished (barring tests) for classic classes, and writing tests is not very inspiring because all functionality is already tested in the standard test suite. However, it doesn't do anything with newstyle classes at all, yet. I've had suprisingly little time since PyCon (it's amazing how not being at the office for two weeks makes people shove work your way -- I guess they realized I couldn't object :) but I'm in the process of grokking newstyle classes. So far, I've been alternating from 'Wow! to 'Au!', and I'll send another email after this one for clarification of a few issues :) Anyway, if anyone has straightforward ideas about how CALL_ATTR should deal with newstyle classes (if at all), please inform me (preferably via SF) or just grab the patch and run with it. I'm still confused about descrgets and where they come from. -- Thomas Wouters <thomas@xs4all.net> Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

On Thu, Apr 17, 2003 at 05:27:22PM +0200, Thomas Wouters wrote:
So far, I've been alternating from 'Wow! to 'Au!', and I'll send another email after this one for clarification of a few issues :)
Nevermind that. A "D'oh" slipped into the stream, and I think I get it now. At least the stuff that wasn't working is working now. I wouldn't mind if someone pointed me to a xxtype.c (newstyle class in C) like we have xxobject and xxsubclass though... So far, it's been so simple I fear I'm missing something. -- Thomas Wouters <thomas@xs4all.net> Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

[Thomas]
And I want the new-style classes version!
Even without so much that problem here, I was buried in email for a week after returning from Python UK. :-)
Yes, please. Here's a quick explanation of descriptors: A descriptor is something that lives in a class' __dict__, and primarily affects instance attribute lookup. A descriptor has a __get__ method (in C this is the tp_descrget function in its type object) and the instance attribute lookup calls this to "bind" the descriptor to a specific instance. This is what turns a function into a bound method object in Python 2.2. In earlier versions, functions were special-cased by the instance getattr code; the special case has been subsumed by looking for a __get__ method. Yes, this means that a plain Python function object is a descriptor! Because the instance getattr code returns whatever __get__ returns as the result of the attribute lookup, this is also how properties work: they have a __get__ method that calls the property-get" function. A descriptor's __get__ method is also called for class attribute lookup (with the instance argument set to NULL or None). And a descsriptor's __set__ method is called for instance attribute assignment; but not for class attribute assignment. Hope this helps! --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum wrote:
Could you put such short overviews somewhere on the Python Wiki ? They sure help in understanding what is going on behind the scenes without having to grep through tons of source code :-) Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Software directly from the Source (#1, Apr 17 2003)
Python/Zope Products & Consulting ... http://www.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
EuroPython 2003, Charleroi, Belgium: 68 days left

Could you put such short overviews somewhere on the Python Wiki ?
I don't have the time for that. When I want to publish stuff like this somewhere, I need to spend time to make it all correct, complete etc.
They sure help in understanding what is going on behind the scenes without having to grep through tons of source code :-)
You should start by reading http://www.python.org/2.2.2/descrintro.html If you still have questions about descriptors after reading that, grepping the source is an option. :-) --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum <guido@python.org> writes:
Besides which, it's already in the docs. Correct, complete, and all that ;-) http://www.python.org/dev/doc/devel/ref/descriptors.html -- Dave Abrahams Boost Consulting www.boost-consulting.com

On Thu, Apr 17, 2003 at 11:53:01AM -0400, Guido van Rossum wrote:
Yes, please. Here's a quick explanation of descriptors:
[ the descriptor describes descriptors ]
Hope this helps!
Well, yes, in that it reminded me to stop looking for how functions get turned into methods. That part is the same for old-style classes, though, and not quite what I'm confused about. What the call_attr patch does is shortcut the instance_getattr functions in a new function, to do just that what is necessary (and no more.) _Py_instance_getmethod() returns NULL for anything that isn't a method, too, letting the slow case handle it. When it does find a would-be method, it returns the unwrapped function. The call_attr function basically does a PyInstance_Check() and a _Py_instance_getmethod(), and calls the returned function. The problem I have with newstyle classes is where to shortcut what. I understand now how to detect a would-be method, but I'm not sure how to get unwrapped attributes. As far as I understand, types can provide their own getattr function with complete control over descriptors, so there isn't much to shortcut. Unless I should make the shortcut depend on the actual value of tp_getattro, as in shortcut only if it actually is PyObject_GenericGetAttr ? In that case, I'm somewhat sceptical about the speed benefit's cost in maintenance, as it would require a near copy of PyObject_GenericGetAttr (which is already a near-copy of a few other functions :) It's also very hard to control any nested getattrs (possible, I think, because the process goes over all bases' dicts and the instance dict.) Or can we reduce the number of steps PyObject_GenericGetAttr goes through if we know we are just looking for a method ? I don't believe so, but I'm not sure. (Looking at PyObject_GenericGetAttr with that in mind, I wonder if there isn't a possible crash there. In the first MRO lookup, looking for descr's, if a non-data-descr is found, it is kept around but not INCREF'd until later, after the instance-dict is searched. Am I wrong in believing the PyDict_GetItem of the instance dict can call Python code ? There isn't even as much as an assert(PyDict_Check(dict)) there.) -- Thomas Wouters <thomas@xs4all.net> Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

On Thu, Apr 17, 2003 at 10:59:56PM +0200, Thomas Wouters wrote:
Well, I went ahead and did that, and uploaded the new patch to SF. The result is somewhat annoying, but explainable: The patch is now 3% _slower_ than an unmodified Python, whereas the patch without support for newstyle classes was a good 5% _faster_ than unmodified. This is both according to PyBench (which doesn't use newstyle classes) and according to 'time timeit.py pass' (which does use newstyle classes.) Timing just 'x.foo()' where 'x' is a newstyle class instance is about 20% faster, against 25-30% for oldstyle classes. The overall slowdown is caused by the fact that the patch only treats PyFunctions (functions written in Python) specially, and not PyMethodDescrs (PyCFunctions wrapped in PyMethodDefs wrapped in a descriptor.) This is because it would still need to instantiate a PyCFunctionObject (a PyObject wrapper for a PyCFunction, which is just a C function-pointer) OR it would need to do all interpretation of METH_* arguments and a bunch of argument-preparing itself. Another possible cause for the slowdown (but almost certainly not as substantial as the type-with-C-function one) is calling an almost-method on a newstyle class; a callable object that is an attribute of a type (or instance of the type) but is not a PyFunction or PyMethodDescr. The way the current mechanisms works, it would have to traverse the MRO and (possibly) check the instance dict twice; first to determine that it's not a PyFunction in _PyObject_Generic_getmethod() and then again in the regular run though PyObject_GenericGetAttr(). Examples of this case would be staticmethods, classmethods, and other callable objects as attributes. I do not believe this is a substantial party though. The slowdown can be fixed in two ways: handing PyMethodDescrs as well, in _PyObject_Generic_getmethod(), or removing the double lookups. Hm, wait, handling PyMethodDescrs may not be as tricky as I thought... hrm... I'll look at it tomorrow, it's time for bed. -- Thomas Wouters <thomas@xs4all.net> Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

On Fri, Apr 18, 2003 at 02:06:50AM +0200, Thomas Wouters wrote:
Hm, wait, handling PyMethodDescrs may not be as tricky as I thought... hrm... I'll look at it tomorrow, it's time for bed.
I did a quick hack to the same effect, and it still came out a 1% loss (so about 6% against the no-newstyle patch) in PyBench and a few timeit tests. Sigh. I guess the non-method overhead is just too large, or there are more almost-methods than I figured. I'll start work on a more lookup-saving _PyObject_Generic_getmethod tomorrow or this weekend (and will probably do _Py_instance_getmethod that way too, while I'm at it.) -- Thomas Wouters <thomas@xs4all.net> Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

On Fri, Apr 18, 2003 at 02:34:31AM +0200, Thomas Wouters wrote:
On Fri, Apr 18, 2003 at 02:06:50AM +0200, Thomas Wouters wrote:
Hm, wait, handling PyMethodDescrs may not be as tricky as I thought... hrm... I'll look at it tomorrow, it's time for bed.
Okay, for those who care about this but aren't on Patches, I just uploaded a new CALL_ATTR patch, version 4. It's actually two separate versions (3 and 4): maintainable, and fast. See the SF patch comment for more details :) However, I spent most of tonight trying to clock the patch, only to come to the conclusion that benchmarks suck. Which I already knew :) PyBench did a reasonable job pointing me towards slowness, but the main slowdowns I see with PyBench I cannot reproduce with timeit.py. I think I stopped trusting PyBench when it reported the patch was 2% slower, but did so 5% faster -- consistently. So, if anyone has any *real* programs they can test the patch with, I would be much obliged. Otherwise we'll have to check it in claiming it gives a 20% performance boost on ... methods of newstyle classes ,,, and 30% on ... methods of old-style classes. :) The patch is here: http://www.python.org/sf/709744 Where-...-reads-"empty-no-argument"-and-,,,-reads-"that-use- -PyObject_GenericGetAttr"-in-very-very-small-letters'ly y'rs, -- Thomas Wouters <thomas@xs4all.net> Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

So, if anyone has any *real* programs they can test the patch
I tried it on some of my apps which moderately exercise both new and old style classes. None of the apps improved and one a 1% worse. Both pybench and pystone were worse by 1%. Also, line 767 in classobject.c has an unreferenced variable, f. Raymond Hettinger

It can, if there's a key whose type has a custom __eq__ or __cmp__. So indeed, if this custom __eq__ is evil enough to delete the corresponding key from the class dict, it could cause descr to point to freed memory. I won't try to construct a case, but it's not impossible. :-( Fixing this would make the code even hairier though... :-(
There isn't even as much as an assert(PyDict_Check(dict)) there.)
All over the place it is assumed and ensured that a types tp_dict and an instance's __dict__ are always real dicts. The only way this could be violated would be by C code defining a type that violates this. --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum <guido@python.org> writes:
Indeed, there are several examples of this sort of thing already in Lib/test/test_mutants.py. Cheers, M. -- If comp.lang.lisp *is* what vendors are relying on to make or break Lisp sales, that's more likely the problem than is the effect of any one of us on such a flimsy marketing strategy... -- Kent M Pitman, comp.lang.lisp

On Friday, Apr 18, 2003, at 01:22 Europe/London, Guido van Rossum wrote: Note that this is quite some time ago :-) I've been reading through some messages that have been ticked for rather too long in my python-dev mail folder...
Indeed not! Here's mine: import gc class Evil(object): def __hash__(self): return hash('attr') def __eq__(self, other): global foo foo = gc.get_referrers(C.attr) del C.attr return 0 class Descr(object): def __get__(self, ob, type=None): return 1 class C(object): attr = Descr() c = C() c.__dict__[Evil()] = 0 c.attr `foo` It's always a bit hard to be sure that stuff like this will actually crash the interpreter, which is what the gimmicks with gc.get_referrers & `foo` are in aid of. This crashes every Python I have lying around on my iBook -- the 2.2[.0] in /usr/bin, heads of release22-maint, release23-maint, the CVS trunk, a random build of 2.2.3c1, Jack's 2.3 build. Etc ;-) I'll check on linux when I get into work.
Fixing this would make the code even hairier though... :-(
Pff, it's not so bad, just a little refcount code jiggling. I'll make a real patch including the above test case soon (once I'm reasonably confident that I'm not leaking references). And then I find out running test_descr in a loop leaks 166(!) references *anyway* (run the attached blah.py in a debug build, and some print test or other that I don't understand fails when run like this, so I fiddled it) Should I have been expecting this, or is it time for some ref-leak hunting? Actually, the process seems to grow -- slowly -- when you loop, so it's probably a real problem. I gather it's best to ask about this sort of thing when Tim is off sick from work :-) Cheers, mwh

[Michael Hudson]
Only if you want help <wink>. Since Zope has worse leak problems than Python, Zope testing has more tools to dig into it: http://cvs.zope.org/Zope3/test.py That's the Zope3 unittest test driver. The TrackRefs class is a handy little beast, which uses a debug build's sys.getobjects() to keep track of how many objects of each *type* exist across test runs, and how many refcounts total they account for. After the first few iterations, TrackRefs doesn't itself distort the results it prints. It will tell you the types of the objects that are leaking, and also whether objects are actually leaking, or that merely refcounts to existing objects are leaking (e.g., a common kind of "leak" is to forget to decref Py_None, but memory doesn't actually grow then). Do use 2.3 if you try this. The "list of all objects" before 2.3 turned out to be missing many objects that tend to leak (None, booleans, any number of type objects, essentially all statically allocated objects), and far fewer are missing in 2.3. It's typically easy to find out which types of objects are leaking this way. Then it's typical to add more code to TraceRefs.update() to get more details. In Zope use, it's surprising how often that still left us scratching our heads! But Zope has lots of extension types coded in C, massive interconnections among them, and none of them originally participated in cyclic gc. Python leaks are usually easier to plug (for example, in Python, they're almost always in new code).

"Tim Peters" <tim.one@comcast.net> writes:
Actually, I found the worst culprit this morning. Some dunderhead had got the decrefs wrong on error return from type_set_bases... I'll check my fix in momentarily.
Cool! I'd blundered my way to something like an ad hoc version of this, next time I'll use something that someone's actually thought about :-)
Do use 2.3 if you try this.
Well, yeah. Turned out the worst leaks weren't in 2.2 code anyway.
Bingo! I did all these tests incorporating my safety fixes to PyObject_GenericGetAttr so I think they're probably solid, too. Cheers, mwh -- The PROPER way to handle HTML postings is to cancel the article, then hire a hitman to kill the poster, his wife and kids, and fuck his dog and smash his computer into little bits. Anything more is just extremism. -- Paul Tomblin, asr

On Thursday, Aug 7, 2003, at 14:08 Europe/London, Michael Hudson wrote:
[I'll observe that trying to wrap running a test in a refcount neutral way is entertaining...]
OK, so I've now blundered my way to the attached, which when run produces this: ints seems to leak 1 references... <type 'long'> 1 4 multi seems to leak 9 references... <type 'tuple'> 1 5 <type 'classobj'> 1 4 <type 'dict'> 1 4 <type 'int'> 1 3 <type 'str'> 0 4 <type 'NoneType'> 0 1 slots seems to leak 10 references... <type 'tuple'> 5 20 <type 'type'> 0 5 errors seems to leak 9 references... <type 'tuple'> 4 16 <type 'type'> 0 5 subtype_resurrection seems to leak 2 references... (the names are all names of functions in test_descr). I might have a look at these over the next few days (and also might ponder other bits of the test suite). Cheers, mwh

Michael Hudson <mwh@python.net> writes:
http://www.python.org/sf/784825 Cheers, mwh -- ... with these conditions cam the realisation that ... nothing turned a perfectly normal healthy individual into a great political or military leader better than irreversible brain damage. -- The Hitch-Hikers Guide to the Galaxy, Episode 11

Thomas Wouters <thomas@xs4all.net>:
The problem I have with newstyle classes is where to shortcut what.
It sounds to me like descriptor objects will need to have a __callattr__ slot added. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+

While we're on the topic -- Guid, how would you feel about the idea of giving built-in function objects the same instance- binding behaviour as interpreted functions? This would help Pyrex considerably, because currently I have to resort to a kludge to make Pyrex-defined functions work as methods. It mostly works, but it has some side effects, such as breaking the most common idiomatic usage of staticmethod() and classmethod(). If built-in functions were more like interpreted functions in this regard, all these problems would go away. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+

There are two ways to "bind" a built-in function to an object. One would be to do what happens for Python functions, which is in effect a currying: f.__get__(obj) yields a function g that when called as g(arg1, ...) calls f(obj, arg1, ...). (In fact, I've recently checked in a change that makes instancemethod a general currying function on the first argument. :-) But the other interpretation, which might be more appropriate for C functions, is that the bound instance is passed to the first argument at the *C* level, usually called self: PyObject * my_c_function(PyObject *self, PyObject *args) { ... } Which one would you like? I think we could do each rather easily (perhaps the first more easily because the type needed to represent the bound method already exist; for the second I think we'd have to introduce a new helper object type). --Guido van Rossum (home page: http://www.python.org/~guido/)

That's the one I'm talking about. I forgot to explain that the problem occurs when I'm creating a *Python* class object and populating it with functions that are supposed to be methods. Currently I have to manually wrap each function in an unbound method object before putting it in the class's __dict__. If that happened automatically on access, I would be able to create Python classes that behave more like the real thing. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+

OK, are you up for submitting a patch? --Guido van Rossum (home page: http://www.python.org/~guido/)

On woensdag, apr 16, 2003, at 17:52 Europe/Amsterdam, Guido van Rossum wrote:
The getargs mods got checked in just this morning, even though I explicitly and rather strongly asked that if these mods be made they be checked in *long* before a release was due:-( This means that all the Mac modules are now 100% dead. The same is probably true for PyObjC. And PyObjC has the added problem that it needs to be compatible with both 2.3b1 and 2.2 (notice that that is "2.2", not "2.2.X": PyObjC has to work with /usr/bin/python that Apple ships, which is 2.2 at the moment). I assume there are format codes that will convert 16 bit and 32 bit integer quantities without any checks on both 2.2 and 2.3, but I haven't investigated yet. And there may be problems with other wrapper packages (win32, wxPython, PyOpenGL) too. I will start fixing things, but there are only 4 real days left before April 25, given easter, so I would strongly urge for postponing the release date for another two weeks or so. -- - Jack Jansen <Jack.Jansen@oratrix.com> http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman -

Sorry, I forgot. Did you make a note of that on the SF patch?
Maybe we should retract the changes to existing format codes that make them more restrictive? That should revive any code that's curerntly dead, right?
That would endanger then entire release schedule to the point of pushing the 2.3 release past the OSCON conference (July 7-11). --Guido van Rossum (home page: http://www.python.org/~guido/)

On donderdag, apr 17, 2003, at 22:17 Europe/Amsterdam, Guido van Rossum wrote:
Yes, I'm pretty sure I did. Thomas also seems to refer to it...
That would be much better. if "l" (lower case ell) would continue to accept anything I wouldn't have to change anything. Of course I've been busy all night fixing code, but apart from a couple of hand-crafted modules I haven't checked anything in yet. I will check it in on a branch later tonight, and then I'll either forget about the branch or merge it, depending on the resolution of this. -- - Jack Jansen <Jack.Jansen@oratrix.com> http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman -

Jack Jansen <Jack.Jansen@oratrix.com> writes:
He did, and I also mentioned it yesterday. OTOH, I had sitting a first version of the patch on SF for a rather long time (shortly after the alpha2 release), asking for feedback, but didn't get any.
Guido has also suggested to keep another code without changes, I cannot remember which one it was, but there is a comment on SF. I have the impression that the new test_getargs2.py test makes it easy to change the behaviour and verify it to anything we want. In case it is too much trouble, why not backout all this again (although someone else would have to do it, I'm basically offline until tuesday), and check in after the b1 release. Sorry, Thomas

That was my fault -- I was too busy. :-(
That was 'h'.
I'll back out the change to 'h', which is the only incompatible change I can see (unless you consider accepting *more* than before an error). Thomas made no changes to 'l', so I'm not sure what that is about -- maybe the problem is with unsigned hex constants? --Guido van Rossum (home page: http://www.python.org/~guido/)

On vrijdag, apr 18, 2003, at 01:14 Europe/Amsterdam, Guido van Rossum wrote:
Right, 'h' turns out to be the problem. I changed a lot of 'l's to 'k's, but it seems this one is the real killer. -- - Jack Jansen <Jack.Jansen@oratrix.com> http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman -

Right, 'h' turns out to be the problem. I changed a lot of 'l's to 'k's, but it seems this one is the real killer.
So now that I rolled back 'h', is there any reason not to keep the rest of these changes? --Guido van Rossum (home page: http://www.python.org/~guido/)

On vrijdag, apr 18, 2003, at 15:25 Europe/Amsterdam, Guido van Rossum wrote:
No, everything is fine as it is now. I'm happy again! -- - Jack Jansen <Jack.Jansen@oratrix.com> http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman -

On vrijdag, apr 18, 2003, at 01:14 Europe/Amsterdam, Guido van Rossum wrote:
Okay, great!! Is this a temporary measure, i.e. is the semantic change to 'h' going to come back after 2.3 is out? -- - Jack Jansen <Jack.Jansen@oratrix.com> http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman -

I don't see why -- it always was a signed short, let it stay that way. --Guido van Rossum (home page: http://www.python.org/~guido/)

On Wed, 16 Apr 2003, Guido van Rossum wrote:
The UnixWare build is way dead right now. (today's CVS) cc -c -K pentium,host,inline,loop_unroll,alloca -DNDEBUG -O -I. -I/opt/src/utils/python/python/dist/src/Include -DPy_BUILD_CORE -o Modules/python.o /opt/src/utils/python/python/dist/src/Modules/python.c UX:acomp: ERROR: "/usr/include/sys/select.h", line 45: identifier redeclared: fd_set UX:acomp: ERROR: "/usr/include/sys/select.h", line 72: identifier redeclared: select gmake: *** [Modules/python.o] Error 1
-- Tim Rice Multitalents (707) 887-1469 tim@multitalents.net

On 22 Apr 2003 at 7:28, Martin v. Löwis wrote:
I'm sorry I'm not in a position to fix it, but I do have an un-opened Unixware Advanced Server 2.01 box set (docs and media) if anyone wants them. Personally, I think Unixware is dead. Novell dropped it ages ago. -- Brad Clements, bkc@murkworks.com (315)268-1000 http://www.murkworks.com (315)268-9812 Fax http://www.wecanstopspam.org/ AOL-IM: BKClements

That doesn't look like a *new* problem to me; if sys/select.h is being included twice, that probably was so for a long time. You may be the only person with access to this platform. Can you find the problem? Was this present in 2.3a2? --Guido van Rossum (home page: http://www.python.org/~guido/)
participants (14)
-
Brad Clements
-
Brett Cannon
-
David Abrahams
-
Greg Ewing
-
Guido van Rossum
-
Jack Jansen
-
M.-A. Lemburg
-
martin@v.loewis.de
-
Michael Hudson
-
Raymond Hettinger
-
Thomas Heller
-
Thomas Wouters
-
Tim Peters
-
Tim Rice