[Guido van Rossum]

I'd like to do a 2.3b1 release someday. Maybe at the end of next week, that would be Friday April 25. If anyone has something that needs to be done before this release go out, please let me know!

Just to make sure since this is the first release that I have CVS commit, we can apply patches to fix bugs without having to worry about it being beta, right? How about new tests? -Brett

Guido van Rossum writes:

I'd like to do a 2.3b1 release someday. Maybe at the end of next week, that would be Friday April 25. If anyone has something that needs to be done before this release go out, please let me know!

I would still like to work on http://www.python.org/sf/595026 support for masks in getargs.c. Jack requested that this change should be implement shortly after the release of 2.3a2, but this is too late now as it seems ;-) What to do? Implement it now and commit it after 2.3b1 is released, or delay this until 2.3 final is released. I have to admit that I'm sure I can implement it for 32-bit Windows, but it would have to be tested (and maybe completed) on other, especially 64-bit platforms as well. And it introduces incompatibilities. BTW: Since you want to release a beta version, what's the state of the FutureWarning about hex/oct constants: will this stay the way it is? Thomas

...

I would still like to work on http://www.python.org/sf/595026 support for masks in getargs.c.

Great!

...

Jack requested that this change should be implement shortly after the release of 2.3a2, but this is too late now as it seems ;-)

What to do?

Do it ASAP.

Ok, working on it.

...

Implement it now and commit it after 2.3b1 is released, or delay this until 2.3 final is released. I have to admit that I'm sure I can implement it for 32-bit Windows, but it would have to be tested (and maybe completed) on other, especially 64-bit platforms as well.

If you can get something rough into 2.3b1, it can be improved while 2.3b2 is cooking.

...

And it introduces incompatibilities.

What kind? I thought it would be a new format code?

Two new format codes ('k' and 'K'), and changes to existing format codes - per your request: | How about the following counterproposal. This also changes some of the | other format codes to be a little more regular. | | Code C type Range check | | b unsigned char 0..UCHAR_MAX | B unsigned char none ** | h unsigned short 0..USHRT_MAX | H unsigned short none ** | i int INT_MIN..INT_MAX | I * unsigned int 0..UINT_MAX | l long LONG_MIN..LONG_MAX | k * unsigned long none | L long long LLONG_MIN..LLONG_MAX | K * unsigned long long none | | Notes: | | * New format codes. | | ** Changed from previous "range-and-a-half" to "none"; the | range-and-a-half checking wasn't particularly useful.

...

BTW: Since you want to release a beta version, what's the state of the FutureWarning about hex/oct constants: will this stay the way it is?

Probably, unless you hve a better idea. :-(

I haven't used warnings very much, but is there a possibility to disable them per module? You get a lot of them if you 'import win32con' for example. Thomas

Guido van Rossum writes:

...

...

...

And it introduces incompatibilities.

What kind? I thought it would be a new format code?

Two new format codes ('k' and 'K'), and changes to existing format codes - per your request:

| How about the following counterproposal. This also changes some of the | other format codes to be a little more regular. | | Code C type Range check | | b unsigned char 0..UCHAR_MAX | B unsigned char none ** | h unsigned short 0..USHRT_MAX | H unsigned short none ** | i int INT_MIN..INT_MAX | I * unsigned int 0..UINT_MAX | l long LONG_MIN..LONG_MAX | k * unsigned long none | L long long LLONG_MIN..LLONG_MAX | K * unsigned long long none | | Notes: | | * New format codes. | | ** Changed from previous "range-and-a-half" to "none"; the | range-and-a-half checking wasn't particularly useful.

Oh of course. None to worry about IMO.

Well, implementing (and testing) these as the main part of the work, and I'm at least halfway through. Thomas

I'd like to do a 2.3b1 release someday. Maybe at the end of next week, that would be Friday April 25. If anyone has something that needs to be done before this release go out, please let me know!

I have a couple of small patches and bugs to review. Should be no problem getting these in this weekend. Am working on a more cache friendly dict lookup strategy. If it is not ready for prime time in the next few days, it will have to wait for Py2.4. A couple of bytecode optimizations may also have to wait for Py2.4. For some reason, Basicblock(nop, jump_if_true) is not always directly substitutable for Basicblock(unary_not, jump_if_false). I suspect the three-way return value for PyObject_IsTrue() but it could be something else. Raymond Hettinger

On Thu, Apr 17, 2003 at 05:27:22PM +0200, Thomas Wouters wrote:

So far, I've been alternating from 'Wow! to 'Au!', and I'll send another email after this one for clarification of a few issues :)

Nevermind that. A "D'oh" slipped into the stream, and I think I get it now. At least the stuff that wasn't working is working now. I wouldn't mind if someone pointed me to a xxtype.c (newstyle class in C) like we have xxobject and xxsubclass though... So far, it's been so simple I fear I'm missing something. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

Guido van Rossum wrote:

Yes, please. Here's a quick explanation of descriptors:

A descriptor is something that lives in a class' __dict__, and primarily affects instance attribute lookup. A descriptor has a __get__ method (in C this is the tp_descrget function in its type object) and the instance attribute lookup calls this to "bind" the descriptor to a specific instance. This is what turns a function into a bound method object in Python 2.2. In earlier versions, functions were special-cased by the instance getattr code; the special case has been subsumed by looking for a __get__ method. Yes, this means that a plain Python function object is a descriptor! Because the instance getattr code returns whatever __get__ returns as the result of the attribute lookup, this is also how properties work: they have a __get__ method that calls the property-get" function.

A descriptor's __get__ method is also called for class attribute lookup (with the instance argument set to NULL or None). And a descsriptor's __set__ method is called for instance attribute assignment; but not for class attribute assignment.

Hope this helps!

Could you put such short overviews somewhere on the Python Wiki ? They sure help in understanding what is going on behind the scenes without having to grep through tons of source code :-) Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Software directly from the Source (#1, Apr 17 2003)

...

...

Python/Zope Products & Consulting ... http://www.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/

EuroPython 2003, Charleroi, Belgium: 68 days left

On woensdag, apr 16, 2003, at 17:52 Europe/Amsterdam, Guido van Rossum wrote:

I'd like to do a 2.3b1 release someday. Maybe at the end of next week, that would be Friday April 25. If anyone has something that needs to be done before this release go out, please let me know!

The getargs mods got checked in just this morning, even though I explicitly and rather strongly asked that if these mods be made they be checked in *long* before a release was due:-( This means that all the Mac modules are now 100% dead. The same is probably true for PyObjC. And PyObjC has the added problem that it needs to be compatible with both 2.3b1 and 2.2 (notice that that is "2.2", not "2.2.X": PyObjC has to work with /usr/bin/python that Apple ships, which is 2.2 at the moment). I assume there are format codes that will convert 16 bit and 32 bit integer quantities without any checks on both 2.2 and 2.3, but I haven't investigated yet. And there may be problems with other wrapper packages (win32, wxPython, PyOpenGL) too. I will start fixing things, but there are only 4 real days left before April 25, given easter, so I would strongly urge for postponing the release date for another two weeks or so. -- - Jack Jansen http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman -

On woensdag, apr 16, 2003, at 20:11 Europe/Amsterdam, Thomas Heller wrote:

| How about the following counterproposal. This also changes some of the | other format codes to be a little more regular. | | Code C type Range check | | b unsigned char 0..UCHAR_MAX | B unsigned char none ** | h unsigned short 0..USHRT_MAX | H unsigned short none ** | i int INT_MIN..INT_MAX | I * unsigned int 0..UINT_MAX | l long LONG_MIN..LONG_MAX | k * unsigned long none | L long long LLONG_MIN..LLONG_MAX | K * unsigned long long none | | Notes: | | * New format codes. | | ** Changed from previous "range-and-a-half" to "none"; the | range-and-a-half checking wasn't particularly useful.

Do I understand correctly that there is no format code that works on both 2.2 and 2.3 that converts 32 bit quantities without complaining (B and H will work for 8 and 16 bit quantities)? That may be a serious problem for PyObjC.... -- - Jack Jansen http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman -

On donderdag, apr 17, 2003, at 22:17 Europe/Amsterdam, Guido van Rossum wrote:

...

...

I'd like to do a 2.3b1 release someday. Maybe at the end of next week, that would be Friday April 25. If anyone has something that needs to be done before this release go out, please let me know!

The getargs mods got checked in just this morning, even though I explicitly and rather strongly asked that if these mods be made they be checked in *long* before a release was due:-(

Sorry, I forgot. Did you make a note of that on the SF patch?

Yes, I'm pretty sure I did. Thomas also seems to refer to it...

...

This means that all the Mac modules are now 100% dead. The same is probably true for PyObjC. And PyObjC has the added problem that it needs to be compatible with both 2.3b1 and 2.2 (notice that that is "2.2", not "2.2.X": PyObjC has to work with /usr/bin/python that Apple ships, which is 2.2 at the moment). I assume there are format codes that will convert 16 bit and 32 bit integer quantities without any checks on both 2.2 and 2.3, but I haven't investigated yet.

Maybe we should retract the changes to existing format codes that make them more restrictive? That should revive any code that's curerntly dead, right?

That would be much better. if "l" (lower case ell) would continue to accept anything I wouldn't have to change anything. Of course I've been busy all night fixing code, but apart from a couple of hand-crafted modules I haven't checked anything in yet. I will check it in on a branch later tonight, and then I'll either forget about the branch or merge it, depending on the resolution of this. -- - Jack Jansen http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman -

Jack Jansen writes:

...

On donderdag, apr 17, 2003, at 22:17 Europe/Amsterdam, Guido van Rossum wrote:

...

...

...

I'd like to do a 2.3b1 release someday. Maybe at the end of next week, that would be Friday April 25. If anyone has something that needs to be done before this release go out, please let me know!

The getargs mods got checked in just this morning, even though I explicitly and rather strongly asked that if these mods be made they be checked in *long* before a release was due:-(

Sorry, I forgot. Did you make a note of that on the SF patch?

Yes, I'm pretty sure I did. Thomas also seems to refer to it...

He did, and I also mentioned it yesterday. OTOH, I had sitting a first version of the patch on SF for a rather long time (shortly after the alpha2 release), asking for feedback, but didn't get any.

That was my fault -- I was too busy. :-(

...

...

...

This means that all the Mac modules are now 100% dead. The same is probably true for PyObjC. And PyObjC has the added problem that it needs to be compatible with both 2.3b1 and 2.2 (notice that that is "2.2", not "2.2.X": PyObjC has to work with /usr/bin/python that Apple ships, which is 2.2 at the moment). I assume there are format codes that will convert 16 bit and 32 bit integer quantities without any checks on both 2.2 and 2.3, but I haven't investigated yet.

Maybe we should retract the changes to existing format codes that make them more restrictive? That should revive any code that's curerntly dead, right?

That would be much better. if "l" (lower case ell) would continue to accept anything I wouldn't have to change anything.

Guido has also suggested to keep another code without changes, I cannot remember which one it was, but there is a comment on SF.

That was 'h'.

I have the impression that the new test_getargs2.py test makes it easy to change the behaviour and verify it to anything we want.

In case it is too much trouble, why not backout all this again (although someone else would have to do it, I'm basically offline until tuesday), and check in after the b1 release.

I'll back out the change to 'h', which is the only incompatible change I can see (unless you consider accepting *more* than before an error). Thomas made no changes to 'l', so I'm not sure what that is about -- maybe the problem is with unsigned hex constants? --Guido van Rossum (home page: http://www.python.org/~guido/)

(Looking at PyObject_GenericGetAttr with that in mind, I wonder if there isn't a possible crash there. In the first MRO lookup, looking for descr's, if a non-data-descr is found, it is kept around but not INCREF'd until later, after the instance-dict is searched. Am I wrong in believing the PyDict_GetItem of the instance dict can call Python code ?

It can, if there's a key whose type has a custom __eq__ or __cmp__. So indeed, if this custom __eq__ is evil enough to delete the corresponding key from the class dict, it could cause descr to point to freed memory. I won't try to construct a case, but it's not impossible. :-( Fixing this would make the code even hairier though... :-(

There isn't even as much as an assert(PyDict_Check(dict)) there.)

All over the place it is assumed and ensured that a types tp_dict and an instance's __dict__ are always real dicts. The only way this could be violated would be by C code defining a type that violates this. --Guido van Rossum (home page: http://www.python.org/~guido/)

In earlier versions, functions were special-cased by the instance getattr code; the special case has been subsumed by looking for a __get__ method. Yes, this means that a plain Python function object is a descriptor!

While we're on the topic -- Guid, how would you feel about the idea of giving built-in function objects the same instance- binding behaviour as interpreted functions? This would help Pyrex considerably, because currently I have to resort to a kludge to make Pyrex-defined functions work as methods. It mostly works, but it has some side effects, such as breaking the most common idiomatic usage of staticmethod() and classmethod(). If built-in functions were more like interpreted functions in this regard, all these problems would go away. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+

While we're on the topic -- Guid, how would you feel about the idea of giving built-in function objects the same instance- binding behaviour as interpreted functions?

This would help Pyrex considerably, because currently I have to resort to a kludge to make Pyrex-defined functions work as methods. It mostly works, but it has some side effects, such as breaking the most common idiomatic usage of staticmethod() and classmethod().

If built-in functions were more like interpreted functions in this regard, all these problems would go away.

There are two ways to "bind" a built-in function to an object. One would be to do what happens for Python functions, which is in effect a currying: f.__get__(obj) yields a function g that when called as g(arg1, ...) calls f(obj, arg1, ...). (In fact, I've recently checked in a change that makes instancemethod a general currying function on the first argument. :-) But the other interpretation, which might be more appropriate for C functions, is that the bound instance is passed to the first argument at the *C* level, usually called self: PyObject * my_c_function(PyObject *self, PyObject *args) { ... } Which one would you like? I think we could do each rather easily (perhaps the first more easily because the type needed to represent the bound method already exist; for the second I think we'd have to introduce a new helper object type). --Guido van Rossum (home page: http://www.python.org/~guido/)

On vrijdag, apr 18, 2003, at 01:14 Europe/Amsterdam, Guido van Rossum wrote:

...

...

...

Maybe we should retract the changes to existing format codes that make them more restrictive? That should revive any code that's curerntly dead, right?

That would be much better. if "l" (lower case ell) would continue to accept anything I wouldn't have to change anything.

Guido has also suggested to keep another code without changes, I cannot remember which one it was, but there is a comment on SF.

That was 'h'.

Right, 'h' turns out to be the problem. I changed a lot of 'l's to 'k's, but it seems this one is the real killer. -- - Jack Jansen http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman -

Guido van Rossum writes:

...

Could you put such short overviews somewhere on the Python Wiki ?

I don't have the time for that. When I want to publish stuff like this somewhere, I need to spend time to make it all correct, complete etc.

Besides which, it's already in the docs. Correct, complete, and all that ;-) http://www.python.org/dev/doc/devel/ref/descriptors.html -- Dave Abrahams Boost Consulting www.boost-consulting.com

Right, 'h' turns out to be the problem. I changed a lot of 'l's to 'k's, but it seems this one is the real killer.

So now that I rolled back 'h', is there any reason not to keep the rest of these changes? --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum writes:

...

(Looking at PyObject_GenericGetAttr with that in mind, I wonder if there isn't a possible crash there. In the first MRO lookup, looking for descr's, if a non-data-descr is found, it is kept around but not INCREF'd until later, after the instance-dict is searched. Am I wrong in believing the PyDict_GetItem of the instance dict can call Python code ?

It can, if there's a key whose type has a custom __eq__ or __cmp__. So indeed, if this custom __eq__ is evil enough to delete the corresponding key from the class dict, it could cause descr to point to freed memory. I won't try to construct a case, but it's not impossible. :-(

Indeed, there are several examples of this sort of thing already in Lib/test/test_mutants.py. Cheers, M. -- If comp.lang.lisp *is* what vendors are relying on to make or break Lisp sales, that's more likely the problem than is the effect of any one of us on such a flimsy marketing strategy... -- Kent M Pitman, comp.lang.lisp

On Fri, Apr 18, 2003 at 02:34:31AM +0200, Thomas Wouters wrote:

On Fri, Apr 18, 2003 at 02:06:50AM +0200, Thomas Wouters wrote:

...

Hm, wait, handling PyMethodDescrs may not be as tricky as I thought... hrm... I'll look at it tomorrow, it's time for bed.

I did a quick hack to the same effect, and it still came out a 1% loss (so about 6% against the no-newstyle patch) in PyBench and a few timeit tests. Sigh. I guess the non-method overhead is just too large, or there are more almost-methods than I figured. I'll start work on a more lookup-saving _PyObject_Generic_getmethod tomorrow or this weekend (and will probably do _Py_instance_getmethod that way too, while I'm at it.)

Okay, for those who care about this but aren't on Patches, I just uploaded a new CALL_ATTR patch, version 4. It's actually two separate versions (3 and 4): maintainable, and fast. See the SF patch comment for more details :) However, I spent most of tonight trying to clock the patch, only to come to the conclusion that benchmarks suck. Which I already knew :) PyBench did a reasonable job pointing me towards slowness, but the main slowdowns I see with PyBench I cannot reproduce with timeit.py. I think I stopped trusting PyBench when it reported the patch was 2% slower, but did so 5% faster -- consistently. So, if anyone has any *real* programs they can test the patch with, I would be much obliged. Otherwise we'll have to check it in claiming it gives a 20% performance boost on ... methods of newstyle classes ,,, and 30% on ... methods of old-style classes. :) The patch is here: http://www.python.org/sf/709744 Where-...-reads-"empty-no-argument"-and-,,,-reads-"that-use- -PyObject_GenericGetAttr"-in-very-very-small-letters'ly y'rs, -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

On Wed, 16 Apr 2003, Guido van Rossum wrote:

I'd like to do a 2.3b1 release someday. Maybe at the end of next week, that would be Friday April 25. If anyone has something that needs to be done before this release go out, please let me know!

The UnixWare build is way dead right now. (today's CVS) cc -c -K pentium,host,inline,loop_unroll,alloca -DNDEBUG -O -I. -I/opt/src/utils/python/python/dist/src/Include -DPy_BUILD_CORE -o Modules/python.o /opt/src/utils/python/python/dist/src/Modules/python.c UX:acomp: ERROR: "/usr/include/sys/select.h", line 45: identifier redeclared: fd_set UX:acomp: ERROR: "/usr/include/sys/select.h", line 72: identifier redeclared: select gmake: *** [Modules/python.o] Error 1

Assigning a SF bug or patch to me and setting the priority to 7 is a good way to get my attention.

--Guido van Rossum (home page: http://www.python.org/~guido/)

_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev

-- Tim Rice Multitalents (707) 887-1469 tim@multitalents.net

The UnixWare build is way dead right now. (today's CVS)

cc -c -K pentium,host,inline,loop_unroll,alloca -DNDEBUG -O -I. -I/opt/src/utils/python/python/dist/src/Include -DPy_BUILD_CORE -o Modules/python.o /opt/src/utils/python/python/dist/src/Modules/python.c UX:acomp: ERROR: "/usr/include/sys/select.h", line 45: identifier redeclared: fd_set UX:acomp: ERROR: "/usr/include/sys/select.h", line 72: identifier redeclared: select gmake: *** [Modules/python.o] Error 1

That doesn't look like a *new* problem to me; if sys/select.h is being included twice, that probably was so for a long time. You may be the only person with access to this platform. Can you find the problem? Was this present in 2.3a2? --Guido van Rossum (home page: http://www.python.org/~guido/)

On Tue, 22 Apr 2003, Brad Clements wrote:

On 22 Apr 2003 at 7:28, Martin v. L�wis wrote:

...

Tim Rice writes:

...

The UnixWare build is way dead right now. (today's CVS)

Any volunteers to fix it?

Regards, Martin

I'm sorry I'm not in a position to fix it, but I do have an un-opened Unixware Advanced Server 2.01 box set (docs and media) if anyone wants them.

Personally, I think Unixware is dead. Novell dropped it ages ago.

"Dropped it" isn't quite correct. They sold it to SCO. -- Tim Rice Multitalents (707) 887-1469 tim@multitalents.net

On Friday, Apr 18, 2003, at 01:22 Europe/London, Guido van Rossum wrote: Note that this is quite some time ago :-) I've been reading through some messages that have been ticked for rather too long in my python-dev mail folder...

...

(Looking at PyObject_GenericGetAttr with that in mind, I wonder if there isn't a possible crash there. In the first MRO lookup, looking for descr's, if a non-data-descr is found, it is kept around but not INCREF'd until later, after the instance-dict is searched. Am I wrong in believing the PyDict_GetItem of the instance dict can call Python code ?

It can, if there's a key whose type has a custom __eq__ or __cmp__. So indeed, if this custom __eq__ is evil enough to delete the corresponding key from the class dict, it could cause descr to point to freed memory. I won't try to construct a case, but it's not impossible. :-(

Indeed not! Here's mine: import gc class Evil(object): def __hash__(self): return hash('attr') def __eq__(self, other): global foo foo = gc.get_referrers(C.attr) del C.attr return 0 class Descr(object): def __get__(self, ob, type=None): return 1 class C(object): attr = Descr() c = C() c.__dict__[Evil()] = 0 c.attr `foo` It's always a bit hard to be sure that stuff like this will actually crash the interpreter, which is what the gimmicks with gc.get_referrers & `foo` are in aid of. This crashes every Python I have lying around on my iBook -- the 2.2[.0] in /usr/bin, heads of release22-maint, release23-maint, the CVS trunk, a random build of 2.2.3c1, Jack's 2.3 build. Etc ;-) I'll check on linux when I get into work.

Fixing this would make the code even hairier though... :-(

Pff, it's not so bad, just a little refcount code jiggling. I'll make a real patch including the above test case soon (once I'm reasonably confident that I'm not leaking references). And then I find out running test_descr in a loop leaks 166(!) references *anyway* (run the attached blah.py in a debug build, and some print test or other that I don't understand fails when run like this, so I fiddled it) Should I have been expecting this, or is it time for some ref-leak hunting? Actually, the process seems to grow -- slowly -- when you loop, so it's probably a real problem. I gather it's best to ask about this sort of thing when Tim is off sick from work :-) Cheers, mwh

"Tim Peters" writes:

[Michael Hudson]

...

... And then I find out running test_descr in a loop leaks 166(!) references *anyway* (run the attached blah.py in a debug build, and some print test or other that I don't understand fails when run like this, so I fiddled it)

Should I have been expecting this, or is it time for some ref-leak hunting? Actually, the process seems to grow -- slowly -- when you loop, so it's probably a real problem. I gather it's best to ask about this sort of thing when Tim is off sick from work :-)

Only if you want help <wink>.

Actually, I found the worst culprit this morning. Some dunderhead had got the decrefs wrong on error return from type_set_bases... I'll check my fix in momentarily.

Since Zope has worse leak problems than Python, Zope testing has more tools to dig into it:

http://cvs.zope.org/Zope3/test.py

That's the Zope3 unittest test driver. The TrackRefs class is a handy little beast, which uses a debug build's sys.getobjects() to keep track of how many objects of each *type* exist across test runs, and how many refcounts total they account for. After the first few iterations, TrackRefs doesn't itself distort the results it prints. It will tell you the types of the objects that are leaking, and also whether objects are actually leaking, or that merely refcounts to existing objects are leaking (e.g., a common kind of "leak" is to forget to decref Py_None, but memory doesn't actually grow then).

Cool! I'd blundered my way to something like an ad hoc version of this, next time I'll use something that someone's actually thought about :-)

Do use 2.3 if you try this.

Well, yeah. Turned out the worst leaks weren't in 2.2 code anyway.

The "list of all objects" before 2.3 turned out to be missing many objects that tend to leak (None, booleans, any number of type objects, essentially all statically allocated objects), and far fewer are missing in 2.3.

It's typically easy to find out which types of objects are leaking this way. Then it's typical to add more code to TraceRefs.update() to get more details. In Zope use, it's surprising how often that still left us scratching our heads! But Zope has lots of extension types coded in C, massive interconnections among them, and none of them originally participated in cyclic gc. Python leaks are usually easier to plug (for example, in Python, they're almost always in new code).

Bingo! I did all these tests incorporating my safety fixes to PyObject_GenericGetAttr so I think they're probably solid, too. Cheers, mwh -- The PROPER way to handle HTML postings is to cancel the article, then hire a hitman to kill the poster, his wife and kids, and fuck his dog and smash his computer into little bits. Anything more is just extremism. -- Paul Tomblin, asr

On Thursday, Aug 7, 2003, at 14:08 Europe/London, Michael Hudson wrote:

...

Since Zope has worse leak problems than Python, Zope testing has more tools to dig into it:

http://cvs.zope.org/Zope3/test.py

That's the Zope3 unittest test driver. The TrackRefs class is a handy little beast, which uses a debug build's sys.getobjects() to keep track of how many objects of each *type* exist across test runs, and how many refcounts total they account for. After the first few iterations, TrackRefs doesn't itself distort the results it prints. It will tell you the types of the objects that are leaking, and also whether objects are actually leaking, or that merely refcounts to existing objects are leaking (e.g., a common kind of "leak" is to forget to decref Py_None, but memory doesn't actually grow then).

[I'll observe that trying to wrap running a test in a refcount neutral way is entertaining...]

Cool! I'd blundered my way to something like an ad hoc version of this, next time I'll use something that someone's actually thought about :-)

OK, so I've now blundered my way to the attached, which when run produces this: ints seems to leak 1 references... 1 4 multi seems to leak 9 references... 1 5 1 4 1 4 1 3 0 4 0 1 slots seems to leak 10 references... 5 20 0 5 errors seems to leak 9 references... 4 16 0 5 subtype_resurrection seems to leak 2 references... (the names are all names of functions in test_descr). I might have a look at these over the next few days (and also might ponder other bits of the test suite). Cheers, mwh