Re: [Python-Dev] Python 1.6 timing (fwd)

Guido> There are several other things I can think of now that were Guido> planned for 1.6: revamped import, rich comparisons, revised Guido> coercions, parallel for loop (for i in L; j in M: ...), Guido> extended slicing for all sequences. I've also been thinking Guido> about making classes be types (not as huge a change as you Guido> think, if you don't allow subclassing built-in types), and Guido> adding a built-in array type suitable for use by NumPy. I've Guido> also received a conservative GC patch that seems to be fairly Guido> easy to apply and has some of Tim Peters' blessing. BAW> All very cool things that could easily wait until 1.7. After BAW> all, what's in a number? GVW writes: I agree on all counts except garbage collection --- I'm half-way through the second day of the Python class I teach at Los Alamos (the people who are funding the Python tool design competition), and it's come up a couple of times. People want to be able to prototype meshes, throw callbacks around without worrying about circularity, and some other things that I don't really understand yet. There's also a couple of smart guys in the class who are wondering about CPython vs. JPython ("So this'll be safe in one version of the language, but not in the other?"), and about marketing ("Help me win a feature comparison against Java in my group..."). There's also been questions about tighter integration of NumPy (e.g. overloading operators rather than calling 'greater()' to do comparison), but I think that's a separate discussion... My $0.02, Greg

[garbage collection] gvwilson@nevex.com wrote:
Guido once posted some proposal of a hybrid system *with* refcounts and some additional garbage collection scheme to match circular things. I believe this is a much better approach than what Java and therefor also JPython does at the moment. Although people might argue differently, I'm pretty sure that reference counting is the stronger concept. By reference counting, the idea of object ownership can be made explicit. This plays a central role in the Corba specification for instance, and I made the same observation when implementing continuations for Stackless Python. Refcounts are no burden but a virtue. Even better: Refcounting can lead to many new optimizations if we pay the cost to make INCREF/DECREF into methods. It has its cost (about 10 percent less pystones), but massive long-term benefits. I'm currently in charge to develop a custom version of Python's builtin types where this concept is used. Everything is refcounted, but without storing the refcounts in the objects. This is possible (proven) and will be shown in my next paper. Conclusion: I vote for a kind of GC that does just what refcounts cannot do, but please keep with the refcounts. cheers - chris -- Christian Tismer :^) <mailto:tismer@appliedbiometrics.com> Virtual Photonics GmbH : Have a break! Take a ride on Python's Carnotstr. 6 : *Starship* http://starship.python.net 10587 Berlin : PGP key -> http://wwwkeys.pgp.net PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF we're tired of banana software - shipped green, ripens at home

Conclusion: I vote for a kind of GC that does just what refcounts cannot do, but please keep with the refcounts.
The patch that I received and that has Tim's <0.5 blessing> does just that. I haven't had the time to understand why it doesn't have his <1.0 blessing>. --Guido van Rossum (home page: http://www.python.org/~guido/)

[Christian]
[Guido]
Primarily because it doesn't reclaim the most common cycles; e.g., cycles among class instances aren't touched. This seems easily repairable, but at an unknown cost (it needs to do the "reachability" transitive closure business from the set of all "suspicious" objects, and instances are never considered suspicious now; adding them will certainly cause a lot more pointer chasing). Apart from that, the code appears unreasonably expensive as written today, using e.g. splay trees instead of hash tables to keep track of objects. The author hasn't said anything more in a bit over two weeks, so I suspect he's off on other things now. The technical approach is sound, but even its inventor (Rafael Lins; Toby Kelsey may have reinvented it on his own, though) stresses that getting it to run fast is difficult. needs-work!-ly y'rs - tim, who hasn't the time to do it

[hristian Tismer]
The GC patch Guido forced <wink> me to look at is based on the observation that it's impossible to create cyclic trash unless a decref leaves a nonzero refcount. So the patch adds a function call to the DECREF macro (if the new refcount is > 0, the object must be added to the set of "suspicious" objects; else the object must be removed from that set). So it roughly adds the cost of a method call to each decref anyway. You would think it adds less <wink>, but "the set" now is represented as a splay tree, so *gobs* of hairy code get executed in either case (splay trees do mutating rotations even on a lookup).
I like 'em too! BTW, Toby posted (at least an earlier version of) the patch to c.l.py, if anyone else wants to dig into it (I may have mentioned before that I'm short on time <wink>).

[Greg Wilson]
[David Ascher]
That's the rich comparison proposal which Guido mentioned.
But there's also been talk about moving (at least) the basic NumPy array type into the core. This would be a Good Thing. Speaking for my employer, however, only Unicode is an Important Thing <wink>. As a developer, I have railed against schedule-driven release cycles. Python tends toward the opposite end of that spectrum, driven by features no matter how bloody long they take. Add Unicode to what's already waiting to go, and that's *good enough* reason for a major release; heck, it's been 9 months & we haven't even had a 1.5.2 bugfix patch. BTW, do the Perl-Porters have real jobs? pay-me-to-do-python-releases-and-you'll-get-a-major-new- release-every-three-days<wink>-ly y'rs - tim

Tim Peters
But there's also been talk about moving (at least) the basic NumPy array type into the core. This would be a Good Thing.
IMNSHO, moving the current NumPy array into the core would be a Bad Thing. Moving a new similar object with cleaned up semantics and better implementation in would be a Good Thing. But it won't happen until 1.7 at the earliest, as the semantics haven't even been agreed on, let alone the code written. --david

[Greg Wilson]
Greg, people who have been exposed to Fortran (this is LANL, right <wink>?) can't possibly have a problem with the concept of "not defined by the standard". Don't sell these as different *versions* of the language, but as different implementations. That's what they are. The Python *language* doesn't define anything about the lifetime of objects. Even when CPython grows "real GC", thanks to refcounting too you'll still be able to *rely* on behaviors in CPython you'll see only accidentally in JPython. You do so at your own risk, same as e.g. you rely on floating point Fortran x+y+z getting evaluated "left to right" at your own risk (BTW, x+y+z *is* "left to right" in Python -- maybe they'll trade that for the lack of GC promises <wink>).

[garbage collection] gvwilson@nevex.com wrote:
Guido once posted some proposal of a hybrid system *with* refcounts and some additional garbage collection scheme to match circular things. I believe this is a much better approach than what Java and therefor also JPython does at the moment. Although people might argue differently, I'm pretty sure that reference counting is the stronger concept. By reference counting, the idea of object ownership can be made explicit. This plays a central role in the Corba specification for instance, and I made the same observation when implementing continuations for Stackless Python. Refcounts are no burden but a virtue. Even better: Refcounting can lead to many new optimizations if we pay the cost to make INCREF/DECREF into methods. It has its cost (about 10 percent less pystones), but massive long-term benefits. I'm currently in charge to develop a custom version of Python's builtin types where this concept is used. Everything is refcounted, but without storing the refcounts in the objects. This is possible (proven) and will be shown in my next paper. Conclusion: I vote for a kind of GC that does just what refcounts cannot do, but please keep with the refcounts. cheers - chris -- Christian Tismer :^) <mailto:tismer@appliedbiometrics.com> Virtual Photonics GmbH : Have a break! Take a ride on Python's Carnotstr. 6 : *Starship* http://starship.python.net 10587 Berlin : PGP key -> http://wwwkeys.pgp.net PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF we're tired of banana software - shipped green, ripens at home

Conclusion: I vote for a kind of GC that does just what refcounts cannot do, but please keep with the refcounts.
The patch that I received and that has Tim's <0.5 blessing> does just that. I haven't had the time to understand why it doesn't have his <1.0 blessing>. --Guido van Rossum (home page: http://www.python.org/~guido/)

[Christian]
[Guido]
Primarily because it doesn't reclaim the most common cycles; e.g., cycles among class instances aren't touched. This seems easily repairable, but at an unknown cost (it needs to do the "reachability" transitive closure business from the set of all "suspicious" objects, and instances are never considered suspicious now; adding them will certainly cause a lot more pointer chasing). Apart from that, the code appears unreasonably expensive as written today, using e.g. splay trees instead of hash tables to keep track of objects. The author hasn't said anything more in a bit over two weeks, so I suspect he's off on other things now. The technical approach is sound, but even its inventor (Rafael Lins; Toby Kelsey may have reinvented it on his own, though) stresses that getting it to run fast is difficult. needs-work!-ly y'rs - tim, who hasn't the time to do it

[hristian Tismer]
The GC patch Guido forced <wink> me to look at is based on the observation that it's impossible to create cyclic trash unless a decref leaves a nonzero refcount. So the patch adds a function call to the DECREF macro (if the new refcount is > 0, the object must be added to the set of "suspicious" objects; else the object must be removed from that set). So it roughly adds the cost of a method call to each decref anyway. You would think it adds less <wink>, but "the set" now is represented as a splay tree, so *gobs* of hairy code get executed in either case (splay trees do mutating rotations even on a lookup).
I like 'em too! BTW, Toby posted (at least an earlier version of) the patch to c.l.py, if anyone else wants to dig into it (I may have mentioned before that I'm short on time <wink>).

[Greg Wilson]
[David Ascher]
That's the rich comparison proposal which Guido mentioned.
But there's also been talk about moving (at least) the basic NumPy array type into the core. This would be a Good Thing. Speaking for my employer, however, only Unicode is an Important Thing <wink>. As a developer, I have railed against schedule-driven release cycles. Python tends toward the opposite end of that spectrum, driven by features no matter how bloody long they take. Add Unicode to what's already waiting to go, and that's *good enough* reason for a major release; heck, it's been 9 months & we haven't even had a 1.5.2 bugfix patch. BTW, do the Perl-Porters have real jobs? pay-me-to-do-python-releases-and-you'll-get-a-major-new- release-every-three-days<wink>-ly y'rs - tim

Tim Peters
But there's also been talk about moving (at least) the basic NumPy array type into the core. This would be a Good Thing.
IMNSHO, moving the current NumPy array into the core would be a Bad Thing. Moving a new similar object with cleaned up semantics and better implementation in would be a Good Thing. But it won't happen until 1.7 at the earliest, as the semantics haven't even been agreed on, let alone the code written. --david

[Greg Wilson]
Greg, people who have been exposed to Fortran (this is LANL, right <wink>?) can't possibly have a problem with the concept of "not defined by the standard". Don't sell these as different *versions* of the language, but as different implementations. That's what they are. The Python *language* doesn't define anything about the lifetime of objects. Even when CPython grows "real GC", thanks to refcounting too you'll still be able to *rely* on behaviors in CPython you'll see only accidentally in JPython. You do so at your own risk, same as e.g. you rely on floating point Fortran x+y+z getting evaluated "left to right" at your own risk (BTW, x+y+z *is* "left to right" in Python -- maybe they'll trade that for the lack of GC promises <wink>).
participants (5)
-
Christian Tismer
-
David Ascher
-
Guido van Rossum
-
gvwilson@nevex.com
-
Tim Peters