Mailman 3 March 2006 - Python-Dev

Fwd: [Python-checkins] r43358 - python/trunk/Modules/itertoolsmodule.c
by Thomas Wouters March 29, 2006

March 29, 2006

Wupsie, this was meant for all of python-dev ;P ---------- Forwarded message ---------- From: Thomas Wouters <thomas(a)python.org> Date: Mar 29, 2006 1:34 AM Subject: Re: [Python-Dev] [Python-checkins] r43358 - python/trunk/Modules/itertoolsmodule.c To: Tim Peters <tim.peters(a)gmail.com> On 3/29/06, Tim Peters <tim.peters(a)gmail.com> wrote: > > ... that throws cycle-gc in a hissy fit; > If by "hissy fit" you mean "gcmodule properly moves generators to the list of objects with finalizers, thereby preventing catastrophes", right, that's an intended hissy fit ;-) Well, I meant 'the durned thing refuses to do what I want it to do', which is clear cycles. I guess the hissy fit was mine ;) Did the best I could above, short of explaining exactly how failing to > identify an object with a finalizer can lead to disaster. Much appreciated. Short course: gc obviously needs to know which objects are and are not > trash. If a finalizer is associated with an object in cyclic trash, > then trash objects are potentially visible _from_ the finalizer, and > therefore can be resurrected by running the finalizer. gc must > therefore identify all trash objects reachable from trash objects with > finalizers, because running finalizers _may_ make all such objects > "not trash" again. Getting any part of that wrong can lead to calling > tp_clear on an object that a finalizer actually resurrects, leading to > symptoms from "hey, all the attributes on my object suddenly vanished > by magic, for no reason at all" to segfaults. > So does that make all cycles involving only objects with finalizers impervious to cycle-gc? I guess it'd have to be that way. That also means I can stop searching for leaks in the C code, as we'll never be able to fix them all, anyway (although 'good code' could manually break many of the cycles.) At least not while keeping the tp_del of generator objects, not to mention avoiding finalizers on other builtin objects. (None seem to have them at the moment, they all do their stuff in tp_dealloc, which I guess doesn't allow objects to reincarnate themselves.) I'll see about fixing the Python code to avoid or explicitly break the cycles. Maybe we can patch regrtest.py to take into account uncollectable cycles, so that it could report them separately from refleaks. I'm going to think about all this first, though. Hrrm-hrrm'ly y'rs, -- Thomas Wouters <thomas(a)python.org> Hi! I'm a finalizer virus! copy me into your tp_del slot to help me spread!

2 3

Re: [Python-Dev] Class decorators
by Phillip J. Eby March 28, 2006

March 28, 2006

At 11:04 AM 3/28/2006 -0800, Guido van Rossum wrote: >That's fine. But there's also the C#/Java POV. Can someone point me to examples of C# class "attributes" and Java annotations that they'd like to use with this mechanism? I would indeed like to see how those use cases compare with mine. > I'm somehow concerned >that any mechanism that puts the syntax inside the class body is >somehow going to have a hackish implementation, That depends on whether there's syntax support, or it's just something that happens at runtime. I'm not opposed to having syntax, I just don't think that putting the syntax outside the class is the best thing for most of my class decorator use cases. Nonetheless, neither special syntax nor hackiness is really required. For example, your proposal here: http://mail.python.org/pipermail/python-dev/2004-April/043913.html was to use a '__decorators__' attribute, which would then be processed by a metaclass. However, this could simply become a standard magic attribute ala __metaclass__, and be processed by the class creation machinery after first invoking the correct metaclass. I don't really *like* that approach, but it requires neither syntax nor hacks, (nor a custom metaclass) and it could be syntax-sugared later. For that matter, if this __decorators__ machinery existed, the existing advisor implementation could become a lot less hacky. The only magical bit at that point would be doing something like: locals = sys._getframe(n).f_locals. locals.setdefault('__decorators__',[]).append(decorator) And this of course could go in a function in the stdlib, rather than have everybody writing their own _getframe() hacks.

1 0

Reminder: Bug Day this Friday, 31st of March
by Georg Brandl March 28, 2006

March 28, 2006

Hello, it's time for the 7th Python Bug Day, just before 2.5 alpha 1 is released. The aim of the bug day is to close as many bugs, patches and feature requests as possible, this time with a focus on small feature additions that can still go into the upcoming 2.5 alpha release. When? ^^^^^ The bug day will take place on Friday, March 31st, running from approximately 1PM to 8PM GMT (9AM to 4PM Eastern time). You don't need to be around all day; feel free to stop by for a few hours and contribute. Where and How? ^^^^^^^^^^^^^^ To join, stop by the IRC channel #python-dev on irc.freenode.net, where efforts will be discussed and coordinated. We'll collaboratively go through the Python bug database at SourceForge and fix things as they come up. IMPORTANT: *No* prior knowledge of the Python source is necessary to participate! You'll get all assistance the developers can offer for starting up with helping, this is in fact a good opportunity to learn the basics. Bug day participation helps the developers and makes Python 2.5 a better release by reducing the backlog of bugs and patches. Plus, it's fun! More information ^^^^^^^^^^^^^^^^ For instructions and more information, see the Wiki page at http://www.python.org/cgi-bin/moinmoin/PythonBugDay Cheers, Georg

1 0

Class decorators
by Greg Ewing March 28, 2006

March 28, 2006

I've just been playing around with metaclasses, and I think I've stumbled across a reason for having class decorators as an alternative to metaclasses for some purposes. The metaclass I wrote was for the purpose of adding a class to a registry, the reason for which isn't important here. It worked, but I was surprised to find that it not only registered the classes that I made it the metaclass of, but all subclasses of those classes as well. I'm not sure whether that's really the behaviour I want, and I can imagine some cases in which it's definitely not what I'd want. The general principle brought out here is that when you use a metaclass, it gets inherited by subclasses, but if we had class decorators, they would only affect to the classes that you explicitly applied them to. I think there are uses for both behaviours. Greg

7 7

Error msgs for new-style division
by Georg Brandl March 28, 2006

March 28, 2006

Hi, (this makes test_ctypes fail, therefore I noticed) currently with -Qnew: >>> 2/0 Traceback (most recent call last): File "<stdin>", line 1, in ? ZeroDivisionError: float division >>> 2L/0 Traceback (most recent call last): File "<stdin>", line 1, in ? ZeroDivisionError: long division or modulo by zero >>> 2/0L Traceback (most recent call last): File "<stdin>", line 1, in ? ZeroDivisionError: long division or modulo by zero This seems a little inconsistent. Do we care? Georg

2 1

Libref sections to put new modules under?
by Phillip J. Eby March 28, 2006

March 28, 2006

Any thoughts on where documentation for the new "contextlib" module should go in the library reference? Some candidates: * Program Frameworks (???) * Development Tools (??) * Python Runtime Services (?) * Python Language Services (??) * Miscellaneous Services (??) A similar issue exists for "ctypes" module, although I imagine an argument could easily be made for putting it with "Optional Operating System Services". wsgiref can probably go in "Internet Protocols and Support", while ElementTree obviously goes under "Structured Markup Processing Tools". What other new modules do we have going in 2.5 that haven't found a home in the documentation yet? I seem to recall some discussion about a decorator library or functools or something of that sort.

5 4

Inconsistency in 2.4.3 for __repr__() returning unicode
by Hye-Shik Chang March 28, 2006

March 28, 2006

We got an inconsistency for __repr__() returning unicode as reported in http://python.org/sf/1459029 : class s1: def __repr__(self): return '\\n' class s2: def __repr__(self): return u'\\n' print repr(s1()), repr(s2()) Until 2.4.2: \n \n 2.4.3: \n \\n \\n looks bit weird but it's correct. As once discussed[1] in python-dev before, if __repr__ returns unicode object, PyObject_Repr encodes it via unicode-escape codec. So, non-latin character also could be in repr neutrally. But our unicode-escape had a bug since when it is introduced. The bug was that it doesn't escape backslashes. Therefore, backslashes wasn't escaped in repr while it sholud be escaped because we used the unicode-escape codec. So, fixing the bug made a behavior inconsistency. How do we resolve the problem? Hye-Shik [1] http://mail.python.org/pipermail/python-dev/2000-July/005353.html

3 3

Re: [Python-Dev] Class decorators
by Phillip J. Eby March 28, 2006

March 28, 2006

At 08:02 PM 3/27/2006 -0800, Guido van Rossum wrote: >Just curious (and lazy): what magic is the implementation using that >makes this work without a custom metaclass? It registers a function as the __metaclass__ by poking it into the f_locals of the frame that's defining the class. This function has two attributes that denote 1) the callback to be invoked, and 2) the previous metaclass, forming a linked list of callbacks. When the class is created, this dummy metaclass function invokes the previous metaclass (or metaclass function) in the chain, then calls its callback on the result and returns that value, thus recursively unwinding the chain of advisors in the right order. (i.e., the same order as the advisors were called in the class body). The only part of the code that's even mildly complex is simulating Python's builtin rules for determining the default metaclass, so that when the recursion bottoms out you end up with an actual class with the right metaclass. And that's not so much complex as tedious. Anyway, the mechanism is orthogonal to the chosen metaclass, and thus can be used with classic classes or even ExtensionClasses if you're sufficiently masochistic. :) And AFAIK it's been in use by Zope 3 for some years now; in fact I think you were still working at Zope Corp. when Steve Alexander came up with the basic idea and I made the first practical implementation.

2 1

Re: [Python-Dev] Expose the array interface in Python 2.5?
by Chris Barker March 27, 2006

March 27, 2006

On 3/17/06, Thomas Heller <theller <at> python.net> wrote: > Accessing Python arrays (Numeric arrays, Numeric array, or Numpy > array) as ctypes arrays, and vice versa, without copying any memory, > would be a good thing. This does bring up a point. I was thinking that a really bare-bones nd-array type would be very useful by itself as a way to exchange data. However, if ctypes were to return nd-arrays from C function calls, then we would want at least a rudimentary way of directly accessing the data. I'd sure like to see indexing and slicing, at at least. Also if ctypes were to allow nd-arrays as a way to pass data into a C function, then you'd need a build-in python way to create the data in the first place. If that's too much for now, I'd still like to have the basic data structure defined in the standard library (or built-in) as soon as possible. Greg Ewing wrote: > It might be all right for writers of new > extensions, but there are existing modules (PIL, ctypes, etc.) > that already have their own way of storing data, and it > seems to me it would be easier for the maintainers of > those modules to add a new interface to the existing > data than to rearrange their internal structure to use > this new C-object. Can we have both? A defined interface, that existing code can be adapted to provide, and a new C-Object, that future code can just use. If the goal is to have as many extension types as possible use the same base object, the sooner a standard object is provided the better. There are those of us in the scientific computing community that would love to just have numpy be part of the standard library. It really is big, and maybe too special purpose for that, but at least having the core array in there would be great. For years, I've been dealing with modules like wxPython that don't understand numpy arrays. As a result, passing in a list of points to be drawn is actually faster than passing in a numpy array of points, even though the numpy array of points has the data in the exact binary representation that wxWidgets expects. The problem is that the wrapper code doesn't know that, because Robin (quite reasonably) didn't want to have wxPython depend on Numeric. While a standard interface could support this, it would be great if wxPython could also do things like pass an image buffer back to Python efficiently. Another point is that n-dimensional arrays really are very useful for all sorts of stuff that have nothing to do with high-performance Numeric computing. I find myself using numpy for almost every little script I write, even though most of it is not performance bounded at all. I suspect that if we get a n-dimensional array type into Python, one that allows n-d slicing, it will see a LOT of use by people who now think they have no use for numpy. My guess is that a 2-dimensional object array would get the most use, but why not support n-d while you're at it? Having an easy and natural way to store and manipulate a "table" of objects, for instance, would be handy for many, many users. I'm still a tiny bit confused about the proposed individual pieces involved, but I'd like to see, in order of priority (and difficulty): 1) A standard n-d array interface (numpy already defines this, but outside of the numpy community, who knows about it?) 2) A standard, n-d array C-Object, inheritable for use by various extension packages. 3) A standard n-d array object, with a python interface for indexing and slicing for access and assignment (modeled after, or better yet taken directly from, numpy). 4) A standard n-d array object that supports array-wise arithmetic and functions (ufuncs, in numpy parlance). There is no reason it has to have anything to do with the current array module. For instance, having the arrays be of fixed size is just fine. It's really not that big a deal to make a new one when you need to change the size, after all, we live with that for immutable objects, like strings. just my $0.02 -Chris

3 2

TRUNK FREEZE for 2.5a1: 0000 UTC, Thursday 30th
by Anthony Baxter March 27, 2006

March 27, 2006

Ok, it's time to rock and roll. The SVN trunk is FROZEN for 2.5a1 from 00:00 UTC on Thursday 30th of March. I'll post again once it's open. Note that new features can keep going in during the alpha cycle, the feature freeze only happens once we hit beta. And we're not going to hit beta until the features we want are in. Please help in making this release as painless as possible by not checking in while the trunk is frozen. Thanks! Anthony -- Anthony Baxter <anthony(a)interlink.com.au> It's never too late to have a happy childhood.

3 3