Mailman 3 August 2006 - Python-Dev

Re: [Python-Dev] GeneratorExit is unintuitive and uneccessary
by Phillip J. Eby 24 Aug '06

24 Aug '06

At 04:10 PM 8/23/2006 -0700, Guido van Rossum wrote: >IIUC this is how return currently works -- in some sense it's an >exception, but it can't be caught, and it won't escape from the >current frame. Ditto for break/continue. > >The generator extensions are still very young, and I'm not against >changing them somewhat in 2.6 or py3k. Perhaps GeneratorExit can be >replaced by a treatment similar to the treatment of return. But I'm >not sure it helps with the original problem; It depends on what you're referring to as "the original problem". If you mean the problem Bob Ippolito opened a bug for, then yes, it does not solve that problem. But I think we agree that *that* problem doesn't actually need solving. However, it *does* solve the problem(s) that Igor Bukanov brought up, namely that GeneratorExit is unintuitive and unnecessary. :) It also fixes the problem of calling close() or GC'ing a generator written for an earlier version of Python, that yields inside a try-except. >In a generator, a yield >in the finally clause still needs to be dealt with somehow. And we should keep the existing PEP-defined behavior for dealing with it. That is, my understanding is that the only part of close()'s definition that would change is the 'throw(GeneratorExit)' bit. I just wish this had been proposed some time much sooner, like maybe not long after I did the original implementation of throw()/close()/etc. During the original discussion, I'd have been more skeptical of a return-based approach. But knowing what I know now about the implementation, making it based on return instead would be trivial. A proof-of-concept patch I just whipped up adds 7 lines to the hook code in ceval.c (to determine whether WHY_RETURN or WHY_EXCEPTION should be used), and 5 lines to gen_close() in genobject.c.

1 0

Re: [Python-Dev] GeneratorExit is unintuitive and uneccessary
by Phillip J. Eby 23 Aug '06

23 Aug '06

I rather like it, actually. I don't recall there being any real reasons to catch a GeneratorExit exception, but even if there were, you could do the equivalent like this: try: closed = True yield 1 closed = False finally: if closed: # special closing code Unfortunately, this is a pretty major change to be making to the specification (PEP 342) when we've just released 2.5c1; it seems like something that requires at least a beta release. But aside from that, it seems like a fairly neat solution to the problem. I'd like to hear from Jython and IronPython representatives regarding implementability for those platforms, though. Our original assumption was that if they could implement throw() then they could clearly implement close(), since close() was defined in terms of throw(). An asynchronous return might be another matter. But if that hurdle is crossable, then I agree that modelling close() as an early return is more intuitive, eliminates a built-in exception, etc. At 03:17 PM 8/22/2006 +0200, Igor Bukanov wrote: >Consider the following example: > >for i in range(3): > try: > print i > break > except: > print "Unexpected exception!" > finally: > print "Finally" > >When executed, it naturally prints > 0 > Finally >since break does not use exceptions to transfer the control and as >such can not be stopped using catch-all exception handler. > >Now consider a similar example using generators: > >def gen(): > for i in range(3): > try: > yield i > except: > print "Unexpected exception!" > finally: > print "Finally" > >for i in gen(): > print i > break > >This example prints: > 0 > Unexpected exception! > Finally > Exception exceptions.RuntimeError: 'generator ignored GeneratorExit' >in <generator object at 0xb7daaa8c> ignored > >Suddenly with generators a program can mess with control transfer >since it uses explicit GeneratorExit which can be caught and ignored. >This is unintuitive IMO. > >This example also suggests how to fix generators. One just need to >change the close method so it would cause return executed right after >the yield instead of throw. > >I.e. replace the current text from >http://www.python.org/dev/peps/pep-0342/ > >4. Add a close() method for generator-iterators, which raises >GeneratorExit at the point where the generator was paused. If the >generator then raises StopIteration (by exiting normally, or due to >already being closed) or GeneratorExit (by not catching the >exception), close() returns to its caller. If the generator yields a >value, a RuntimeError is raised. If the generator raises any other >exception, it is propagated to the caller. close() does nothing if the >generator has already exited due to an exception or normal exit. > >by simpler one: > >4. Add a close() method for generator-iterators, which executes normal >return at the point where the generator was paused. If the generator >then raises StopIteration (by exiting normally, or due to already >being closed), close() returns to its caller. If the generator yields >a value, a RuntimeError is raised. If the generator raises any other >exception, it is propagated to the caller. close() does nothing if the >generator has already exited due to an exception or normal exit. > >This not only fixes the above discrepancy between normal flow control >and generators, removes GeneratorExit and simplifies the generator >protocol, but also bring a new feature allowing to have easy to grasp >feature table of the iterator methods: > >next: continue after yield >throw: raise after yield >close: return after yield > >Regards, Igor >_______________________________________________ >Python-Dev mailing list >Python-Dev(a)python.org >http://mail.python.org/mailman/listinfo/python-dev >Unsubscribe: >http://mail.python.org/mailman/options/python-dev/pje%40telecommunity.com

3 3

Re: [Python-Dev] [Python-checkins] r51525 - in python/trunk:Lib/test/test_float.py Objects/floatobject.c
by martin＠v.loewis.de 23 Aug '06

23 Aug '06

Zitat von Tim Peters <tim.peters(a)gmail.com>: > Huh. It's been a (mildly controversial, but intentional all the same) > feature that Python tries to raise raise OverflowError on overflowing > libm operations. Doesn't work all that well, since there's no > consistency across platforms about when libm sets errno, or to what > (when it does). The other question is whether pow() is guaranteed to return inf when it sets errno to ERANGE; I believe there is no such guarantee. So I think the change should be reverted. Regards, Martin

2 1

[Python-checkins] r51525 - in python/trunk: Lib/test/test_float.py Objects/floatobject.c
by Thomas Wouters 23 Aug '06

23 Aug '06

Since Alex isn't on python-dev, forwarding this for his convenience (he said he wanted to reply.) Tim also replied to his checkin, but I didn't forward that message. ---------- Forwarded message ---------- From: Thomas Wouters <thomas(a)python.org> Date: Aug 24, 2006 12:16 AM Subject: Re: [Python-checkins] r51525 - in python/trunk: Lib/test/test_float.py Objects/floatobject.c To: "alex. martelli" <python-checkins(a)python.org> On 8/23/06, alex.martelli <python-checkins(a)python.org> wrote: > > > --- python/trunk/Objects/floatobject.c (original) > +++ python/trunk/Objects/floatobject.c Wed Aug 23 22:42:02 2006 > @@ -821,12 +821,12 @@ > ix = pow(iv, iw); > PyFPE_END_PROTECT(ix) > Py_ADJUST_ERANGE1(ix); > - if (errno != 0) { > + /* we need to ignore ERANGE here and just return inf */ > + if (errno != 0 && errno != ERANGE) { > /* We don't expect any errno value other than ERANGE, but > * the range of libm bugs appears unbounded. > */ > - PyErr_SetFromErrno(errno == ERANGE ? PyExc_OverflowError : > - PyExc_ValueError); > + PyErr_SetFromErrno(PyExc_ValueError); > return NULL; I don't think this can be right. The returnvalue of pow() in the case of ERANGE isn't defined anywhere that I can find, at least. How can we assume it is +Infinity? As I tried to say over the visiphone, you can't even use compile-time or startup-time checks for the behaviour because 'undefined' also means it need not be consistent, either. The best we could do, if we really wanted to return +inf instead of raising OverflowError (which I don't, but don't really care about either), is generate +Infinity in some guaranteed way. I'm sure Tim can come up with a convenient, portable way < 0.9 wink>. But I'd prefer to keep it the way it was: x*x and x**2 don't always do the same thing. Floats have a lot more confusing features like that, such as 10*x - 9*x need not be x. I don't see the added value of trying to make it consistent in just this case, even if it's portable. -- Thomas Wouters <thomas(a)python.org> Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -- Thomas Wouters <thomas(a)python.org> Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

2 1

Re: [Python-Dev] [4suite] cDomlette deallocation bug?
by Jeremy Kloth 23 Aug '06

23 Aug '06

I'm CC python-dev as this may be a bug in the gc module. Although this may be more of a c.l.p. post, I think that this usage shouldn't cause segfaults regardless. On Tuesday 22 August 2006 3:06 pm, Robert Fleming wrote: > [step 1 unnecessary] > 2. test program (test.c): > /******** snip ********/ > #include <Python.h> > > #include <stdio.h> > > int main(int argc, char* argv[]) > { > int i; > for (i = 0; i<10; i++) > { > printf("%d\n", i); > > Py_Initialize(); > (void)PyRun_SimpleString("import sys\n" > "sys.path.append('.')\n" > "import cDomlettec"); Change the string to just "import gc" and the segfault still happens. > Py_Finalize(); > } > > return EXIT_SUCCESS; > } > /****** snip ********/ > > 3. compile with: > gcc -I/usr/include/python2.4 test.c -lpython2.4 > > 4. run program; output is: > 0 > 1 > Segmentation fault > > Am I doing something wrong? I'm pretty new to embedding Python, but I'm > hoping to be able to cleanly load and unload 4Suite this way It just so happens that cDomlettec imports gc internally which, by the change above shows, causes the segfault. Hopefully someone with more knowledge of GC internals can comment on this. -- Jeremy Kloth http://4suite.org/

1 0

GeneratorExit is unintuitive and uneccessary
by Igor Bukanov 23 Aug '06

23 Aug '06

Consider the following example: for i in range(3): try: print i break except: print "Unexpected exception!" finally: print "Finally" When executed, it naturally prints 0 Finally since break does not use exceptions to transfer the control and as such can not be stopped using catch-all exception handler. Now consider a similar example using generators: def gen(): for i in range(3): try: yield i except: print "Unexpected exception!" finally: print "Finally" for i in gen(): print i break This example prints: 0 Unexpected exception! Finally Exception exceptions.RuntimeError: 'generator ignored GeneratorExit' in <generator object at 0xb7daaa8c> ignored Suddenly with generators a program can mess with control transfer since it uses explicit GeneratorExit which can be caught and ignored. This is unintuitive IMO. This example also suggests how to fix generators. One just need to change the close method so it would cause return executed right after the yield instead of throw. I.e. replace the current text from http://www.python.org/dev/peps/pep-0342/ 4. Add a close() method for generator-iterators, which raises GeneratorExit at the point where the generator was paused. If the generator then raises StopIteration (by exiting normally, or due to already being closed) or GeneratorExit (by not catching the exception), close() returns to its caller. If the generator yields a value, a RuntimeError is raised. If the generator raises any other exception, it is propagated to the caller. close() does nothing if the generator has already exited due to an exception or normal exit. by simpler one: 4. Add a close() method for generator-iterators, which executes normal return at the point where the generator was paused. If the generator then raises StopIteration (by exiting normally, or due to already being closed), close() returns to its caller. If the generator yields a value, a RuntimeError is raised. If the generator raises any other exception, it is propagated to the caller. close() does nothing if the generator has already exited due to an exception or normal exit. This not only fixes the above discrepancy between normal flow control and generators, removes GeneratorExit and simplifies the generator protocol, but also bring a new feature allowing to have easy to grasp feature table of the iterator methods: next: continue after yield throw: raise after yield close: return after yield Regards, Igor

3 3

String formatting / unicode 2.5 bug?
by John J Lee 23 Aug '06

23 Aug '06

Is this a bug? # run with 2.4 and then with 2.5 (I'm running release25-maint:51410) class a(object): def __getattribute__(self, name): print "accessing %r.%s" % (self, name) return object.__getattribute__(self, name) def __str__(self): print "__str__" return u'hi' print repr(str(a())) print print repr("%s" % a()) John

3 7

Re: [Python-Dev] [Python-checkins] r51458 - peps/trunk/pep-0000.txt peps/trunk/pep-0362.txt
by Nick Coghlan 22 Aug '06

22 Aug '06

brett.cannon wrote: > Added: peps/trunk/pep-0362.txt > +Relation With Other PEPs > +======================== > + > +"Keyword-Only Arguments [#pep-3102]_ > +------------------------------------ > + > +If keyword-only parameters come into existence, the Parameter object > +will require modification. A ``keyword_only`` attribute will be added > +that holds a boolean representing whether the parameter is > +keyword-only or not. Why an extra attribute instead of using "x.position is None"? Unlike default_value, None is not otherwise a legitimate value for the position attribute. > +Where should keyword-only parameters be stored in the Signature object? > +----------------------------------------------------------------------- > + > +If PEP 3102 [#pep-3102]_ ends up being accepted, there is the > +possibility that storing keyword-only parameters in a set instead of > +in the ``parameters`` attribute of the Signature object makes more > +sense. Since keyword-only parameters do not have any semantic meaning > +in terms of their position within the signature, there is no direct > +semantic gain in storing it in the parameter list. > + > +Storing keyword-only paramaters in a set makes it much more explicit > +that keyword-only parameters have no inherent order. It does have the > +drawback, though, that if one wants to process all parameters at once > +they would need to perform extra work to make sure to go through both > +the parameter list and set. Throwing the keyword arguments in a set still loses information that was present in the source code. Preserving the ordering in the signature object means that the author of the function can control the order in which the keyword arguments are displayed by introspection-based help utilities (e.g. putting "more important" arguments first. Cheers, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org

2 2

What should the focus for 2.6 be?
by Guido van Rossum 22 Aug '06

22 Aug '06

I've been thinking a bit about a focus for the 2.6 release. We are now officially starting parallel development of 2.6 and 3.0. I really don't expect that we'll be able to merge the easily into the 3.0 branch much longer, so effectively 3.0 will be a fork of 2.5. I wonder if it would make sense to focus in 2.6 on making porting of 2.6 code to 3.0 easier, rather than trying to introduce new features in 2.6. We've done releases without new language features before; notable 2.3 didn't add anything new (except making a few __future__ imports redundant) and concentrated on bugfixes, performance, and library additions. Some projects that could be undertaken in 2.6: - add warnings when apply() is used - add warnings when string exceptions or non-BaseException-derived exceptions are used (this is already planned in PEP 252, which has a very specific roll-out plan) - add warnings when has_key() is used - add warnings when the result of dict.keys(), .values(), .items() is used for anything else than iterating over it - a warning if a class defines has_key() but not __contains__(). - add __contains__ to dbm and gdbm - add warnings to modules and built-ins that will die in 3.0 Some of these warnings should be suppressed by default, but enabled by a command line option. We should also do the work on the standard library to avoid the warnings: get rid of apply(), use 'x in d' instead of 'd.has_key(x)', etc. I've recently done some of this work in the 3.0 branch (e.g. dbm/gdbm are fresh in my memory). Another area that could use a lot of work (and from which 3.0 could also benefit directly) is converting all unit tests to using either unittest.py or doctest.py. There are still at least 40 tests written in the old "compare the output with one we prepared earlier" style. Of course, if people are chomping at the bit to implement certain new features (and those features are generally approved of) then I don't want to stop them; but I would recommend that our effort may better be focused on smoothing the 2.6/3.0 transition rather than looking for new features to add to 2.6. I am often approached by people who object against this or that feature proposal not because they dislike the proposed feature in particular, but because they believe Python is already large enough, and they worry that if we keep adding features at the current pace, it will soon become too unwieldy, and hence harder to learn or to keep in one's brain. I am very sympathetic to this concern (and you should be too!). This is one of the reasons that so much of the Python 3000 work is about ripping out old stuff and simplifying / unifiying things. Dropping two common data types (long and unicode -- of course they will really be merged into their simpler counterparts int and str, but it means much less to learn/remember) is one example. Ripping out classic classes and string exceptions another. Removing dead library modules a third. -- --Guido van Rossum (home page: http://www.python.org/~guido/)

9 10

Type of range object members
by Alexander Belopolsky 21 Aug '06

21 Aug '06

The range object is currently defined in Objects/rangeobject.c as typedef struct { PyObject_HEAD long start; long step; long len; } rangeobject; Is this consistent with PEP 353, or should Py_ssize_t be used instead of long? It looks like some of the code in rangeobject.c is already Py_ssize_t aware (see range_item and range_length), but it assumes that it is safe to cast long to ssize_t and back.

14 51