Mailman 3 July 2006 - Python-Dev

sys.settrace() in Python 2.3 vs. 2.4
by Josiah Carlson 01 Jul '06

01 Jul '06

I've previously asked on python-list, but have recieved no responses or explanations. Maybe someone here with a better memory can help, and I apologize for asking a somewhat off-topic question about such an archaic version of Python. According to my reading of Python 2.3 docs, the call to goo() should exit with a KeyboardInterrupt... import sys def goo(): while 1: pass count = [100] def foo(frame, event, arg): count[0] -= 1 if not count[0]: raise KeyboardInterrupt return foo sys.settrace(foo) goo() In Python 2.3, the above call to goo() doesn't return. Adding print statements suggests that foo() is only called for each line executed in goo() once. It exits with a KeyboardInterrupt in 2.4, as expected, where foo() is called for essentially every operaition performed. Does anyone have an idea why this is the case? I've checked the release notes for both 2.3 and 2.4 and found no discussion of trace functions in them or in sourceforge (I could be entering the wrong search terms, of course). Any pointers as to why there is a difference would be appreciated. Thank you, - Josiah

3 4

how long to wait for expat to incorporate a fix to prevent a crasher?
by Brett Cannon 01 Jul '06

01 Jul '06

Lib/test/crashers/xml_parsers.py is a crasher that involves expat (bug report at http://python.org/sf/1296433). What is at issue here is that there is a 'for' loop in expat where the status of the parser is not checked. Because of this, the loop continues on its merry way, which is a problem because pyexpat sets all handlers to 0 upon error and the 'for' loop executes a handler. =) We all know what happens if you try to execute memory location 0x0. Anyway, the fault is not on our end since expat should be checking the status of the parser before going around the loop again instead of blindly assuming that everything is fine after a characterDataHandler() call (especially since there is no error return code and there is a parser status flag for this exact reason). I have filed a bug report at http://sourceforge.net/support/tracker.php?aid=1515266 and attached a possible patch. The question is how long do we wait for the expat developers to patch and do a micro release? Do we just leave this possible crasher in and just rely entirely on the expat developers, or do we patch our copy and use that until they get around to doing their next version push? -Brett

3 4

Empty Subscript PEP on Wiki - keep or toss?
by skip＠pobox.com 01 Jul '06

01 Jul '06

Noam Raphael posted an empty subscript PEP on the Python Wiki: http://wiki.python.org/moin/EmptySubscriptListPEP It's not linked to by any other pages on the wiki. Is there a reason it wasn't added to the peps repository? Skip

4 4

For sandboxing: alternative to crippling file()
by Brett Cannon 01 Jul '06

01 Jul '06

I have gotten some questions from people about why cripple 'file' (and probably 'socket' if they cared), asking whey I didn't just remove the 'file' built-in from built-ins. Problem is that I still want to provide some protection for files. So an option I have been thinking of is making sure 'file' does not end up in built-ins by just not inserting it into tstate->interp->builtins (see Include/pystate.h to see what other fields there are; also will answer Trent's question about how "heavy", roughly, interpreters are). Then, there can be a file delegator class that can, at the C level, store a reference to the actual 'file' object that is open. Finally, handling whether a path is legal or not can be handled by open(). And the open() thing is the key here. Guido always talks about how open() should be treated more like a factory function that could some day return the proper object based on its argument. Well, perhaps we should start doing that and add support for HTTP addresses and IP addresses. Then the file and networking settings can be viewed more as global settings to be followed for file handling instead of specific restrictions placed on the 'file' and socket types. My worry, as has been from the start, is containing 'file'. The ``del __builtins__`` bug for 'rexec' started me as skittish towards hiding stuff from the built-in namespace. And knowing how easy it tends to be to get at objects and types in Python in general makes me worry even more about hiding objects and types properly from people (within reason, of course; if you want to allow blatent 'file' access we should be able to give it to you somehow). But perhaps removing 'file' from the builtin dict in the PyInterpreterState state is enough to hide the type. So, my question to python-dev, is: 1) Is removing 'file' from the builtins dict in PyInterpreterState (and maybe some other things) going to be safe enough to sufficiently hide 'file' confidently (short of someone being stupid in their C extension module and exposing 'file' directly)? 2) Changing open() to return C-implemented delegate objects for files (and thus won't type check, but this is Python so I am not worried about that too much) and delegate socket objects for IP and URL addresses. Basically, do people think doing this instead of modifying the 'file' object directly and crippling is better and safer in terms of possible security breach issues from implementing this? -Brett

6 9

ImportWarning flood
by Ralf W. Grosse-Kunstleve 01 Jul '06

01 Jul '06

http://docs.python.org/dev/whatsnew/other-lang.html says: > One error that Python programmers sometimes make is forgetting to > include an __init__.py module in a package directory. Debugging this > mistake can be confusing, and usually requires running Python with the > -v switch to log all the paths searched. In Python 2.5, a new > ImportWarning warning is raised when an import would have picked up a > directory as a package but no __init__.py was found. I am getting tons of "ImportWarning: Not importing directory". See below for examples. It is impractical for me to reorganize our directory structure. I'd be busy for a week or more and people would probably scream at me because all the paths have changed. Are there other options to get rid of the warnings? Thanks! Ralf /net/rosie/scratch1/rwgk/dist/libtbx/libtbx/command_line/scons.py:1: ImportWarning: Not importing directory '/net/rosie/scratch1/rwgk/dist/libtbx': missing __init__.py from libtbx.utils import Sorry /net/rosie/scratch1/rwgk/py25b1/build/python/lib/python2.5/random.py:43: ImportWarning: Not importing directory '/net/rosie/scratch1/rwgk/dist/cctbx/math': missing __init__.py from math import log as _log, exp as _exp, pi as _pi, e as _e, ceil as _ceil /net/rosie/scratch1/rwgk/py25b1/build/python/lib/python2.5/random.py:43: ImportWarning: Not importing directory '/net/rosie/scratch1/rwgk/dist/scitbx/math': missing __init__.py from math import log as _log, exp as _exp, pi as _pi, e as _e, ceil as _ceil scons: Reading SConscript files ... /net/rosie/scratch1/rwgk/dist/scons/src/engine/SCons/Tool/__init__.py:112: ImportWarning: Not importing directory '/net/rosie/scratch1/rwgk/dist/scons/src/engine/SCons/Tool/CVS': missing __init__.py file, path, desc = imp.find_module(self.name, smpath) /net/rosie/scratch1/rwgk/dist/phenix/phenix/__init__.py:1: ImportWarning: Not importing directory '/net/rosie/scratch1/rwgk/dist/libtbx': missing __init__.py try: import libtbx.forward_compatibility /net/rosie/scratch1/rwgk/dist/phenix/phenix/refinement/__init__.py:1: ImportWarning: Not importing directory '/net/rosie/scratch1/rwgk/dist/iotbx': missing __init__.py import iotbx.phil /net/rosie/scratch1/rwgk/dist/iotbx/iotbx/phil.py:1: ImportWarning: Not importing directory '/net/rosie/scratch1/rwgk/dist/cctbx': missing __init__.py from cctbx import sgtbx /net/rosie/scratch1/rwgk/dist/cctbx/cctbx/array_family/flex.py:1: ImportWarning: Not importing directory '/net/rosie/scratch1/rwgk/dist/scitbx': missing __init__.py import scitbx.array_family.flex /net/rosie/scratch1/rwgk/dist/scitbx/scitbx/array_family/flex.py:2: ImportWarning: Not importing directory '/net/rosie/scratch1/rwgk/dist/boost': missing __init__.py import boost.optional /net/rosie/scratch1/rwgk/dist/libtbx/libtbx/utils.py:226: ImportWarning: Not importing directory '/net/rosie/scratch1/rwgk/dist/mmtbx': missing __init__.py try: module = __import__(module_path) etc. etc. __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com

14 43

Re: [Python-Dev] ImportWarning flood
by Guido van Rossum 01 Jul '06

01 Jul '06

It's up to the release manager now to decide whether the pitchforks at Google or the pitchforks in the larger Python community are sharper. ;-) --Guido (ducks) On 6/30/06, Shane Hathaway <shane(a)hathawaymix.org> wrote: > Guido van Rossum wrote: > > On 6/30/06, Jean-Paul Calderone <exarkun(a)divmod.com> wrote: > >> How about if someone grovels through import.c and figures out how to make > >> the warning information only show up if the import actually fails? > > > > That would work I think. But it's not easy. > > I just posted a patch intended to solve this. With my patch, > find_module() collects the warnings in a list and only generates > ImportWarning if no package is found. It works, but I have not done > extensive testing. > > I also discovered and tried to resolve what appear to be memory leaks > involving the "copy" variable. If the ImportWarning patch doesn't fly, > I'll be happy to post a different patch that only fixes the leaks. > > The patch is #1515361: > > https://sourceforge.net/tracker/index.php?func=detail&aid=1515361&group_id=… > > Shane > > -- --Guido van Rossum (home page: http://www.python.org/~guido/)

4 3

Re: [Python-Dev] PEP 328 and PEP 338, redux
by Phillip J. Eby 01 Jul '06

01 Jul '06

At 08:08 AM 6/27/2006 -0700, Guido van Rossum wrote: >Bad idea IMO. The __name__ == "__main__" rule is so ingrained, you >don't want to mess with it. Actually, maybe we *do* want to, for this usage. Note that until Python 2.5, it was not possible to do "python -m nested.module", so this change merely prevents *existing* modules from being run this way -- when they could not have been before! So, such modules would require a minor change to run under -m. Is this actually a problem, or is it a new feature?

8 12

Bug in stringobject?
by Georg Brandl 01 Jul '06

01 Jul '06

In string_replace, there is if (PyString_Check(from)) { /* Can this be made a '!check' after the Unicode check? */ } #ifdef Py_USING_UNICODE if (PyUnicode_Check(from)) return PyUnicode_Replace((PyObject *)self, from, to, count); #endif else if (PyObject_AsCharBuffer(from, &tmp_s, &tmp_len)) return NULL; [the same check with "to"] return (PyObject *)replace((PyStringObject *) self, (PyStringObject *) from, (PyStringObject *) to, count); Can this be correct if from or to isn't a string object, but a char buffer compatible object? Georg

5 14

Cleanup of test harness for Python
by Frank Wierzbicki 01 Jul '06

01 Jul '06

Hello all, According to the thread that includes http://mail.python.org/pipermail/python-dev/2006-June/065727.html there will be some effort in 2.6 to make the tests in Python more consistent. I would like to help with that effort, partly to sneak in some checks for CPython internal tests that should be excluded from Jython, but mainly to understand the future implementation of Python for which the tests provide the only real spec. Which of the current tests is closest to an "ideal" test, so I can use it as a model? Thanks, -Frank Wierzbicki

3 2