Mailman 3 June 2006 - Python-Dev

For sandboxing: alternative to crippling file()
by Brett Cannon July 1, 2006

July 1, 2006

I have gotten some questions from people about why cripple 'file' (and probably 'socket' if they cared), asking whey I didn't just remove the 'file' built-in from built-ins. Problem is that I still want to provide some protection for files. So an option I have been thinking of is making sure 'file' does not end up in built-ins by just not inserting it into tstate->interp->builtins (see Include/pystate.h to see what other fields there are; also will answer Trent's question about how "heavy", roughly, interpreters are). Then, there can be a file delegator class that can, at the C level, store a reference to the actual 'file' object that is open. Finally, handling whether a path is legal or not can be handled by open(). And the open() thing is the key here. Guido always talks about how open() should be treated more like a factory function that could some day return the proper object based on its argument. Well, perhaps we should start doing that and add support for HTTP addresses and IP addresses. Then the file and networking settings can be viewed more as global settings to be followed for file handling instead of specific restrictions placed on the 'file' and socket types. My worry, as has been from the start, is containing 'file'. The ``del __builtins__`` bug for 'rexec' started me as skittish towards hiding stuff from the built-in namespace. And knowing how easy it tends to be to get at objects and types in Python in general makes me worry even more about hiding objects and types properly from people (within reason, of course; if you want to allow blatent 'file' access we should be able to give it to you somehow). But perhaps removing 'file' from the builtin dict in the PyInterpreterState state is enough to hide the type. So, my question to python-dev, is: 1) Is removing 'file' from the builtins dict in PyInterpreterState (and maybe some other things) going to be safe enough to sufficiently hide 'file' confidently (short of someone being stupid in their C extension module and exposing 'file' directly)? 2) Changing open() to return C-implemented delegate objects for files (and thus won't type check, but this is Python so I am not worried about that too much) and delegate socket objects for IP and URL addresses. Basically, do people think doing this instead of modifying the 'file' object directly and crippling is better and safer in terms of possible security breach issues from implementing this? -Brett

6 9

ImportWarning flood
by Ralf W. Grosse-Kunstleve July 1, 2006

July 1, 2006

http://docs.python.org/dev/whatsnew/other-lang.html says: > One error that Python programmers sometimes make is forgetting to > include an __init__.py module in a package directory. Debugging this > mistake can be confusing, and usually requires running Python with the > -v switch to log all the paths searched. In Python 2.5, a new > ImportWarning warning is raised when an import would have picked up a > directory as a package but no __init__.py was found. I am getting tons of "ImportWarning: Not importing directory". See below for examples. It is impractical for me to reorganize our directory structure. I'd be busy for a week or more and people would probably scream at me because all the paths have changed. Are there other options to get rid of the warnings? Thanks! Ralf /net/rosie/scratch1/rwgk/dist/libtbx/libtbx/command_line/scons.py:1: ImportWarning: Not importing directory '/net/rosie/scratch1/rwgk/dist/libtbx': missing __init__.py from libtbx.utils import Sorry /net/rosie/scratch1/rwgk/py25b1/build/python/lib/python2.5/random.py:43: ImportWarning: Not importing directory '/net/rosie/scratch1/rwgk/dist/cctbx/math': missing __init__.py from math import log as _log, exp as _exp, pi as _pi, e as _e, ceil as _ceil /net/rosie/scratch1/rwgk/py25b1/build/python/lib/python2.5/random.py:43: ImportWarning: Not importing directory '/net/rosie/scratch1/rwgk/dist/scitbx/math': missing __init__.py from math import log as _log, exp as _exp, pi as _pi, e as _e, ceil as _ceil scons: Reading SConscript files ... /net/rosie/scratch1/rwgk/dist/scons/src/engine/SCons/Tool/__init__.py:112: ImportWarning: Not importing directory '/net/rosie/scratch1/rwgk/dist/scons/src/engine/SCons/Tool/CVS': missing __init__.py file, path, desc = imp.find_module(self.name, smpath) /net/rosie/scratch1/rwgk/dist/phenix/phenix/__init__.py:1: ImportWarning: Not importing directory '/net/rosie/scratch1/rwgk/dist/libtbx': missing __init__.py try: import libtbx.forward_compatibility /net/rosie/scratch1/rwgk/dist/phenix/phenix/refinement/__init__.py:1: ImportWarning: Not importing directory '/net/rosie/scratch1/rwgk/dist/iotbx': missing __init__.py import iotbx.phil /net/rosie/scratch1/rwgk/dist/iotbx/iotbx/phil.py:1: ImportWarning: Not importing directory '/net/rosie/scratch1/rwgk/dist/cctbx': missing __init__.py from cctbx import sgtbx /net/rosie/scratch1/rwgk/dist/cctbx/cctbx/array_family/flex.py:1: ImportWarning: Not importing directory '/net/rosie/scratch1/rwgk/dist/scitbx': missing __init__.py import scitbx.array_family.flex /net/rosie/scratch1/rwgk/dist/scitbx/scitbx/array_family/flex.py:2: ImportWarning: Not importing directory '/net/rosie/scratch1/rwgk/dist/boost': missing __init__.py import boost.optional /net/rosie/scratch1/rwgk/dist/libtbx/libtbx/utils.py:226: ImportWarning: Not importing directory '/net/rosie/scratch1/rwgk/dist/mmtbx': missing __init__.py try: module = __import__(module_path) etc. etc. __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com

14 43

Re: [Python-Dev] PEP 328 and PEP 338, redux
by Phillip J. Eby July 1, 2006

July 1, 2006

At 08:08 AM 6/27/2006 -0700, Guido van Rossum wrote: >Bad idea IMO. The __name__ == "__main__" rule is so ingrained, you >don't want to mess with it. Actually, maybe we *do* want to, for this usage. Note that until Python 2.5, it was not possible to do "python -m nested.module", so this change merely prevents *existing* modules from being run this way -- when they could not have been before! So, such modules would require a minor change to run under -m. Is this actually a problem, or is it a new feature?

8 12

Bug in stringobject?
by Georg Brandl July 1, 2006

July 1, 2006

In string_replace, there is if (PyString_Check(from)) { /* Can this be made a '!check' after the Unicode check? */ } #ifdef Py_USING_UNICODE if (PyUnicode_Check(from)) return PyUnicode_Replace((PyObject *)self, from, to, count); #endif else if (PyObject_AsCharBuffer(from, &tmp_s, &tmp_len)) return NULL; [the same check with "to"] return (PyObject *)replace((PyStringObject *) self, (PyStringObject *) from, (PyStringObject *) to, count); Can this be correct if from or to isn't a string object, but a char buffer compatible object? Georg

5 14

Cleanup of test harness for Python
by Frank Wierzbicki July 1, 2006

July 1, 2006

Hello all, According to the thread that includes http://mail.python.org/pipermail/python-dev/2006-June/065727.html there will be some effort in 2.6 to make the tests in Python more consistent. I would like to help with that effort, partly to sneak in some checks for CPython internal tests that should be excluded from Jython, but mainly to understand the future implementation of Python for which the tests provide the only real spec. Which of the current tests is closest to an "ideal" test, so I can use it as a model? Thanks, -Frank Wierzbicki

3 2

traceback regression
by Jim Jewett June 30, 2006

June 30, 2006

python.org/sf/1515343 fixes python.org/sf/1515163, which is a new-in-2.5 regression. On the one hand, the regression only affects >>> raise "string1", "string2" which is both obscure and deprecated. On the other hand, it is a regression, and it is something I bumped into while working with unittest. [Note that I probably won't be checking email again for a week, so I'm afraid I won't be very responsive to comments.] -jJ

1 0

Python memory model (low level)
by Nick Maclaren June 30, 2006

June 30, 2006

I Have been thinking about software floating point, and there are some aspects of Python and decimal that puzzle me. Basically, they are things that are wanted for this sort of thing and seem to be done in very contorted ways, so I may have missed something. Firstly, can Python C code assume no COMPACTING garbage collector, or should it allow for things shifting under its feet? Secondly, is there any documentation on the constraints and necessary ritual when allocating chunks of raw data and/or types of variable size? Decimal avoids the latter. Thirdly, I can't find an efficient way for object-mangling code to access class data and/or have some raw data attached to a class (as distinct from an instance). Fourthly, can I assume that no instance of a class will remain active AFTER the class disappears? This would mean that it could use a pointer to class-level raw data. I can explain why all of those are the 'right' way to approach the problem, at an abstract level, but it is quite possible that Python does not support the abstract model of class implementation that I am thinking of. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1(a)cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679

4 4

Pickle implementation questions
by Bruce Christensen June 30, 2006

June 30, 2006

In developing a cPickle module for IronPython that's as compatible as possible with CPython, these questions have come up: - Where are object.__reduce__ and object.__reduce_ex__ defined, and how does copy_reg._reduce_ex fit into the picture? PEP 307 states that the default __reduce__ implementation for new-style classes implemented in Python is copy_reg._reduce. However, in Python 2.4.3 dir(copy_reg) indicates that it has no _reduce method. (Tangentially, is there a way to inspect a method_descriptor object to determine the function it's bound to?) - When the optional constructor argument is passed to copy_reg.pickle, where is it stored and how is it used by pickle? - What does copy_reg.constructor() do? Thanks! --Bruce

5 9

Re: [Python-Dev] Python memory model (low level)
by Nick Maclaren June 30, 2006

June 30, 2006

"Tim Peters" <tim.peters(a)gmail.com> wrote: [ Many useful answers ] Thanks very much! That helps. Here are a few points where we are at cross-purposes. I am talking about the C level. What I am thinking of is the standard method of implementing the complicated housekeeping of a class (e.g. inheritance) in Python, and the basic operations in C (for efficiency). The model that I would like to stick to is that the Python layer never knows about the actual object implementation, and the C never knows about the housekeeping. The housekeeping would include the class derivation, which would (inter alia) fix the size of a number. The C code would need to allocate some space to store various constants and workspace, shared between all instances of the derived class. This would be accessible from the object it returns. Each instance would be of a length specified by its derivation (i.e. like Decimal), but would be constant for all members of the class (i.e. not like long). So it would be most similar to tuple in that respect. Operations like addition would copy the pointer to the class data from the arguments, and ones like creation would need to be passed the appropriate class and whatever input data they need. I believe that, using the above approach, it would be possible to achieve good efficiency with very little C - certainly, it has worked in other languages. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1(a)cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679

2 1

Re: [Python-Dev] Proposal to eliminate PySet_Fini
by Kristján V. Jónsson June 30, 2006

June 30, 2006

> > That was a purely altruistic proposal. I've already > discovered that sets are finalized and that some code that > works with dict emulating a set may not work with a set. It > will not make much difference for me if my proposal will be > implemented in 2.6 or even in 3.0, but the sooner it will > happen the fewer people will stumble on the same problem that > I did. I also feel that dummy allocated on the heap and the > free set list are complicating the code with no gain. > Can this not be resolved by carefully adjusting the order of finalization? If code can be bootstrapped it can be strootbapped. As a side note, is there a finalization order list for imported modules? Kristján

3 2