Python-Dev June 2007

python-dev@python.org

66 participants
76 discussions

What's going on with the check-in emails?

by Alexandre Vassalotti

Hi, It seems there is a problem with check-in emails -- i.e., none have been sent since r56057 (and the svn tree is at r56098 right now). Does someone has a hint what's going on? Thanks, -- Alexandre

13 years, 6 months

Re: [Python-Dev] object capability; func_closure; __subclasses__

by Phillip J. Eby

At 05:23 PM 6/28/2007 +0100, tav wrote: >Any pointers on removing members via ctypes front? > >Whilst I can understand even the most obscure aspects of your python >code fine, I'm not familiar with C/ctypes... What you want is to get access to the type's real dictionary, not the proxy. Then you can just delete '__subclasses__' from the dictionary using Python code. Here's some code that does the trick: from ctypes import pythonapi, POINTER, py_object getdict = pythonapi._PyObject_GetDictPtr getdict.restype = POINTER(py_object) getdict.argtypes = [py_object] def dictionary_of(ob): dptr = getdict(ob) if dptr and dptr.contents: return dptr.contents.value 'dictionary_of' returns either a dictionary object, or None if the object has no dictionary. You can then simply delete any unwanted contents. However, you should *never use this* to assign __special__ methods, as Python will not change the type slots correctly. Heck, you should probably never use this, period. :) Usage example: print "before", type.__subclasses__ del dictionary_of(type)['__subclasses__'] print "after", type.__subclasses__ This will print something like: before <method '__subclasses__' of 'type' objects> after Traceback (most recent call last): File "ctypes_dicto.py", line 14, in <module> print "after", type.__subclasses__ AttributeError: type object 'type' has no attribute '__subclasses__' et voila. You should also be able to delete unwanted function type attributes like this:: from types import FunctionType del dictionary_of(FunctionType)['func_closure'] del dictionary_of(FunctionType)['func_code'] Of course, don't blame me if any of this code fries your computer and gives you a disease, doesn't work with new versions of Python, etc. etc. It works for me on Windows and Linux with Python 2.3, 2.4 and 2.5. It may also work with 3.0, but remember that func_* attributes have different names there.

13 years, 6 months

Re: [Python-Dev] object capability; func_closure; __subclasses__

by Phillip J. Eby

At 10:20 AM 6/28/2007 -0700, Robert Brewer wrote: >tav wrote: > > But, all I am asking for is to not expose func_closure (and perhaps > > some of the other func_*) as members of FunctionType -- isn't it > > possible to add functionality to the ``new`` module which would allow > > one to read/write func_closure? > >Would func_closure then also be removed from the FunctionType >constructor arg list? That wouldn't be necessary, since restricted code is probably not going to be allowed access to new in the first place. We're talking about removing read access to the closure *attribute* only. (And write access to func_code would also have to be removed, else you could replace the bytecode in order to grant yourself read access to the closure contents...) >If so, would I be expected to create a function >object and then use the "new" module to supply its func_closure? Nope. The idea here is that the new module would grow utility functions like get_closure, get_code, set_code, get_subclasses, etc. The 'inspect' module would then use these functions to do its job, and I would use them for generic function stuff.

13 years, 6 months

Return error codes from getaddrinfo.

by Alan Kennedy

Dear all, I'm seeking enlightenment on the error codes returned by the socket.getaddrinfo() function. Consider the following on python 2.5 on Windows >>> import urllib >>> urllib.urlopen("http://nonexistent") [snip traceback] IOError: [Errno socket error] (11001, 'getaddrinfo failed') So the error number is 11001. But when I try to find a symbolic constant in the errno module corresponding to this error number, I can't find one. >>> import errno >>> errno.errorcode[11] 'EAGAIN' >>> errno.errorcode[11001] Traceback (most recent call last): File "<stdin>", line 1, in <module> KeyError: 11001 Looking through the C source for the socket module doesn't provide any clarity (although my C is a little rusty). That module has a special function, set_gaierror(), for handling error returns from getaddrinfo. But I can't see if or how the resulting error codes relate to the errno module. Is there supposed to be symbolic constants in the errno module corresponding to getaddrinfo errors? I want jython to use the same errno symbolic constants as cpython, to ease portability of code. Regards, Alan.

13 years, 6 months

Re: [Python-Dev] object capability; func_closure; __subclasses__

by Phillip J. Eby

At 02:04 AM 6/28/2007 +0100, tav wrote: >rehi all, > >I have been looking at the object capability + Python discussions for >a while now, and was wondering what the use cases for >``object.__subclasses__`` and ``FunctionType.func_closure`` were? > >I don't see __subclasses__ used anywhere in the standard library. And >whilst I can see exposing func_closure as being useful in terms of >"cloning/modifying" an existing function, isn't it possible to do that >without making it introspectable? You know, I find it particularly interesting that, as far as I can tell, nobody who proposes making changes to the Python language to add security, ever seems to offer any comparison or contrast of their approaches to Zope's -- which doesn't require any changes to the language. :) >Now, whilst probably wrong, I can see myself being able to create a >minimal object capability system in pure python if those 2 "features" >disappeared. Am I missing something obvious that prevents me from >doing that? Well, you're missing a simpler approach to protecting functions, anyway. The '__call__' attribute of functions is still callable, but doesn't provide any access to func_closure, func_code, etc. I believe this trick also works for bound method objects. I suspect that you could also use ctypes to remove or alter the type.__subclasses__ member, though I suppose you might not consider that to be "pure" Python any more. However, if you use a definition of pure that allows for stdlib modules, then perhaps it works. :) >Can we get rid of them for Python 2.6? Or even 2.5.2? Is anyone >besides PJE actually using them? ;p I wouldn't object (no pun intended) to moving the type.__subclasses__ method to say, the 'new' or 'gc' modules, since you wouldn't want to make those available to restricted code, but then they'd still be available for folks who need/want them. 'gc' has similar capabilities (again no pun intended) anyway. However, ISTM that this should be a 3.0 change rather than a 2.x one, even so. With regard to the func_closure thing, I'd actually like to make it *writable* as well as readable, and I don't mean just to change the contents of the cells. But, since you can use .__call__ to make a capability without access to func_closure, it doesn't seem like you really need to remove func_closure.

13 years, 6 months

Re: [Python-Dev] object capability; func_closure; __subclasses__

by Phillip J. Eby

At 04:14 PM 6/28/2007 +0100, tav wrote: > > Well, there's no __self__ in 2.3 or 2.4; I guess that was added > in 2.5. Darn. > >anyone know *why* it was added? > > > >Or, setting __call__.__doc__ ? > > > > What does that do? > >ah, i just wanted a way of providing documentation, and __call__'s >__doc__ isn't writable... > > > If it works, you could probably do the same thing to remove > __call__.__self__. > >will look into that too... > > > In 3.0, I don't mind if the access method moves, I just want to keep > > the access. OTOH, I don't really care about __call__.__self__, since > > I got along fine without it in 2.3/2.4 and didn't know it had been > > added in 2.5. :) > >w00p! > >so, suggestions as to how one can go about getting those 2 access >methods moved? Post a proposal on the Python-3000 list and supply patches to do the moves.

13 years, 6 months

Re: [Python-Dev] object capability; func_closure; __subclasses__

by Phillip J. Eby

At 01:09 PM 6/28/2007 +0100, tav wrote: >>You know, I find it particularly interesting that, as far as I can >>tell, nobody who proposes making changes to the Python language to >>add security, ever seems to offer any comparison or contrast of their >>approaches to Zope's -- which doesn't require any changes to the >>language. :) > >Whilst it is definitely possible to build up a object capability >system with a pruned down version of Zope 3's proxy + checker >mechanism, it ends up in a system which is quite performance >intensive. All those proxies being wrapped/unwrapped/checked... > >In contrast, the mechanism I am looking at here simply requires >*moving* just 2 attributes *out* of the core builtins... > >Quite cheap, simple and effective, no? > >>Well, you're missing a simpler approach to protecting functions, >>anyway. The '__call__' attribute of functions is still callable, but >>doesn't provide any access to func_closure, func_code, etc. I >>believe this trick also works for bound method objects. > >Whilst that would have been a nice trick, what about __call__.__self__ ? Well, there's no __self__ in 2.3 or 2.4; I guess that was added in 2.5. Darn. >Or, setting __call__.__doc__ ? What does that do? >>I suspect that you could also use ctypes to remove or alter the >>type.__subclasses__ member, though I suppose you might not consider >>that to be "pure" Python any more. However, if you use a definition >>of pure that allows for stdlib modules, then perhaps it works. :) > >Ah, thanks! Will look into that. If it works, you could probably do the same thing to remove __call__.__self__. >I don't object to making func_closure writable either. In fact, as >someone who has been following your work on generic functions from way >before RuleDispatch, I really want to see PEP 3124 in 3.0 > >But, all I am asking for is to not expose func_closure (and perhaps >some of the other func_*) as members of FunctionType -- isn't it >possible to add functionality to the ``new`` module which would allow >one to read/write func_closure? In 3.0, I don't mind if the access method moves, I just want to keep the access. OTOH, I don't really care about __call__.__self__, since I got along fine without it in 2.3/2.4 and didn't know it had been added in 2.5. :)

13 years, 6 months

csv changed from python 2.4 to 2.5

by Christian K

Hi, I could not find documentation of the following change in python2.5. What is the reason for that? Python 2.4.4 (#2, Apr 12 2007, 21:03:11) [GCC 4.1.2 (Ubuntu 4.1.2-0ubuntu4)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import csv >>> d=csv.get_dialect('excel') >>> d.delimiter = ',' >>> ck@kiste:/media/hda6/home/ck/prog/peak-o-mat/trunk$ python2.5 Python 2.5.1 (r251:54863, May 2 2007, 16:56:35) [GCC 4.1.2 (Ubuntu 4.1.2-0ubuntu4)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import csv >>> d=csv.get_dialect('excel') >>> d.delimiter = ',' Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: readonly attribute >>> the following however works: Python 2.5.1 (r251:54863, May 2 2007, 16:56:35) [GCC 4.1.2 (Ubuntu 4.1.2-0ubuntu4)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import csv >>> d = csv.excel >>> d.delimiter = ',' >>> Christian

13 years, 6 months

object capability; func_closure; __subclasses__

by tav

rehi all, I have been looking at the object capability + Python discussions for a while now, and was wondering what the use cases for ``object.__subclasses__`` and ``FunctionType.func_closure`` were? I don't see __subclasses__ used anywhere in the standard library. And whilst I can see exposing func_closure as being useful in terms of "cloning/modifying" an existing function, isn't it possible to do that without making it introspectable? Years ago, Ka-Ping Yee pointed out: http://mail.python.org/pipermail/python-dev/2003-March/034284.html Derived from this we get: # capability.py functions def Namespace(*args, **kwargs): for arg in args: kwargs[arg.__name__] = arg def get(key): return kwargs.get(key) return get class Getter(object): def __init__(self, getter): self.getter = getter def __repr__(self): return self.getter('__repr__') or object.__repr__(self) def __getattr__(self, attr): return self.getter(attr) # io.py module def FileReader(name): file = open(name, 'r') def __repr__(): return '<FileReader: %r>' % name def read(bufsize=-1): return file.read(bufsize) def close(): return file.close() return Getter(Namespace(__repr__, read, close)) ---- Now, a process A -- which has full access to all objects -- can do: >>> motd = FileReader('/etc/motd') And pass it to "process B" operating in a limited scope, which can then call: >>> motd.read() >>> motd.close() But not: >>> motd = type(motd)(motd.name, 'w') which would have been possible *had* motd been created as a ``file`` type by calling: ``open('/etc/motd', 'r')``. Now, there are probably a million holes in this approach, but as long as process B's __import__ is sanitised and it operates in a "limited" scope with regards to references to other functionality, this seems to be relatively secure. However, this is where __subclasses__ and func_closure get in the way. With object.__subclasses__ (as Brett points out), all defined classes/types are available -- including the ``file`` type we were trying to deny process B access to! Is it necessary to expose this attribute publically? And, likewise with func_closure, one can do motd.read.func_closure[0].cell_contents and get hold of the original ``file`` object. Is it absolutely necessary to expose func_closure in this way? Now, whilst probably wrong, I can see myself being able to create a minimal object capability system in pure python if those 2 "features" disappeared. Am I missing something obvious that prevents me from doing that? Can we get rid of them for Python 2.6? Or even 2.5.2? Is anyone besides PJE actually using them? ;p Thanks in advance for your thoughts. -- love, tav founder and ceo, esp metanational llp plex:espians/tav | tav(a)espians.com | +44 (0) 7809 569 369

13 years, 6 months

handling the granularity of possible Py3K warnings in 2.6

by Brett Cannon

My rewrite of import is written for 2.6. But I am going to try to bootstrap it into 3.0. But I want to bootstrap into 2.6 if it works out well in 3.0. That means coding in 2.6 but constantly forward-porting to 3.0. In other words I am going to be a guinea pig of our transition plan. And being the guinea pig means I want to know where we want to take the Py3K warnings in 2.6. As of right now the -3 option causes some DeprecationWarnings to be raised (mostly about callable and dict.has_key). But I was thinking that a certain level of granularity existed amongst what should be warned against. There is syntactic vs. semantic. For syntactic there is stuff you can do in versions of Python older than 2.6 (e.g., backtick removal), stuff you can only do in 2.6 (e.g., new exception syntax), or stuff that you can't do at all without a __future__ statement (e.g., 'print'). For semantics, there is removal (e.g., callable), and then there is new semantics (dict.items). What I was thinking was having these various types of changes be represented by proper warnings. That allows for using the -W option to control what warnings you care about. So I envision something like: + Py3KWarning + Py3KSyntaxWarning + some reasonable name for stuff that can be done in 2.6. + some name for stuff that can be done in older than 2.6. + something for stuff like the 'print' removal that require a __future__ statement. + Py3KSemanticWarning + Py3KDeprecationWarning + Py3KChangedSemanticsWarning (or whatever name you prefer) The key point is that when I am forward-porting I want to easily tell what syntax changes I can deal with now in older, e.g. 2.5 code, stuff that I can change directly in 2.6, and stuff that requires 2to3 or a __future__ statement. Similar idea for semantic changes. That way I can do this all in steps. Does this sound reasonable to people at all? -Brett

13 years, 6 months

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

Python-Dev June 2007