Sun Mar 28 21:25:37 CEST 2010

Well, it's been a couple of days, and nobody has stepped forward to 
speak in favor of my aggressive PyCapsule schedule.  So I will bow to 
the wishes of those who have spoken up, and scale it back.  
Specifically, I propose writing a patch that will:

    * allow the CObject API to (unsafely) dereference a capsule
    * add support for the new files added by capsule to the Visual
      Studio builds (fixes an unrelated second complaint about my checkin)
    * add a -3 warning for calls to CObject
    * add a PendingDeprecation warning to CObject

I've gotten in the habit of doing code reviews at work, and I now 
greatly prefer working that way.  Can I get a volunteer to review the 
patch when it's ready?   Please email me directly.  Thanks!

M.-A. Lemburg wrote:
> Just as reminder of the process we have in place for such changes:
> Please discuss any major breakage on python-dev before checking in
> the patch.
>   

I'm aware this is a good idea.  I simply didn't consider this a major 
breakage.  Recompiling against the 2.7 header files fixes it for 
everybody.  (Except external users of pyexpat, if any exist.  Google 
doesn't show any, though this is not proof that they don't exist.)

If you suggest that any breakage with previous versions is worth 
mentioning, no matter how small, then I'll remember that in the future.  
Certainly the Python community has had many thrilling and dynamic 
conversations over minutae, so I guess it wouldn't be that surprising if 
this were true.

>> 1. The CObject API isn't safe.  It's easy to crash Python 2.6 in just a
>> few lines by mixing and matching CObjects.  Switching Python to capsules
>> prevents a class of exploits.  I've included a script at the bottom of
>> this message that demonstrates three such crashes.  The script runs in
>> Python 2 and 3, but 3.1 doesn't crash because it's using capsules.
>>     
>
> That's good, but then again: deliberate wrong use of APIs will
> always cause crashes and at least I don't know of any report about
> PyCObjects posing a problem in all their years of existence.
>   
> [...] we've been happy with this vulnerability for years, just
> as we've been happy with the fact that it's easy to crash the
> VM by passing hand-crafted byte-code to it, or using ctypes to
> call an OS function with the wrong parameters, etc.
>   

I'm surprised you find that acceptable.  I thought the community was in 
agreement: our goal is to make the interpreter uncrashable from pure Python.

I would describe ctypes as "impure" Python.  If you wanted to allow 
untrusted developers to run code through a Python interpreter, you'd 
harden it: take away ctypes, and the ability to feed in arbitrary 
bytecode, and reduce the stack limit.  You probably wouldn't take away 
"datetime", "cStringIO", and "cPickle"--yet I can crash 2.6 using those.

>> Third, I've been pondering writing a set of preprocessor macros, shipped
>> in their own header file distributed independently of Python and
>> released to the public domain, that would make it easy to use either
>> CObjects or capsules depending on what version of Python you were
>> compiling against.  Obviously, using these macros would require a source
>> code change in the third-party library.  But these macros would make it
>> a five-minute change.  This could compliment the first or second
>> approaches.
>>     
> I'm not sure whether that would worth the effort. 3rd party modules
> for Python 2.x are likely not going to use them, since they typically
> have to support more than just Python 2.7.
>   

The point of these macros would be to support older versions of Python.  
The macros would boil down to either PyCapsule or CObject calls, 
depending on what version of Python you were compiling against.  For 
Python 2.6 and before it would alway use CObjects.  One set of macros 
would switch to PyCapsule when building against 2.7; another would use 
CObject permanently in 2.x.  (Or maybe it should switch in 2.8?  2.9?  
But now we're in the realm of speculative fiction.)

I'd be happy to write these if there was demand, but my sneaking 
suspicion is that nobody would use them.  If you're genuinely interested 
in these macros please email me privately.

/larry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100328/d888c7a6/attachment-0001.html>