[Python-3000] [Python-Dev] inst_persistent_id

[Jim Fulton]

On Jan 23, 2008, at 4:30 PM, Alexandre Vassalotti wrote:

> On Jan 22, 2008 10:30 AM, Jim Fulton <jim at zope.com> wrote:
>>
>> On Jan 20, 2008, at 10:39 PM, Alexandre Vassalotti wrote:
>>
>>> On Jan 20, 2008 4:14 PM, Jim Fulton <jim at zope.com> wrote:
>>>>
>>>> Interesting. Does that mean that cPickle can be extended through
>>>> inheritance?
>>>>
>>>
>>> Well, Pickler and Unpickler can be subclassed. I am not sure they  
>>> can
>>> be really "extended," since, unlike the Python version, the C  
>>> version
>>> of Pickler and Unpickler only exposes a minimal API.
>>
>> Then I don't understand what you mean by:
>>
>> "I removed the differences between the Python and C implementation of
>> pickle, and thus allowing the C version to be transparently used,
>> instead of the Python one, when it is available."
>>
>> People *do* routinely subclass pickle.Pickler and pickle.Unpickler  
>> and
>> if this isn't possible with cPickle, then it can't be substituted for
>> pickle.  If making cPickle.Pickler and cPickle.Unpickler subclassible
>> in the same way as the pickle classes makes cPickle much slower, then
>> I don't think the "transparency" is worth the tradeoff.
>>
>
> I am not sure what you mean by "cPickle.Pickler and cPickle.Unpickler
> subclassible in the same way as the pickle classes." It is possible to
> subclass the C implementation. However, the C implementation does not
> expose pickle "private" methods, such as Pickler.save_int and
> Unpickler.load_tuple. This is fine because these were never documented
> as part of the interface of pickle.

That doesn't mean that they aren't overridden.  I've overridden them  
in specialized applications.  I suspect that others have as well.


> Only Pickler.dump,
> Pickler.clear_memo and Unpickler.load are supported by the Python
> implementation. All these are supported by the C implementation as
> well.

How about find_class?  I actually prefer cPickle's way of handling this.

> So, what kind of "extensibility" are you looking for the C
> implementation of pickle?

I'm not really looking for anything that isn't there already.  The  
python pickle version is more extensible that cPickle and I find that  
valuable.  I don't want to make cPickle more flexible.  I'd willing to  
trade off speed and flexibility depending on the application.  I don't  
want to lose either in the interest of unification.

> Quick note, one of the original motives for making the C
> implementation of pickle transparent was to make it more "friendly"
> for alternate implementations of Python (e.g., Jython, IronPython,
> etc).

I don't know what that means.  I don't know that Jython or IronPython  
need to support cPickle. Honestly, I'd be happy to see a *much*  
smaller standard library and, IMO, the standard library doesn't need  
to include pickle or cPickle.

Having said that, I would agree that there are certain aspects of  
their external APIs that ought to be unified.  A good example is  
handling of globals.


>>> Well, I haven't changed cPickle much to make it subclassable. So,  
>>> the
>>> impact is minimal. Currently, the only performance change is due the
>>> removal of the special-cases for PyFile and cStringIO "buffers."
>>
>> What is the benefit of removing this special casing? What is the  
>> cost?
>>
>
> As far as know, most of PyFile is gone in Py3k -- i.e., it been
> castrated beyond recognition [1]. Therefore, the special case for
> PyFile didn't improve the performance (in fact, it probably made
> things go slower).

I'll have to skeptically take your work for it.  I take this to mean  
that there is no C API to write to file objects or their underlying  
files.  If there is a C API, then I'd be surprised if it wasn't faster  
than going though a Python API.  My initial skepticism of this change  
was along the lines of "if it aint broke, don't fix it".  It sounds  
like it was broke, so it makes sense to rip it out and only re- 
implement it if it makes sense later.

> I removed the special case for cStringIO for
> similiar reason -- i.e., cStringIO will be replaced by the respective
> C implementations of io.BytesIO and io.StringIO. I also did it because
> it allowed me to keep the part of pickle that aren't speed critical
> written in Python, thus cleaning out almost 1000 lines of code. The
> performance cost of this is minimal and constant.


OK, for the same reasons.

Jim

--
Jim Fulton
Zope Corporation