Segmenting a pickle stream without unpickling

Paul Rubin http
Fri May 19 11:04:55 CEST 2006

Boris Borcic <bborcic at> writes:
> def pickles(my_stream) :
>      from cPickle import load,dumps
>      while 1 :
>          yield dumps(load(my_stream))
> without the overhead associated with unpickling objects
> just to pickle them again ?

I think you'd have to write something special.  The unpickler parses
as it goes along, and all the dispatch actions build up objects.
You'd have to write a set of actions that just read past the
representations.  I think there's no way to know where an object ends
without parsing it, including parsing any objects nested inside it.

More information about the Python-list mailing list