Load a list subset with pickle?

[Peng Yu]

On Tue, Oct 13, 2009 at 1:23 PM, Robert Kern <robert.kern at gmail.com> wrote:
> On 2009-10-13 13:00 PM, Peng Yu wrote:
>>
>> I use pickle to dump a long list. But when I load it, I only want to
>> load the first a few elements in the list. I am wondering if there is
>> a easy way to do so? Thank you!
>
> Not by pickling the list. However, you can concatenate pickles, so you could
> just pickle each item from the list in order to the same file and only
> unpickle the first few.
>
> In [1]: import cPickle
>
> In [2]: from cStringIO import StringIO
>
> In [3]: very_long_list = range(10)
>
> In [4]: f = StringIO()
>
> In [5]: for item in very_long_list:
>   ...:     cPickle.dump(item, f)
>   ...:
>   ...:
>
> In [6]: f.seek(0,0)
>
> In [7]: cPickle.load(f)
> Out[7]: 0
>
> In [8]: cPickle.load(f)
> Out[8]: 1
>
> In [9]: cPickle.load(f)
> Out[9]: 2
>
> In [10]: cPickle.load(f)
> Out[10]: 3

How do I determine if I have loaded all the elements? I use the
following code. I'm wondering if there is any better solution than
this.


###############
import pickle

alist = [1, 2.0, 3, 4+6j]

output=open('serialize_list.output/serialize_list.pkl', 'wb')
for e in alist:
  pickle.dump(e, output)
output.close()

input=open('serialize_list.output/serialize_list.pkl', 'rb')

try:
  while 1:
    e = pickle.load(input)
    print e
except EOFError:
  pass