cPickle alternative?

[Michael Peuser]

So stupid of me  :-(((

Now here are the benchmarks I got from Drochems dataset. I think it should
sufice to use the binary mode of 2.2. (I checked the 2.3 data on a different
disk the other day - that made them not comparable!! I now use the same disk
for the tests.)

Timings (2.2.2):
Generation of data:   18 secs
Dunping:  3 secs
Loading:  18,5 sec
Filesize: 5,5 MB

Binary dump: 2,4
Binary load:   3
Filesize:  2,8 MB

2.3
Generation of data: 9 secs
Dumping:  2,4
Loading:   2,8


Binary dump:  1
Binary load:   1,9
Filesize:  2,8 MB

Mode 2 dump:  0,9
Mode 2 load:   1,7
Filesize:  2,6 MB

The musch faster time for generating the data in 2.3 could be due to an
improved random generator (?) That had alwys been quite slow..

Kindly
Michael P



"Tim Evans" <t.evans at paradise.net.nz> schrieb im Newsbeitrag
news:87r83l3jmj.fsf at cassandra.evansnet...
> "Michael Peuser" <mpeuser at web.de> writes:
>
> > Hi Drochem,
> >
> > (1) Your dataset seems to break the binary cPickle mode ;-) (I tried it
with
> > the "new Pickle" in 2.3 - same result: "EOF error" when loading back...)
May
> > be there is someone interested in fixing this ....
> [snip]
> > > f=open ("test","w")
> [snip]
> > > f=open ("test")
> [snip]
>
> Note that on windows, you must open binary files using binary mode
> when reading and writing them, like so:
>
>     f = open('test', 'wb')
>     f = open('test', 'rb')
>                      ^^^^
>
> If you don't do this binary data will be corrupted by the automatic
> conversion of '\n' to '\r\n' by win32.  This is very likely what is
> causing the above error.
>
> --
> Tim Evans