[Python-Dev] Python 3.4, marshal dumps slower (version 3 protocol)

[tds333 at gmail.com]

On 28.01.2014 10:23, Barry Warsaw wrote:
> On Jan 28, 2014, at 09:17 AM, tds333 at gmail.com wrote:
>
>> yes I know the main usage is to generate pyc files. But marshal is also used
>> for other stuff and is the fastest built in serialization method. For some
>> use cases it makes sense to use it instead of pickle or others. And people
>> use it not only to generate pyc files.
> marshall is not guaranteed to be backward compatible between Python versions,
> so it's generally not a good idea to use it for serialization.
>

Yes I know. And because of that I use it only if nothing persists and 
the exchange is between
the same Python version (even the same architecture and Interpreter type).
But there are use cases for inter process communication with no 
persistence and no need
to serialize custom classes and so on. And if speed matters and security 
is not the problem
you use the marshal module to serialize data.

Assume something like multiprocessing for Windows (no fork available) 
and only a pipe to exchange
a lot of simple data and pickle is to slow. (Sometimes distributed to 
other computers.)

Another use case can be a persistent cache with ultra fast serialization 
(dump/load) needs but
not with critical data normally stored in a database. Can be regenerated 
easily if Python version
changes from main data. (think pyc files are such a use case)

I have tested a lot of modules for some needs (JSON, Thrift, 
MessagePack, Pickle, ProtoBuffers, ...)
all are very useful and has their usage scenario.
The same applies to marshal if all the limitations are no problem for you.
(I've read the manual and have some knowledge about the limitations)

But all these serialization modules are not as fast as marshal. (for my 
use case)

I hear you and registered the warning about this. And will not complain 
if something will be incompatible. :-)

If someone knows something faster to serialize basic Python types. I'm 
glad to use it.


Regards,

Wolfgang