object references/memory access
nagle at animats.com
Tue Jul 3 19:50:05 CEST 2007
> Paul Rubin wrote:
>>dlomsak <dlomsak at gmail.com> writes:
>>>knowledge of the topic to help. If the above are not possible but you
>>>have a really good idea for zipping large amounts of data from one
>>>program to another, I'd like to hear it.
> Well, I was using the regular pickle at first but then I switched to
> just using repr() / eval() because the resulting string doesn't have
> all the extra 's1=' and all that so it cuts down on the amount of data
> I have to send for large returns when you cut out all of that
> formatting. The speed of the above method is pretty high even for
> really large returns and it works fine for a list of dictionaries.
OK, that's where the time is going. It's not the interprocess
communication cost, it's the marshalling cost. "repr/eval" is not
an efficient way to marshall. Try using "pack" and "unpack", if
you control both ends of the connection.
More information about the Python-list