[Twisted-Python] AmpList: length limitations
Hi all! Stuck with AmpList again :( Im trying to send/retrieve data much larger then 64K and choose AmpList cause it "Convert a list of dictionaries into a list of AMP boxes". My original data looks like a long list of dicts [ {1}, {2}, ..., {x} ]. I convert it to [ {'data': 'json encoded 1,2 dicts'}, {'data': 'json encoded 3,4 dicts}', ... ] hoping EACH element in converted list will be converted to AmpBox and retriever on wire. However, still got TooLong exception on AmpBox.serialize(). AmpBox trying to serialize all my converted list but not each dict in it separately. Does this behavior ok, or this is a bug? -- WBR, Vlad
On 03:54 pm, vlad.shevchenko@gmail.com wrote:
Hi all!
Stuck with AmpList again :( Im trying to send/retrieve data much larger then 64K and choose AmpList cause it "Convert a list of dictionaries into a list of AMP boxes".
My original data looks like a long list of dicts [ {1}, {2}, ..., {x} ]. I convert it to [ {'data': 'json encoded 1,2 dicts'}, {'data': 'json encoded 3,4 dicts}', ... ] hoping EACH element in converted list will be converted to AmpBox and retriever on wire.
However, still got TooLong exception on AmpBox.serialize(). AmpBox trying to serialize all my converted list but not each dict in it separately.
Does this behavior ok, or this is a bug?
When you use an AmpList, the value is still serialized as a single key's value string, so the 64k limit still applies, even if it has sub- structures of its own, none of which exceed the limit. At the Twisted sprint hosted by Flumotion last weekend, some people were working on adding real streaming support to AMP. I think most of the progress made was in understanding how the implementation should work, though. The associated ticket is #2529.
-- WBR, Vlad
Why does not serialize each AmpBox separately? On Thu, Apr 1, 2010 at 12:45 AM, <exarkun@twistedmatrix.com> wrote:
On 03:54 pm, vlad.shevchenko@gmail.com wrote:
Hi all!
Stuck with AmpList again :( Im trying to send/retrieve data much larger then 64K and choose AmpList cause it "Convert a list of dictionaries into a list of AMP boxes".
My original data looks like a long list of dicts [ {1}, {2}, ..., {x} ]. I convert it to [ {'data': 'json encoded 1,2 dicts'}, {'data': 'json encoded 3,4 dicts}', ... ] hoping EACH element in converted list will be converted to AmpBox and retriever on wire.
However, still got TooLong exception on AmpBox.serialize(). AmpBox trying to serialize all my converted list but not each dict in it separately.
Does this behavior ok, or this is a bug?
When you use an AmpList, the value is still serialized as a single key's value string, so the 64k limit still applies, even if it has sub- structures of its own, none of which exceed the limit.
At the Twisted sprint hosted by Flumotion last weekend, some people were working on adding real streaming support to AMP. I think most of the progress made was in understanding how the implementation should work, though.
The associated ticket is #2529.
-- WBR, Vlad
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
-- WBR, Vlad Shevchenko
participants (2)
-
exarkun＠twistedmatrix.com -
Vlad Shevchenko