MemoryError in data conversion
Peter Otten
__peter__ at web.de
Mon Apr 14 09:59:48 EDT 2014
Mok-Kong Shen wrote:
> Am 14.04.2014 09:46, schrieb Peter Otten:
>
>> You ran into a limitation of the compiler. For us to suggest a workaround
>> you'd have to explain why you want to convert the list returned from
>> buildhuffmantree() into python source code and back.
>
> That list gives the Huffman encoding tree for compressing a given piece
> of source text. I am writing a Python code to implement an algorithm
> (not new, being first sketched in the literature since decades but yet
> having no publically available implementation as far as I am aware) of
> encryption processing that has Huffman data compression as its major
> constituent. Now, for good security against cryptanalysis, this list
> (which has to be included in the ciphertext for decryption by the
> recipient) has to be well scrambled in some way. I choose to use 8-bit
> bytes as units for the scrambling. Hence I convert the list to a
> bytearray for performing scrambling. On decryption I reverse the
> scrambling and get back the original bytearray and use ast to recover
> from it the list so as to be able to do the decompression. Hopefully
> this description is sufficiently clear.
You could use json, but you may run into the same problem with that, too
(only later):
>>> import json
>>> items = []
>>> for i in range(1000):
... s = json.dumps(items)
... items = [items]
...
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
File "/usr/lib/python3.3/json/__init__.py", line 236, in dumps
return _default_encoder.encode(obj)
File "/usr/lib/python3.3/json/encoder.py", line 191, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/usr/lib/python3.3/json/encoder.py", line 249, in iterencode
return _iterencode(o, 0)
RuntimeError: maximum recursion depth exceeded while encoding a JSON object
>>> i
995
The safest option is probably to serialize the original flist and slist, and
use them to create the tree on the fly.
More information about the Python-list
mailing list