[Tutor] Serializing bytestring as JSON

[Cameron Simpson]

On 10Aug2020 05:04, S D <zebra05 at gmail.com> wrote:
>I built an RPC service based on the Nameko framework. The service class
>implements a method which takes a list of strings as an argument and
>returns a dictionary with each original string as a key and the string
>Huffman encoded as a value. The encoding works, but I cannot send the
>return value across the network because the encoded bytestrings cannot be
>serialized to JSON.
>
>If I import the service class and test the methods using PyTest, this
>problem is not reproduced.
>
>How can I return a Huffman encoded bytestring over a wire as JSON?

Aren't Huffman codes variable bit lengths?

But anyway, presuming you've got bytes you have a few choices. The basic 
deal is that you need to turn the values into a type which can be JSON 
serialised, so a string seems simplest. But any pair of reversible 
functions would do, converting bytes to something JSON serialisable, and 
the reverse.

So you want something like:

    # original dict with str->bytes mapping
    src_dict = { 'a': b'zzzz', ... }
    jsonable_dict = { k: bytes_as_str(v) for k, v in src_dict.items() }
    ... send JSON now ...

    ... receive some JSON ...
    # a received JSON dict with b'zzzz' expressed as some string
    serialised_dict = { 'a': 'bsbsbsbsbs', ...}
    # decode the strings back into original bytes
    dst_dict = { k: str_as_bytes(v) for k, v in serialised_dict.items() }

So you just need to define your bytes_as_str() and str_as_bytes() 
functions.

The simplest thing might be to pretend the bytes are in fact the binary 
transcription of some 8-bit character set. Such as 'iso8859-1'. AN 8 bit 
character set will neatly map 1-to-1 into a byte, since a byte is 8 
bits. It doesn't matter what character set we choose provided that its 
encode and decode functions put a single character into a single byte 
and vice versa.

So you might make:

    def bytes_as_str(bs):
        return bs.decode('iso8859-1')

and the converse str_as_bytes() to reverse it.

Cheers,
Cameron Simpson <cs at cskk.id.au>