sys.argv as a list of bytes

[Peter Otten]

Olive wrote:

> In Unix the operating system pass argument as a list of C strings. But
> C strings does corresponds to the bytes notions of Python3. Is it
> possible to have sys.argv as a list of bytes ? What happens if I pass
> to a program an argumpent containing funny "character", for example
> (with a bash shell)?
> 
> python -i ./test.py $'\x01'$'\x05'$'\xFF'

Python has a special errorhandler, "surrogateescape" to deal with bytes that are not 
valid UTF-8. If you try to print such a string you get an error:

$ python3 -c'import sys; print(repr(sys.argv[1]))' $'\x01'$'\x05'$'\xFF'
'\x01\x05\udcff'
$ python3 -c'import sys; print(sys.argv[1])' $'\x01'$'\x05'$'\xFF'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
UnicodeEncodeError: 'utf-8' codec can't encode character '\udcff' in position 2: surrogates not allowed

It is still possible to get the original bytes:

$ python3 -c'import sys; print(sys.argv[1].encode("utf-8", "surrogateescape"))' $'\x01'$'\x05'$'\xFF'
b'\x01\x05\xff'