[Python-ideas] RFC: bytestring as a str representation [was: a new bytestring type?]

INADA Naoki songofacandy at gmail.com
Wed Jan 8 12:31:10 CET 2014


On Wed, Jan 8, 2014 at 7:34 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:

> On Wed, 8 Jan 2014 09:50:30 +0900
> INADA Naoki <songofacandy at gmail.com>
> wrote:
> >
> > textdata = b"hello"
>
> textdata shouldn't be a bytes object! If it's text it's a str.
>
>
PyMySQL and MySQL-python supports both of unicode text and encoded text.
So bytes may be text in MySQL if it inserted into TEXT or VARCHAR column.



> > bindata = b"abc\xff\x00"
> > query = "UPDATE table SET textcol=%s bincol=%s"
> >
> > print build_query(query, textdata, bindata)
> >
> >
> > I can't port this to Python 3.
>
> I'm sure you can port it. Just decode your bindata using
> surrogateescape:
>
>   bindata = bindata.decode('utf8', 'surrogateescape')
>
> and then encode the query at the end:
>
>   query = query.encode('utf8', 'surrogateescape')
>
> It will be a little slower, though.
>

You're right. I've not considered using surrogateescape here.

But MySQL connection may be not utf8. It's default latin1 and you can use
many encoding.
Some encoding doesn't ensure roundtrip. In such encoding,

bindata = bindata.decode('sjis', 'surrogateescape')
query = query % bindata
query.encode('sjis', 'surrogateescape')

may break bindata.

I may be able to ascii for decoding when mysql uses ascii compatible
encoding.

bindata = bindata.decode('ascii', 'surrogateescape')
query = query % bindata
query.encode('sjis', 'surrogateescape')

But I think decode/encode with surrogateescape is not only slow, but also
dangerous when using
encoding except ascii or utf8.



>
> Regards
>
> Antoine
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 
INADA Naoki  <songofacandy at gmail.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140108/047f5605/attachment.html>


More information about the Python-ideas mailing list