[Python-Dev] RFC: PEP 460: Add bytes % args and bytes.format(args) to Python 3.5

[Nick Coghlan]

On 12 January 2014 01:15, M.-A. Lemburg <mal at egenix.com> wrote:
> On 11.01.2014 14:54, Georg Brandl wrote:
>> Am 11.01.2014 14:49, schrieb Georg Brandl:
>>> Am 11.01.2014 10:44, schrieb Stephen Hansen:
>>>
>>>> I mean, its not like the "bytes" type lacks knowledge of the subset of bytes
>>>> that happen to be 7-bit ascii-compatible and can't perform text-ish operations
>>>> on them--
>>>>
>>>>   Python 3.3.3 (v3.3.3:c3896275c0f6, Nov 18 2013, 21:18:40) [MSC v.1600 32 bit
>>>> (Intel)] on win32
>>>>   Type "help", "copyright", "credits" or "license" for more information.
>>>>   >>> b"stephen hansen".title()
>>>>   b'Stephen Hansen'
>>>>
>>>> How is this not a practical recognition that yes, while bytes are byte streams
>>>> and not text, a huge subset of bytes are text-y, and as long as we maintain the
>>>> barrier between higher characters and implicit conversion therein, we're fine?
>>>>
>>>> I don't see the difference here. There is a very real, practical need to
>>>> interpolate bytes. This very real, practical need includes the very real
>>>> recognition that converting 12345 to b'12345' is not something weird, unusual,
>>>> and subject to the thorny issues of Encodings. It is not violating the doctrine
>>>> of separation of powers between Text and Bytes.
>>>
>>> This. Exactly. Thanks for putting it so nicely, Stephen.
>>
>> To elaborate: if the bytes type didn't have all this ASCII-aware functionality
>> already, I think we would have (and be using) a dedicated "asciistr" type right
>> now.  But it has the functionality, and it's way too late to remove it.
>
> I think we need to step back a little from the purist view
> of things and give more emphasis on the "practicality beats
> purity" Zen.
>
> I complete agree with Stephen, that bytes are in fact often
> an encoding of text. If that text is ASCII compatible, I don't
> see any reason why we should not continue to expose the C lib
> standard string APIs available for text manipulations on bytes.
>
> We don't have to be pedantic about the bytes/text separation.
> It doesn't help in real life.

Yes, it bloody well does. The number of people who have told me that
using Python 3 is what allowed them to finally understand how Unicode
works vastly exceeds the number of wire protocol and file format devs
that have complained about working with binary formats being
significantly less tolerant of the "it's really like ASCII text"
mindset.

We are NOT going back to the confusing incoherent mess that is the
Python 2 model of bolting Unicode onto the side of POSIX:
http://python-notes.curiousefficiency.org/en/latest/python3/questions_and_answers.html#what-actually-changed-in-the-text-model-between-python-2-and-python-3

While that was an *expedient* (and, in fact, necessary) solution at
the time, the fact it is still thoroughly confusing people 13 years
later shows it is not a *comprehensible* solution.

> If you give programmers the choice they will - most of the time -
> do the right thing. If you don't give them the tools, they'll
> work around the missing features in a gazillion different
> ways of which many will probably miss a few edge cases.
>
> bytes already have most of the 8-bit string methods from Python 2,
> so it doesn't hurt adding some more of the missing features
> from Python 2 on top to make life easier for people dealing
> with multiple/unknown encoding data.

Because people that aren't happy with the current bytes type
persistently refuse to experiment with writing their own extension
type to figure out what the API should look like. Jamming speculative
API design into the core text model without experimenting in a third
party extension first is a straight up stupid idea.

Anyone that is pushing for this should be checking out Benno's first
draft experimental prototype for asciistr and be working on getting it
passing the test suite I created:
https://github.com/jeamland/asciicompat

The "Wah, you broke it and now I have completely forgotten how to
create custom types, so I'm just going to piss and moan until somebody
else fixes it" infantilism of the past five years in this regard has
frankly pissed me off.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia