Mailman 3 Fwd: PEP 467: Minor API improvements for bytes & bytearray - Python-Dev

newer
PEP 4000 to explicitly declare we...

Fwd: PEP 467: Minor API improvements for bytes & bytearray

older
os.walk() is going to be *fast*...

Ian Cordasco

17 Aug 2014 17 Aug '14

6:40 p.m.

On Aug 17, 2014 12:17 PM, "Donald Stufft" wrote:

...

...
On Aug 17, 2014, at 1:07 PM, Raymond Hettinger wrote:

On Aug 17, 2014, at 1:41 AM, Nick Coghlan wrote:

...
If I see "bytearray(10)" there is nothing there that suggests "this creates an array of length 10 and initialises it to zero" to me. I'd be more inclined to guess it would be equivalent to "bytearray([10])".

"bytearray.zeros(10)", on the other hand, is relatively clear, independently of user expectations.

Zeros would have been great but that should have been done originally. The time to get API design right is at inception. Now, you're just breaking code and invalidating any published examples.

...
...
Another thought is that the core devs should be very reluctant to deprecate anything we don't have to while the 2 to 3 transition is still in progress. Every new deprecation of APIs that existed in Python 2.7 just adds another obstacle to converting code. Individually, the differences are trivial. Collectively, they present a good reason to never migrate code to Python 3.

This is actually one of the inconsistencies between the Python 2 and 3 binary APIs:

However, bytearray(n) is the same in both Python 2 and Python 3. Changing it in Python 3 increases the gulf between the two.

The further we let Python 3 diverge from Python 2, the less likely that people will convert their code and the harder you make it to write code that runs under both.

FWIW, I've been teaching Python full time for three years. I cover the use of bytearray(n) in my classes and not a single person out of 3000+ engineers have had a problem with it. I seriously question the PEP's assertion that there is a real problem to be solved (i.e. that people are baffled by bytearray(bufsiz)) and that the problem is sufficiently painful to warrant the headaches that go along with API changes.

The other proposal to add bytearray.byte(3) should probably be named bytearray.from_byte(3) for clarity. That said, I question whether there is actually a use case for this. I have never seen seen code that has a need to create a byte array of length one from a single integer. For the most part, the API will be easiest to learn if it matches what we do for lists and for array.array.

Sorry Nick, but I think you're making the API worse instead of better. This API isn't perfect but it isn't flat-out broken either. There is some unfortunate asymmetry between bytes() and bytearray() in Python 2, but that ship has sailed. The current API for Python 3 is pretty good (though there is still a tension between wanting to be like lists and like strings both at the same time).

Raymond

P.S. The most important problem in the Python world now is getting Python 2 users to adopt Python 3. The core devs need to develop a strong distaste for anything that makes that problem harder.

For the record I’ve had all of the problems that Nick states and I’m +1 on this change.

I've run into these problems as well, but I'm swayed by Raymond's argument regarding bytearray's constructor. I wouldn't be adverse to adding zeroes (for some parity between bytes and bytearray) to that but I'm not sure deprecating te behaviour of bytearray's constructor is necessary. (Whilst on my phone I only replied to Donald, so I'm forwarding this to the list.)

Show replies by date

Barry Warsaw

17 Aug 17 Aug

9:41 p.m.

New subject: Fwd: PEP 467: Minor API improvements for bytes & bytearray

I think the biggest API "problem" is that default iteration returns integers instead of bytes. That's a real pain. I'm not sure .iterbytes() is the best name for spelling iteration over bytes instead of integers though. Given that we can't change __iter__(), I personally would perhaps prefer a simple .bytes property over which if you iterated you would receive bytes, e.g.

...

...
...
data = bytes([1, 2, 3]) for i in data: ... print(i) ... 1 2 3 for b in data.bytes: ... print(b) ... b'\x01' b'\x02' b'\x03'

There are no backward compatibility issues with this of course. As for the single-int-ctor forms, they're inconvenient and arguably "wrong", but I think we can live with it. OTOH, I don't see any harm by adding the .zeros() alternative constructor. I'd probably want to spell the .byte() alternative constructor .from_int() but I also don't think the status quo (or .byte()) is that much of a usability problem. The API churn problem comes about when you start wanting to deprecate the single-int-ctor form. *If* that part gets adopted, it should have a really long deprecation cycle, IMO. Cheers, -Barry

Markus Unterwaditzer

9:55 p.m.

New subject: Fwd: PEP 467: Minor API improvements for bytes & bytearray

On Sun, Aug 17, 2014 at 05:41:10PM -0400, Barry Warsaw wrote:

...

I think the biggest API "problem" is that default iteration returns integers instead of bytes. That's a real pain.

I agree, this behavior required some helper functions while porting Werkzeug to Python 3 AFAIK.

...

I'm not sure .iterbytes() is the best name for spelling iteration over bytes instead of integers though. Given that we can't change __iter__(), I personally would perhaps prefer a simple .bytes property over which if you iterated you would receive bytes, e.g.

I'd rather be for a .bytes() method, to match the .values(), and .keys() methods on dictionaries. -- Markus

Nick Coghlan

10:48 p.m.

On 18 Aug 2014 08:04, "Markus Unterwaditzer" wrote:

...

On Sun, Aug 17, 2014 at 05:41:10PM -0400, Barry Warsaw wrote:

...
I think the biggest API "problem" is that default iteration returns

integers

...

...
instead of bytes. That's a real pain.

I agree, this behavior required some helper functions while porting Werkzeug to Python 3 AFAIK.

...
I'm not sure .iterbytes() is the best name for spelling iteration over

bytes

...
instead of integers though. Given that we can't change __iter__(), I personally would perhaps prefer a simple .bytes property over which if you iterated you would receive bytes, e.g.

I'd rather be for a .bytes() method, to match the .values(), and .keys() methods on dictionaries.

Calling it bytes is too confusing: for x in bytes(data): ... for x in bytes(data).bytes() When referring to bytes, which bytes do you mean, the builtin or the method? iterbytes() isn't especially attractive as a method name, but it's far more explicit about its purpose. Cheers, Nick.

...

-- Markus _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe:

https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com

Barry Warsaw

10:52 p.m.

New subject: Fwd: PEP 467: Minor API improvements for bytes & bytearray

On Aug 18, 2014, at 08:48 AM, Nick Coghlan wrote:

...

Calling it bytes is too confusing:

for x in bytes(data): ...

for x in bytes(data).bytes()

When referring to bytes, which bytes do you mean, the builtin or the method?

iterbytes() isn't especially attractive as a method name, but it's far more explicit about its purpose.

I don't know. How often do you really instantiate the bytes object there in the for loop? -Barry

Nick Coghlan

11:12 p.m.

On 18 Aug 2014 08:55, "Barry Warsaw" wrote:

...

On Aug 18, 2014, at 08:48 AM, Nick Coghlan wrote:

...
Calling it bytes is too confusing:

for x in bytes(data): ...

for x in bytes(data).bytes()

When referring to bytes, which bytes do you mean, the builtin or the

method?

...

...
iterbytes() isn't especially attractive as a method name, but it's far

more

...
explicit about its purpose.

I don't know. How often do you really instantiate the bytes object there in the for loop?

I'm talking more generally - do you *really* want to be explaining that "bytes" behaves like a tuple of integers, while "bytes.bytes" behaves like a tuple of bytes? Namespaces are great and all, but using the same name for two different concepts is still inherently confusing. Cheers, Nick.

...

-Barry

_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe:

https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com

...

Barry Warsaw

11:55 p.m.

New subject: Fwd: PEP 467: Minor API improvements for bytes & bytearray

On Aug 18, 2014, at 09:12 AM, Nick Coghlan wrote:

...

I'm talking more generally - do you *really* want to be explaining that "bytes" behaves like a tuple of integers, while "bytes.bytes" behaves like a tuple of bytes?

I would explain it differently though, using concrete examples. data = bytes(...) for i in data: # iterate over data as integers for i in data.bytes: # iterate over data as bytes But whatever. I just wish there was something better than iterbytes. -Barry

Nick Coghlan

18 Aug 18 Aug

12:08 a.m.

On 18 Aug 2014 09:57, "Barry Warsaw" wrote:

...

On Aug 18, 2014, at 09:12 AM, Nick Coghlan wrote:

...
I'm talking more generally - do you *really* want to be explaining that "bytes" behaves like a tuple of integers, while "bytes.bytes" behaves

...

...
a tuple of bytes?

I would explain it differently though, using concrete examples.

data = bytes(...) for i in data: # iterate over data as integers for i in data.bytes: # iterate over data as bytes

But whatever. I just wish there was something better than iterbytes.

There's actually another aspect to your idea, independent of the naming: exposing a view rather than just an iterator. I'm going to have to look at the implications for memoryview, but it may be a good way to go (and would align with the iterator -> view changes in dict). Cheers, Nick.

...

-Barry

_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe:

https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com

...

Barry Warsaw

12:22 a.m.

New subject: Fwd: PEP 467: Minor API improvements for bytes & bytearray

On Aug 18, 2014, at 10:08 AM, Nick Coghlan wrote:

...

There's actually another aspect to your idea, independent of the naming: exposing a view rather than just an iterator. I'm going to have to look at the implications for memoryview, but it may be a good way to go (and would align with the iterator -> view changes in dict).

Yep! Maybe that will inspire a better spelling. :) Cheers, -Barry

Guido van Rossum

12:45 a.m.

On Sun, Aug 17, 2014 at 5:22 PM, Barry Warsaw wrote:

...

On Aug 18, 2014, at 10:08 AM, Nick Coghlan wrote:

...
There's actually another aspect to your idea, independent of the naming: exposing a view rather than just an iterator. I'm going to have to look at the implications for memoryview, but it may be a good way to go (and would align with the iterator -> view changes in dict).

Yep! Maybe that will inspire a better spelling. :)

+1. It's just as much about b[i] as it is about "for c in b", so a view sounds right. (The view would have to be mutable for bytearrays and for writable memoryviews.) On the rest, it's sounding more and more as if we will just need to live with both bytes(1000) and bytearray(1000). A warning sounds worse than a deprecation to me. bytes.zeros(n) sounds fine to me; I value similar interfaces for bytes and bytearray pretty highly. I'm lukewarm on bytes.byte(c); but bytes([c]) does bother me because a size one list is (or at least feels) more expensive to allocate than a size one bytes object. So, okay. -- --Guido van Rossum (python.org/~guido)

Nick Coghlan

19 Aug 19 Aug

12:25 p.m.

On 18 August 2014 10:45, Guido van Rossum wrote:

...

On Sun, Aug 17, 2014 at 5:22 PM, Barry Warsaw wrote:

...
On Aug 18, 2014, at 10:08 AM, Nick Coghlan wrote:

...
There's actually another aspect to your idea, independent of the naming: exposing a view rather than just an iterator. I'm going to have to look at the implications for memoryview, but it may be a good way to go (and would align with the iterator -> view changes in dict).

Yep! Maybe that will inspire a better spelling. :)

+1. It's just as much about b[i] as it is about "for c in b", so a view sounds right. (The view would have to be mutable for bytearrays and for writable memoryviews.)

On the rest, it's sounding more and more as if we will just need to live with both bytes(1000) and bytearray(1000). A warning sounds worse than a deprecation to me.

I'm fine with keeping bytearray(1000), since that works the same way in both Python 2 & 3, and doesn't seem likely to be invoked inadvertently. I'd still like to deprecate "bytes(1000)", since that does different things in Python 2 & 3, while "b'\x00' * 1000" does the same thing in both. $ python -c 'print("{!r}\n{!r}".format(bytes(10), b"\x00" * 10))' '10' '\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00' $ python3 -c 'print("{!r}\n{!r}".format(bytes(10), b"\x00" * 10))' b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00' b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00' Hitting the deprecation warning in single-source code would seem to be a strong hint that you have a bug in one version or the other rather than being intended behaviour.

...

bytes.zeros(n) sounds fine to me; I value similar interfaces for bytes and bytearray pretty highly.

With "bytearray(1000)" sticking around indefinitely, I'm less concerned about adding a "zeros" constructor.

...

I'm lukewarm on bytes.byte(c); but bytes([c]) does bother me because a size one list is (or at least feels) more expensive to allocate than a size one bytes object. So, okay.

So, here's an interesting thing I hadn't previously registered: we actually already have a fairly capable "bytesview" option, and have done since Stefan implemented "memoryview.cast" in 3.3. The trick lies in the 'c' format character for the struct module, which is parsed as a length 1 bytes object rather than as an integer:

...

...
...
data = bytearray(b"Hello world") bytesview = memoryview(data).cast('c') list(bytesview) [b'H', b'e', b'l', b'l', b'o', b' ', b'w', b'o', b'r', b'l', b'd'] b''.join(bytesview) b'Hello world' bytesview[0:5] = memoryview(b"olleH").cast('c') list(bytesview) [b'o', b'l', b'l', b'e', b'H', b' ', b'w', b'o', b'r', b'l', b'd'] b''.join(bytesview) b'olleH world'

For the read-only case, it covers everything (iteration, indexing, slicing), for the writable view case, it doesn't cover changing the shape of the target array, and it doesn't cover assigning arbitrary buffer objects (you need to wrap them in a similar cast for memoryview to allow the assignment). It's hardly the most *intuitive* spelling though - I was one of the reviewers for Stefan's memoryview rewrite back in 3.3, and I only made the connection today when looking to see how a view object like the one we were discussing elsewhere in the thread might be implemented as a facade over arbitrary memory buffers, rather than being specific to bytes and bytearray. If we went down the "bytesview" path, then a single new facade would cover not only the 3 builtins (bytes, bytearray, memoryview) but also any *other* buffer exporting type. If we so chose (at some point in the future, not as part of this PEP), such a type could allow additional bytes operations (like "count", "startswith" or "index") to be applied to arbitrary regions of memory without making a copy. We can't add those other operations to memoryview, since they don't make sense for an n-dimensional array. Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

Guido van Rossum

4:46 p.m.

On Tue, Aug 19, 2014 at 5:25 AM, Nick Coghlan wrote:

...

On 18 August 2014 10:45, Guido van Rossum wrote:

...
On Sun, Aug 17, 2014 at 5:22 PM, Barry Warsaw wrote:

...
On Aug 18, 2014, at 10:08 AM, Nick Coghlan wrote:

...
There's actually another aspect to your idea, independent of the

naming:

...
...
exposing a view rather than just an iterator. I'm going to have to look at the implications for memoryview, but it may be a good way to go (and would align with the iterator -> view changes in dict).

Yep! Maybe that will inspire a better spelling. :)

+1. It's just as much about b[i] as it is about "for c in b", so a view sounds right. (The view would have to be mutable for bytearrays and for writable memoryviews.)

On the rest, it's sounding more and more as if we will just need to live with both bytes(1000) and bytearray(1000). A warning sounds worse than a deprecation to me.

I'm fine with keeping bytearray(1000), since that works the same way in both Python 2 & 3, and doesn't seem likely to be invoked inadvertently.

I'd still like to deprecate "bytes(1000)", since that does different things in Python 2 & 3, while "b'\x00' * 1000" does the same thing in both.

I think any argument based on what "bytes" does in Python 2 is pretty weak, since Python 2's bytes is just an alias for str, so it has tons of behavior that differ -- why single this out? In Python 3, I really like bytes and bytearray to be as similar as possible, and that includes the constructor.

...

$ python -c 'print("{!r}\n{!r}".format(bytes(10), b"\x00" * 10))' '10' '\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00' $ python3 -c 'print("{!r}\n{!r}".format(bytes(10), b"\x00" * 10))' b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00' b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'

Hitting the deprecation warning in single-source code would seem to be a strong hint that you have a bug in one version or the other rather than being intended behaviour.

...
bytes.zeros(n) sounds fine to me; I value similar interfaces for bytes and bytearray pretty highly.

With "bytearray(1000)" sticking around indefinitely, I'm less concerned about adding a "zeros" constructor.

That's fine.

...

...
I'm lukewarm on bytes.byte(c); but bytes([c]) does bother me because a size one list is (or at least feels) more expensive to allocate than a size one bytes object. So, okay.

So, here's an interesting thing I hadn't previously registered: we actually already have a fairly capable "bytesview" option, and have done since Stefan implemented "memoryview.cast" in 3.3. The trick lies in the 'c' format character for the struct module, which is parsed as a length 1 bytes object rather than as an integer:

...
...
...
data = bytearray(b"Hello world") bytesview = memoryview(data).cast('c') list(bytesview) [b'H', b'e', b'l', b'l', b'o', b' ', b'w', b'o', b'r', b'l', b'd'] b''.join(bytesview) b'Hello world' bytesview[0:5] = memoryview(b"olleH").cast('c') list(bytesview) [b'o', b'l', b'l', b'e', b'H', b' ', b'w', b'o', b'r', b'l', b'd'] b''.join(bytesview) b'olleH world'

For the read-only case, it covers everything (iteration, indexing, slicing), for the writable view case, it doesn't cover changing the shape of the target array, and it doesn't cover assigning arbitrary buffer objects (you need to wrap them in a similar cast for memoryview to allow the assignment).

It's hardly the most *intuitive* spelling though - I was one of the reviewers for Stefan's memoryview rewrite back in 3.3, and I only made the connection today when looking to see how a view object like the one we were discussing elsewhere in the thread might be implemented as a facade over arbitrary memory buffers, rather than being specific to bytes and bytearray.

Maybe the 'future' package can offer an iterbytes or bytesview implemented this way?

...

If we went down the "bytesview" path, then a single new facade would cover not only the 3 builtins (bytes, bytearray, memoryview) but also any *other* buffer exporting type. If we so chose (at some point in the future, not as part of this PEP), such a type could allow additional bytes operations (like "count", "startswith" or "index") to be applied to arbitrary regions of memory without making a copy.

Why call out "without making a copy" for operations that naturally don't have to copy anything?

...

We can't add those other operations to memoryview, since they don't make sense for an n-dimensional array.

I'm sorry for your efforts, but I'm getting more and more lukewarm about the entire PEP. -- --Guido van Rossum (python.org/~guido)

Antoine Pitrou

18 Aug 18 Aug

1:40 a.m.

Le 17/08/2014 20:08, Nick Coghlan a écrit :

...

On 18 Aug 2014 09:57, "Barry Warsaw" mailto:barry@python.org> wrote:

...
On Aug 18, 2014, at 09:12 AM, Nick Coghlan wrote:

...
I'm talking more generally - do you *really* want to be explaining that "bytes" behaves like a tuple of integers, while "bytes.bytes"

behaves like

...
...
a tuple of bytes?

I would explain it differently though, using concrete examples.

data = bytes(...) for i in data: # iterate over data as integers for i in data.bytes: # iterate over data as bytes

But whatever. I just wish there was something better than iterbytes.

There's actually another aspect to your idea, independent of the naming: exposing a view rather than just an iterator.

So that view would actually be the bytes object done right? Funny :-) Will it have lazy slicing? Regards Antoine.

Donald Stufft

1:48 a.m.

New subject: Fwd: PEP 467: Minor API improvements for bytes & bytearray

from __future__ import bytesdoneright? :D -- Donald Stufft donald@stufft.io On Sun, Aug 17, 2014, at 09:40 PM, Antoine Pitrou wrote:

...

Le 17/08/2014 20:08, Nick Coghlan a écrit :

...
On 18 Aug 2014 09:57, "Barry Warsaw" mailto:barry@python.org> wrote:

...
On Aug 18, 2014, at 09:12 AM, Nick Coghlan wrote:

...
I'm talking more generally - do you *really* want to be explaining that "bytes" behaves like a tuple of integers, while "bytes.bytes"

behaves like

...
...
a tuple of bytes?

I would explain it differently though, using concrete examples.

data = bytes(...) for i in data: # iterate over data as integers for i in data.bytes: # iterate over data as bytes

But whatever. I just wish there was something better than iterbytes.

There's actually another aspect to your idea, independent of the naming: exposing a view rather than just an iterator.

So that view would actually be the bytes object done right? Funny :-) Will it have lazy slicing?

Regards

Antoine.

_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/donald%40stufft.io

Barry Warsaw

2:17 p.m.

New subject: Fwd: PEP 467: Minor API improvements for bytes & bytearray

On Aug 17, 2014, at 09:48 PM, Donald Stufft wrote:

...

from __future__ import bytesdoneright? :D

Synonymous to: bytes = bytesdoneright or maybe from bytesdoneright import bytes :) -Barry

Chris McDonough

2:51 a.m.

New subject: Fwd: PEP 467: Minor API improvements for bytes & bytearray

On 08/17/2014 09:40 PM, Antoine Pitrou wrote:

...

Le 17/08/2014 20:08, Nick Coghlan a écrit :

...
On 18 Aug 2014 09:57, "Barry Warsaw" mailto:barry@python.org> wrote:

...
On Aug 18, 2014, at 09:12 AM, Nick Coghlan wrote:

...
I'm talking more generally - do you *really* want to be explaining

that

...
...
"bytes" behaves like a tuple of integers, while "bytes.bytes" behaves like a tuple of bytes?

I would explain it differently though, using concrete examples.

data = bytes(...) for i in data: # iterate over data as integers for i in data.bytes: # iterate over data as bytes

But whatever. I just wish there was something better than iterbytes.

There's actually another aspect to your idea, independent of the naming: exposing a view rather than just an iterator.

So that view would actually be the bytes object done right? Funny :-) Will it have lazy slicing?

bytes.sorry()? ;-) - C

Chris Barker

4:04 p.m.

On Sun, Aug 17, 2014 at 2:41 PM, Barry Warsaw wrote:

...

I think the biggest API "problem" is that default iteration returns integers instead of bytes. That's a real pain.

what is really needed for this NOT to be a pain is a byte scalar. numpy has a scalar type for every type it supports -- this is a GOOD THING (tm): In [53]: a = np.array((3,4,5), dtype=np.uint8) In [54]: a Out[54]: array([3, 4, 5], dtype=uint8) In [55]: a[1] Out[55]: 4 In [56]: type(a[1]) Out[56]: numpy.uint8 In [57]: a[1].shape Out[57]: () The lack of a character type is a major source of "type errors" in python (the whole list of strings vs a single string problem -- both return a sequence when you index into them or iterate over them) Anyway, the character ship has long since sailed, but maybe a byte scalar would be a good idea? And FWIW, I think the proposal does make for a better, cleaner API. Whether that's worth the deprecation is not clear to me, though as someone whose been on the verge of making the leap to 3.* for ages, this isn't going to make any difference. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov

Terry Reedy

8:06 p.m.

On 8/18/2014 12:04 PM, Chris Barker wrote:

...

On Sun, Aug 17, 2014 at 2:41 PM, Barry Warsaw mailto:barry@python.org> wrote:

I think the biggest API "problem" is that default iteration returns integers instead of bytes. That's a real pain.

what is really needed for this NOT to be a pain is a byte scalar.

The byte scalar is an int in range(256). Bytes is an array of such.

...

numpy has a scalar type for every type it supports -- this is a GOOD THING (tm):

In [53]: a = np.array((3,4,5), dtype=np.uint8)

In [54]: a Out[54]: array([3, 4, 5], dtype=uint8)

In [55]: a[1] Out[55]: 4

In [56]: type(a[1]) Out[56]: numpy.uint8

In [57]: a[1].shape Out[57]: ()

The lack of a character type is a major source of "type errors" in python (the whole list of strings vs a single string problem -- both return a sequence when you index into them or iterate over them)

This is exactly what iterbytes would do -- yields bytes of size 1.

...

Anyway, the character ship has long since sailed, but maybe a byte scalar would be a good idea?

And FWIW, I think the proposal does make for a better, cleaner API.

-- Terry Jan Reedy

Chris Barker

8:37 p.m.

On Mon, Aug 18, 2014 at 1:06 PM, Terry Reedy wrote:

...

The byte scalar is an int in range(256). Bytes is an array of such.

then why the complaint about iterating over bytes producing ints? Ye,s a byte owuld be pretty much teh same as an int, but it would have restrictions - useful ones. numpy has a scalar type for every type it supports -- this is a GOOD

...

...
THING (tm): In [56]: type(a[1]) Out[56]: numpy.uint8

In [57]: a[1].shape Out[57]: ()

The lack of a character type is a major source of "type errors" in python (the whole list of strings vs a single string problem -- both return a sequence when you index into them or iterate over them)

This is exactly what iterbytes would do -- yields bytes of size 1.

as I understand it, it would yield a bytes object of length one -- that is a sequence that _happens_ to only have one item in it -- not the same thing. Note above. In numpy, when you index out of a 1-d array you get a scalar -- with shape == () -- not a 1-d array of length 1. And this is useful, as it provide s clear termination point when you drill down through multiple dimensions. I often wish I could do that with nested lists with strings at the bottom. [1,2,3] is a sequence of numbers "this" is a sequence of characters -- oops, not it's not, it's a sequence of sequences of sequences of ... I think it would be cleaner if bytes was a sequence of a scalar byte object. This is a bigger deal for numpy, what with its n-dimensional arrays and many reducing operations, but the same principles apply. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov

3531

Age (days ago)

3533

Last active (days ago)

List overview

Download

18 comments

10 participants

participants (10)

Antoine Pitrou
Barry Warsaw
Chris Barker
Chris McDonough
Donald Stufft
Guido van Rossum
Ian Cordasco
Markus Unterwaditzer
Nick Coghlan
Terry Reedy

Fwd: PEP 467: Minor API improvements for bytes & bytearray

tags

participants (10)