Mailman 3 Smoothing transition to Python 3 - Python-ideas

newer
Proposal to add const-ness type...

Smoothing transition to Python 3

Neil Schemenauer

3 Jun 2016 3 Jun '16

1:17 p.m.

I'm nearly finished porting a decently size application from Python 2 to Python 3. It has been a lot of work. I am dubious as to whether the port was really the best use of time. I can imagine there is still millions, possibly billions of lines of Python 2 code that has not yet been converted. Further, my guess is the lines of Python 2 code are still growing faster than the lines of Python 3 compatible code. IMHO, we need to do better in making it easier for people to port code. Here is a thought that occured to me. Create a patched version of Python 3.x, making a stepping stone version for people porting from Python 2. I know this would have been useful for me. Specific changes that would be helpful, all generating warnings so code can be eventually fixed: - comparision with None, smaller than all other types - comparision of distinct types: use Python 2 behavior (i.e. order by type name) - mixing of unicode strings with byte strings: decode/encode using latin-1 - dict objects: make keys() items() values() return special sequence that warns if iterated over multiple times or indexed as sequence Changes that are not necessary as porting code is easy: - print function syntax - try/except syntax - standard module renaming - __next__/next() - long type, literal syntax - true division as default

Show replies by date

Ethan Furman

3 Jun 3 Jun

8:33 p.m.

On 06/03/2016 06:17 AM, Neil Schemenauer wrote:

...

Here is a thought that occured to me. Create a patched version of Python 3.x, making a stepping stone version for people porting from Python 2.

This is highly unlikely. I believe the efforts being made to make porting easier is in the optional type annotations, and using them is a double win: - clarifies intent by specifying expected types going in and expected types coming back out - type checker can then find certain classes of bugs to help with porting -- ~Ethan~

Giampaolo Rodola'

4 Jun 4 Jun

12:13 a.m.

On Fri, Jun 3, 2016 at 3:17 PM, Neil Schemenauer < nas-pythonideas@arctrix.com> wrote:

...

I'm nearly finished porting a decently size application from Python 2 to Python 3. It has been a lot of work. I am dubious as to whether the port was really the best use of time. I can imagine there is still millions, possibly billions of lines of Python 2 code that has not yet been converted. Further, my guess is the lines of Python 2 code are still growing faster than the lines of Python 3 compatible code. IMHO, we need to do better in making it easier for people to port code.

Here is a thought that occured to me. Create a patched version of Python 3.x, making a stepping stone version for people porting from Python 2. I know this would have been useful for me. Specific changes that would be helpful, all generating warnings so code can be eventually fixed:

- comparision with None, smaller than all other types

- comparision of distinct types: use Python 2 behavior (i.e. order by type name)

- mixing of unicode strings with byte strings: decode/encode using latin-1

To me this sounds more like going back to Python 2 rather than facilitating the transition, especially #3. Unfortunately there's no middle ground in there: it's either "do this or that" and that's why it's so painful. Incidentally, all these things you mentioned are bad practices which may hide bugs you may not even be aware of. The more I think about the porting subject, the more I realize that the difficulty mainly resides in how good your Python 2 code base is. Python 3 attempted to be a better language (mainly) by becoming stricter. As such, a badly written Python 2 code base is harder to port because Python 3 stands in your way acting as a referee for all the mistakes you've made in the past, forcing you to fix them all at once. That's its biggest advantage and damnation at the same time. -- Giampaolo - http://grodola.blogspot.com

Random832

12:41 a.m.

On Fri, Jun 3, 2016, at 20:13, Giampaolo Rodola' wrote:

...

On Fri, Jun 3, 2016 at 3:17 PM, Neil Schemenauer < nas-pythonideas@arctrix.com> wrote:

...
- mixing of unicode strings with byte strings: decode/encode using latin-1

To me this sounds more like going back to Python 2 rather than facilitating the transition, especially #3.

What about moving forward to unify the types? For example, we could go with the Emacs way: A single string abstract type*, which is a sequence whose elements can be Unicode characters, or raw non-ASCII bytes, all of which are distinct from each other (Emacs' representation is to assign the high bytes "code points" between "U+3FFF80" and "U+3FFFFF"). *Emacs has two concrete types: "byte strings" which can contain no non-ASCII characters, and "unicode strings" which use UTF-8 (plus those extra code points) underlying representation [various indexing operations are O(N)]. Python would use the FSR as it is now, along with perhaps a "byte string" type which likewise can contain only ASCII characters and high bytes. It might be worthwhile to have a 16-bit mode for "BMP + high bytes" which omits, say, surrogates. With only one type, mixing them becomes the easiest thing in the world.

Chris Barker

1:19 a.m.

On Fri, Jun 3, 2016 at 5:41 PM, Random832 wrote:

...

*Emacs has two concrete types: "byte strings" which can contain no non-ASCII characters, and "unicode strings" which use UTF-8 (plus those extra code points)

Uhm, I'm confused, isn't the exactly what py3 has now? byte strings and unicode strings? _maybe_ you want a little more interoperability, but those types are there. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov

Random832

5:24 p.m.

On Fri, Jun 3, 2016, at 21:19, Chris Barker wrote:

...

On Fri, Jun 3, 2016 at 5:41 PM, Random832 wrote:

...
*Emacs has two concrete types: "byte strings" which can contain no non-ASCII characters, and "unicode strings" which use UTF-8 (plus those extra code points)

Uhm, I'm confused, isn't the exactly what py3 has now? byte strings and unicode strings?

_maybe_ you want a little more interoperability, but those types are there.

Er, no, I'm talking about two concrete implementations of the abstract string type. Same as how Python has four: ASCII, Latin-1, UCS-2, and UCS-4.

Steven D'Aprano

9:05 a.m.

On Fri, Jun 03, 2016 at 08:41:14PM -0400, Random832 wrote:

...

What about moving forward to unify the types? For example, we could go with the Emacs way: A single string abstract type*, which is a sequence whose elements can be Unicode characters, or raw non-ASCII bytes, all of which are distinct from each other (Emacs' representation is to assign the high bytes "code points" between "U+3FFF80" and "U+3FFFFF").

I am fascinated by this concept. I think it might help solve the problem of "mixed text and bytes" files, but at the cost of throwing memory at it. Reading a file in binary mode would return a sequence of 32-bit "Unicode-plus-bytes" code points, rather than 8-bit bytes. Working in bytes would be more expensive, unless you happened to be lucky enough to only be dealing with bytes with the high bit cleared. Basically, instead of having two types: bytes: valid values are \x00 through \xFF; text: valid values are U+0000 through U+10FFFF we'd have one: text+bytes interpreted as: U+0000 through U+007F: bytes, or Unicode, depending on context; U+0080 through U+10FFFF: only Unicode; U+3FFF80 through U+3FFFFF: bytes \x80 through \xFF. Values outside of those ranges are presumably impossible. I'm not sure what implications there are for codecs. The downside is that reading from a binary file would give a sequence of code points that require four bytes rather than one (except in the unusual case that *no* byte had had its high-bit set). Nor could you tell the difference between a bunch of ASCII bytes or ASCII text. But maybe we could live with that?

...

*Emacs has two concrete types: "byte strings" which can contain no non-ASCII characters, and "unicode strings" which use UTF-8 (plus those extra code points) underlying representation [various indexing operations are O(N)]. Python would use the FSR as it is now, along with perhaps a "byte string" type which likewise can contain only ASCII characters and high bytes.

Presumably for backwards compatibility we would keep bytes as they are now, and either add a new Mixed string type, or modify str to be mixed. I'm still not entirely sure what the implications for encodings would be, but this is a promising idea with respect to mixed text/bytes. -- Steve

Stephen J. Turnbull

4:36 p.m.

Re: mixing text and bytes: We have mixed bytes and text already (PEP 383 surrogate escape codecs), as well as a space-efficient representation of text (PEP 393). Python already has improved versions of Emacs's facilities. PEPs tl;dr version: Emacs's rawbytes representation is obnoxiously expensive (minimum 4 bytes per "raw byte"), Python's half as much (PEP 383 surrogates). PEP 393 gives 1-byte representation not only of pure ASCII (as characters), but Latin-1 as well. I'm not sure if Emacs still implements the horrible as-unibyte APIs (which give you access to the internal represention, guaranteeing you'll cause data corruption, and sometimes even crashes), but they've long been deprecated and I suppose Python 3 can do something similar with memory views. (It was never automatic in Emacs, and it was extremely dangerous.) As for the OPs other desiderata, they are all similarly bad ideas, and they *won't help in porting* because Python 3 will never get them, Guido has many times said so (except perhaps None as "bottom" in all comparisons, which as far as I know hasn't been categorically rejected). Steve

Steven D'Aprano

7:31 a.m.

On Fri, Jun 03, 2016 at 06:17:30AM -0700, Neil Schemenauer wrote:

...

Here is a thought that occured to me. Create a patched version of Python 3.x, making a stepping stone version for people porting from Python 2. I know this would have been useful for me. Specific changes that would be helpful, all generating warnings so code can be eventually fixed:

- comparision with None, smaller than all other types

Sometimes I think that would be useful as a feature, not just for 2.x compatibility...

...

- comparision of distinct types: use Python 2 behavior (i.e. order by type name) - mixing of unicode strings with byte strings: decode/encode using latin-1

These would be a reversion to harmful behaviour and I guarantee they would be abused. But I don't understand why you want 3.x to raise a warning? It will raise an exception, which you then fix. How is this better than a warning that you will ignore? If this is purely a budget issue ("my boss has given me six weeks to port, but it will take eight, however if we can get warnings and suppress them, we'll squeeze in under budget") then you have my sympathy but not much else.

...

- dict objects: make keys() items() values() return special sequence that warns if iterated over multiple times or indexed as sequence

This is surely unnecessary -- you just need to mechanically change your dict iteration to use dict.viewkeys() etc in Python 2, fix any problems, then mechanically remove the "view" in Python 3. -- Steve

Nick Coghlan

8:05 a.m.

On 3 Jun 2016 13:17, "Neil Schemenauer" wrote:

...

I'm nearly finished porting a decently size application from Python 2 to Python 3. It has been a lot of work. I am dubious as to whether the port was really the best use of time. I can imagine there is still millions, possibly billions of lines of Python 2 code that has not yet been converted. Further, my guess is the lines of Python 2 code are still growing faster than the lines of Python 3 compatible code. IMHO, we need to do better in making it easier for people to port code.

Here is a thought that occured to me. Create a patched version of Python 3.x, making a stepping stone version for people porting from Python 2. I know this would have been useful for me. Specific changes that would be helpful, all generating warnings so code can be eventually fixed:

- comparision with None, smaller than all other types

Static type checkers should already be able to find these cases today, allowing them to be fixed incrementally pre-migration.

...

- comparision of distinct types: use Python 2 behavior (i.e. order by type name)

As above (and we definitely won't revert it - it's one of the more appreciated changes reported by educators)

...

- mixing of unicode strings with byte strings: decode/encode using latin-1

Converting with latin-1 would be even less correct than Python 2's behaviour of converting with ASCII (since it would allow arbitrary binary data to be implicitly interpreted as text) Type checkers don't generally help here yet, but are expected to in the future.

...

- dict objects: make keys() items() values() return special sequence that warns if iterated over multiple times or indexed as sequence

These can be mechanically converted by futurize in a way that avoids a redundant copy on Python 2 (modernize will insert a redundant call to list for the Python 2 case) Cheers, Nick.

Neil Schemenauer

8:12 a.m.

On 2016-06-04, Nick Coghlan wrote:

...

On 3 Jun 2016 13:17, "Neil Schemenauer" wrote:

...
- comparision with None, smaller than all other types

Static type checkers should already be able to find these cases today, allowing them to be fixed incrementally pre-migration.

Really? Please point me to them as I have about 100k lines of code that needs to be checked. I can't imagine how a static checker could find this.

...

...
- comparision of distinct types: use Python 2 behavior (i.e. order by type name)

As above (and we definitely won't revert it - it's one of the more appreciated changes reported by educators)

I'm not proposing to revert it. Sometimes I wonder if people actually read my proposal. I want a version of Python between 2.7.x and 3.x that allows it with a warning. Do you not see how such a version of Python is useful? Why was the whole __future__ mechanism designed in the first place?

...

...
- mixing of unicode strings with byte strings: decode/encode using latin-1

Converting with latin-1 would be even less correct than Python 2's behaviour of converting with ASCII (since it would allow arbitrary binary data to be implicitly interpreted as text)

I'm not sure how to best handle this, maybe UTF-8 would be better. An implicit conversion only happens in the cases where Python 3 raises a TypeError. Getting rid of the conversion is easy, make the object either a str or bytes and call encode/decode as necessary.

...

Type checkers don't generally help here yet, but are expected to in the future.

Python 3 was released in 2008. How long until we have these tools that help people to port their code?

...

...
- dict objects: make keys() items() values() return special sequence that warns if iterated over multiple times or indexed as sequence

These can be mechanically converted by futurize in a way that avoids a redundant copy on Python 2 (modernize will insert a redundant call to list for the Python 2 case)

I think you are correct. Blindly putting list() calls around the method calls works. People can change their Python 2 code to iter* if they want to avoid that.

Paul Moore

3:52 p.m.

On 4 June 2016 at 09:12, Neil Schemenauer wrote:

...

...
As above (and we definitely won't revert it - it's one of the more appreciated changes reported by educators)

I'm not proposing to revert it. Sometimes I wonder if people actually read my proposal. I want a version of Python between 2.7.x and 3.x that allows it with a warning.

Who do you propose develops that version? Will you do so to help the next group of people in your situation? Even though there has been a formal pronouncement that Python 2.8 will not happen, someone could produce a fork, call it "transition Python" and add all the features you're asking for. Would people use it? I don't know. But the people who've asked for such a thing in the past have apparently never been sufficiently convinced it was a worthwhile effort to put their own time into it. People with large Python 2 codebases don't have to port. The lack of ongoing support for Python 2 from the core devs can be handled by other means (pay your distribution vendor for extended support, accept that Python 2 will freeze - it's not as if it's going to stop working - or maintain your own patches for Python 2.7, lots of options). Which, of course, may or may not suit your needs. The big part of the equation that's yet to become a major issue is libraries. At some point, library authors will start to face the decision of whether maintaining Python 2 support is worth it. Maybe only once Python 2 is no longer supported by the core devs, I don't know. But I don't see any major libraries that are *not* supporting Python 3 (even Twisted, probably the highest-profile example of a library that's had major problems porting, is getting Python 3 support). So the Python 2 community faces the prospect of what to do when libraries like pip, requests, numpy, ... stop supporting Python 2. (Equally, it would be a big issue if one of those major libraries announced that it would no longer support Python 3, but my instinct says that's unlikely). So it seems to me that for Python 2 users, the options are: 1. Stay where you are: Advantages - no porting costs, no risk of introducing new bugs in the port Disadvantages - platform will become less well supported (may cost money for support), key dependencies may also desupport Python 2, adding to the support question. 2. Port to Python 3: Advantages - Ongoing free support from the Python community, possibly better or cheaper vendor support Disadvantages - Porting cost, possibility of introducing new bugs during the port. 3. Your proposal of creating a transitional version: Advantages - ??? Porting costs are smaller? But there's the cost of developing the transitional version (or paying someone to do so). Unless you were hoping someone would do that for free, or you think that spreading the cost over a lot of people with the same needs will reduce the cost sufficiently... Disadvantages - At the moment, this option doesn't exist. So delays before you can even use it. And potential users (who might contribute help to the effort) may not be willing to wait. Any others? For me the killer there is "porting cost" which is why I say you don't have to port. The costs may easily outweigh the benefits. Maybe in your case, they did but nobody realised that in time. Maybe there's not enough impartial information on how to quantify the costs. I don't know. Paul

Neil Schemenauer

11:34 a.m.

On 2016-06-04, Paul Moore wrote:

...

Who do you propose develops that version? Will you do so to help the next group of people in your situation?

I'm proposing to help. Here is a very early start: https://github.com/nascheme/ppython

...

But the people who've asked for such a thing in the past have apparently never been sufficiently convinced it was a worthwhile effort to put their own time into it.

Given the choice of porting your code to Python 3 or developing this transitional Python to help you, it is easy to see why people choose the former option. However, if you look at the millions of lines of Python 2 code that is yet to be ported, I think the idea has some merit. Neil

Paul Moore

7:28 p.m.

On 4 June 2016 at 12:34, Neil Schemenauer wrote:

...

On 2016-06-04, Paul Moore wrote:

...
Who do you propose develops that version? Will you do so to help the next group of people in your situation?

I'm proposing to help. Here is a very early start:

https://github.com/nascheme/ppython

Cool!

...

...
But the people who've asked for such a thing in the past have apparently never been sufficiently convinced it was a worthwhile effort to put their own time into it.

Given the choice of porting your code to Python 3 or developing this transitional Python to help you, it is easy to see why people choose the former option. However, if you look at the millions of lines of Python 2 code that is yet to be ported, I think the idea has some merit.

Sounds like you know where you want to go with this, I hope you get the interest to keep the project going. I can't say I'd recommend people go this route myself, but maybe that's because I'm not looking at projects on the scale you are - typically the things I've been involved in haven't been that hard to port (I work mostly on libraries, and working in the common Python 2/3 subset, with the aid of libraries like six, has been a perfectly practical solution for me). Paul

Nick Coghlan

5 Jun 5 Jun

5:14 a.m.

On 4 Jun 2016 8:06 am, "Neil Schemenauer" wrote:

...

On 2016-06-04, Nick Coghlan wrote:

...
On 3 Jun 2016 13:17, "Neil Schemenauer"

wrote:

...

...
...
- comparision with None, smaller than all other types

Static type checkers should already be able to find these cases today, allowing them to be fixed incrementally pre-migration.

Really? Please point me to them as I have about 100k lines of code that needs to be checked. I can't imagine how a static checker could find this.

It depends on APIs being correctly flagged as returning Optional results (although I don't know if the current crop of type checkers are clever enough to know Optional values aren't orderable)

...

...
...
- comparision of distinct types: use Python 2 behavior (i.e. order by type name)

As above (and we definitely won't revert it - it's one of the more appreciated changes reported by educators)

I'm not proposing to revert it. Sometimes I wonder if people actually read my proposal. I want a version of Python between 2.7.x and 3.x that allows it with a warning.

Do you not see how such a version of Python is useful? Why was the whole __future__ mechanism designed in the first place?

The -3 switch in Python 2.7 should already warn about these cases. If it doesn't, new -3 warnings are permitted in 2.7 maintenance releases.

...

...
...
- mixing of unicode strings with byte strings: decode/encode using latin-1

Converting with latin-1 would be even less correct than Python 2's behaviour of converting with ASCII (since it would allow arbitrary binary data to be implicitly interpreted as text)

I'm not sure how to best handle this, maybe UTF-8 would be better. An implicit conversion only happens in the cases where Python 3 raises a TypeError. Getting rid of the conversion is easy, make the object either a str or bytes and call encode/decode as necessary.

...
Type checkers don't generally help here yet, but are expected to in the future.

Python 3 was released in 2008. How long until we have these tools that help people to port their code?

We already have a number of them, many linked from the porting guide in the main documentation. python-future.org is one of the most comprehensive in providing a Python 3 like experience atop Python 2 (but also most invasive for existing Python 2 projects). Red Hat's Python maintenance team are working on a more explicitly minimalistic porting guide based on their experiences porting Fedora components, sometimes in the context of upstream projects that are more interested in keeping Python 2.4 support than they are in Python 3: http://portingguide.readthedocs.io/en/latest/ For projects with the code coverage needed to facilitate significant refactorings, these existing tools should be sufficient in most cases. Where the static analysers being worked on by Dropbox, Google, etc are likely to help most is with code bases that *don't* have that kind of test coverage - good static analysis tools can help compensate for the gaps in test coverage. Cheers, Nick.

Pavol Lisy

8 Jun 8 Jun

7:34 a.m.

2016-06-05 7:14 GMT+02:00, Nick Coghlan :

...

Red Hat's Python maintenance team are working on a more explicitly minimalistic porting guide based on their experiences porting Fedora components, sometimes in the context of upstream projects that are more interested in keeping Python 2.4 support than they are in Python 3: http://portingguide.readthedocs.io/en/latest/

do you have some drafts how to write (new) code supporting python from 2.4 - 2.7 with intention to port it to python3 in future?

Ben Finney

8:04 a.m.

Pavol Lisy writes:

...

do you have some drafts how to write (new) code supporting python from 2.4 - 2.7 with intention to port it to python3 in future?

I would advise: * Write the code targeting Python 3.5 primarily. * Back-port the code from the Python 3 source to Python 2. This is a much easier transition because, instead of hunting hard-to-find mistakes (e.g. the broken “text and bytes are the same” assumption in Python 2 code), instead it entails adding some temporary work-arounds. If the code works on Python 2 with your unit tests, your job is done. -- \ “I got fired from my job the other day. They said my | `\ personality was weird. … That's okay, I have four more.” | _o__) —Bug-Eyed Earl, _Red Meat_ | Ben Finney

Nick Coghlan

9 Jun 9 Jun

10:20 p.m.

On 8 June 2016 at 00:34, Pavol Lisy wrote:

...

2016-06-05 7:14 GMT+02:00, Nick Coghlan :

...
Red Hat's Python maintenance team are working on a more explicitly minimalistic porting guide based on their experiences porting Fedora components, sometimes in the context of upstream projects that are more interested in keeping Python 2.4 support than they are in Python 3: http://portingguide.readthedocs.io/en/latest/

do you have some drafts how to write (new) code supporting python from 2.4 - 2.7 with intention to port it to python3 in future?

Unfortunately, the only practical solution we've found for that case is to fork the code base until the maintainers of the original project are willing to raise the minimum version requirement to Python 2.6: http://portingguide.readthedocs.io/en/latest/process.html#drop-python-2-5-an... While it's theoretically possible to support 2.4+ and 3.x in the same code base, it's painful in practice due to the lack of the forward compatibility features added in Python 2.6 (such as the Python 3 style syntax for name binding in except clauses) For applications (rather than libraries), an alternate recommendation is to bundle a Python 2.7 runtime on platforms where the system Python is ancient, but whether or not that's a suitable solution will depend a great deal on the specifics of the application. My own view is that it doesn't make sense for *new* software (or new versions of existing software) being written in 2016 to support a version of Python that has been end-of-life for more than 7 years (last release December 2008, first release November 2004), just to allow folks to continue avoiding upgrading a base operating system that is itself now two major releases behind (assuming your interest in Python 2.4 support is driven by RHEL/CentOS 5 users). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

Carl Meyer

10:35 p.m.

On 06/09/2016 04:20 PM, Nick Coghlan wrote:

...

On 8 June 2016 at 00:34, Pavol Lisy wrote:

...
2016-06-05 7:14 GMT+02:00, Nick Coghlan :

...
Red Hat's Python maintenance team are working on a more explicitly minimalistic porting guide based on their experiences porting Fedora components, sometimes in the context of upstream projects that are more interested in keeping Python 2.4 support than they are in Python 3: http://portingguide.readthedocs.io/en/latest/

do you have some drafts how to write (new) code supporting python from 2.4 - 2.7 with intention to port it to python3 in future?

Unfortunately, the only practical solution we've found for that case is to fork the code base until the maintainers of the original project are willing to raise the minimum version requirement to Python 2.6: http://portingguide.readthedocs.io/en/latest/process.html#drop-python-2-5-an...

While it's theoretically possible to support 2.4+ and 3.x in the same code base, it's painful in practice due to the lack of the forward compatibility features added in Python 2.6 (such as the Python 3 style syntax for name binding in except clauses)

While I agree that it's by far the best option to raise the minimum supported version to 2.6 (or even 2.7) before trying to straddle to 3, we shouldn't make it sound impossible to straddle all the way back to 2.4. It's painful, for sure, but every issue is surmountable if you're dedicated enough. E.g. if you need to capture the exception in an `except` clause, you use `sys.exc_info()` instead. Pip did that for years :-) Carl

Nick Coghlan

10 Jun 10 Jun

12:24 a.m.

On 9 June 2016 at 15:35, Carl Meyer wrote:

...

On 06/09/2016 04:20 PM, Nick Coghlan wrote:

...
While it's theoretically possible to support 2.4+ and 3.x in the same code base, it's painful in practice due to the lack of the forward compatibility features added in Python 2.6 (such as the Python 3 style syntax for name binding in except clauses)

While I agree that it's by far the best option to raise the minimum supported version to 2.6 (or even 2.7) before trying to straddle to 3, we shouldn't make it sound impossible to straddle all the way back to 2.4. It's painful, for sure, but every issue is surmountable if you're dedicated enough. E.g. if you need to capture the exception in an `except` clause, you use `sys.exc_info()` instead. Pip did that for years :-)

Right, but there's also a time related aspect to this advice: most folks enthusiastic enough about Python 3 to adopt those kinds of invasive measures already support it, and Python 2.4 is now several years older than it was when folks first started adding Python 3 support to their projects (even RHEL 5 is reaching the point where Red Hat starts requiring the Extended Life Support addon for ongoing support beyond March 2017). That means that for anyone that's still holding off on porting due to a desire to retain Python 2.4 compatibility, "wait 12 months or so, and then reassess our need to support Python 2.4" is likely to make more sense than putting a lot of work into supporting Python 2.4 and 3.x in the same code base. Similarly, maintaining a Python 3 compatibility changeset with a reasonable hope of being able to merge it back into the parent project sometime in the next year is a significantly more attractive option than creating and maintaining a full fork of the project indefinitely. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

Pavol Lisy

6:29 a.m.

2016-06-10 2:24 GMT+02:00, Nick Coghlan :

...

Right, but there's also a time related aspect to this advice: most folks enthusiastic enough about Python 3 to adopt those kinds of invasive measures already support it, and Python 2.4 is now several years older than it was when folks first started adding Python 3 support to their projects (even RHEL 5 is reaching the point where Red Hat starts requiring the Extended Life Support addon for ongoing support beyond March 2017).

If I am not wrong then RHEL5 is stil about 2.4. 2016-06-10 0:20 GMT+02:00, Nick Coghlan :

...

Unfortunately, the only practical solution we've found for that case is to fork the code base until the maintainers of the original project are willing to raise the minimum version requirement to Python 2.6: http://portingguide.readthedocs.io/en/latest/process.html#drop-python-2-5-an...

If you need support 2.4 it is highly probably because very conservative customer or environement. Conservative environment could not approve to raise higher version. :/

...

While it's theoretically possible to support 2.4+ and 3.x in the same code base, it's painful in practice due to the lack of the forward compatibility features added in Python 2.6 (such as the Python 3 style syntax for name binding in except clauses)

It is why I like to see some guidlines how to do it. One practical possibility is (sorry for blasphemy) to (temporary!) drop support for python3. We need to understand that it is not only "fight" for python3 but also "fight" for python at all. So people who "want" to write code in 2.4 are not enemies. --- Btw. six library (now in version 1.10) dropped support for python 2.4 in version 1.5. and I am surprised. (They/we had probably to rename it to seven_and_half too :) Interesting for me is reason why they/we did it -> "Removed support for Python 2.4. This is because py.test no longer supports 2.4."

Paul Moore

8:47 a.m.

On 10 June 2016 at 07:29, Pavol Lisy wrote:

...

It is why I like to see some guidlines how to do it. One practical possibility is (sorry for blasphemy) to (temporary!) drop support for python3.

That's certainly an option. If you're writing an in-house application, then it's entirely reasonable to decide not to support a version of Python you don't plan to go to. If you're writing in-house libraries, saying "our company is not yet ready to support Python 3 for our internal code" is perfectly reasonable. If you're publishing code for external use, you have to decide whether alienating customers who want Python 3 support is worth it - that's again your decision. Only you know how many such customers you have. There is no requirement on anyone to support Python 3. However, there's a growing trend [1] to think of public software that doesn't support Python 3 as stagnant, and for people to look for "more up to date" replacements. That's something you have to consider when deciding not to support Python 3. Paul [1] We can debate endlessly on how fast the growth is. But even Twisted, the most highly visible project that I know of to struggle with supporting Python 3, has felt enough pressure to invest massive amounts of time in the porting effort.

Terry Reedy

5:43 p.m.

On 6/10/2016 4:47 AM, Paul Moore wrote:

...

On 10 June 2016 at 07:29, Pavol Lisy wrote:

...
It is why I like to see some guidlines how to do it. One practical possibility is (sorry for blasphemy) to (temporary!) drop support for python3.

That's certainly an option. If you're writing an in-house application, then it's entirely reasonable to decide not to support a version of Python you don't plan to go to. If you're writing in-house libraries, saying "our company is not yet ready to support Python 3 for our internal code" is perfectly reasonable.

If you're publishing code for external use, you have to decide whether alienating customers who want Python 3 support is worth it - that's again your decision. Only you know how many such customers you have. There is no requirement on anyone to support Python 3.

Or to support Python 2 ;-). For a standalone application, it almost does not matter which Python it runs on, as long as the needed version is available for the machine, and 2.7 and some 3.x are available for most anything now. However,

...

there's a growing trend [1] to think of public software that doesn't support Python 3 as stagnant, and for people to look for "more up to date" replacements. That's something you have to consider when deciding not to support Python 3.

Paul

[1] We can debate endlessly on how fast the growth is. But even Twisted, the most highly visible project that I know of to struggle with supporting Python 3, has felt enough pressure to invest massive amounts of time in the porting effort.

-- Terry Jan Reedy

Nikolaus Rath

3:01 a.m.

On Jun 09 2016, Carl Meyer wrote:

...

On 06/09/2016 04:20 PM, Nick Coghlan wrote:

...
On 8 June 2016 at 00:34, Pavol Lisy wrote:

...
2016-06-05 7:14 GMT+02:00, Nick Coghlan :

...
Red Hat's Python maintenance team are working on a more explicitly minimalistic porting guide based on their experiences porting Fedora components, sometimes in the context of upstream projects that are more interested in keeping Python 2.4 support than they are in Python 3: http://portingguide.readthedocs.io/en/latest/

do you have some drafts how to write (new) code supporting python from 2.4 - 2.7 with intention to port it to python3 in future?

Unfortunately, the only practical solution we've found for that case is to fork the code base until the maintainers of the original project are willing to raise the minimum version requirement to Python 2.6: http://portingguide.readthedocs.io/en/latest/process.html#drop-python-2-5-an...

While it's theoretically possible to support 2.4+ and 3.x in the same code base, it's painful in practice due to the lack of the forward compatibility features added in Python 2.6 (such as the Python 3 style syntax for name binding in except clauses)

While I agree that it's by far the best option to raise the minimum supported version to 2.6 (or even 2.7) before trying to straddle to 3, we shouldn't make it sound impossible to straddle all the way back to 2.4. It's painful, for sure, but every issue is surmountable if you're dedicated enough.

Sure, e.g. you could implement a Python 3 runtime in Python 2.4. It's Turing-complete after all. SCNR. Best, -Nikolaus -- GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F »Time flies like an arrow, fruit flies like a Banana.«

tritium-list＠sdamon.com

4 Jun 4 Jun

10:33 a.m.

...

-----Original Message----- From: Python-ideas [mailto:python-ideas-bounces+tritium- list=sdamon.com@python.org] On Behalf Of Neil Schemenauer Sent: Friday, June 3, 2016 9:18 AM To: python-ideas@python.org Subject: [Python-ideas] Smoothing transition to Python 3

...

- comparision with None, smaller than all other types

In a hypothetical python 2.8, the python 3 behavior would be in my backport list...

...

- comparision of distinct types: use Python 2 behavior (i.e. order by type name)

...and this

...

- mixing of unicode strings with byte strings: decode/encode using latin-1

...and this

...

- dict objects: make keys() items() values() return special sequence that warns if iterated over multiple times or indexed as sequence

...and this. In fact most of the things you list here are the GOOD ideas that python 3 enforces that reduces bugs when avoided in python 2. What would actually help the transition, in my world-view at least is * A bytes type like the string type in python 2 (*without implicit conversion!*) There are too many real world use cases that the bytes type makes painful, including anything dealing with networking. * an alias to the string type named 'unicode' (this just makes polyglot a whole heck of a lot less stressful... yes I do this myself, it's annoying, if it was there by default, like bytes is in 2.7, it would make life a lot easier. One just never just never references `str`) * a "magic" mapping from old to new module names. In my experience, this is actually a bigger pain than it looks. I had discussed, informally on IRC, the concept of building a hypothetical 'final version of python 2' that is, in fact, libpython3 with a python2 parser and shims in front of it. The general response was the theme of the entire python 3 transition story: "I don't see the value added." This was in response to more than just my crazy idea, but to the entire process and existence of python 3. I cannot disagree; the entire situation is a lot of effort for very little value added. Maybe a 'more compatible python 3' is a solution, maybe not. I have seen the pypi download numbers[1], and it is not encouraging for python 3; python 2 outstrips python 3 by a factor of 10 in terms of package downloads on linux, and a factor of 6 on mac. The only platform where python 3 has a real story against 2 is windows. If python 3 really is to be the future, a lot more needs to be done to bring the vast majority of users on board. Telling them "Just port your code" is not good enough. [1] https://s.caremad.io/WPVkP3Ruhg/ (query: https://bpaste.net/show/56443668b83f)

Neil Schemenauer

8:27 a.m.

On 2016-06-04, tritium-list@sdamon.com wrote:

...

In fact most of the things you list here are the GOOD ideas that python 3 enforces that reduces bugs when avoided in python 2.

Sure, and I'm not proposing that standard Python 3.x change the behavior.

...

What would actually help the transition, in my world-view at least is

* A bytes type like the string type in python 2 (*without implicit conversion!*) There are too many real world use cases that the bytes type makes painful, including anything dealing with networking.

The Python 3 bytes type should gain whatever features it needs to make things not painful. The %-based formating in 3.5 is a big one. Is there something else you miss?

...

* an alias to the string type named 'unicode' (this just makes polyglot a whole heck of a lot less stressful... yes I do this myself, it's annoying, if it was there by default, like bytes is in 2.7, it would make life a lot easier. One just never just never references `str`)

Maybe too late now but there should have been 'unicode', 'basestring' as aliases for 'str'.

...

* a "magic" mapping from old to new module names. In my experience, this is actually a bigger pain than it looks.

I would like to add this to my "pragmatic" version.

...

The general response was the theme of the entire python 3 transition story: "I don't see the value added."

Yes, and here we are. Python 3 is not yet winning and I'm not sure it will. I believe Dropbox, Facebook and Google are all still using Python 2. If porting code was so easy, why are they not moved over? I see VMWare released some new IoT SDK: https://github.com/vmware/liota This is new code, written this year. It is not compatible with Python 3 as far as I see. I can't understand why people don't see we have a problem. Neil

Jason Fried

3:52 p.m.

As said during pycon, Facebook is using Python 3 and all new code is Python 3 by default. Google is working on it. Sounds like VMware just released a big ole load of tech debt On Sat, Jun 4, 2016 at 08:20 Neil Schemenauer wrote:

...

On 2016-06-04, tritium-list@sdamon.com wrote:

...
In fact most of the things you list here are the GOOD ideas that python 3 enforces that reduces bugs when avoided in python 2.

Sure, and I'm not proposing that standard Python 3.x change the behavior.

...
What would actually help the transition, in my world-view at least is

* A bytes type like the string type in python 2 (*without implicit conversion!*) There are too many real world use cases that the bytes type makes painful, including anything dealing with networking.

The Python 3 bytes type should gain whatever features it needs to make things not painful. The %-based formating in 3.5 is a big one. Is there something else you miss?

...
* an alias to the string type named 'unicode' (this just makes polyglot a whole heck of a lot less stressful... yes I do this myself, it's annoying, if it was there by default, like bytes is in 2.7, it would make life a lot easier. One just never just never references `str`)

Maybe too late now but there should have been 'unicode', 'basestring' as aliases for 'str'.

...
* a "magic" mapping from old to new module names. In my experience, this is actually a bigger pain than it looks.

I would like to add this to my "pragmatic" version.

...
The general response was the theme of the entire python 3 transition story: "I don't see the value added."

Yes, and here we are. Python 3 is not yet winning and I'm not sure it will. I believe Dropbox, Facebook and Google are all still using Python 2. If porting code was so easy, why are they not moved over? I see VMWare released some new IoT SDK:

https://github.com/vmware/liota

This is new code, written this year. It is not compatible with Python 3 as far as I see. I can't understand why people don't see we have a problem.

Neil _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

tritium-list＠sdamon.com

11:29 p.m.

...

-----Original Message----- From: Python-ideas [mailto:python-ideas-bounces+tritium- list=sdamon.com@python.org] On Behalf Of Neil Schemenauer Sent: Saturday, June 4, 2016 4:27 AM To: python-ideas@python.org Subject: Re: [Python-ideas] Smoothing transition to Python 3

On 2016-06-04, tritium-list@sdamon.com wrote:

...
In fact most of the things you list here are the GOOD ideas that python 3 enforces that reduces bugs when avoided in python 2.

Sure, and I'm not proposing that standard Python 3.x change the behavior.

...
What would actually help the transition, in my world-view at least is

* A bytes type like the string type in python 2 (*without implicit conversion!*) There are too many real world use cases that the bytes type makes painful, including anything dealing with networking.

The Python 3 bytes type should gain whatever features it needs to make things not painful. The %-based formating in 3.5 is a big one. Is there something else you miss?

b'Foo'[0] returning b'F' instead of the int 70

...

...
* an alias to the string type named 'unicode' (this just makes polyglot a whole heck of a lot less stressful... yes I do this myself, it's annoying, if it was there by default, like bytes is in 2.7, it would make life a lot easier. One just never just never references `str`)

Maybe too late now but there should have been 'unicode', 'basestring' as aliases for 'str'.

...
* a "magic" mapping from old to new module names. In my experience, this is actually a bigger pain than it looks.

I would like to add this to my "pragmatic" version.

...
The general response was the theme of the entire python 3 transition story: "I don't see the value added."

Yes, and here we are. Python 3 is not yet winning and I'm not sure it will. I believe Dropbox, Facebook and Google are all still using Python 2. If porting code was so easy, why are they not moved over? I see VMWare released some new IoT SDK:

https://github.com/vmware/liota

This is new code, written this year. It is not compatible with Python 3 as far as I see. I can't understand why people don't see we have a problem.

Neil _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

Guido van Rossum

11:37 p.m.

The bytes -> int behavior is widely considered a mistake. We're just not able to fix it without yet another round of layoffs ^W deprecations. And I'm not ready for that -- not even Python 4 should be allowed to change this unilaterally. Though maybe we could do something with a __future__ import. On Sat, Jun 4, 2016 at 4:29 PM, wrote:

...

...
-----Original Message----- From: Python-ideas [mailto:python-ideas-bounces+tritium- list=sdamon.com@python.org] On Behalf Of Neil Schemenauer Sent: Saturday, June 4, 2016 4:27 AM To: python-ideas@python.org Subject: Re: [Python-ideas] Smoothing transition to Python 3

On 2016-06-04, tritium-list@sdamon.com wrote:

...
In fact most of the things you list here are the GOOD ideas that python 3 enforces that reduces bugs when avoided in python 2.

Sure, and I'm not proposing that standard Python 3.x change the behavior.

...
What would actually help the transition, in my world-view at least is

* A bytes type like the string type in python 2 (*without implicit conversion!*) There are too many real world use cases that the bytes type makes painful, including anything dealing with networking.

The Python 3 bytes type should gain whatever features it needs to make things not painful. The %-based formating in 3.5 is a big one. Is there something else you miss?

b'Foo'[0] returning b'F' instead of the int 70

...
...
* an alias to the string type named 'unicode' (this just makes polyglot a whole heck of a lot less stressful... yes I do this myself, it's annoying, if it was there by default, like bytes is in 2.7, it would make life a lot easier. One just never just never references `str`)

Maybe too late now but there should have been 'unicode', 'basestring' as aliases for 'str'.

...
* a "magic" mapping from old to new module names. In my experience, this is actually a bigger pain than it looks.

I would like to add this to my "pragmatic" version.

...
The general response was the theme of the entire python 3 transition story: "I don't see the value added."

Yes, and here we are. Python 3 is not yet winning and I'm not sure it will. I believe Dropbox, Facebook and Google are all still using Python 2. If porting code was so easy, why are they not moved over? I see VMWare released some new IoT SDK:

https://github.com/vmware/liota

This is new code, written this year. It is not compatible with Python 3 as far as I see. I can't understand why people don't see we have a problem.

Neil _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

-- --Guido van Rossum (python.org/~guido)

tritium-list＠sdamon.com

5 Jun 5 Jun

12:17 a.m.

Unless you changed your mind about version numbing (and the digit count in each slot of a version number), I don’t think we will be considering breaking changes till python 5 :) I am totally unopposed to locking these kinds of shims/hacks/shames in a __future__ (__past__?) import.

...

-----Original Message----- From: gvanrossum@gmail.com [mailto:gvanrossum@gmail.com] On Behalf Of Guido van Rossum Sent: Saturday, June 4, 2016 7:38 PM To: Alexander Walters Cc: Neil Schemenauer ; Python-Ideas Subject: Re: [Python-ideas] Smoothing transition to Python 3

The bytes -> int behavior is widely considered a mistake. We're just not able to fix it without yet another round of layoffs ^W deprecations. And I'm not ready for that -- not even Python 4 should be allowed to change this unilaterally. Though maybe we could do something with a __future__ import.

On Sat, Jun 4, 2016 at 4:29 PM, wrote:

...
...
-----Original Message----- From: Python-ideas [mailto:python-ideas-bounces+tritium- list=sdamon.com@python.org] On Behalf Of Neil Schemenauer Sent: Saturday, June 4, 2016 4:27 AM To: python-ideas@python.org Subject: Re: [Python-ideas] Smoothing transition to Python 3

On 2016-06-04, tritium-list@sdamon.com wrote:

...
In fact most of the things you list here are the GOOD ideas that python 3 enforces that reduces bugs when avoided in python 2.

Sure, and I'm not proposing that standard Python 3.x change the behavior.

...
What would actually help the transition, in my world-view at least is

* A bytes type like the string type in python 2 (*without implicit conversion!*) There are too many real world use cases that the bytes type makes painful, including anything dealing with networking.

The Python 3 bytes type should gain whatever features it needs to make things not painful. The %-based formating in 3.5 is a big one. Is there something else you miss?

b'Foo'[0] returning b'F' instead of the int 70

...
...
* an alias to the string type named 'unicode' (this just makes polyglot a whole heck of a lot less stressful... yes I do this myself, it's annoying, if it was there by default, like bytes is in 2.7, it would make life a lot easier. One just never just never references `str`)

Maybe too late now but there should have been 'unicode', 'basestring' as aliases for 'str'.

...
* a "magic" mapping from old to new module names. In my experience, this is actually a bigger pain than it looks.

I would like to add this to my "pragmatic" version.

...
The general response was the theme of the entire python 3 transition story: "I don't see the value added."

Yes, and here we are. Python 3 is not yet winning and I'm not sure it will. I believe Dropbox, Facebook and Google are all still using Python 2. If porting code was so easy, why are they not moved over? I see VMWare released some new IoT SDK:

https://github.com/vmware/liota

This is new code, written this year. It is not compatible with Python 3 as far as I see. I can't understand why people don't see we have a problem.

Neil _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

-- --Guido van Rossum (python.org/~guido)

Random832

12:23 a.m.

On Sat, Jun 4, 2016, at 19:37, Guido van Rossum wrote:

...

The bytes -> int behavior is widely considered a mistake. We're just not able to fix it without yet another round of layoffs ^W deprecations. And I'm not ready for that -- not even Python 4 should be allowed to change this unilaterally. Though maybe we could do something with a __future__ import.

I did have some thoughts (in the discussion a week or so ago about int division returning float vs Fraction) on how a __future__ flag (which have to be file scoped) effecting a change in the behavior of an operator of a class could be implemented without a __truediv__/__floordiv__ dichotomy intended to stick around forever. In this case: Change the actual behavior of bytes.__getitem__ (direct callers aren't important enough to matter) to return a 1-length bytes instance. Add a compile flag (enabled by default, disabled by the future import) which causes foo[bar] to become __oldgetitem__(foo, bar) where: def __oldgetitem__(obj, i): if isinstance(obj, bytes) and not isinstance(i, slice): return ord(obj.__getitem__(i)) else: return obj.__getitem__(i) Or instead of literally calling a function, we could add another byte code that directly implements this behavior. The question is how big the performance hit is and how important is the performance of code that doesn't use the future import, considering dynamic typing means every [] operator (except perhaps those statically knowable to be slices) goes through this.

tritium-list＠sdamon.com

12:28 a.m.

In my experience (and if this is wrong, ignore the rest of this message), python 3 is faster than python 2. Since these are hacks primarily to get people off of python 2... you only really have to hit python 2 speeds. A minor performance hit is acceptable.

...

-----Original Message----- From: Python-ideas [mailto:python-ideas-bounces+tritium- list=sdamon.com@python.org] On Behalf Of Random832 Sent: Saturday, June 4, 2016 8:24 PM To: python-ideas@python.org Subject: Re: [Python-ideas] Smoothing transition to Python 3

On Sat, Jun 4, 2016, at 19:37, Guido van Rossum wrote:

...
The bytes -> int behavior is widely considered a mistake. We're just not able to fix it without yet another round of layoffs ^W deprecations. And I'm not ready for that -- not even Python 4 should be allowed to change this unilaterally. Though maybe we could do something with a __future__ import.

I did have some thoughts (in the discussion a week or so ago about int division returning float vs Fraction) on how a __future__ flag (which have to be file scoped) effecting a change in the behavior of an operator of a class could be implemented without a __truediv__/__floordiv__ dichotomy intended to stick around forever.

In this case: Change the actual behavior of bytes.__getitem__ (direct callers aren't important enough to matter) to return a 1-length bytes instance. Add a compile flag (enabled by default, disabled by the future import) which causes foo[bar] to become __oldgetitem__(foo, bar) where:

def __oldgetitem__(obj, i): if isinstance(obj, bytes) and not isinstance(i, slice): return ord(obj.__getitem__(i)) else: return obj.__getitem__(i)

Or instead of literally calling a function, we could add another byte code that directly implements this behavior. The question is how big the performance hit is and how important is the performance of code that doesn't use the future import, considering dynamic typing means every [] operator (except perhaps those statically knowable to be slices) goes through this. _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

Neil Schemenauer

6 Jun 6 Jun

7:35 p.m.

New subject: bytes indexing behavior

On 2016-06-04, Guido van Rossum wrote:

...

The bytes -> int behavior is widely considered a mistake. We're just not able to fix it without yet another round of layoffs ^W deprecations. And I'm not ready for that -- not even Python 4 should be allowed to change this unilaterally. Though maybe we could do something with a __future__ import.

Maybe the following would work: - add a new method to 'bytes' that returns a view object with the current index/iteration behavior - enable a deprecation warning for code that uses indexing/iteration on bytes - when sufficient time has passed, revert to Python 2 behavior for indexing/iteration Another, probably crazy and unworkable idea: - have bytes indexing/iteration return a special type that behaves like an int or length one byte. - ord() of this object would return a real int - code that utilizes this object as an int would generate a warning (suggest adding an ord() call to fix code). - eventually just return length one byte strings

Sjoerd Job Postmus

8:09 p.m.

New subject: bytes indexing behavior

I like this idea! >>> k = b'hello world' >>> v = k[0] >>> type(v) >>> ord(v) 104 >>> v + 5 __main__:1: DeprecationWarning: bytes items will become byte instances. Use ord(). 109 On the other hand, maybe 'hello'[0] should be a ch(a)r instead of str? The 'wrapping' class could be a `bytearray`? >>> k = bytearray(b'hello') >>> k[0] 104 >>> k[2:4] bytearray(b'll') The only 'downside' is that a bytearray is mutable.

...

On 6 Jun 2016, at 21:35, Neil Schemenauer wrote:

...
On 2016-06-04, Guido van Rossum wrote: The bytes -> int behavior is widely considered a mistake. We're just not able to fix it without yet another round of layoffs ^W deprecations. And I'm not ready for that -- not even Python 4 should be allowed to change this unilaterally. Though maybe we could do something with a __future__ import.

Maybe the following would work:

- add a new method to 'bytes' that returns a view object with the current index/iteration behavior

- enable a deprecation warning for code that uses indexing/iteration on bytes

- when sufficient time has passed, revert to Python 2 behavior for indexing/iteration

Another, probably crazy and unworkable idea:

- have bytes indexing/iteration return a special type that behaves like an int or length one byte.

- ord() of this object would return a real int

- code that utilizes this object as an int would generate a warning (suggest adding an ord() call to fix code).

- eventually just return length one byte strings _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

Guido van Rossum

8:28 p.m.

New subject: bytes indexing behavior

I think the approach using a new method for the old behavior is workable. Maybe we should add another (temporary) new method for the new behavior. The magic object approach is too magical. We should do the same for bytearray and memoryview. On Mon, Jun 6, 2016 at 12:35 PM, Neil Schemenauer < nas-pythonideas@arctrix.com> wrote:

...

On 2016-06-04, Guido van Rossum wrote:

...
The bytes -> int behavior is widely considered a mistake. We're just not able to fix it without yet another round of layoffs ^W deprecations. And I'm not ready for that -- not even Python 4 should be allowed to change this unilaterally. Though maybe we could do something with a __future__ import.

Maybe the following would work:

- add a new method to 'bytes' that returns a view object with the current index/iteration behavior

- enable a deprecation warning for code that uses indexing/iteration on bytes

- when sufficient time has passed, revert to Python 2 behavior for indexing/iteration

Another, probably crazy and unworkable idea:

- have bytes indexing/iteration return a special type that behaves like an int or length one byte.

- ord() of this object would return a real int

- code that utilizes this object as an int would generate a warning (suggest adding an ord() call to fix code).

- eventually just return length one byte strings

-- --Guido van Rossum (python.org/~guido)

Serhiy Storchaka

7 Jun 7 Jun

4:41 a.m.

New subject: bytes indexing behavior

On 06.06.16 23:28, Guido van Rossum wrote:

...

I think the approach using a new method for the old behavior is workable. Maybe we should add another (temporary) new method for the new behavior.

The magic object approach is too magical.

We should do the same for bytearray and memoryview.

I often use bytearray and memoryview as arrays of ints.

Guido van Rossum

4:48 a.m.

New subject: bytes indexing behavior

On Monday, June 6, 2016, Serhiy Storchaka wrote:

...

On 06.06.16 23:28, Guido van Rossum wrote:

...
I think the approach using a new method for the old behavior is workable. Maybe we should add another (temporary) new method for the new behavior.

The magic object approach is too magical.

We should do the same for bytearray and memoryview.

I often use bytearray and memoryview as arrays of ints.

Really short ints. :-). But you should really use the array module instead. Anyway we should do this for all three or for none. --Guido -- --Guido (mobile)

Serhiy Storchaka

5:06 a.m.

New subject: bytes indexing behavior

On 07.06.16 07:48, Guido van Rossum wrote:

...

On Monday, June 6, 2016, Serhiy Storchaka mailto:storchaka@gmail.com> wrote:

On 06.06.16 23:28, Guido van Rossum wrote:

I think the approach using a new method for the old behavior is workable. Maybe we should add another (temporary) new method for the new behavior.

The magic object approach is too magical.

We should do the same for bytearray and memoryview.

I often use bytearray and memoryview as arrays of ints.

Really short ints. :-). But you should really use the array module instead.

Yes, ints of known width (even 1-bit ints in sre_compile.py). Using the array module requires additional copying. Aren't bytearray and memoryview here for avoiding unnecessary copying?

...

Anyway we should do this for all three or for none.

I think representing bytes as an array of ints was good decision. If you need indexing to return a substring, you should use str instead. It is as well memory efficient thanks to PEP 393.

Jonathan Goble

5:48 a.m.

New subject: bytes indexing behavior

On Tue, Jun 7, 2016 at 1:06 AM, Serhiy Storchaka wrote:

...

On 07.06.16 07:48, Guido van Rossum wrote:

...
On Monday, June 6, 2016, Serhiy Storchaka mailto:storchaka@gmail.com> wrote:

On 06.06.16 23:28, Guido van Rossum wrote:

I think the approach using a new method for the old behavior is workable. Maybe we should add another (temporary) new method for the new behavior.

The magic object approach is too magical.

We should do the same for bytearray and memoryview.

I often use bytearray and memoryview as arrays of ints.

Really short ints. :-). But you should really use the array module instead.

Yes, ints of known width (even 1-bit ints in sre_compile.py). Using the array module requires additional copying. Aren't bytearray and memoryview here for avoiding unnecessary copying?

Actually, array.array implements the buffer protocol, and memoryview can operate on any object that implements the buffer protocol. Thus, you should be able to create a memoryview of an array.array instance and operate on that (and in fact, you can, as I just tested it in an interactive session).

Stephen J. Turnbull

7:01 p.m.

New subject: bytes indexing behavior

Serhiy Storchaka writes:

...

I think representing bytes as an array of ints was good decision. If you need indexing to return a substring, you should use str instead. It is as well memory efficient thanks to PEP 393.

You can do this by using latin-1 as the codec, but that's pretty unpleasant, because of the risk of combining with another str and getting mojibake. I have long thought that it would be interesting to have a codec and an extension to PEP 393 that gives "asciibytes" behavior. That is, the codec simply slops the bytes into the 8-bit storage of a string, but when joined with another string the result types are: asciibytes other arg result has 8bit type type yes pure ascii asciibytes yes asciibytes asciibytes yes other str str with 8bit bytes from asciibytes encoded as PEP 383 surrogateescape (note: promotes latin1 to 2-byte-wide) no whatever whatever I think Nick actually had a module that worked pretty much like this, but he never pushed it. I've never had time to reason out the possible failure modes, though, or the performance issues. And it's not an itch I personally need to scratch. I believe (but haven't proved) that the failure modes with the above operation table are the same as for str containing PEP 383 surrogates. I'm not sure what other issues you might run into. Also, I'm not sure it's reasonable to have an asciibytes with no 8bit bytes. Steve

Nick Coghlan

8:07 p.m.

New subject: bytes indexing behavior

On 7 June 2016 at 12:01, Stephen J. Turnbull wrote:

...

Serhiy Storchaka writes:

...
I think representing bytes as an array of ints was good decision. If you need indexing to return a substring, you should use str instead. It is as well memory efficient thanks to PEP 393.

You can do this by using latin-1 as the codec, but that's pretty unpleasant, because of the risk of combining with another str and getting mojibake.

I have long thought that it would be interesting to have a codec and an extension to PEP 393 that gives "asciibytes" behavior. That is, the codec simply slops the bytes into the 8-bit storage of a string, but when joined with another string the result types are:

asciibytes other arg result has 8bit type type yes pure ascii asciibytes yes asciibytes asciibytes yes other str str with 8bit bytes from asciibytes encoded as PEP 383 surrogateescape (note: promotes latin1 to 2-byte-wide) no whatever whatever

I think Nick actually had a module that worked pretty much like this, but he never pushed it. I've never had time to reason out the possible failure modes, though, or the performance issues. And it's not an itch I personally need to scratch.

Benno Rice, rather than me (although I gave Benno the idea): https://github.com/jeamland/asciicompat Managing extra C dependencies is a pain though, and it's a dubious idea at best, so neither of us seriously pushed for anyone to use it. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

Steven D'Aprano

3:57 a.m.

New subject: bytes indexing behavior

On Mon, Jun 06, 2016 at 12:35:30PM -0700, Neil Schemenauer wrote:

...

Maybe the following would work:

- add a new method to 'bytes' that returns a view object with the current index/iteration behavior

- enable a deprecation warning for code that uses indexing/iteration on bytes

- when sufficient time has passed, revert to Python 2 behavior for indexing/iteration

Sounds reasonable. That could even be a __future__ import, so that those without a big investment in the current behaviour could get the new behaviour straight away. -- Steve

Antoine Pitrou

11:27 a.m.

New subject: bytes indexing behavior

Hi, While I prefer Python 2's indexing behaviour, IMHO that ship has sailed. Python 3.5 is probably seeing massive adoption now, and by introducing another behaviour switch we only confuse users and make their lives miserable. Let's move on and stop obsessing about errors made in the past. Every language has some of those. Regards Antoine.

Joao S. O. Bueno

2:39 p.m.

New subject: bytes indexing behavior

On 7 June 2016 at 08:27, Antoine Pitrou wrote:

...

Hi,

While I prefer Python 2's indexing behaviour, IMHO that ship has sailed. Python 3.5 is probably seeing massive adoption now, and by introducing another behaviour switch we only confuse users and make their lives miserable.

Let's move on and stop obsessing about errors made in the past. Every language has some of those.

+1 Why not just stick in a new attribute over withch one can iterate as 1 byte "bytes sequence" instead? Backwards compatible, and everyone gets happy: for code, onebyte in zip(mybytes, mybytes.as_bytes): assert code == ord(onebyte) I just can't view how any other thing could justify a major breakearage in backward compatibility that would take another 10-12 years to settle. Maybe just add an "as_ints" iterator as well, and deprecate interation on the object itself - but reverse the behavior should just be a major no no as in "python 4 scale" major no no! And on the other hand, 'bytearray's have always behaved this way, even in Python 2 - and it is actually quite handy.

...

Regards

Antoine.

Ethan Furman

3:59 p.m.

New subject: bytes indexing behavior

On 06/07/2016 04:27 AM, Antoine Pitrou wrote:

...

While I prefer Python 2's indexing behaviour, IMHO that ship has sailed. Python 3.5 is probably seeing massive adoption now, and by introducing another behaviour switch we only confuse users and make their lives miserable.

I agree on both counts. Wasn't there a proposal/issue about adding some convenience methods to bytes to help ease this pain? My quick search did not find it.

...

Let's move on and stop obsessing about errors made in the past. Every language has some of those.

Yup. -- ~Ethan~

Nick Coghlan

7:57 p.m.

New subject: bytes indexing behavior

On 7 June 2016 at 08:59, Ethan Furman wrote:

...

On 06/07/2016 04:27 AM, Antoine Pitrou wrote:

...
While I prefer Python 2's indexing behaviour, IMHO that ship has sailed. Python 3.5 is probably seeing massive adoption now, and by introducing another behaviour switch we only confuse users and make their lives miserable.

I agree on both counts. Wasn't there a proposal/issue about adding some convenience methods to bytes to help ease this pain? My quick search did not find it.

It was in PEP 467: https://www.python.org/dev/peps/pep-0467/#addition-of-optimised-iterator-met... If someone wanted to take that PEP and drive it through to resolution, I'd be happy to hand it over (my recollection is that the sticking point in the previous discussion was the proposed constructor changes, so dropping those may make it easier to get the additional method accepted). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

Ethan Furman

8:31 p.m.

New subject: bytes indexing behavior

On 06/07/2016 12:57 PM, Nick Coghlan wrote:

...

On 7 June 2016 at 08:59, Ethan Furman wrote:

...
On 06/07/2016 04:27 AM, Antoine Pitrou wrote:

...
While I prefer Python 2's indexing behaviour, IMHO that ship has sailed. Python 3.5 is probably seeing massive adoption now, and by introducing another behaviour switch we only confuse users and make their lives miserable.

I agree on both counts. Wasn't there a proposal/issue about adding some convenience methods to bytes to help ease this pain? My quick search did not find it.

It was in PEP 467: https://www.python.org/dev/peps/pep-0467/#addition-of-optimised-iterator-met...

If someone wanted to take that PEP and drive it through to resolution, I'd be happy to hand it over (my recollection is that the sticking point in the previous discussion was the proposed constructor changes, so dropping those may make it easier to get the additional method accepted).

Ah, thanks. Split new thread to resolve that PEP. -- ~Ethan~

Michael Selik

6 Jul 6 Jul

6:37 p.m.

New subject: bytes indexing behavior

On Tue, Jun 7, 2016 at 4:30 PM Ethan Furman wrote:

...

On 06/07/2016 12:57 PM, Nick Coghlan wrote:

...
https://www.python.org/dev/peps/pep-0467/#addition-of-optimised-iterator-met...

...
If someone wanted to take that PEP and drive it through to resolution, I'd be happy to hand it over (my recollection is that the sticking point in the previous discussion was the proposed constructor changes, so dropping those may make it easier to get the additional method accepted).

Split new thread to resolve that PEP.

What's the status of this PEP? Does anyone dislike the suggestion of adding an iterbytes method? I ran into this annoyance again yesterday. The natural way to ensure that all bytes in a bytes object are a particular value (in this case b'z') is: all(byte == b'z' for byte in bytestring) It reads like natural language. Using ``ord`` would work in Python3, but then my code would not be Python2-compatible. And it'd uselessly repeat the call to ord a bunch of times. all(byte == ord(b'z') for byte in bytestring) Instead it seems the best way given the current behavior is to write: len(bytestring) == bytestring.count(b'z') While I wait for PEP 467, can anyone suggest a better way to write that?

Franklin? Lee

8 Jul 8 Jul

5:03 a.m.

New subject: bytes indexing behavior

On Jul 6, 2016 2:40 PM, "Michael Selik" wrote:

...

On Tue, Jun 7, 2016 at 4:30 PM Ethan Furman wrote:

...
On 06/07/2016 12:57 PM, Nick Coghlan wrote:

...

...

...
...
If someone wanted to take that PEP and drive it through to resolution, I'd be happy to hand it over (my recollection is that the sticking point in the previous discussion was the proposed constructor changes, so dropping those may make it easier to get the additional method accepted).

Split new thread to resolve that PEP.

What's the status of this PEP? Does anyone dislike the suggestion of adding an iterbytes method?

I ran into this annoyance again yesterday. The natural way to ensure that all bytes in a bytes object are a particular value (in this case b'z') is:

all(byte == b'z' for byte in bytestring)

It reads like natural language. Using ``ord`` would work in Python3, but

https://www.python.org/dev/peps/pep-0467/#addition-of-optimised-iterator-met... then my code would not be Python2-compatible. And it'd uselessly repeat the call to ord a bunch of times.

...

all(byte == ord(b'z') for byte in bytestring)

Instead it seems the best way given the current behavior is to write:

len(bytestring) == bytestring.count(b'z')

While I wait for PEP 467, can anyone suggest a better way to write that?

How about set(bytestring) == set(b'z')

Michael Selik

4:26 p.m.

New subject: bytes indexing behavior

On Fri, Jul 8, 2016, 1:03 AM Franklin? Lee wrote:

...

On Jul 6, 2016 2:40 PM, "Michael Selik" wrote:

...
Instead it seems the best way given the current behavior is to write:

len(bytestring) == bytestring.count(b'z')

While I wait for PEP 467, can anyone suggest a better way to write that?

How about set(bytestring) == set(b'z')

Ah, that makes sense. I'd thought of using a set literal on the right-hand, but for some reason using set constructor slipped my mind.

...

Franklin? Lee

9 Jul 9 Jul

8:17 p.m.

New subject: bytes indexing behavior

On Jul 8, 2016 12:26 PM, "Michael Selik" wrote:

...

On Fri, Jul 8, 2016, 1:03 AM Franklin? Lee

...

...
On Jul 6, 2016 2:40 PM, "Michael Selik" wrote:

...
Instead it seems the best way given the current behavior is to write:

len(bytestring) == bytestring.count(b'z')

While I wait for PEP 467, can anyone suggest a better way to write

wrote: that?

...

...
How about set(bytestring) == set(b'z')

Ah, that makes sense. I'd thought of using a set literal on the right-hand, but for some reason using set constructor slipped my mind.

The previous method didn't allow short-circuiting. We can use `all` without calling `ord` each time, by the obvious way. z = b'z'[0] #or next(iter(b'z')) all(byte == z for byte in bytestring) Make it a function. def eqall(iterable, value): return all(x == value for x in iterable) eqall(bytestring, b'z'[0]) Rewriting in functional programming style. def eqall(iterable, value): return all(map(value.__eq__, iterable))

Pavol Lisy

10 Jul 10 Jul

6:29 a.m.

New subject: bytes indexing behavior

On 7/9/16, Franklin? Lee wrote:

...

On Jul 8, 2016 12:26 PM, "Michael Selik" wrote:

...
On Fri, Jul 8, 2016, 1:03 AM Franklin? Lee

...
...
On Jul 6, 2016 2:40 PM, "Michael Selik" wrote:

...
Instead it seems the best way given the current behavior is to write:

len(bytestring) == bytestring.count(b'z')

While I wait for PEP 467, can anyone suggest a better way to write

wrote: that?

...
...
How about set(bytestring) == set(b'z')

Ah, that makes sense. I'd thought of using a set literal on the right-hand, but for some reason using set constructor slipped my mind.

The previous method didn't allow short-circuiting. We can use `all` without calling `ord` each time, by the obvious way. z = b'z'[0] #or next(iter(b'z')) all(byte == z for byte in bytestring)

Make it a function. def eqall(iterable, value): return all(x == value for x in iterable)

eqall(bytestring, b'z'[0])

Rewriting in functional programming style. def eqall(iterable, value): return all(map(value.__eq__, iterable))

Or if you want to accept more values: def inall(iterable, values): return all(x in values for x in iterable) inall(bytestring, list(b'zZ')) PS. If you got this mail twice pls sorry - something goes wrong in my mail client.

Thomas Güttler

7 Jun 7 Jun

10:42 a.m.

Am 04.06.2016 um 10:27 schrieb Neil Schemenauer:

...

I can't understand why people don't see we have a problem.

Imagine you are father of a three years old boy. Some stranger tells you: "your kid is ugly!" How does the average man react? Some get sad, some get angry. Only very few would ask: Why do you think my kid is ugly? I am not married with Python. Up to now I see no alternative for me, but I guess sooner or later I will switch to a different language. I see only few benefits from porting my code to Python3. I will use Python2 at least for the next 12 months. Regards, Thomas Güttler -- Thomas Guettler http://www.thomas-guettler.de/

Steven D'Aprano

11:34 a.m.

On Tue, Jun 07, 2016 at 12:42:44PM +0200, Thomas Güttler wrote:

...

I am not married with Python. Up to now I see no alternative for me, but I guess sooner or later I will switch to a different language.

I see only few benefits from porting my code to Python3. I will use Python2 at least for the next 12 months.

That's your right, of course. Some people love to experiment with new languages, some don't. But... What will this other language be, and what does it offer that makes it better than Python 3? When you move to Language X, will you re-write your code to X, or leave it in Python 2 forever? If the answer is "re-write in X", do you think that will be easier than porting it to Python 3? If the answer is "leave it in Python 2 forever", then why can't you do that for your existing code and write new code in Python 3? -- Steve

Thomas Güttler

6:27 p.m.

Am 07.06.2016 um 13:34 schrieb Steven D'Aprano:

...

On Tue, Jun 07, 2016 at 12:42:44PM +0200, Thomas Güttler wrote:

...
I am not married with Python. Up to now I see no alternative for me, but I guess sooner or later I will switch to a different language.

I see only few benefits from porting my code to Python3. I will use Python2 at least for the next 12 months.

That's your right, of course. Some people love to experiment with new languages, some don't. But...

What will this other language be, and what does it offer that makes it better than Python 3?

I would love to see "compile to javascript" support. I would love to see a package management that is easy to use and which focuses on simple data structures.

...

When you move to Language X, will you re-write your code to X, or leave it in Python 2 forever?

I guess I will use language X for a new project.

...

If the answer is "re-write in X", do you think that will be easier than porting it to Python 3?

If the answer is "leave it in Python 2 forever", then why can't you do that for your existing code and write new code in Python 3?

I could write new code in Python3, but I see big no benefit. (I see benefit, but no big) Does Python3 run in web browsers? Or is it possible to compile it to javascript? I guess I will port my python2 code to python3 sooner or later. If Django drops support for Python2, then I will do it. Regards, Thomas Güttler -- http://www.thomas-guettler.de/

Oleg Broytman

6:37 p.m.

On Tue, Jun 07, 2016 at 08:27:52PM +0200, Thomas G??ttler wrote:

...

I would love to see "compile to javascript" support.

http://www.brython.info/ http://www.transcrypt.org/ http://pypyjs.org/ https://github.com/azazel75/metapensiero.pj https://github.com/amirouche/pythonium

...

I would love to see a package management that is easy to use and which focuses on simple data structures.

virtualenv + pip Oleg. -- Oleg Broytman http://phdru.name/ phd@phdru.name Programmers don't die, they just GOSUB without RETURN.

Ian Kelly

6:37 p.m.

On Tue, Jun 7, 2016 at 12:27 PM, Thomas Güttler wrote:

...

Does Python3 run in web browsers? Or is it possible to compile it to javascript?

http://brython.info/ https://pypi.python.org/pypi/pythonium http://pypyjs.org/ http://www.skulpt.org/ Brython explicitly claims Python 3 support. I don't know how close the others are.

Ian Foote

7:08 p.m.

On 07/06/16 19:27, Thomas Güttler wrote:

...

Am 07.06.2016 um 13:34 schrieb Steven D'Aprano:

...
On Tue, Jun 07, 2016 at 12:42:44PM +0200, Thomas Güttler wrote:

...
I am not married with Python. Up to now I see no alternative for me, but I guess sooner or later I will switch to a different language.

I see only few benefits from porting my code to Python3. I will use Python2 at least for the next 12 months.

That's your right, of course. Some people love to experiment with new languages, some don't. But...

What will this other language be, and what does it offer that makes it better than Python 3?

I would love to see "compile to javascript" support.

I would love to see a package management that is easy to use and which focuses on simple data structures.

...
When you move to Language X, will you re-write your code to X, or leave it in Python 2 forever?

I guess I will use language X for a new project.

...
If the answer is "re-write in X", do you think that will be easier than porting it to Python 3?

If the answer is "leave it in Python 2 forever", then why can't you do that for your existing code and write new code in Python 3?

I could write new code in Python3, but I see big no benefit. (I see benefit, but no big)

Does Python3 run in web browsers? Or is it possible to compile it to javascript?

I guess I will port my python2 code to python3 sooner or later. If Django drops support for Python2, then I will do it.

Regards, Thomas Güttler

Batavia (https://github.com/pybee/batavia) is worth keeping an eye on (for browser support), as is the rest of the BeeWare suite (http://pybee.org/). Regards, Ian F

Nick Coghlan

8:21 p.m.

On 7 June 2016 at 11:27, Thomas Güttler wrote:

...

Am 07.06.2016 um 13:34 schrieb Steven D'Aprano:

...
On Tue, Jun 07, 2016 at 12:42:44PM +0200, Thomas Güttler wrote:

...
I am not married with Python. Up to now I see no alternative for me, but I guess sooner or later I will switch to a different language.

I see only few benefits from porting my code to Python3. I will use Python2 at least for the next 12 months.

That's your right, of course. Some people love to experiment with new languages, some don't. But...

What will this other language be, and what does it offer that makes it better than Python 3?

I would love to see "compile to javascript" support.

Not a problem for python-dev to tackle - there are plenty of Python-in-the-browser projects, and they don't need to wait for python-dev's permission (since they're taking the existing language and making it available in a new context, rather than changing the language). They'd only need to come back to python-dev if they wanted language level support for browser concepts like WebWorkers and the Domain Object Model.

...

I would love to see a package management that is easy to use and which focuses on simple data structures.

Also not python-dev's problem - that one falls on distutils-sig, PyPA and the PSF (but unfortunately is never going to be entirely simple given Python's broad scope of use).

...

Does Python3 run in web browsers? Or is it possible to compile it to javascript?

Again, these are software distribution questions, not language design questions.

...

I guess I will port my python2 code to python3 sooner or later. If Django drops support for Python2, then I will do it.

Exactly, and this is what we expect most folks with problems that are already well solved by Python 2 to do: it won't be the availability of alternatives that prompts them to find something else (since there are already a number of those in the form of both Python 3 and other language ecosystems), it will be folks ceasing to support the Python 2 ecosystem for free. By the time upstream Python 2.7 support ends in 2020, there will have been ~14 years of parallel development on both Python 2.x and 3.x, and ~12 years of parallel support. This isn't a new phenomenon - it's one that's seen regularly at the operating system level, to the point where people will pay exhorbitant amounts of money to keep old software supported, even though they could attain both happier staff and lower operating costs by doing a bit more forward planning and budgeting for their infrastructure upgrades. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

Thomas Güttler

8 Jun 8 Jun

11:02 a.m.

New subject: package management is not python-dev's problem :-(

...

...
I would love to see a package management that is easy to use and which focuses on simple data structures.

Also not python-dev's problem - that one falls on distutils-sig, PyPA and the PSF (but unfortunately is never going to be entirely simple given Python's broad scope of use).

This statement make me sad. Why is this not python-dev's problem? Regards, Thomas Güttler -- Thomas Guettler http://www.thomas-guettler.de/

Paul Moore

11:10 a.m.

New subject: package management is not python-dev's problem :-(

On 8 June 2016 at 12:02, Thomas Güttler wrote:

...

...
...
I would love to see a package management that is easy to use and which focuses on simple data structures.

Also not python-dev's problem - that one falls on distutils-sig, PyPA and the PSF (but unfortunately is never going to be entirely simple given Python's broad scope of use).

This statement make me sad. Why is this not python-dev's problem?

Because distutils-sig is the central point for packaging related issues. Paul

Donald Stufft

11:14 a.m.

New subject: package management is not python-dev's problem :-(

...

On Jun 8, 2016, at 7:02 AM, Thomas Güttler wrote:

...
...
I would love to see a package management that is easy to use and which focuses on simple data structures.

Also not python-dev's problem - that one falls on distutils-sig, PyPA and the PSF (but unfortunately is never going to be entirely simple given Python's broad scope of use).

This statement make me sad. Why is this not python-dev's problem?

Historically python-dev hasn’t been involved much in it and almost all of the tooling for packaging is developed externally to Python itself. This is generally a strength, since packaging tools tend to work best when they’re not lied to the lifetime of the language itself. In the abstract, it *could* be python-dev’s problem, but Python as a community didn’t choose that solution, it chose to delegate that task outside of it and I (though I am biased) don’t think that trying to move it into the python-dev domain brings much in the way of benefits. — Donald Stufft

Nick Coghlan

9 Jun 9 Jun

9:32 p.m.

New subject: package management is not python-dev's problem :-(

On 8 June 2016 at 04:14, Donald Stufft wrote:

...

...
On Jun 8, 2016, at 7:02 AM, Thomas Güttler wrote:

...
...
I would love to see a package management that is easy to use and which focuses on simple data structures.

Also not python-dev's problem - that one falls on distutils-sig, PyPA and the PSF (but unfortunately is never going to be entirely simple given Python's broad scope of use).

This statement make me sad. Why is this not python-dev's problem?

Historically python-dev hasn’t been involved much in it and almost all of the tooling for packaging is developed externally to Python itself. This is generally a strength, since packaging tools tend to work best when they’re not lied to the lifetime of the language itself.

To help put some specifics on that: * for the core runtime and standard library, having features appear only in new Python versions is normal and expected, and the related update cycles are measured in months (maintenance releases) or years (feature releases) * for build and distribution tools, it's highly desirable to offer a consistent feature set across all widely deployed Python versions (since there is only one PyPI shared amongst all versions), and the related update cycles tend to be measured in weeks (maintenance releases) or months (feature releases) As such, distutils-sig/PyPA maintain Python 2/3 compatible tooling on behalf of not only CPython, but also PyPy, Jython, IronPython, and other sufficiently compatible alternate implementations. There are some *people* that participate in both communities (including folks that are both PyPA contributors and CPython core developers), and there are some issues that remain specifically python-dev's responsibility (such as providing recent versions of pip by default as part of CPython installations), but the day-to-day design discussions are separate. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

Ben Finney

8 Jun 8 Jun

8:12 p.m.

New subject: package management is not python-dev's problem :-(

Thomas Güttler writes:

...

...
...
I would love to see a package management that is easy to use and which focuses on simple data structures.

Also not python-dev's problem - that one falls on distutils-sig, PyPA and the PSF (but unfortunately is never going to be entirely simple given Python's broad scope of use).

This statement make me sad. Why is this not python-dev's problem?

Because the management of packaging and distribution tools is not very much related to development of the core language. The other teams mentioned above do a demonstrably better job, so ‘python-dev’ gets out of their way. -- \ “Don't be misled by the enormous flow of money into bad defacto | `\ standards for unsophisticated buyers using poor adaptations of | _o__) incomplete ideas.” —Alan Kay | Ben Finney

Guido van Rossum

7 Jun 7 Jun

3:26 p.m.

On Tue, Jun 7, 2016 at 3:42 AM, Thomas Güttler

...

wrote:

...

Am 04.06.2016 um 10:27 schrieb Neil Schemenauer:

...
I can't understand why people don't see we have a problem.

Imagine you are father of a three years old boy. Some stranger tells you: "your kid is ugly!"

Just in case you didn't realize this can be turned around too, that's exactly how it feels to *me* when people complain about Python 3.

...

How does the average man react? Some get sad, some get angry.

Only very few would ask: Why do you think my kid is ugly?

I am not married with Python. Up to now I see no alternative for me, but I guess sooner or later I will switch to a different language.

I see only few benefits from porting my code to Python3. I will use Python2 at least for the next 12 months.

That sounds like a threat (or another underhanded insult), but it's really up to you. You have until 2020, and in fact after that date Python 2 won't suddenly implode -- but none of the Python core devs who currently still keep 2.7 alive will help you maintain it. Of course it's open source so you are free to fork it any time. Just give your fork a distinctive name so potential users aren't confused about what they'll be getting and where they'd have to go for support. -- --Guido van Rossum (python.org/~guido)

Michael Selik

5:32 p.m.

On Tue, Jun 7, 2016 at 11:26 AM Guido van Rossum wrote:

...

Of course it's open source so you are free to fork it any time. Just give your fork a distinctive name so potential users aren't confused about what they'll be getting and where they'd have to go for support

If only we were talking about Python 4 vs 5, then the obvious name would be "Python 4evah"

Oleg Broytman

5:35 p.m.

On Tue, Jun 07, 2016 at 05:32:06PM +0000, Michael Selik wrote:

...

On Tue, Jun 7, 2016 at 11:26 AM Guido van Rossum wrote:

...
Of course it's open source so you are free to fork it any time. Just give your fork a distinctive name so potential users aren't confused about what they'll be getting and where they'd have to go for support

If only we were talking about Python 4 vs 5, then the obvious name would be "Python 4evah"

Python 2therescue Oleg. -- Oleg Broytman http://phdru.name/ phd@phdru.name Programmers don't die, they just GOSUB without RETURN.

Thomas Güttler

8 Jun 8 Jun

10:56 a.m.

Am 07.06.2016 um 17:26 schrieb Guido van Rossum:

...

On Tue, Jun 7, 2016 at 3:42 AM, Thomas Güttler mailto:guettliml@thomas-guettler.de> wrote:

Am 04.06.2016 um 10:27 schrieb Neil Schemenauer: > I can't understand why people don't see > we have a problem.

Imagine you are father of a three years old boy. Some stranger tells you: "your kid is ugly!"

Just in case you didn't realize this can be turned around too, that's exactly how it feels to *me* when people complain about Python 3.

Yes, this hurts. But nevertheless you listen and try to understand the reasons. That's why Python is successful. Thank you Guido!

...

How does the average man react? Some get sad, some get angry.

Only very few would ask: Why do you think my kid is ugly?

I am not married with Python. Up to now I see no alternative for me, but I guess sooner or later I will switch to a different language.

I see only few benefits from porting my code to Python3. I will use Python2 at least for the next 12 months.

That sounds like a threat (or another underhanded insult), but it's really up to you.

From Wikipedia: ... a devil's advocate is someone who, given a certain argument, takes a position they do not necessarily agree with (or simply an alternative position from the accepted norm), for the sake of debate or to explore the thought further. Back to Python3: I compare the 3.2 and current docs about "Porting Python 2 Code to Python 3" https://docs.python.org/3.2/howto/pyporting.html https://docs.python.org/3.5/howto/pyporting.html The 3.2 version started with "Choosing a Strategy" the current docs start with "1. ..., 2. ..., 3. ..." I read the porting docs years ago, and I was not able to choose a strategy. I was unsure. (sheeps don't like fog) I think, at least sometimes, docs are more important than code. The current docs give me a nice roadmap. I will follow them, but not in 2016. Regards, Thomas Güttler Regards, Thomas Güttler -- Thomas Guettler http://www.thomas-guettler.de/

Brett Cannon

3:48 p.m.

On Wed, Jun 8, 2016, 03:57 Thomas Güttler wrote:

...

Am 07.06.2016 um 17:26 schrieb Guido van Rossum:

...
On Tue, Jun 7, 2016 at 3:42 AM, Thomas Güttler < guettliml@thomas-guettler.de mailto:guettliml@thomas-guettler.de> wrote:

Am 04.06.2016 um 10:27 schrieb Neil Schemenauer: > I can't understand why people don't see > we have a problem.

Imagine you are father of a three years old boy. Some stranger tells you: "your kid is ugly!"

Just in case you didn't realize this can be turned around too, that's exactly how it feels to *me* when people complain about Python 3.

Yes, this hurts. But nevertheless you listen and try to understand the reasons. That's why Python is successful. Thank you Guido!

...
How does the average man react? Some get sad, some get angry.

Only very few would ask: Why do you think my kid is ugly?

I am not married with Python. Up to now I see no alternative for me, but I guess sooner or later I will switch to a different language.

I see only few benefits from porting my code to Python3. I will use Python2 at least for the next 12 months.

That sounds like a threat (or another underhanded insult), but it's

really up to you.

From Wikipedia: ... a devil's advocate is someone who, given a certain argument, takes a position they do not necessarily agree with (or simply an alternative position from the accepted norm), for the sake of debate or to explore the thought further.

The point is that you don't state you're playing devil's advocate and you phrase your statements in such a way that they can be (mis)construed as an insult. There's nothing wrong with stating the other view of a discussion, but you still have to take care to state it politely.

...

Back to Python3: I compare the 3.2 and current docs about "Porting Python 2 Code to Python 3"

https://docs.python.org/3.2/howto/pyporting.html https://docs.python.org/3.5/howto/pyporting.html

The 3.2 version started with "Choosing a Strategy" the current docs start with "1. ..., 2. ..., 3. ..."

I read the porting docs years ago, and I was not able to choose a strategy. I was unsure. (sheeps don't like fog) I think, at least sometimes, docs are more important than code.

The current docs give me a nice roadmap. I will follow them, but not in 2016.

I changed the docs when it became obvious the community had chosen the 2/3 support structure for porting code. -brett

...

Regards, Thomas Güttler

Regards, Thomas Güttler

-- Thomas Guettler http://www.thomas-guettler.de/ _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

2846

Age (days ago)

2883

Last active (days ago)

List overview

Download

68 comments

30 participants

participants (30)

Antoine Pitrou
Ben Finney
Brett Cannon
Carl Meyer
Chris Barker
Donald Stufft
Ethan Furman
Franklin? Lee
Giampaolo Rodola'
Guido van Rossum
Ian Foote
Ian Kelly
Jason Fried
Joao S. O. Bueno
Jonathan Goble
Michael Selik
Neil Schemenauer
Nick Coghlan
Nikolaus Rath
Oleg Broytman
Paul Moore
Pavol Lisy
Random832
Serhiy Storchaka
Sjoerd Job Postmus
Stephen J. Turnbull
Steven D'Aprano
Terry Reedy
Thomas Güttler
tritium-list＠sdamon.com

Smoothing transition to Python 3

Neil Schemenauer

Neil Schemenauer

Neil Schemenauer

Neil Schemenauer

Neil Schemenauer

Ian Kelly

tags

participants (30)