[Python-Dev] Division of tool labour in porting Python 2 code to 2/3

[Brett Cannon]

On Fri Jun 06 2014 at 2:29:13 PM, Terry Reedy <tjreedy at udel.edu> wrote:

> On 6/6/2014 12:37 PM, Brett Cannon wrote:
> > After Glyph and Alex's email about their asks for assisting in writing
> > Python 2/3 code, it got me thinking about where in the toolchain various
> > warnings and such should go in order to help direct energy to help
> > develop whatever future toolchain to assist in porting.
> >
> > There seems to be three places where issues are/can be caught once a
> > project has embarked down the road of 2/3 source compatibility:
> >
> >  1. -3 warnings
> >  2. Some linter tool
> >  3. Failing tests
> >
> > -3 warnings are things that we know are flat-out wrong and do not cause
> > massive compatibility issues in the stdlib. For instance, warning that
> > buffer() is not in Python 3 is a py3k warning -- Glyph made a mistake
> > when he asked for it as a new warning -- is a perfect example of
> > something that isn't excessively noisy and won't cause issues when
> > people run with it.
> >
> > But what about warning about classic classes? The stdlib is full of them
> > and they were purposefully left alone for compatibility reasons. But
> > there is a subtle semantic difference between classic and new-style
> > classes,
>
> A non-subtle difference is that old style classes do not have .__new__.
> I just ran into this when backporting an Idle test to 2.7. (I rewrote
> the test to avoid diverging the code). In retrospect, perhaps we should
> have added a global 'new-class future' -C switch, like -Q, and made sure
> that stdlib worked either way. People running 2and3 code could then run
> 2.x with the switch. Is is possible to add this now?
>

I consider changing the CLI out of bounds in a bugfix release as it's part
of the API of Python.


>
>  > and so 2/3 code should consider switching
>
> I do not not understand what you mean without a switch or furture
> statement available to switch from old to new in 2.7.
>

Run a 2to3 fixer that changes all of their classes to new-style.


>
> > (this is when people
> > chime in saying "this is why we want a 2.8 release!", but that still
> > isn't happening). If this were made a py3k warning in 2.7 then the
> > stdlib itself would spew out warnings which we can't change due to
> > compatibility, so that makes it not useful
> > (http://bugs.python.org/issue21231).
>
> Don't issue the warning if the class is in the stdlib.
> If the warning is issued *after* creating class C:
>
> f = C.__module__.__file__
> if classic(C) and (not 'lib' in f or 'site_packages' in f):
>    warn(...)
>
> On Windows, the directory is 'Lib'; I presume it is lowercased
> everywhere. If not, adjust.
>

That's just asking for trouble. I don't want to be import-dependent like
that in the stdlib.


>
>  > But as part of a lint tool specific
> > to Python 2.7 that kind of warning would not be an issue and is easily
> > managed and integrated into CI setups to make sure there are no
> regressions.
> >
> > Lastly, there are things like string/unicode comparisons.
> > http://bugs.python.org/issue21401 has a patch from VIctor which warns
> > when comparing strings and unicode in Python 2.7. Much like the classic
> > classes example, the stdlib becomes rather noisy due to APIs that handle
> > either/or, etc. But unlike the classic classes example, you just can't
> > systematically verify that two variables are always going to be str vs.
> > unicode in Python 2.7 if they aren't literals. If people want to
> > implement type constraint graphs for 2.7 code to help find them then
> > that's great, but I personally don't have that kind of time. In this
> > instance it would seem like relying on a project's unit tests to find
> > this sort of problem is the best option.
> >
> > With those three levels in mind, where do we draw the line between these
> > levels? Take for instance the print statement. Right now there is no
> > warning with -3. Do we add one and then update the 2.7 stdlib to prevent
> > warnings being generated by the stdlib?
>
> Make conditional as with class.
>
> We *could* change 'print s' to the exactly equivalent 'print(s)'
> (perhaps half the cases); 'print r, s' to "print('%s %s' % (r,s)),
> 'print 'xxxx', y' to "print('xxxx %s' % y), and so on. However, 'print
>  >>self.stdout, x', etc, does not translate to a pseudo-call. It would
> need transltion to "self.stdout.write(x+'\n')".  Grepping 2.7.6 lib/*.py
> for print gives 1341 hits, with at least 1000 being actual print
> statements.
>

Yep, which is why I don't want to do a 2to3 run on the stdlib to get rid of
them. I also want to minimize conditional checks as it leads to potential
issues of people thinking it's okay not to change things when there are in
actually differences (e.g. I don't want to promote classic classes or
native strings if it can be helped for the vast majority of users).

-Brett


>
>  > Or do we add it to some linter
>  > tool to pick up when people accidentally leave one in their code?
>
> > The reason I ask is since this is clear I'm willing to spearhead the
> > tooling work we talked about at the language summit to make sure there's
> > a clear path for people wanting to port which is as easy as (reasonably)
> > possible, but I don't want to start on it until I have a clear
> > indication of what people are going to be okay with.
>
>
> --
> Terry Jan Reedy
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> brett%40python.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140606/170fc076/attachment.html>