Mailman 3 PEP 563 and Python 3.10. - python-committers

PEP 563 and Python 3.10.

Thomas Wouters

20 Apr 2021 20 Apr '21

11:57 a.m.

(Starting a new thread so as not to derail any of the ongoing discussions.)

Thanks, everyone, for your thoughts on Python 3.10 and the impact of PEP 563 (postponed evaluation of annotations) becoming the default. The Steering Council has considered the issue carefully, along with many of the proposed alternatives and solutions, and we’ve decided that at this point, we simply can’t risk the compatibility breakage of PEP 563. We need to roll back the change that made stringified annotations the default, at least for 3.10. (Pablo is already working on this.)

To be clear, we are not reverting PEP 563 itself. The future import will keep working like it did since Python 3.7. We’re delaying making PEP 563 string-based annotations the default until Python 3.11. This will give us time to find a solution that works for everyone (or to find a feasible upgrade path for users who currently rely on evaluated annotations). Some considerations that led us to this decision:

PEP 563’s default change is clearly too disruptive to downstream users and third-party libraries to happen right now. We can’t risk breaking even a small subset of the FastAPI/pydantic users, not to mention other uses of evaluated type annotations that we’re not aware of yet.
PEP 563 provides no warning to users of the feature it’s disabling. Without that, we can’t expect users to be aware of the upcoming breakage. The lack of a warning was by design, and made sense in a world where type annotations were only consumed by static type checkers --- but that’s not actually the situation we’re in. There are clearly existing real-world, run-time uses of type annotations that would be adversely affected by this change.
Originally, PEP 563 was scheduled to take effect in Python 4, and this changed recently (after the discussion in the Language Summit of 2020). It's possible that third-party libraries and users didn’t plan to react in the current time frame as they were not aware of this change in timing.
There isn’t enough time to properly discuss PEP 649 or any of the alternatives before the beta 1 deadline, and we really need to make sure we don’t compound errors here. We need to look for a long term solution, which isn’t possible while still maintaining the release deadlines of Python 3.10. That means we’re also deferring PEP 649 to Python 3.11.

In the Steering Council’s unanimous opinion, rolling back the default flip for stringified annotations in Python 3.10 is the least disruptive of all the options.

We need to continue discussing the issue and potential solutions, since this merely postpones the problem until 3.11. (For the record, postponing the change further is not off the table, either, for example if the final decision is to treat evaluated annotations as a deprecated feature, with warnings on use.)

For what it’s worth, the SC is also considering what we can do to reduce the odds of something like this happening again, but that’s a separate consideration, and a multi-faceted one at that.

For the Steering Council, Thomas.

Thomas Wouters thomas@python.org

Hi! I'm an email virus! Think twice before sending your email to help me spread!

Attachments:

attachment.htm (text/html — 3.4 KB)

Show replies by date

Guido van Rossum

20 Apr 20 Apr

12:07 p.m.

Thanks to the Steering Council! You have the wisdom of Solomon. Rolling back the code that made PEP 563 the default behavior is the only sensible solution for 3.10.

On Tue, Apr 20, 2021 at 11:58 AM Thomas Wouters thomas@python.org wrote:

...

(Starting a new thread so as not to derail any of the ongoing discussions.)

Thanks, everyone, for your thoughts on Python 3.10 and the impact of PEP 563 (postponed evaluation of annotations) becoming the default. The Steering Council has considered the issue carefully, along with many of the proposed alternatives and solutions, and we’ve decided that at this point, we simply can’t risk the compatibility breakage of PEP 563. We need to roll back the change that made stringified annotations the default, at least for 3.10. (Pablo is already working on this.)

To be clear, we are not reverting PEP 563 itself. The future import will keep working like it did since Python 3.7. We’re delaying making PEP 563 string-based annotations the default until Python 3.11. This will give us time to find a solution that works for everyone (or to find a feasible upgrade path for users who currently rely on evaluated annotations). Some considerations that led us to this decision:

PEP 563’s default change is clearly too disruptive to downstream users and third-party libraries to happen right now. We can’t risk breaking even a small subset of the FastAPI/pydantic users, not to mention other uses of evaluated type annotations that we’re not aware of yet.

PEP 563 provides no warning to users of the feature it’s disabling. Without that, we can’t expect users to be aware of the upcoming breakage. The lack of a warning was by design, and made sense in a world where type annotations were only consumed by static type checkers --- but that’s not actually the situation we’re in. There are clearly existing real-world, run-time uses of type annotations that would be adversely affected by this change.

Originally, PEP 563 was scheduled to take effect in Python 4, and this changed recently (after the discussion in the Language Summit of 2020). It's possible that third-party libraries and users didn’t plan to react in the current time frame as they were not aware of this change in timing.

There isn’t enough time to properly discuss PEP 649 or any of the alternatives before the beta 1 deadline, and we really need to make sure we don’t compound errors here. We need to look for a long term solution, which isn’t possible while still maintaining the release deadlines of Python 3.10. That means we’re also deferring PEP 649 to Python 3.11.

In the Steering Council’s unanimous opinion, rolling back the default flip for stringified annotations in Python 3.10 is the least disruptive of all the options.

We need to continue discussing the issue and potential solutions, since this merely postpones the problem until 3.11. (For the record, postponing the change further is not off the table, either, for example if the final decision is to treat evaluated annotations as a deprecated feature, with warnings on use.)

For what it’s worth, the SC is also considering what we can do to reduce the odds of something like this happening again, but that’s a separate consideration, and a multi-faceted one at that.

For the Steering Council, Thomas.

Thomas Wouters thomas@python.org

Hi! I'm an email virus! Think twice before sending your email to help me spread!

python-committers mailing list -- python-committers@python.org To unsubscribe send an email to python-committers-leave@python.org https://mail.python.org/mailman3/lists/python-committers.python.org/ Message archived at https://mail.python.org/archives/list/python-committers@python.org/message/C... Code of Conduct: https://www.python.org/psf/codeofconduct/

-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...

Larry Hastings

1:27 p.m.

I commend the Steering Council for its wise decision. I'm sure that once the Python community spends more time considering this issue, and innovating new solutions, we can come up with a path forward that'll be good news for everybody.

Best wishes,

//arry/

On 4/20/21 11:57 AM, Thomas Wouters wrote:

...

(Starting a new thread so as not to derail any of the ongoing discussions.)

Thanks, everyone, for your thoughts on Python 3.10 and the impact of PEP 563 (postponed evaluation of annotations) becoming the default. The Steering Council has considered the issue carefully, along with many of the proposed alternatives and solutions, and we’ve decided that at this point, we simply can’t risk the compatibility breakage of PEP 563. We need to roll back the change that made stringified annotations the default, at least for 3.10. (Pablo is already working on this.)

To be clear, we are not reverting PEP 563 itself. The future import will keep working like it did since Python 3.7. We’re delaying making PEP 563 string-based annotations the default until Python 3.11. This will give us time to find a solution that works for everyone (or to find a feasible upgrade path for users who currently rely on evaluated annotations). Some considerations that led us to this decision:

- PEP 563’s default change is clearly too disruptive to downstream users and third-party libraries to happen right now. We can’t risk breaking even a small subset of the FastAPI/pydantic users, not to mention other uses of evaluated type annotations that we’re not aware of yet. - PEP 563 provides no warning to users of the feature it’s disabling. Without that, we can’t expect users to be aware of the upcoming breakage. The lack of a warning was by design, and made sense in a world where type annotations were only consumed by static type checkers --- but that’s not actually the situation we’re in. There are clearly existing real-world, run-time uses of type annotations that would be adversely affected by this change. - Originally, PEP 563 was scheduled to take effect in Python 4, and this changed recently (after the discussion in the Language Summit of 2020). It's possible that third-party libraries and users didn’t plan to react in the current time frame as they were not aware of this change in timing. - There isn’t enough time to properly discuss PEP 649 or any of the alternatives before the beta 1 deadline, and we really need to make sure we don’t compound errors here. We need to look for a long term solution, which isn’t possible while still maintaining the release deadlines of Python 3.10. That means we’re also deferring PEP 649 to Python 3.11.

In the Steering Council’s unanimous opinion, rolling back the default flip for stringified annotations in Python 3.10 is the least disruptive of all the options.

We need to continue discussing the issue and potential solutions, since this merely postpones the problem until 3.11. (For the record, postponing the change further is not off the table, either, for example if the final decision is to treat evaluated annotations as a deprecated feature, with warnings on use.)

For what it’s worth, the SC is also considering what we can do to reduce the odds of something like this happening again, but that’s a separate consideration, and a multi-faceted one at that.

For the Steering Council, Thomas.

Thomas Wouters mailto:thomas@python.org>

Hi! I'm an email virus! Think twice before sending your email to help me spread!

python-committers mailing list -- python-committers@python.org To unsubscribe send an email to python-committers-leave@python.org https://mail.python.org/mailman3/lists/python-committers.python.org/ Message archived at https://mail.python.org/archives/list/python-committers@python.org/message/C... Code of Conduct: https://www.python.org/psf/codeofconduct/

Terry Reedy

3:08 p.m.

On 4/20/2021 2:57 PM, Thomas Wouters wrote:

...

We need to roll back the change that made stringified annotations the default, at least for 3.10. (Pablo is already working on this.)

I also agree and thank Pablo in advance.

...

- PEP 563 provides no warning to users of the feature it’s disabling.

Removals or major changes to callables usually get deprecation warnings. These are relatively easy to add by adding code to the object, to be triggered at runtime. I believe something similar can be done with modules, triggered on import, but I have never been involved with such.

Warning about other behavior changes may be needed just as much, but are likely to be harder. I think removing the ability to get object annotations is such a case. It was and needed and likely will be for 3.10, depending on what we decide for 3.11. But the need for a warning would have been and likely will be dependent on the presence of a __future__ import. So the compiler will have to be involved somehow.

The addition of with statements, for instance, was quite different and on balance did not need a deprecation statement. The breaking behavior change was removing 'with' as a candidate for name binding. Detecting such bindings would have been relatively expensive, while the consequence of being caught unaware was having to change a name later rather than sooner, but not losing the underlying code behavior.

The nearest parallel I can think of to this syntax-related type change is the switch for string exceptions to exception subclass exceptions. I believe that at the end, raise 'somestring' got a deprecation notice. This difference is the the change was in the syntax input, where as the annotation change is in the somewhat hidden syntax output.

...

We need to continue discussing the issue and potential solutions, since

...

this merely postpones the problem until 3.11. (For the record, postponing the change further is not off the table, either, for example

...

if the final decision is to treat evaluated annotations as a deprecated

...

feature, with warnings on use.)

Yes, discussion should continue now both while the issue is 'hot' and because adding an appropriate deprecation warning in 3.10.0 requires a decision before it becomes final.

...

For what it’s worth, the SC is also considering what we can do to reduce the odds of something like this happening again, but that’s a separate consideration, and a multi-faceted one at that.

Broadly speaking, statements are functions and their output, as well as input, should be subject to our normal deprecation and suggested replacement policy.

-- Terry Jan Reedy

Thomas Wouters

21 Apr 21 Apr

3:16 a.m.

(For the record, I'm not responding for the whole SC here.)

On Wed, Apr 21, 2021 at 12:08 AM Terry Reedy tjreedy@udel.edu wrote:

...

On 4/20/2021 2:57 PM, Thomas Wouters wrote:

...

PEP 563 provides no warning to users of the feature it’s disabling.

Removals or major changes to callables usually get deprecation warnings. These are relatively easy to add by adding code to the object, to be triggered at runtime. I believe something similar can be done with modules, triggered on import, but I have never been involved with such.

Warning about other behavior changes may be needed just as much, but are likely to be harder. I think removing the ability to get object annotations is such a case. It was and needed and likely will be for 3.10, depending on what we decide for 3.11. But the need for a warning would have been and likely will be dependent on the presence of a __future__ import. So the compiler will have to be involved somehow.

The addition of with statements, for instance, was quite different and on balance did not need a deprecation statement. The breaking behavior change was removing 'with' as a candidate for name binding. Detecting such bindings would have been relatively expensive, while the consequence of being caught unaware was having to change a name later rather than sooner, but not losing the underlying code behavior.

I'm not sure what point you're trying to make here. Use of 'with' as identifier *was* detected. They *were* warned about. It wasn't particularly expensive. This is why __future__.with_statement exists. In Python 2.5, using 'with' (and 'as') as identifiers issued a warning. You couldn't use 'with' as a statement without using the future import.

The idea that we should warn before significant changes to behaviour -- documented behaviour, like function annotations being evaluated at definition time, or behaviour commonly depended on, like 'with' being allowed as an identifier because it wasn't explicitly listed as a keyword -- is not new. It's covered by PEP 387: https://www.python.org/dev/peps/pep-0387/

The question isn't really whether we should raise a warning, or whether we can, the question is how much effort it's going to be. I'm sure it could be done by, for example, making a new API (for example, putting stringified annotations in a different attribute) and warning on any use of the old API (if we decide to go forward with PEP 563 over the alternatives). That would be more disruptive to code already adjusted for stringified annotations, so perhaps there are better alternatives.

It may be that we need to adjust the expectations around PEP 387 (our own and our users') if we find that making desirable changes becomes difficult. Perhaps we need different safeguards against incompatible changes than "make sure users have a chance to see warnings for at least 2 releases". For example, perhaps we need a separate validation tool, something that will analyse code and tell the user if their code is going to be safe for a particular Python release. That would be a big undertaking, since it's not enough to provide such a tool, we also have to make very clear that users need to use it if they want to be compatible with future Python releases. (I just say 'users' here, but we all know there are very many different kinds of users of Python, so it's not as simple as putting out a blog post either.)

In the meantime we should strive to honour the commitments we made in PEP 387 and elsewhere, and not pull the rug out from under users.

The nearest parallel I can think of to this syntax-related type change

...

is the switch for string exceptions to exception subclass exceptions. I believe that at the end, raise 'somestring' got a deprecation notice. This difference is the the change was in the syntax input, where as the annotation change is in the somewhat hidden syntax output.

...
We need to continue discussing the issue and potential solutions, since

...
this merely postpones the problem until 3.11. (For the record, postponing the change further is not off the table, either, for example

...
if the final decision is to treat evaluated annotations as a deprecated

...
feature, with warnings on use.)

Yes, discussion should continue now both while the issue is 'hot' and because adding an appropriate deprecation warning in 3.10.0 requires a decision before it becomes final.

...
For what it’s worth, the SC is also considering what we can do to reduce the odds of something like this happening again, but that’s a separate consideration, and a multi-faceted one at that.

Broadly speaking, statements are functions and their output, as well as input, should be subject to our normal deprecation and suggested replacement policy.

What I meant by that paragraph was more along the lines of making sure third-party uses of Python feel comfortable bringing these issues to our attention earlier.

-- Thomas Wouters thomas@python.org

Hi! I'm an email virus! Think twice before sending your email to help me spread!

M.-A. Lemburg

4:05 a.m.

On 21.04.2021 12:16, Thomas Wouters wrote:

...

The idea that we should warn before significant changes to behaviour -- documented behaviour, like function annotations being evaluated at definition time, or behaviour commonly depended on, like 'with' being allowed as an identifier because it wasn't explicitly listed as a keyword -- is not new. It's covered by PEP 387: https://www.python.org/dev/peps/pep-0387/

Perhaps we should reconsider making deprecation warnings only visible by explicitly enabling them and instead make them visible by default.

This would create more noise for users, but for the better, since planned changes then become more visible and can be addressed either by silencing the warning (and opening a ticket to get the change addressed) or by fixing the code in a new release.

If package authors were to get into the habit of doing the silencing for their users after opening a ticket, that would probably make the whole process more streamlined and effective.

-- Marc-Andre Lemburg eGenix.com

Professional Python Services directly from the Experts (#1, Apr 21 2021)

...

...
...
Python Projects, Coaching and Support ... https://www.egenix.com/ Python Product Development ... https://consulting.egenix.com/

::: We implement business ideas - efficiently in both time and costs :::

eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 https://www.egenix.com/company/contact/ https://www.malemburg.com/

Paul Moore

4:14 a.m.

On Wed, 21 Apr 2021 at 12:05, M.-A. Lemburg mal@egenix.com wrote:

...

Perhaps we should reconsider making deprecation warnings only visible by explicitly enabling them and instead make them visible by default.

This would create more noise for users, but for the better, since planned changes then become more visible and can be addressed either by silencing the warning (and opening a ticket to get the change addressed) or by fixing the code in a new release.

We've tried this in the past, and the problem is that it hits the wrong people. Users typically can't do anything directly about the warnings, other than report them to the offending packages. So the person hit by the warning then has an indefinite wait while the upstream package fixes the issue (either fully, or just by temporarily suppressing the warning) and releases a new version (which depending on the project release cycles and processes, may not be a straightforward replacement for the previous version).

...

If package authors were to get into the habit of doing the silencing for their users after opening a ticket, that would probably make the whole process more streamlined and effective.

If that was what actually happened, then maybe this would work. But unfortunately this is open source, and many projects haven't got the resources to make emergency releases to silence a warning for their users.

Paul

M.-A. Lemburg

4:24 a.m.

On 21.04.2021 13:14, Paul Moore wrote:

...

On Wed, 21 Apr 2021 at 12:05, M.-A. Lemburg mal@egenix.com wrote:

...
Perhaps we should reconsider making deprecation warnings only visible by explicitly enabling them and instead make them visible by default.

This would create more noise for users, but for the better, since planned changes then become more visible and can be addressed either by silencing the warning (and opening a ticket to get the change addressed) or by fixing the code in a new release.

We've tried this in the past, and the problem is that it hits the wrong people. Users typically can't do anything directly about the warnings, other than report them to the offending packages. So the person hit by the warning then has an indefinite wait while the upstream package fixes the issue (either fully, or just by temporarily suppressing the warning) and releases a new version (which depending on the project release cycles and processes, may not be a straightforward replacement for the previous version).

Isn't that an educational problem ? Adjusting reporting of warnings isn't all that hard:

https://docs.python.org/3/library/warnings.html#the-warnings-filter

Perhaps it's just a usability issue. We could have venvs help us a bit with this by e.g. making such settings "global" per venv, without the user having to configure PYTHONWARNINGS or writing a sitecustomize.py for this purpose.

...

...
If package authors were to get into the habit of doing the silencing for their users after opening a ticket, that would probably make the whole process more streamlined and effective.

If that was what actually happened, then maybe this would work. But unfortunately this is open source, and many projects haven't got the resources to make emergency releases to silence a warning for their users.

True, but at the same time, we often find that deprecations are not visible enough by these projects, which then causes a problem further down the road when the deprecation then gets turned into a breaking change.

-- Marc-Andre Lemburg eGenix.com

Professional Python Services directly from the Experts (#1, Apr 21 2021)

...

...
...
Python Projects, Coaching and Support ... https://www.egenix.com/ Python Product Development ... https://consulting.egenix.com/

::: We implement business ideas - efficiently in both time and costs :::

Victor Stinner

4:34 a.m.

On Wed, Apr 21, 2021 at 1:24 PM M.-A. Lemburg mal@egenix.com wrote:

...

Isn't that an educational problem ? Adjusting reporting of warnings isn't all that hard:

A common practical problem is a project CI which pulls the most recent verisons of 3rd party dependencies and suddenly break if a new deprecation warning is raised by such project.

It's not convenient to have to ignore warnings in every single dependencies, especially from *indirect* dependencies.

It would be better to have a simple way to only emit warnings in a set of packages.

...

True, but at the same time, we often find that deprecations are not visible enough by these projects, which then causes a problem further down the road when the deprecation then gets turned into a breaking change.

Who is responsible of fixing deprecation warnings? Python core developers who introduce the warnings, developers using a module, or maintainers of the module? I have no answer to that question. Usually, the answer is: it depends :-) Everyone has their own agenda, and reducing the technical debt is rarely the top priority ;-)

Victor

Night gathers, and now my watch begins. It shall not end until my death.

Antoine Pitrou

5:55 a.m.

New subject: DeprecationWarning silent by default

Le 21/04/2021 à 13:34, Victor Stinner a écrit :

...

On Wed, Apr 21, 2021 at 1:24 PM M.-A. Lemburg mal@egenix.com wrote:

...
Isn't that an educational problem ? Adjusting reporting of warnings isn't all that hard:

A common practical problem is a project CI which pulls the most recent verisons of 3rd party dependencies and suddenly break if a new deprecation warning is raised by such project.

If a warning breaks CI, does it mean that the CI set non-default warnings flags (i.e. turn them into errors)? But we're talking about the default flags.

As a concrete data point, some packages in the scientific ecosystem have decided to use FutureWarning instead of DeprecationWarning, precisely because DeprecationWarnings are silent by default and users never see them. This is an ecosystem where users are often non-professional developers, and the fact that DeprecationWarning is hidden by default hits them hard, because they are not aware that they need to change something.

Regards

Antoine.

Paul Moore

4:35 a.m.

On Wed, 21 Apr 2021 at 12:24, M.-A. Lemburg mal@egenix.com wrote:

...

Isn't that an educational problem ? Adjusting reporting of warnings isn't all that hard:

https://docs.python.org/3/library/warnings.html#the-warnings-filter

Perhaps it's just a usability issue. We could have venvs help us a bit with this by e.g. making such settings "global" per venv, without the user having to configure PYTHONWARNINGS or writing a sitecustomize.py for this purpose.

Maybe. In my own personal experience, I hit this sort of thing when using tools. Consider for example black - if that triggered a warning, I'd report it, but then what would I do? Edit my copy of black (possibly in multiple environments) to suppress the warning? Block the warning globally which means I then don't see it for other projects and hence don't report it to them? Work out the precise incantation to suppress it just for black?

In practice, I just moan a lot about the warning, and vote to suppress warnings by default next time the question comes up :-)

So yes, maybe it's an education/usability issue, but if so it's one that's hard to fix. If we can work out a way for users (who may well have limited programming knowledge) to just "push a button" to say "I reported that issue to black, now stop bothering me about it for this version of black (at least on this PC)" then that would be great. But at the moment I don't believe it's that simple.

Paul

M.-A. Lemburg

7:52 a.m.

On 21.04.2021 13:35, Paul Moore wrote:

...

On Wed, 21 Apr 2021 at 12:24, M.-A. Lemburg mal@egenix.com wrote:

...
Isn't that an educational problem ? Adjusting reporting of warnings isn't all that hard:

https://docs.python.org/3/library/warnings.html#the-warnings-filter

Perhaps it's just a usability issue. We could have venvs help us a bit with this by e.g. making such settings "global" per venv, without the user having to configure PYTHONWARNINGS or writing a sitecustomize.py for this purpose.

Maybe. In my own personal experience, I hit this sort of thing when using tools. Consider for example black - if that triggered a warning, I'd report it, but then what would I do? Edit my copy of black (possibly in multiple environments) to suppress the warning? Block the warning globally which means I then don't see it for other projects and hence don't report it to them? Work out the precise incantation to suppress it just for black?

In practice, I just moan a lot about the warning, and vote to suppress warnings by default next time the question comes up :-)

So yes, maybe it's an education/usability issue, but if so it's one that's hard to fix. If we can work out a way for users (who may well have limited programming knowledge) to just "push a button" to say "I reported that issue to black, now stop bothering me about it for this version of black (at least on this PC)" then that would be great. But at the moment I don't believe it's that simple.

I wouldn't give up so fast :-)

If we'd make warning message more instructive and point people to a simple command they could run to suppress the warning that should take away a lot of the pain.

Sketching here:

""" DeprecationWarning: The package x.y.z is using a deprecated feature tadada at module.py:34. If you get a change, please report this to the maintainers. In order to suppress warnings for this package, please run: python3 -m warnings.ignore('::DeprecationWarning:package') """

The ignore() function would then add a warning filter to pyvenv.cfg, if available, or output instructions on how to setup the environment via the a usercustomize.py, sitecustomize.py or PYTHONWARNINGS env var.

If you want a click-for-help approach, we could also put a URL into the message, which then points to a page with specific instructions (rendered to be specific to the warning instance via a Django view for example).

-- Marc-Andre Lemburg eGenix.com

Professional Python Services directly from the Experts (#1, Apr 21 2021)

...

...
...
Python Projects, Coaching and Support ... https://www.egenix.com/ Python Product Development ... https://consulting.egenix.com/

::: We implement business ideas - efficiently in both time and costs :::

Victor Stinner

4:29 a.m.

Hi,

In this case, we need more advanced warnings filters to only show deprecation warnings in the "current application code", and ignore deprecation warnings from any other module. Is there a way to create an entry point in setuptools which says "this application uses the package xxx"?

Since there is no simple solution to this problem and it was decided to hide warnings by default, I added an explicit "Python Development Mode" to Python 3.7 which shows warnings, but also enables some expensive runtime debug checks. https://docs.python.org/dev/library/devmode.html

"The Python Development Mode introduces additional runtime checks that are too expensive to be enabled by default. It should not be more verbose than the default if the code is correct; new warnings are only emitted when an issue is detected."

I suggest running your test suite using "python -X dev -Werror" command line, or PYTHONDEVMODE=1 and PYTHONWARNINGS=error environment variables. It's possible to ignore warnings from third party projects using warnings filters. I put this suggestion at the start of the What's New in Python 3.9 document: https://docs.python.org/dev/whatsnew/3.9.html#you-should-check-for-deprecati...

Should we put such suggestion elsewhere? Or repeat it in each What's New in Python X.Y document?

Victor

On Wed, Apr 21, 2021 at 1:14 PM Paul Moore p.f.moore@gmail.com wrote:

...

On Wed, 21 Apr 2021 at 12:05, M.-A. Lemburg mal@egenix.com wrote:

...
Perhaps we should reconsider making deprecation warnings only visible by explicitly enabling them and instead make them visible by default.

This would create more noise for users, but for the better, since planned changes then become more visible and can be addressed either by silencing the warning (and opening a ticket to get the change addressed) or by fixing the code in a new release.

We've tried this in the past, and the problem is that it hits the wrong people. Users typically can't do anything directly about the warnings, other than report them to the offending packages. So the person hit by the warning then has an indefinite wait while the upstream package fixes the issue (either fully, or just by temporarily suppressing the warning) and releases a new version (which depending on the project release cycles and processes, may not be a straightforward replacement for the previous version).

...
If package authors were to get into the habit of doing the silencing for their users after opening a ticket, that would probably make the whole process more streamlined and effective.

If that was what actually happened, then maybe this would work. But unfortunately this is open source, and many projects haven't got the resources to make emergency releases to silence a warning for their users.

Paul

python-committers mailing list -- python-committers@python.org To unsubscribe send an email to python-committers-leave@python.org https://mail.python.org/mailman3/lists/python-committers.python.org/ Message archived at https://mail.python.org/archives/list/python-committers@python.org/message/X... Code of Conduct: https://www.python.org/psf/codeofconduct/

-- Night gathers, and now my watch begins. It shall not end until my death.

Anthony Baxter

4:26 a.m.

While I have not been involved in the release process for like 15 years or more, I would like to point out that breaking changes mean the distros are less likely to ship them, and be less likely to trust updates.

Trying to get RH &c to stop shipping 1.5.2 was a huge effort.

Always, any time when you might need to break compat it's a huge risk.

On Wed, 21 Apr 2021, 4:58 am Thomas Wouters, thomas@python.org wrote:

...

(Starting a new thread so as not to derail any of the ongoing discussions.)

Thanks, everyone, for your thoughts on Python 3.10 and the impact of PEP 563 (postponed evaluation of annotations) becoming the default. The Steering Council has considered the issue carefully, along with many of the proposed alternatives and solutions, and we’ve decided that at this point, we simply can’t risk the compatibility breakage of PEP 563. We need to roll back the change that made stringified annotations the default, at least for 3.10. (Pablo is already working on this.)

To be clear, we are not reverting PEP 563 itself. The future import will keep working like it did since Python 3.7. We’re delaying making PEP 563 string-based annotations the default until Python 3.11. This will give us time to find a solution that works for everyone (or to find a feasible upgrade path for users who currently rely on evaluated annotations). Some considerations that led us to this decision:

PEP 563’s default change is clearly too disruptive to downstream users and third-party libraries to happen right now. We can’t risk breaking even a small subset of the FastAPI/pydantic users, not to mention other uses of evaluated type annotations that we’re not aware of yet.

PEP 563 provides no warning to users of the feature it’s disabling. Without that, we can’t expect users to be aware of the upcoming breakage. The lack of a warning was by design, and made sense in a world where type annotations were only consumed by static type checkers --- but that’s not actually the situation we’re in. There are clearly existing real-world, run-time uses of type annotations that would be adversely affected by this change.

Originally, PEP 563 was scheduled to take effect in Python 4, and this changed recently (after the discussion in the Language Summit of 2020). It's possible that third-party libraries and users didn’t plan to react in the current time frame as they were not aware of this change in timing.

There isn’t enough time to properly discuss PEP 649 or any of the alternatives before the beta 1 deadline, and we really need to make sure we don’t compound errors here. We need to look for a long term solution, which isn’t possible while still maintaining the release deadlines of Python 3.10. That means we’re also deferring PEP 649 to Python 3.11.

In the Steering Council’s unanimous opinion, rolling back the default flip for stringified annotations in Python 3.10 is the least disruptive of all the options.

We need to continue discussing the issue and potential solutions, since this merely postpones the problem until 3.11. (For the record, postponing the change further is not off the table, either, for example if the final decision is to treat evaluated annotations as a deprecated feature, with warnings on use.)

For what it’s worth, the SC is also considering what we can do to reduce the odds of something like this happening again, but that’s a separate consideration, and a multi-faceted one at that.

For the Steering Council, Thomas.

Thomas Wouters thomas@python.org

Hi! I'm an email virus! Think twice before sending your email to help me spread!

python-committers mailing list -- python-committers@python.org To unsubscribe send an email to python-committers-leave@python.org https://mail.python.org/mailman3/lists/python-committers.python.org/ Message archived at https://mail.python.org/archives/list/python-committers@python.org/message/C... Code of Conduct: https://www.python.org/psf/codeofconduct/

1098

Age (days ago)

1099

Last active (days ago)

List overview

Download

13 comments

9 participants

participants (9)

Anthony Baxter
Antoine Pitrou
Guido van Rossum
Larry Hastings
M.-A. Lemburg
Paul Moore
Terry Reedy
Thomas Wouters
Victor Stinner

PEP 563 and Python 3.10.

For the Steering Council, Thomas.

For the Steering Council, Thomas.

For the Steering Council, Thomas.

Victor

For the Steering Council, Thomas.

tags

participants (9)