Hi Nick, First, thanks a lot for your detailed answer, it was very instructive to me. My answers below. 2016-07-03 6:09 GMT+02:00 Nick Coghlan <ncoghlan@gmail.com>:
On 2 July 2016 at 16:17, Ludovic Gasc <gmludo@gmail.com> wrote:
Hi everybody,
I fully understand that AsyncIO is a drop in the ocean of CPython, you're working to prepare the entire 3.5.3 release for December, not yet ready. However, you might create a 3.5.2.1 release with only this AsyncIO fix ?
That would be more work than just doing a 3.5.3 release, though - the problem isn't with the version number bump, it's with asking the release team to do additional work without clearly explaining the rationale for the request (more on that below). While some parts of the release process are automated, there's still a lot of steps to run through by a number of different people: https://www.python.org/dev/peps/pep-0101/.
Thanks for the link, I didn't know this PEP, it was interesting to read.
The first key question to answer in this kind of situation is: "Is there code that will run correctly on 3.5.1 that will now fail on 3.5.2?" (i.e. it's a regression introduced by the asyncio and coroutine changes in the point release rather than something that was already broken in 3.5.0 and 3.5.1).
If the answer is "No", then it doesn't inhibit the 3.5.2 rollout in any way, and folks can wait until 3.5.3 for the fix.
However, if the answer is "Yes, it's a new regression in 3.5.2" (as in this case), then the next question becomes "Is there an agreed resolution for the regression?"
The answer to that is currently "No" - Yury's PR against the asyncio repo is still being discussed.
Once the answer to that question is "Yes", *then* the question of releasing a high priority fix in a Python 3.5.3 release can be properly considered by answering the question "Of the folks using asyncio, what proportion of them are likely to encounter problems in upgrading to Python 3.5.2, and is there a workaround they can apply or alternate approach they can use to avoid the problem?".
At the moment, Yury's explanation of the fix in the PR is (understandably) addressed at getting the problem resolved within the context of asyncio, and hence just describes the particular APIs affected, and the details of the incorrect behaviour. While that's an important step in the process, it doesn't provide a clear assessment of the *consequences* of the bug aimed at folks that aren't themselves deeply immersed in using asyncio, so we can't tell if the problem is "Some idiomatic code frequently recommended in user facing examples and used in third party asyncio based libraries may hang client processes" (which would weigh in favour of an early 3.5.3 release before people start encountering the regression in practice) or "Some low level API's not recommended for general use may hang if used in a particular non-idiomatic combination only likely to be encountered by event loop implementors" (which would suggest it may be OK to stick with the normal maintenance release cadence).
To my basic understanding, it seems to have race conditions to open sockets. If my understanding is true, it's a little bit the heart of AsyncIO is affected ;-) If you search about loop.sock_connect in Github, you've found a lot of results https://github.com/search?l=python&q=loop.sock_connect&ref=searchresults&type=Code&utf8=%E2%9C%93 Moreover, if Yury, one of contributors of AsyncIO: https://github.com/python/asyncio/graphs/contributors and uvloop creator has sent an e-mail about that, I'm tented to believe him. It's why a little bit scared by that, even if we don't have a lot of AsyncIO's users, especially with the latest release. However, Google Trends might give us a good overview of relative users we have, compare to Twisted, Gevent and Tornado: https://www.google.com/trends/explore#q=asyncio%2C%20%2Fm%2F02xknvd%2C%20gevent%2C%20%2Fm%2F07s58h4&date=1%2F2016%2012m&cmpt=q&tz=Etc%2FGMT-2
If 3.5.2.1 or 3.5.3 are impossible to release before december,
Early maintenance releases are definitely possible, but the consequences of particular regressions need to be put into terms that make sense to the release team, which generally means stepping up from "APIs X, Y, and Z broke in this way" to "Users doing A, B, and C will be affected in this way".
As an example of a case where an early maintenance release took place: several years ago, Python 2.6.3 happened to break both "from logging import *" (due to a missing entry in test___all__ letting an error in logging.__all__ through) and building extension modules with setuptools (due to a change in a private API that setuptools was monkeypatching). Those were considered significant enough for the 2.6.4 release to happen early.
Ok, we'll see first what's the decision will emerge about this pull request in AsyncIO.
what are the alternative solutions for AsyncIO users ? 1. Use 3.5.1 and hope that Linux distributions won't use 3.5.2 ?
Linux distributions have mechanisms to carry patches (indeed, selective application of patches is one of the main benefits of using system packages over upstream ones), so any distro that rebases on 3.5.2 can be encouraged to add the fix once it lands regardless of whether or not Larry approves an additional maintenance release outside the normal cadence.
Good to know. It means that it should be more Mac and Windows users who are concerned about this bug, especially new comers, because they download directly from python.org website. Depends on the pull request decision, it might be also a warning message on downloads page to explain to use 3.5.1 instead of 3.5.2 if you want to use AsyncIO. Cheers,
Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia