Python-Dev May 2018

python-dev@python.org

103 participants
89 discussions

Reminder: Please elaborate commit messages

by Victor Stinner

Hi, In https://bugs.python.org/issue33531, Andrew Svetlov wrote "Fixed failed sendfile tests on Windows (at least I hope so)." without giving any bpo number or a commit number. So I looked at latest commits and I found: --- commit e2537521916c5bf88fcf54d4654ff1bcd332be4a Author: Andrew Svetlov <andrew.svetlov(a)gmail.com> Date: Mon May 21 12:03:45 2018 +0300 Fix asyncio flaky tests (#7023) --- Please try to write better error messages for people who will dig into the Git history in 1, 5 or 10 years: * Usually, it's better to open a bug. Here you could give the full error message, mention on which platform the test fails, etc. * Mention which tests are affected * Maybe even give an extract of the error message of the fixed test in the commit message I know that it's more effort and fixing flaky tests is annoying and may require multiple iterations, but again, please think to people who will have to read the Git history later... I was able able to rebuild the context of this commit from a comment of https://bugs.python.org/issue33531 Victor

2 years, 11 months

Re: [Python-Dev] PEP: 576 Title: Rationalize Built-in function classes

by Jeroen Demeyer

On 2018-05-19 11:15, mark wrote: > PEP 576 aims to fulfill the same goals as PEP 575 (this is a copy of my comments on GitHub before this PEP was official) **Performance** Most importantly, changing bound methods of extension types from builtin_function_or_method to bound_method will yield a performance loss. It might be possible to mitigate this somewhat by adding specific optimizations for calling bound_method. However, that would add extra complexity and it will probably still be slower than the existing code. And I would also like to know whether it will be possible for custom built-in function subclasses to implement __get__ to change a function into a method (like Python functions) and whether/how the LOAD_METHOD opcode will work in that case. **Introspection** When I want "introspection support", that goes beyond the call signature. Also inspect.getfile should be supported. Currently, that simply raises an exception for built-in functions. I think it's important to specify the semantics of inspect.isfunction. Given that you don't mention it, I assume that inspect.isfunction will continue to return True only for Python functions. But that way, these new function classes won't behave like Python functions. > fully backwards compatible. I wonder why you think it is "fully backwards compatible". Just like PEP 575, you are changing the classes of certain objects. I think it's fairer to say that both PEP 575 and PEP 576 might cause minor backwards compatibility issues. I certainly don't think that PEP 576 is significantly more backwards compatible than PEP 575. PS: in your PEP, you write "bound_method" but I guess you mean "method". PEP 575 proposes to rename "method" to "bound_method". Jeroen.

2 years, 11 months

Re: [Python-Dev] PEP 575 (Unifying function/method classes) update

by Jeroen Demeyer

On 2018-05-16 17:31, Petr Viktorin wrote: > The larger a change is, the harder it is to understand I already disagree here... I'm afraid that you are still confusing the largeness of the *change* with the complexity of the *result* after the change was implemented. A change that *removes* complexity should be considered a good thing, even if it's a large change. That being said, if you want me to make smaller changes, I could do it. But I would do it for *you* personally because I'm afraid that other people might rightly complain that I'm making things too complicated. So I would certainly like some feedback from others on this point. > Less disruptive changes tend to have a better backwards compatibility story. Maybe in very general terms, yes. But I believe that the "disruptive" changes that I'm making will not contribute to backwards incompatibility. Adding new ml_flags flags shouldn't break anything and adding a base class shouldn't either (I doubt that there is code relying on the fact that type(len).__base__ is object). In my opinion, the one change that is most likely to cause backwards compatibility problems is changing the type of bound methods of extension types. And that change is even in the less disruptive PEP 576. > Mark Shannon has an upcoming PEP with an alternative to some of the > issues. I'm looking forward to a serious discussion about that. However, from a first reading, I'm not very optimistic about its performance implications. > Currently, the "outside" of a function (how it looks when introspected) > is tied to the "inside" (what happens internally when it's called). > Can we better enable pydoc/IPython developers to tackle introspection > problems without wading deep in the internals and call optimizations? I proposed complete decoupling in https://bugs.python.org/issue30071 and that was rejected. Anyway, decoupling of introspection is not the essence of this PEP. This PEP is really about allowing custom built-in function subclasses. That's the hard part where CPython internals come in. So I suggest that we leave the discussion about introspection and focus on the function classes. > But, it still has to inherit from base_function to "look like a > function". Can we remove that limitation in favor of duck typing? Duck typing is a Python thing, I don't know what "duck typing" would mean on the C level. We could change the existing isinstance(..., base_function) check by a different fast check. For example, we (together with the Cython devs) have been pondering about a new "type" field, say tp_cfunctionoffset pointing to a certain C field in the object structure. That would work but it would not be so fundamentally different from the current PEP. *PS*: On friday, I'm leaving for 2 weeks on holidays. So if I don't reply to comments on PEP 575 or alternative proposals, don't take it as a lack of interest. Jeroen.

2 years, 11 months

Failing tests on master (asyncio, multiprocessing)

by Christian Heimes

Hi, several of my PRs as well as local tests have started failing recently. On my local Fedora 27 machine, four sendfile related tests of test_asyncio's BaseLoopSockSendfileTests suite are failing reproducible. For example Travis CI job https://travis-ci.org/python/cpython/jobs/380852981 fails in: * test_stdin_broken_pipe (test.test_asyncio.test_subprocess.SubprocessFastWatcherTests) * test_stdin_broken_pipe (test.test_asyncio.test_subprocess.SubprocessSafeWatcherTests) * test_ignore (test.test_multiprocessing_forkserver.TestIgnoreEINTR) Could somebody have a look, please? Christian

2 years, 11 months

Re: [Python-Dev] please help triage VSTS failures

by Nathaniel Smith

On Fri, May 18, 2018 at 1:13 PM, Steve Dower <steve.dower(a)python.org> wrote: > According to the VSTS dev team, an easy “rerun this build” button and > filtering by changed paths are coming soon, which should clean things up. If you're talking to them, please ask them to make sure that the "rerun this build" button doesn't erase the old log. (That's what it does on Travis. Appveyor is better.) The problem is that when you have a flaky/intermittent failure, your todo list is always (a) rerun the build so at least it's not blocking whatever this unrelated change is, (b) file some sort of bug, or comment on some existing bug, and link to the log to help track down the intermittent failure. If you click the "rebuild" button on Travis, then it solves (a), while deleting the information you need for (b) – and for rare intermittent bugs you might not have much information to go on besides that build log. -n -- Nathaniel J. Smith -- https://vorpus.org

2 years, 11 months

Re: [Python-Dev] please help triage VSTS failures

by Terry Reedy

On 5/18/2018 4:13 PM, Steve Dower wrote: > Close/reopen PR is the best way to trigger a rebuild right now. It may be the way to retrigger VSTS, but if one want to merge, and either of Travis or AppVeyor pass, tossing the success is a foolish thing to do. Either may fail on a rebuild. -- Terry Jan Reedy

2 years, 11 months

Re: [Python-Dev] please help triage VSTS failures

by Yury Selivanov

On Fri, May 18, 2018 at 4:15 PM Steve Dower <steve.dower(a)python.org> wrote: [..] > The asyncio instability is apparently really hard to fix. There were 2-3 people looking into it yesterday on one of the other systems, but apparently we haven’t solved it yet (my guess is lingering state from a previous test). The multissl script was my fault for not realising that we don’t use it on 3.6 builds, but that should be fixed already. Close/reopen PR is the best way to trigger a rebuild right now. I asked Andrew Svetlov to help with asyncio CI triage. Hopefully we'll resolve most of them early next week. Yury

2 years, 11 months

please help triage VSTS failures

by Gregory P. Smith

These both look like VSTS infrastructure falling over on PRs: https://python.visualstudio.com/cpython/_build?buildId=522 https://python.visualstudio.com/cpython/_build?buildId=523 I don't see anywhere that gives information about the failures. (*) These CI failures on different platforms are both for the same change, on different branches, for a documentation only change. That passed on the other branches and platforms for the same change. -gps (*) I refuse to "Download logs as a zip file". I'm in a web browser, if the information I might need is potentially buried somewhere in a zip file of logs, that is a waste of my time. I'm not going to do it. The web UI *needs* to find and display the relevant failure info from any logs directly.

2 years, 11 months

Re: [Python-Dev] [Python-checkins] bpo-33522: Enable CI builds on Visual Studio Team Services (GH-6865) (GH-6925)

by Gregory P. Smith

Why did this commit modify .py files, unittests, and test.support? That is inappropriate for something claiming to merely enable a CI platform. -gps On Thu, May 17, 2018 at 6:50 AM Steve Dower <webhook-mailer(a)python.org> wrote: > > https://github.com/python/cpython/commit/0d8f83f59c8f4cc7fe125434ca4ecdcac1… > commit: 0d8f83f59c8f4cc7fe125434ca4ecdcac111810f > branch: 3.6 > author: Steve Dower <steve.dower(a)microsoft.com> > committer: GitHub <noreply(a)github.com> > date: 2018-05-17T09:46:00-04:00 > summary: > > bpo-33522: Enable CI builds on Visual Studio Team Services (GH-6865) > (GH-6925) > > files: > A .vsts/docs-release.yml > A .vsts/docs.yml > A .vsts/linux-buildbot.yml > A .vsts/linux-coverage.yml > A .vsts/linux-deps.yml > A .vsts/linux-pr.yml > A .vsts/macos-buildbot.yml > A .vsts/macos-pr.yml > A .vsts/windows-buildbot.yml > A .vsts/windows-pr.yml > A Misc/NEWS.d/next/Build/2018-05-15-12-44-50.bpo-33522.mJoNcA.rst > A Misc/NEWS.d/next/Library/2018-05-16-17-05-48.bpo-33548.xWslmx.rst > M Doc/make.bat > M Lib/tempfile.py > M Lib/test/support/__init__.py > M Lib/test/test_asyncio/test_base_events.py > M Lib/test/test_bdb.py > M Lib/test/test_pathlib.py > M Lib/test/test_poplib.py > M Lib/test/test_selectors.py > M PCbuild/rt.bat > M Tools/ssl/multissltests.py > > diff --git a/.vsts/docs-release.yml b/.vsts/docs-release.yml > new file mode 100644 > index 000000000000..e90428a42494 > --- /dev/null > +++ b/.vsts/docs-release.yml > @@ -0,0 +1,43 @@ > +# Current docs for the syntax of this file are at: > +# > https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettin… > + > +name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) > + > +queue: > + name: Hosted Linux Preview > + > +#variables: > + > +steps: > +- checkout: self > + clean: true > + fetchDepth: 5 > + > +- script: sudo apt-get update && sudo apt-get install -qy --force-yes > texlive-full > + displayName: 'Install LaTeX' > + > +- task: UsePythonVersion@0 > + displayName: 'Use Python 3.6 or later' > + inputs: > + versionSpec: '>=3.6' > + > +- script: python -m pip install sphinx blurb python-docs-theme > + displayName: 'Install build dependencies' > + > +- script: make dist PYTHON=python SPHINXBUILD='python -m sphinx' > BLURB='python -m blurb' > + workingDirectory: '$(build.sourcesDirectory)/Doc' > + displayName: 'Build documentation' > + > +- task: PublishBuildArtifacts@1 > + displayName: 'Publish build' > + inputs: > + PathToPublish: '$(build.sourcesDirectory)/Doc/build' > + ArtifactName: build > + publishLocation: Container > + > +- task: PublishBuildArtifacts@1 > + displayName: 'Publish dist' > + inputs: > + PathToPublish: '$(build.sourcesDirectory)/Doc/dist' > + ArtifactName: dist > + publishLocation: Container > diff --git a/.vsts/docs.yml b/.vsts/docs.yml > new file mode 100644 > index 000000000000..efa1e871656d > --- /dev/null > +++ b/.vsts/docs.yml > @@ -0,0 +1,43 @@ > +# Current docs for the syntax of this file are at: > +# > https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettin… > + > +name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) > + > +queue: > + name: Hosted Linux Preview > + > +trigger: > + branches: > + include: > + - master > + - 3.7 > + - 3.6 > + paths: > + include: > + - Doc/* > + > +#variables: > + > +steps: > +- checkout: self > + clean: true > + fetchDepth: 5 > + > +- task: UsePythonVersion@0 > + displayName: 'Use Python 3.6 or later' > + inputs: > + versionSpec: '>=3.6' > + > +- script: python -m pip install sphinx~=1.6.1 blurb python-docs-theme > + displayName: 'Install build dependencies' > + > +- script: make check suspicious html PYTHON=python > + workingDirectory: '$(build.sourcesDirectory)/Doc' > + displayName: 'Build documentation' > + > +- task: PublishBuildArtifacts@1 > + displayName: 'Publish build' > + inputs: > + PathToPublish: '$(build.sourcesDirectory)/Doc/build' > + ArtifactName: build > + publishLocation: Container > diff --git a/.vsts/linux-buildbot.yml b/.vsts/linux-buildbot.yml > new file mode 100644 > index 000000000000..d75d7f57650e > --- /dev/null > +++ b/.vsts/linux-buildbot.yml > @@ -0,0 +1,71 @@ > +# Current docs for the syntax of this file are at: > +# > https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettin… > + > +name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) > + > +queue: > + name: Hosted Linux Preview > + > +trigger: > + branches: > + include: > + - master > + - 3.7 > + - 3.6 > + paths: > + exclude: > + - Doc/* > + - Tools/* > + > +variables: > + # Copy-pasted from linux-deps.yml until template support arrives > + OPENSSL: 1.1.0g > + OPENSSL_DIR: "$(build.sourcesDirectory)/multissl/openssl/$(OPENSSL)" > + > + > +steps: > +- checkout: self > + clean: true > + fetchDepth: 5 > + > +#- template: linux-deps.yml > + > +# See > https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettin… > +# For now, we copy/paste the steps > +- script: echo "deb-src http://archive.ubuntu.com/ubuntu/ xenial main" > > /etc/apt/sources.list.d/python.list && sudo apt-get update > + displayName: 'Update apt-get lists' > + > +- script: echo ##vso[task.prependpath]$(OPENSSL_DIR) > + displayName: 'Add $(OPENSSL_DIR) to PATH' > +- script: > > + sudo apt-get -yq install > + build-essential > + zlib1g-dev > + libbz2-dev > + liblzma-dev > + libncurses5-dev > + libreadline6-dev > + libsqlite3-dev > + libssl-dev > + libgdbm-dev > + tk-dev > + lzma > + lzma-dev > + liblzma-dev > + libffi-dev > + uuid-dev > + displayName: 'Install dependencies' > +- script: python3 Tools/ssl/multissltests.py --steps=library > --base-directory $(build.sourcesDirectory)/multissl --openssl $(OPENSSL) > --system Linux > + displayName: 'python multissltests.py' > + > +- script: ./configure --with-pydebug > + displayName: 'Configure CPython (debug)' > + > +- script: make -s -j4 > + displayName: 'Build CPython' > + > +- script: make pythoninfo > + displayName: 'Display build info' > + > +- script: make buildbottest TESTOPTS="-j4 -uall,-cpu" > + displayName: 'Tests' > diff --git a/.vsts/linux-coverage.yml b/.vsts/linux-coverage.yml > new file mode 100644 > index 000000000000..3657b1720ee2 > --- /dev/null > +++ b/.vsts/linux-coverage.yml > @@ -0,0 +1,77 @@ > +# Current docs for the syntax of this file are at: > +# > https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettin… > + > +name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) > + > +queue: > + name: Hosted Linux Preview > + > +trigger: > + branches: > + include: > + - master > + - 3.7 > + - 3.6 > + paths: > + exclude: > + - Doc/* > + - Tools/* > + > +variables: > + # Copy-pasted from linux-deps.yml until template support arrives > + OPENSSL: 1.1.0g > + OPENSSL_DIR: "$(build.sourcesDirectory)/multissl/openssl/$(OPENSSL)" > + > +steps: > +- checkout: self > + clean: true > + fetchDepth: 5 > + > +#- template: linux-deps.yml > + > +# See > https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettin… > +# For now, we copy/paste the steps > +- script: echo "deb-src http://archive.ubuntu.com/ubuntu/ xenial main" > > /etc/apt/sources.list.d/python.list && sudo apt-get update > + displayName: 'Update apt-get lists' > + > +- script: echo ##vso[task.prependpath]$(OPENSSL_DIR) > + displayName: 'Add $(OPENSSL_DIR) to PATH' > +- script: > > + sudo apt-get -yq install > + build-essential > + zlib1g-dev > + libbz2-dev > + liblzma-dev > + libncurses5-dev > + libreadline6-dev > + libsqlite3-dev > + libssl-dev > + libgdbm-dev > + tk-dev > + lzma > + lzma-dev > + liblzma-dev > + libffi-dev > + uuid-dev > + displayName: 'Install dependencies' > +- script: python3 Tools/ssl/multissltests.py --steps=library > --base-directory $(build.sourcesDirectory)/multissl --openssl $(OPENSSL) > --system Linux > + displayName: 'python multissltests.py' > + > + > +- script: ./configure --with-pydebug > + displayName: 'Configure CPython (debug)' > + > +- script: make -s -j4 > + displayName: 'Build CPython' > + > +- script: ./python -m venv venv && ./venv/bin/python -m pip install -U > coverage > + displayName: 'Set up virtual environment' > + > +- script: ./venv/bin/python -m test.pythoninfo > + displayName: 'Display build info' > + > +- script: ./venv/bin/python -m coverage run --pylib -m test > --fail-env-changed -uall,-cpu -x test_multiprocessing_fork -x > test_multiprocessing_forkserver -x test_multiprocessing_spawn -x > test_concurrent_futures > + displayName: 'Tests with coverage' > + > +- script: source ./venv/bin/activate && bash <(curl -s > https://codecov.io/bash) > + displayName: 'Publish code coverage results' > diff --git a/.vsts/linux-deps.yml b/.vsts/linux-deps.yml > new file mode 100644 > index 000000000000..b6c8a3690ea1 > --- /dev/null > +++ b/.vsts/linux-deps.yml > @@ -0,0 +1,36 @@ > +# Note: this file is not currently used, but when template support comes > to VSTS it > +# will be referenced from the other scripts.. > + > +# Current docs for the syntax of this file are at: > +# > https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettin… > + > +parameters: > + OPENSSL: 1.1.0g > + OPENSSL_DIR: "$(build.sourcesDirectory)/multissl/openssl/$(OPENSSL)" > + > +steps: > +- script: echo "deb-src http://archive.ubuntu.com/ubuntu/ xenial main" > > /etc/apt/sources.list.d/python.list && sudo apt-get update > + displayName: 'Update apt-get lists' > + > +- script: echo ##vso[task.prependpath]$(OPENSSL_DIR) > + displayName: 'Add $(OPENSSL_DIR) to PATH' > +- script: > > + sudo apt-get -yq install > + build-essential > + zlib1g-dev > + libbz2-dev > + liblzma-dev > + libncurses5-dev > + libreadline6-dev > + libsqlite3-dev > + libssl-dev > + libgdbm-dev > + tk-dev > + lzma > + lzma-dev > + liblzma-dev > + libffi-dev > + uuid-dev > + displayName: 'Install dependencies' > +- script: python3 Tools/ssl/multissltests.py --steps=library > --base-directory $(build.sourcesDirectory)/multissl --openssl $(OPENSSL) > --system Linux > + displayName: 'python multissltests.py' > diff --git a/.vsts/linux-pr.yml b/.vsts/linux-pr.yml > new file mode 100644 > index 000000000000..7f4d458f5a7c > --- /dev/null > +++ b/.vsts/linux-pr.yml > @@ -0,0 +1,75 @@ > +# Current docs for the syntax of this file are at: > +# > https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettin… > + > +name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) > + > +queue: > + name: Hosted Linux Preview > + > +trigger: > + branches: > + include: > + - master > + - 3.7 > + - 3.6 > + paths: > + exclude: > + - Doc/* > + - Tools/* > + > +variables: > + # Copy-pasted from linux-deps.yml until template support arrives > + OPENSSL: 1.1.0g > + OPENSSL_DIR: "$(build.sourcesDirectory)/multissl/openssl/$(OPENSSL)" > + > +steps: > +- checkout: self > + clean: true > + fetchDepth: 5 > + > +#- template: linux-deps.yml > + > +# See > https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettin… > +# For now, we copy/paste the steps > +- script: echo "deb-src http://archive.ubuntu.com/ubuntu/ xenial main" > > /etc/apt/sources.list.d/python.list && sudo apt-get update > + displayName: 'Update apt-get lists' > + > +- script: echo ##vso[task.prependpath]$(OPENSSL_DIR) > + displayName: 'Add $(OPENSSL_DIR) to PATH' > +- script: > > + sudo apt-get -yq install > + build-essential > + zlib1g-dev > + libbz2-dev > + liblzma-dev > + libncurses5-dev > + libreadline6-dev > + libsqlite3-dev > + libssl-dev > + libgdbm-dev > + tk-dev > + lzma > + lzma-dev > + liblzma-dev > + libffi-dev > + uuid-dev > + displayName: 'Install dependencies' > +- script: python3 Tools/ssl/multissltests.py --steps=library > --base-directory $(build.sourcesDirectory)/multissl --openssl $(OPENSSL) > --system Linux > + displayName: 'python multissltests.py' > + > + > +- script: ./configure --with-pydebug > + displayName: 'Configure CPython (debug)' > + > +- script: make -s -j4 > + displayName: 'Build CPython' > + > +- script: make pythoninfo > + displayName: 'Display build info' > + > +# Run patchcheck and fail if anything is discovered > +- script: ./python Tools/scripts/patchcheck.py --travis true > + displayName: 'Run patchcheck.py' > + > +- script: make buildbottest TESTOPTS="-j4 -uall,-cpu" > + displayName: 'Tests' > diff --git a/.vsts/macos-buildbot.yml b/.vsts/macos-buildbot.yml > new file mode 100644 > index 000000000000..8a4f6ba8cb8b > --- /dev/null > +++ b/.vsts/macos-buildbot.yml > @@ -0,0 +1,37 @@ > +# Current docs for the syntax of this file are at: > +# > https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettin… > + > +name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) > + > +queue: > + name: Hosted macOS Preview > + > +trigger: > + branches: > + include: > + - master > + - 3.7 > + - 3.6 > + paths: > + exclude: > + - Doc/* > + - Tools/* > + > +#variables: > + > +steps: > +- checkout: self > + clean: true > + fetchDepth: 5 > + > +- script: ./configure --with-pydebug --with-openssl=/usr/local/opt/openssl > + displayName: 'Configure CPython (debug)' > + > +- script: make -s -j4 > + displayName: 'Build CPython' > + > +- script: make pythoninfo > + displayName: 'Display build info' > + > +- script: make buildbottest TESTOPTS="-j4 -uall,-cpu" > + displayName: 'Tests' > diff --git a/.vsts/macos-pr.yml b/.vsts/macos-pr.yml > new file mode 100644 > index 000000000000..8a4f6ba8cb8b > --- /dev/null > +++ b/.vsts/macos-pr.yml > @@ -0,0 +1,37 @@ > +# Current docs for the syntax of this file are at: > +# > https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettin… > + > +name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) > + > +queue: > + name: Hosted macOS Preview > + > +trigger: > + branches: > + include: > + - master > + - 3.7 > + - 3.6 > + paths: > + exclude: > + - Doc/* > + - Tools/* > + > +#variables: > + > +steps: > +- checkout: self > + clean: true > + fetchDepth: 5 > + > +- script: ./configure --with-pydebug --with-openssl=/usr/local/opt/openssl > + displayName: 'Configure CPython (debug)' > + > +- script: make -s -j4 > + displayName: 'Build CPython' > + > +- script: make pythoninfo > + displayName: 'Display build info' > + > +- script: make buildbottest TESTOPTS="-j4 -uall,-cpu" > + displayName: 'Tests' > diff --git a/.vsts/windows-buildbot.yml b/.vsts/windows-buildbot.yml > new file mode 100644 > index 000000000000..5ec4522796ce > --- /dev/null > +++ b/.vsts/windows-buildbot.yml > @@ -0,0 +1,49 @@ > +# Current docs for the syntax of this file are at: > +# > https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettin… > + > +name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) > + > +queue: > + name: Hosted VS2017 > + parallel: 2 > + matrix: > + amd64: > + buildOpt: -p x64 > + outDirSuffix: amd64 > + win32: > + buildOpt: > + outDirSuffix: win32 > + > +trigger: > + branches: > + include: > + - master > + - 3.7 > + - 3.6 > + paths: > + exclude: > + - Doc/* > + - Tools/* > + > +variables: > + # Relocate build outputs outside of source directory to make cleaning > faster > + Py_IntDir: $(Build.BinariesDirectory)\obj > + # UNDONE: Do not build to a different directory because of broken tests > + Py_OutDir: $(Build.SourcesDirectory)\PCbuild > + EXTERNAL_DIR: $(Build.BinariesDirectory)\externals > + > +steps: > +- checkout: self > + clean: true > + fetchDepth: 5 > + > +- script: PCbuild\build.bat -e $(buildOpt) > + displayName: 'Build CPython' > + > +- script: python.bat -m test.pythoninfo > + displayName: 'Display build info' > + > +- script: PCbuild\rt.bat -q -uall -u-cpu -rwW --slowest --timeout=1200 -j0 > + displayName: 'Tests' > + env: > + PREFIX: $(Py_OutDir)\$(outDirSuffix) > diff --git a/.vsts/windows-pr.yml b/.vsts/windows-pr.yml > new file mode 100644 > index 000000000000..5ec4522796ce > --- /dev/null > +++ b/.vsts/windows-pr.yml > @@ -0,0 +1,49 @@ > +# Current docs for the syntax of this file are at: > +# > https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettin… > + > +name: $(BuildDefinitionName)_$(Date:yyyyMMdd)$(Rev:.rr) > + > +queue: > + name: Hosted VS2017 > + parallel: 2 > + matrix: > + amd64: > + buildOpt: -p x64 > + outDirSuffix: amd64 > + win32: > + buildOpt: > + outDirSuffix: win32 > + > +trigger: > + branches: > + include: > + - master > + - 3.7 > + - 3.6 > + paths: > + exclude: > + - Doc/* > + - Tools/* > + > +variables: > + # Relocate build outputs outside of source directory to make cleaning > faster > + Py_IntDir: $(Build.BinariesDirectory)\obj > + # UNDONE: Do not build to a different directory because of broken tests > + Py_OutDir: $(Build.SourcesDirectory)\PCbuild > + EXTERNAL_DIR: $(Build.BinariesDirectory)\externals > + > +steps: > +- checkout: self > + clean: true > + fetchDepth: 5 > + > +- script: PCbuild\build.bat -e $(buildOpt) > + displayName: 'Build CPython' > + > +- script: python.bat -m test.pythoninfo > + displayName: 'Display build info' > + > +- script: PCbuild\rt.bat -q -uall -u-cpu -rwW --slowest --timeout=1200 -j0 > + displayName: 'Tests' > + env: > + PREFIX: $(Py_OutDir)\$(outDirSuffix) > diff --git a/Doc/make.bat b/Doc/make.bat > index 6cb315fda405..c69cfae31941 100644 > --- a/Doc/make.bat > +++ b/Doc/make.bat > @@ -5,18 +5,21 @@ pushd %~dp0 > > set this=%~n0 > > -call ..\PCBuild\find_python.bat %PYTHON% > -if not defined SPHINXBUILD if defined PYTHON ( > +call ..\PCbuild\find_python.bat %PYTHON% > + > +if not defined PYTHON set PYTHON=py > + > +if not defined SPHINXBUILD ( > %PYTHON% -c "import sphinx" > nul 2> nul > if errorlevel 1 ( > echo Installing sphinx with %PYTHON% > - %PYTHON% -m pip install sphinx > + %PYTHON% -m pip install sphinx python-docs-theme > if errorlevel 1 exit /B > ) > set SPHINXBUILD=%PYTHON% -c "import sphinx, sys; sys.argv[0] = > 'sphinx-build'; sphinx.main()" > ) > > -if not defined BLURB if defined PYTHON ( > +if not defined BLURB ( > %PYTHON% -c "import blurb" > nul 2> nul > if errorlevel 1 ( > echo Installing blurb with %PYTHON% > @@ -26,7 +29,6 @@ if not defined BLURB if defined PYTHON ( > set BLURB=%PYTHON% -m blurb > ) > > -if not defined PYTHON set PYTHON=py > if not defined SPHINXBUILD set SPHINXBUILD=sphinx-build > if not defined BLURB set BLURB=blurb > > diff --git a/Lib/tempfile.py b/Lib/tempfile.py > index 38738082b996..2cb5434ba7b5 100644 > --- a/Lib/tempfile.py > +++ b/Lib/tempfile.py > @@ -173,7 +173,9 @@ def _candidate_tempdir_list(): > > # Failing that, try OS-specific locations. > if _os.name == 'nt': > - dirlist.extend([ r'c:\temp', r'c:\tmp', r'\temp', r'\tmp' ]) > + dirlist.extend([ _os.path.expanduser(r'~\AppData\Local\Temp'), > + _os.path.expandvars(r'%SYSTEMROOT%\Temp'), > + r'c:\temp', r'c:\tmp', r'\temp', r'\tmp' ]) > else: > dirlist.extend([ '/tmp', '/var/tmp', '/usr/tmp' ]) > > diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py > index 867124b63e24..e46394e89d1f 100644 > --- a/Lib/test/support/__init__.py > +++ b/Lib/test/support/__init__.py > @@ -366,6 +366,20 @@ def _rmtree_inner(path): > _force_run(fullname, os.unlink, fullname) > _waitfor(_rmtree_inner, path, waitall=True) > _waitfor(lambda p: _force_run(p, os.rmdir, p), path) > + > + def _longpath(path): > + try: > + import ctypes > + except ImportError: > + # No ctypes means we can't expands paths. > + pass > + else: > + buffer = ctypes.create_unicode_buffer(len(path) * 2) > + length = ctypes.windll.kernel32.GetLongPathNameW(path, buffer, > + len(buffer)) > + if length: > + return buffer[:length] > + return path > else: > _unlink = os.unlink > _rmdir = os.rmdir > @@ -392,6 +406,9 @@ def _rmtree_inner(path): > _rmtree_inner(path) > os.rmdir(path) > > + def _longpath(path): > + return path > + > def unlink(filename): > try: > _unlink(filename) > @@ -2333,13 +2350,15 @@ def can_xattr(): > if not hasattr(os, "setxattr"): > can = False > else: > - tmp_fp, tmp_name = tempfile.mkstemp() > + tmp_dir = tempfile.mkdtemp() > + tmp_fp, tmp_name = tempfile.mkstemp(dir=tmp_dir) > try: > with open(TESTFN, "wb") as fp: > try: > # TESTFN & tempfile may use different file systems > with > # different capabilities > os.setxattr(tmp_fp, b"user.test", b"") > + os.setxattr(tmp_name, b"trusted.foo", b"42") > os.setxattr(fp.fileno(), b"user.test", b"") > # Kernels < 2.6.39 don't respect setxattr flags. > kernel_version = platform.release() > @@ -2350,6 +2369,7 @@ def can_xattr(): > finally: > unlink(TESTFN) > unlink(tmp_name) > + rmdir(tmp_dir) > _can_xattr = can > return can > > diff --git a/Lib/test/test_asyncio/test_base_events.py > b/Lib/test/test_asyncio/test_base_events.py > index 830f0d84a9d4..42c0707e8f21 100644 > --- a/Lib/test/test_asyncio/test_base_events.py > +++ b/Lib/test/test_asyncio/test_base_events.py > @@ -1750,5 +1750,6 @@ def runner(loop): > outer_loop.close() > > > + > if __name__ == '__main__': > unittest.main() > diff --git a/Lib/test/test_bdb.py b/Lib/test/test_bdb.py > index abefe6c4e57a..a36667869718 100644 > --- a/Lib/test/test_bdb.py > +++ b/Lib/test/test_bdb.py > @@ -417,15 +417,17 @@ def __init__(self, test_case, skip=None): > self.dry_run = test_case.dry_run > self.tracer = Tracer(test_case.expect_set, skip=skip, > dry_run=self.dry_run, test_case=test_case.id > ()) > + self._original_tracer = None > > def __enter__(self): > # test_pdb does not reset Breakpoint class attributes on exit :-( > reset_Breakpoint() > + self._original_tracer = sys.gettrace() > return self.tracer > > def __exit__(self, type_=None, value=None, traceback=None): > reset_Breakpoint() > - sys.settrace(None) > + sys.settrace(self._original_tracer) > > not_empty = '' > if self.tracer.set_list: > diff --git a/Lib/test/test_pathlib.py b/Lib/test/test_pathlib.py > index db53a8f202dc..bf9467e96e09 100644 > --- a/Lib/test/test_pathlib.py > +++ b/Lib/test/test_pathlib.py > @@ -1531,7 +1531,7 @@ def test_resolve_common(self): > # resolves to 'dirB/..' first before resolving to parent of > dirB. > self._check_resolve_relative(p, P(BASE, 'foo', 'in', 'spam'), > False) > # Now create absolute symlinks > - d = tempfile.mkdtemp(suffix='-dirD') > + d = support._longpath(tempfile.mkdtemp(suffix='-dirD')) > self.addCleanup(support.rmtree, d) > os.symlink(os.path.join(d), join('dirA', 'linkX')) > os.symlink(join('dirB'), os.path.join(d, 'linkY')) > diff --git a/Lib/test/test_poplib.py b/Lib/test/test_poplib.py > index ca9bc6217509..234c855545c2 100644 > --- a/Lib/test/test_poplib.py > +++ b/Lib/test/test_poplib.py > @@ -217,11 +217,12 @@ def start(self): > def run(self): > self.active = True > self.__flag.set() > - while self.active and asyncore.socket_map: > - self.active_lock.acquire() > - asyncore.loop(timeout=0.1, count=1) > - self.active_lock.release() > - asyncore.close_all(ignore_all=True) > + try: > + while self.active and asyncore.socket_map: > + with self.active_lock: > + asyncore.loop(timeout=0.1, count=1) > + finally: > + asyncore.close_all(ignore_all=True) > > def stop(self): > assert self.active > diff --git a/Lib/test/test_selectors.py b/Lib/test/test_selectors.py > index 852b2feb45fd..14ce91f3768c 100644 > --- a/Lib/test/test_selectors.py > +++ b/Lib/test/test_selectors.py > @@ -450,7 +450,14 @@ def test_above_fd_setsize(self): > self.skipTest("FD limit reached") > raise > > - self.assertEqual(NUM_FDS // 2, len(s.select())) > + try: > + fds = s.select() > + except OSError as e: > + if e.errno == errno.EINVAL and sys.platform == 'darwin': > + # unexplainable errors on macOS don't need to fail the > test > + self.skipTest("Invalid argument error calling poll()") > + raise > + self.assertEqual(NUM_FDS // 2, len(fds)) > > > class DefaultSelectorTestCase(BaseSelectorTestCase): > diff --git > a/Misc/NEWS.d/next/Build/2018-05-15-12-44-50.bpo-33522.mJoNcA.rst > b/Misc/NEWS.d/next/Build/2018-05-15-12-44-50.bpo-33522.mJoNcA.rst > new file mode 100644 > index 000000000000..f44862f0c454 > --- /dev/null > +++ b/Misc/NEWS.d/next/Build/2018-05-15-12-44-50.bpo-33522.mJoNcA.rst > @@ -0,0 +1,2 @@ > +Enable CI builds on Visual Studio Team Services at > +https://python.visualstudio.com/cpython > diff --git > a/Misc/NEWS.d/next/Library/2018-05-16-17-05-48.bpo-33548.xWslmx.rst > b/Misc/NEWS.d/next/Library/2018-05-16-17-05-48.bpo-33548.xWslmx.rst > new file mode 100644 > index 000000000000..65585c152987 > --- /dev/null > +++ b/Misc/NEWS.d/next/Library/2018-05-16-17-05-48.bpo-33548.xWslmx.rst > @@ -0,0 +1 @@ > +tempfile._candidate_tempdir_list should consider common TEMP locations > diff --git a/PCbuild/rt.bat b/PCbuild/rt.bat > index 808102f826d3..212befc95b06 100644 > --- a/PCbuild/rt.bat > +++ b/PCbuild/rt.bat > @@ -7,7 +7,7 @@ rem -q "quick" -- normally the tests are run twice, the > first time > rem after deleting all the .pyc files reachable from Lib/. > rem -q runs the tests just once, and without deleting .pyc files. > rem -x64 Run the 64-bit build of python (or python_d if -d was specified) > -rem from the 'amd64' dir instead of the 32-bit build in this dir. > +rem When omitted, uses %PREFIX% if set or the 32-bit build > rem All leading instances of these switches are shifted off, and > rem whatever remains (up to 9 arguments) is passed to regrtest.py. > rem For example, > @@ -28,28 +28,29 @@ rem rt -u "network,largefile" > setlocal > > set pcbuild=%~dp0 > -set prefix=%pcbuild%win32\ > set suffix= > set qmode= > set dashO= > set regrtestargs= > +set exe= > > :CheckOpts > if "%1"=="-O" (set dashO=-O) & shift & goto CheckOpts > if "%1"=="-q" (set qmode=yes) & shift & goto CheckOpts > if "%1"=="-d" (set suffix=_d) & shift & goto CheckOpts > -if "%1"=="-x64" (set prefix=%pcbuild%amd64\) & shift & goto CheckOpts > +if "%1"=="-x64" (set prefix=%pcbuild%amd64) & shift & goto CheckOpts > if NOT "%1"=="" (set regrtestargs=%regrtestargs% %1) & shift & goto > CheckOpts > > -set exe=%prefix%python%suffix%.exe > -set cmd="%exe%" %dashO% -Wd -E -bb -m test %regrtestargs% > +if not defined prefix set prefix=%pcbuild%win32 > +set exe=%prefix%\python%suffix%.exe > +set cmd="%exe%" %dashO% -u -Wd -E -bb -m test %regrtestargs% > if defined qmode goto Qmode > > echo Deleting .pyc files ... > "%exe%" "%pcbuild%rmpyc.py" > > echo Cleaning _pth files ... > -if exist %prefix%*._pth del %prefix%*._pth > +if exist %prefix%\*._pth del %prefix%\*._pth > > echo on > %cmd% > diff --git a/Tools/ssl/multissltests.py b/Tools/ssl/multissltests.py > index ba4529ae0611..f3241cd6071c 100755 > --- a/Tools/ssl/multissltests.py > +++ b/Tools/ssl/multissltests.py > @@ -123,6 +123,11 @@ > action='store_true', > help="Don't run tests, only compile _ssl.c and _hashopenssl.c." > ) > +parser.add_argument( > + '--system', > + default='', > + help="Override the automatic system type detection." > +) > > > class AbstractBuilder(object): > @@ -150,6 +155,7 @@ def __init__(self, version, compile_args=(), > # build directory (removed after install) > self.build_dir = os.path.join( > self.src_dir, self.build_template.format(version)) > + self.system = args.system > > def __str__(self): > return "<{0.__class__.__name__} for {0.version}>".format(self) > @@ -254,9 +260,13 @@ def _build_src(self): > cwd = self.build_dir > cmd = ["./config", "shared", > "--prefix={}".format(self.install_dir)] > cmd.extend(self.compile_args) > - self._subprocess_call(cmd, cwd=cwd) > + env = None > + if self.system: > + env = os.environ.copy() > + env['SYSTEM'] = self.system > + self._subprocess_call(cmd, cwd=cwd, env=env) > # Old OpenSSL versions do not support parallel builds. > - self._subprocess_call(["make", "-j1"], cwd=cwd) > + self._subprocess_call(["make", "-j1"], cwd=cwd, env=env) > > def _make_install(self, remove=True): > self._subprocess_call(["make", "-j1", "install"], > cwd=self.build_dir) > > _______________________________________________ > Python-checkins mailing list > Python-checkins(a)python.org > https://mail.python.org/mailman/listinfo/python-checkins >

2 years, 11 months

Normalisation of unicode and keywords

by Steven D'Aprano

Stephan Houben noticed that Python apparently allows identifiers to be keywords, if you use Unicode "mathematical bold" letters. His explanation is that the identifier is normalised, but not until after keywords are checked for. So this works: class Spam: locals()['if'] = 1 Spam.𝐢𝐟 # U+1D422 U+1D41F # returns 1 Of course Spam.if fails with SyntaxError. Should this work? Is this a bug, a feature, or an accident of implementation we can ignore? -- Steve

2 years, 11 months

← Newer
1
2
3
4
5
6
7
8
9
Older →

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

Python-Dev May 2018