Mailman 3 July 2020 - Python-checkins

bpo-41412 and bpo-40948: Windows installer updates (GH-21656)
by Steve Dower 28 Jul '20

28 Jul '20

https://github.com/python/cpython/commit/37a06cbe5c17c2aa6ad938339fd42531a8… commit: 37a06cbe5c17c2aa6ad938339fd42531a8a0bea0 branch: master author: Steve Dower <steve.dower(a)python.org> committer: GitHub <noreply(a)github.com> date: 2020-07-28T16:35:46+01:00 summary: bpo-41412 and bpo-40948: Windows installer updates (GH-21656) Prevent installation on Windows 8 and earlier. Download UCRT on demand when required (non-updated Windows 8.1 only) Add reference to py launcher to post-install message files: A Misc/NEWS.d/next/Windows/2020-07-28-11-55-43.bpo-41412.ME20KB.rst A Misc/NEWS.d/next/Windows/2020-07-28-12-39-32.bpo-40948.ISUFO6.rst M Tools/msi/bundle/Default.thm M Tools/msi/bundle/Default.wxl M Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp M Tools/msi/bundle/packagegroups/crt.wxs diff --git a/Misc/NEWS.d/next/Windows/2020-07-28-11-55-43.bpo-41412.ME20KB.rst b/Misc/NEWS.d/next/Windows/2020-07-28-11-55-43.bpo-41412.ME20KB.rst new file mode 100644 index 0000000000000..274264ad876d3 --- /dev/null +++ b/Misc/NEWS.d/next/Windows/2020-07-28-11-55-43.bpo-41412.ME20KB.rst @@ -0,0 +1,2 @@ +The installer will now fail to install on Windows 7 and Windows 8. Further, +the UCRT dependency is now always downloaded on demand. diff --git a/Misc/NEWS.d/next/Windows/2020-07-28-12-39-32.bpo-40948.ISUFO6.rst b/Misc/NEWS.d/next/Windows/2020-07-28-12-39-32.bpo-40948.ISUFO6.rst new file mode 100644 index 0000000000000..f8831d8c13701 --- /dev/null +++ b/Misc/NEWS.d/next/Windows/2020-07-28-12-39-32.bpo-40948.ISUFO6.rst @@ -0,0 +1 @@ +Improve post-install message to direct people to the "py" command. diff --git a/Tools/msi/bundle/Default.thm b/Tools/msi/bundle/Default.thm index 1c0bd08eec941..7c37d2cafb399 100644 --- a/Tools/msi/bundle/Default.thm +++ b/Tools/msi/bundle/Default.thm @@ -116,7 +116,7 @@ <Text X="185" Y="11" Width="-11" Height="36" FontId="1" DisablePrefix="yes">#(loc.SuccessHeader)</Text> <Image X="0" Y="0" Width="178" Height="382" ImageFile="SideBar.png"/> - <Hypertext Name="SuccessText" X="205" Y="71" Width="-71" Height="150" FontId="3" DisablePrefix="yes"></Hypertext> + <Hypertext Name="SuccessText" X="205" Y="71" Width="-71" Height="180" FontId="3" DisablePrefix="yes"></Hypertext> <Button Name="SuccessMaxPathButton" X="185" Y="-70" Width="-11" Height="81" TabStop="yes" FontId="3" HexStyle="0xE" HideWhenDisabled="yes">#(loc.SuccessMaxPathButton)</Button> diff --git a/Tools/msi/bundle/Default.wxl b/Tools/msi/bundle/Default.wxl index 43506d61004a5..f0088d4e13e6b 100644 --- a/Tools/msi/bundle/Default.wxl +++ b/Tools/msi/bundle/Default.wxl @@ -89,7 +89,7 @@ Select Customize to review current options.</String> <String Id="ShortInstallLauncherAllUsersLabel">Install &launcher for all users (recommended)</String> <String Id="PrecompileLabel">&Precompile standard library</String> <String Id="Include_symbolsLabel">Download debugging &symbols</String> - <String Id="Include_debugLabel">Download debu&g binaries (requires VS 2015 or later)</String> + <String Id="Include_debugLabel">Download debu&g binaries (requires VS 2017 or later)</String> <String Id="ProgressHeader">[ActionLikeInstallation] Progress</String> <String Id="ProgressLabel">[ActionLikeInstalling]:</String> @@ -105,11 +105,11 @@ Select Customize to review current options.</String> <String Id="SuccessLaunchButton">&Launch</String> <String Id="SuccessRestartText">You may need to restart your computer to finish updating files.</String> <String Id="SuccessRestartButton">&Restart</String> - <String Id="SuccessInstallMessage">Special thanks to Mark Hammond, without whose years of freely shared Windows expertise, Python for Windows would still be Python for DOS. + <String Id="SuccessInstallMessage">New to Python? Start with the <a href="https://docs.python.org/[ShortVersion]/tutorial/index.html">online tutorial</a> and <a href="https://docs.python.org/[ShortVersion]/index.html">documentation</a>. At your terminal, type "py" to launch Python, or search for Python in your Start menu. -New to Python? Start with the <a href="https://docs.python.org/[ShortVersion]/tutorial/index.html">online tutorial</a> and <a href="https://docs.python.org/[ShortVersion]/index.html">documentation</a>. +See <a href="https://docs.python.org/[ShortVersion]/whatsnew/[ShortVersion].html">what's new</a> in this release, or find more info about <a href="https://docs.python.org/[ShortVersion]/using/windows.html">using Python on Windows</a>. -See <a href="https://docs.python.org/[ShortVersion]/whatsnew/[ShortVersion].html">what's new</a> in this release.</String> +Special thanks to Mark Hammond, without whose years of freely shared Windows expertise, Python for Windows would still be Python for DOS.</String> <String Id="SuccessModifyMessage">Thank you for using [WixBundleName].</String> <String Id="SuccessRepairMessage">Thank you for using [WixBundleName]. @@ -123,26 +123,9 @@ Feel free to email <a href="mailto:python-list@python.org">python-list@pyt <String Id="FailureRestartButton">&Restart</String> <String Id="FailureExistingInstall">Unable to install [WixBundleName] due to an existing install. Use Programs and Features to modify, repair or remove [WixBundleName].</String> - <String Id="FailureWin7MissingSP1">Windows 7 Service Pack 1 and all applicable updates are required to install [WixBundleName]. - -Please <a href="https://www.bing.com/search?q=how%20to%20install%20windows%207%20service%20…">update your machine</a> and then restart the installation.</String> - <String Id="FailureVistaMissingSP2">Windows Vista Service Pack 2 and all applicable updates are required to install [WixBundleName]. - -Please <a href="https://www.bing.com/search?q=how%20to%20install%20windows%20vista%20servic…">update your machine</a> and then restart the installation.</String> - <String Id="FailureXPOrEarlier">Windows Vista or later is required to install and use [WixBundleName]. - -Visit <a href="https://www.python.org/">python.org</a> to download Python 3.4.</String> - - <String Id="FailureWS2K8R2MissingSP1">Windows Server 2008 R2 Service Pack 1 and all applicable updates are required to install [WixBundleName]. - -Please <a href="https://www.bing.com/search?q=how%20to%20install%20windows%20server%202008%…">update your machine</a> and then restart the installation.</String> - <String Id="FailureWS2K8MissingSP2">Windows Server 2008 Service Pack 2 and all applicable updates are required to install [WixBundleName]. - -Please <a href="https://www.bing.com/search?q=how%20to%20install%20windows%20server%202008%…">update your machine</a> and then restart the installation.</String> - <String Id="FailureWS2K3OrEarlier">Windows Server 2008 SP2 or later is required to install and use [WixBundleName]. - -Visit <a href="https://www.python.org/">python.org</a> to download Python 3.4.</String> + <String Id="FailureOldOS">At least Windows 8.1 or Windows Server 2012 are required to install [WixBundleName] +Visit <a href="https://www.python.org/">python.org</a> to download an earlier version of Python.</String> <String Id="SuccessMaxPathButton">Disable path length limit</String> <String Id="SuccessMaxPathButtonNote">Changes your machine configuration to allow programs, including Python, to bypass the 260 character "MAX_PATH" limitation.</String> </WixLocalization> diff --git a/Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp b/Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp index fbdf7b6424d61..3c54e401330cf 100644 --- a/Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp +++ b/Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp @@ -3011,65 +3011,35 @@ class PythonBootstrapperApplication : public CBalBaseBootstrapperApplication { BalLog(BOOTSTRAPPER_LOG_LEVEL_STANDARD, "Target OS is Windows Server 2012 or later"); return; } else if (IsWindowsVersionOrGreater(6, 1, 1)) { - HMODULE hKernel32 = GetModuleHandleW(L"kernel32"); - if (hKernel32 && !GetProcAddress(hKernel32, "AddDllDirectory")) { - BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Detected Windows Server 2008 R2 without KB2533623"); - BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "KB2533623 update is required to continue."); - /* The "MissingSP1" error also specifies updates are required */ - LocGetString(_wixLoc, L"#(loc.FailureWS2K8R2MissingSP1)", &pLocString); - } else { - BalLog(BOOTSTRAPPER_LOG_LEVEL_STANDARD, "Target OS is Windows Server 2008 R2 or later"); - return; - } + BalLog(BOOTSTRAPPER_LOG_LEVEL_STANDARD, "Detected Windows Server 2008 R2"); } else if (IsWindowsVersionOrGreater(6, 1, 0)) { BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Detected Windows Server 2008 R2"); - BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Service Pack 1 is required to continue installation"); - LocGetString(_wixLoc, L"#(loc.FailureWS2K8R2MissingSP1)", &pLocString); - } else if (IsWindowsVersionOrGreater(6, 0, 2)) { - BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Target OS is Windows Server 2008 SP2 or later"); - return; } else if (IsWindowsVersionOrGreater(6, 0, 0)) { BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Detected Windows Server 2008"); - BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Service Pack 2 is required to continue installation"); - LocGetString(_wixLoc, L"#(loc.FailureWS2K8MissingSP2)", &pLocString); } else { BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Detected Windows Server 2003 or earlier"); - BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Windows Server 2008 SP2 or later is required to continue installation"); - LocGetString(_wixLoc, L"#(loc.FailureWS2K3OrEarlier)", &pLocString); } + BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Windows Server 2012 or later is required to continue installation"); } else { - if (IsWindows8OrGreater()) { - BalLog(BOOTSTRAPPER_LOG_LEVEL_STANDARD, "Target OS is Windows 8 or later"); + if (IsWindows10OrGreater()) { + BalLog(BOOTSTRAPPER_LOG_LEVEL_STANDARD, "Target OS is Windows 10 or later"); return; - } else if (IsWindows7SP1OrGreater()) { - HMODULE hKernel32 = GetModuleHandleW(L"kernel32"); - if (hKernel32 && !GetProcAddress(hKernel32, "AddDllDirectory")) { - BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Detected Windows 7 SP1 without KB2533623"); - BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "KB2533623 update is required to continue."); - /* The "MissingSP1" error also specifies updates are required */ - LocGetString(_wixLoc, L"#(loc.FailureWin7MissingSP1)", &pLocString); - } else { - BalLog(BOOTSTRAPPER_LOG_LEVEL_STANDARD, "Target OS is Windows 7 SP1 or later"); - return; - } - } else if (IsWindows7OrGreater()) { - BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Detected Windows 7 RTM"); - BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Service Pack 1 is required to continue installation"); - LocGetString(_wixLoc, L"#(loc.FailureWin7MissingSP1)", &pLocString); - } else if (IsWindowsVistaSP2OrGreater()) { - BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Target OS is Windows Vista SP2"); + } else if (IsWindows8Point1OrGreater()) { + BalLog(BOOTSTRAPPER_LOG_LEVEL_STANDARD, "Target OS is Windows 8.1"); return; + } else if (IsWindows8OrGreater()) { + BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Detected Windows 8"); + } else if (IsWindows7OrGreater()) { + BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Detected Windows 7"); } else if (IsWindowsVistaOrGreater()) { - BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Detected Windows Vista RTM or SP1"); - BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Service Pack 2 is required to continue installation"); - LocGetString(_wixLoc, L"#(loc.FailureVistaMissingSP2)", &pLocString); + BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Detected Windows Vista"); } else { BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Detected Windows XP or earlier"); - BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Windows Vista SP2 or later is required to continue installation"); - LocGetString(_wixLoc, L"#(loc.FailureXPOrEarlier)", &pLocString); } + BalLog(BOOTSTRAPPER_LOG_LEVEL_ERROR, "Windows 8.1 or later is required to continue installation"); } + LocGetString(_wixLoc, L"#(loc.FailureOldOS)", &pLocString); if (pLocString && pLocString->wzText) { BalFormatString(pLocString->wzText, &_failedMessage); } diff --git a/Tools/msi/bundle/packagegroups/crt.wxs b/Tools/msi/bundle/packagegroups/crt.wxs index 61b756fc2558b..f931da1403b22 100644 --- a/Tools/msi/bundle/packagegroups/crt.wxs +++ b/Tools/msi/bundle/packagegroups/crt.wxs @@ -4,7 +4,7 @@ <PackageGroup Id="crt"> <MsiPackage Id="ucrt_AllUsers" SourceFile="ucrt.msi" - Compressed="$(var.CompressMSI)" + Compressed="no" DownloadUrl="$(var.DownloadUrl)" ForcePerMachine="yes" InstallCondition="InstallAllUsers and not CRTInstalled and (Include_core or Include_exe or Include_pip) and not LauncherOnly"> @@ -12,7 +12,7 @@ </MsiPackage> <MsiPackage Id="ucrt_JustForMe" SourceFile="ucrt.msi" - Compressed="$(var.CompressMSI)" + Compressed="no" DownloadUrl="$(var.DownloadUrl)" ForcePerMachine="no" InstallCondition="not InstallAllUsers and not CRTInstalled and (Include_core or Include_exe or Include_pip) and not LauncherOnly">

1 0

bpo-41328: Replace mention of Hudson CI with Travis CI and AppVeyor (GH-21653)
by Miss Islington (bot) 28 Jul '20

28 Jul '20

https://github.com/python/cpython/commit/59cfba326801a1fd809836c16887f53765… commit: 59cfba326801a1fd809836c16887f53765f9680f branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: GitHub <noreply(a)github.com> date: 2020-07-28T07:57:11-07:00 summary: bpo-41328: Replace mention of Hudson CI with Travis CI and AppVeyor (GH-21653) (cherry picked from commit 5e3826785dcc64f8e1a8a7bde11b88fbb40943be) Co-authored-by: Dmytro Litvinov <litvinov.dmytro.it(a)gmail.com> files: M Doc/library/unittest.rst diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst index d9f080a862ff2..67c5fa6117374 100644 --- a/Doc/library/unittest.rst +++ b/Doc/library/unittest.rst @@ -73,7 +73,7 @@ test runner for those new to unit testing. For production environments it is recommended that tests be driven by a continuous integration system such as `Buildbot <https://buildbot.net/>`_, `Jenkins <https://jenkins.io/>`_ - or `Hudson <http://hudson-ci.org/>`_. + or `Travis-CI <https://travis-ci.com>`_, or `AppVeyor <https://www.appveyor.com/>`_. .. _unittest-minimal-example:

1 0

bpo-41328: Replace mention of Hudson CI with Travis CI and AppVeyor (GH-21653)
by Dmytro Litvinov 28 Jul '20

28 Jul '20

https://github.com/python/cpython/commit/5e3826785dcc64f8e1a8a7bde11b88fbb4… commit: 5e3826785dcc64f8e1a8a7bde11b88fbb40943be branch: master author: Dmytro Litvinov <litvinov.dmytro.it(a)gmail.com> committer: GitHub <noreply(a)github.com> date: 2020-07-28T07:48:32-07:00 summary: bpo-41328: Replace mention of Hudson CI with Travis CI and AppVeyor (GH-21653) files: M Doc/library/unittest.rst diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst index 0dddbd25d991b..a52df9ee01578 100644 --- a/Doc/library/unittest.rst +++ b/Doc/library/unittest.rst @@ -73,7 +73,7 @@ test runner for those new to unit testing. For production environments it is recommended that tests be driven by a continuous integration system such as `Buildbot <https://buildbot.net/>`_, `Jenkins <https://jenkins.io/>`_ - or `Hudson <http://hudson-ci.org/>`_. + or `Travis-CI <https://travis-ci.com>`_, or `AppVeyor <https://www.appveyor.com/>`_. .. _unittest-minimal-example:

1 0

bpo-35328: Set VIRTUAL_ENV_PROMPT at venv activation (GH-21587)
by Zackery Spytz 28 Jul '20

28 Jul '20

https://github.com/python/cpython/commit/c82dda1e08c4b74ca24f88d6a549d93108… commit: c82dda1e08c4b74ca24f88d6a549d93108c319cf branch: master author: Zackery Spytz <zspytz(a)gmail.com> committer: GitHub <noreply(a)github.com> date: 2020-07-28T09:41:57+01:00 summary: bpo-35328: Set VIRTUAL_ENV_PROMPT at venv activation (GH-21587) Co-Authored-By: Baptiste Darthenay <baptiste.darthenay(a)gmail.com> files: A Misc/NEWS.d/next/Library/2020-07-21-16-20-55.bpo-35328.jXovHb.rst M Lib/venv/scripts/common/Activate.ps1 M Lib/venv/scripts/common/activate M Lib/venv/scripts/nt/activate.bat M Lib/venv/scripts/nt/deactivate.bat M Lib/venv/scripts/posix/activate.csh M Lib/venv/scripts/posix/activate.fish diff --git a/Lib/venv/scripts/common/Activate.ps1 b/Lib/venv/scripts/common/Activate.ps1 index a3bc6fb1f05bf..51fc55c4ec7cb 100644 --- a/Lib/venv/scripts/common/Activate.ps1 +++ b/Lib/venv/scripts/common/Activate.ps1 @@ -96,6 +96,11 @@ function global:deactivate ([switch]$NonDestructive) { Remove-Item -Path env:VIRTUAL_ENV } + # Just remove VIRTUAL_ENV_PROMPT altogether. + if (Test-Path -Path Env:VIRTUAL_ENV_PROMPT) { + Remove-Item -Path env:VIRTUAL_ENV_PROMPT + } + # Just remove the _PYTHON_VENV_PROMPT_PREFIX altogether: if (Get-Variable -Name "_PYTHON_VENV_PROMPT_PREFIX" -ErrorAction SilentlyContinue) { Remove-Variable -Name _PYTHON_VENV_PROMPT_PREFIX -Scope Global -Force @@ -228,6 +233,7 @@ if (-not $Env:VIRTUAL_ENV_DISABLE_PROMPT) { Write-Host -NoNewline -ForegroundColor Green "($_PYTHON_VENV_PROMPT_PREFIX) " _OLD_VIRTUAL_PROMPT } + $env:VIRTUAL_ENV_PROMPT = $Prompt } # Clear PYTHONHOME diff --git a/Lib/venv/scripts/common/activate b/Lib/venv/scripts/common/activate index 45af3536aa191..6fbc2b8801da0 100644 --- a/Lib/venv/scripts/common/activate +++ b/Lib/venv/scripts/common/activate @@ -28,6 +28,7 @@ deactivate () { fi unset VIRTUAL_ENV + unset VIRTUAL_ENV_PROMPT if [ ! "${1:-}" = "nondestructive" ] ; then # Self destruct! unset -f deactivate @@ -56,6 +57,8 @@ if [ -z "${VIRTUAL_ENV_DISABLE_PROMPT:-}" ] ; then _OLD_VIRTUAL_PS1="${PS1:-}" PS1="__VENV_PROMPT__${PS1:-}" export PS1 + VIRTUAL_ENV_PROMPT="__VENV_PROMPT__" + export VIRTUAL_ENV_PROMPT fi # This should detect bash and zsh, which have a hash command that must diff --git a/Lib/venv/scripts/nt/activate.bat b/Lib/venv/scripts/nt/activate.bat index af4c7e0abacb1..c1c3c82ee37f1 100644 --- a/Lib/venv/scripts/nt/activate.bat +++ b/Lib/venv/scripts/nt/activate.bat @@ -25,6 +25,7 @@ if defined _OLD_VIRTUAL_PATH set PATH=%_OLD_VIRTUAL_PATH% if not defined _OLD_VIRTUAL_PATH set _OLD_VIRTUAL_PATH=%PATH% set PATH=%VIRTUAL_ENV%\__VENV_BIN_NAME__;%PATH% +set VIRTUAL_ENV_PROMPT=__VENV_PROMPT__ :END if defined _OLD_CODEPAGE ( diff --git a/Lib/venv/scripts/nt/deactivate.bat b/Lib/venv/scripts/nt/deactivate.bat index 1205c618686fb..62a39a7584f4d 100644 --- a/Lib/venv/scripts/nt/deactivate.bat +++ b/Lib/venv/scripts/nt/deactivate.bat @@ -17,5 +17,6 @@ if defined _OLD_VIRTUAL_PATH ( set _OLD_VIRTUAL_PATH= set VIRTUAL_ENV= +set VIRTUAL_ENV_PROMPT= :END diff --git a/Lib/venv/scripts/posix/activate.csh b/Lib/venv/scripts/posix/activate.csh index 68a0dc74e1a3c..d6f697c55ed81 100644 --- a/Lib/venv/scripts/posix/activate.csh +++ b/Lib/venv/scripts/posix/activate.csh @@ -3,7 +3,7 @@ # Created by Davide Di Blasi <davidedb(a)gmail.com>. # Ported to Python 3.3 venv by Andrew Svetlov <andrew.svetlov(a)gmail.com> -alias deactivate 'test $?_OLD_VIRTUAL_PATH != 0 && setenv PATH "$_OLD_VIRTUAL_PATH" && unset _OLD_VIRTUAL_PATH; rehash; test $?_OLD_VIRTUAL_PROMPT != 0 && set prompt="$_OLD_VIRTUAL_PROMPT" && unset _OLD_VIRTUAL_PROMPT; unsetenv VIRTUAL_ENV; test "\!:*" != "nondestructive" && unalias deactivate' +alias deactivate 'test $?_OLD_VIRTUAL_PATH != 0 && setenv PATH "$_OLD_VIRTUAL_PATH" && unset _OLD_VIRTUAL_PATH; rehash; test $?_OLD_VIRTUAL_PROMPT != 0 && set prompt="$_OLD_VIRTUAL_PROMPT" && unset _OLD_VIRTUAL_PROMPT; unsetenv VIRTUAL_ENV; unsetenv VIRTUAL_ENV_PROMPT; test "\!:*" != "nondestructive" && unalias deactivate' # Unset irrelevant variables. deactivate nondestructive @@ -18,6 +18,7 @@ set _OLD_VIRTUAL_PROMPT="$prompt" if (! "$?VIRTUAL_ENV_DISABLE_PROMPT") then set prompt = "__VENV_PROMPT__$prompt" + setenv VIRTUAL_ENV_PROMPT "__VENV_PROMPT__" endif alias pydoc python -m pydoc diff --git a/Lib/venv/scripts/posix/activate.fish b/Lib/venv/scripts/posix/activate.fish index 54b9ea5676b66..e40a1d714894f 100644 --- a/Lib/venv/scripts/posix/activate.fish +++ b/Lib/venv/scripts/posix/activate.fish @@ -20,6 +20,7 @@ function deactivate -d "Exit virtual environment and return to normal shell env end set -e VIRTUAL_ENV + set -e VIRTUAL_ENV_PROMPT if test "$argv[1]" != "nondestructive" # Self-destruct! functions -e deactivate @@ -61,4 +62,5 @@ if test -z "$VIRTUAL_ENV_DISABLE_PROMPT" end set -gx _OLD_FISH_PROMPT_OVERRIDE "$VIRTUAL_ENV" + set -gx VIRTUAL_ENV_PROMPT "__VENV_PROMPT__" end diff --git a/Misc/NEWS.d/next/Library/2020-07-21-16-20-55.bpo-35328.jXovHb.rst b/Misc/NEWS.d/next/Library/2020-07-21-16-20-55.bpo-35328.jXovHb.rst new file mode 100644 index 0000000000000..f4d1c6511d0d1 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-07-21-16-20-55.bpo-35328.jXovHb.rst @@ -0,0 +1,2 @@ +Set the environment variable ``VIRTUAL_ENV_PROMPT`` at :mod:`venv` +activation.

1 0

[3.8] bpo-41045: Document debug feature of f-strings ('=') (GH-21509) (#21647)
by Guido van Rossum 27 Jul '20

27 Jul '20

https://github.com/python/cpython/commit/3baff21f5bb8db7fa1c15788a8f82fa040… commit: 3baff21f5bb8db7fa1c15788a8f82fa040a90d5d branch: 3.8 author: Guido van Rossum <guido(a)python.org> committer: GitHub <noreply(a)github.com> date: 2020-07-27T16:22:16-07:00 summary: [3.8] bpo-41045: Document debug feature of f-strings ('=') (GH-21509) (#21647) Co-Authored-By: Rishi <rishi93dev(a)gmail.com> (cherry picked from commit 13efaec) Co-authored-by: amaajemyfren <32741226+amaajemyfren(a)users.noreply.github.com> files: A Misc/NEWS.d/next/Documentation/2020-07-27-20-46-17.bpo-41045.GFF6Ul.rst M Doc/reference/lexical_analysis.rst diff --git a/Doc/reference/lexical_analysis.rst b/Doc/reference/lexical_analysis.rst index d0e7fde0f5ae3..677b7e20f3776 100644 --- a/Doc/reference/lexical_analysis.rst +++ b/Doc/reference/lexical_analysis.rst @@ -637,9 +637,11 @@ and formatted string literals may be concatenated with plain string literals. single: string; formatted literal single: string; interpolated literal single: f-string + single: fstring single: {} (curly brackets); in formatted string literal single: ! (exclamation); in formatted string literal single: : (colon); in formatted string literal + single: = (equals); for help in debugging using string literals .. _f-strings: Formatted string literals @@ -659,7 +661,7 @@ for the contents of the string is: .. productionlist:: f_string: (`literal_char` | "{{" | "}}" | `replacement_field`)* - replacement_field: "{" `f_expression` ["!" `conversion`] [":" `format_spec`] "}" + replacement_field: "{" `f_expression` ["="] ["!" `conversion`] [":" `format_spec`] "}" f_expression: (`conditional_expression` | "*" `or_expr`) : ("," `conditional_expression` | "," "*" `or_expr`)* [","] : | `yield_expression` @@ -671,10 +673,11 @@ The parts of the string outside curly braces are treated literally, except that any doubled curly braces ``'{{'`` or ``'}}'`` are replaced with the corresponding single curly brace. A single opening curly bracket ``'{'`` marks a replacement field, which starts with a -Python expression. After the expression, there may be a conversion field, -introduced by an exclamation point ``'!'``. A format specifier may also -be appended, introduced by a colon ``':'``. A replacement field ends -with a closing curly bracket ``'}'``. +Python expression. To display both the expression text and its value after +evaluation, (useful in debugging), an equal sign ``'='`` may be added after the +expression. A conversion field, introduced by an exclamation point ``'!'`` may +follow. A format specifier may also be appended, introduced by a colon ``':'``. +A replacement field ends with a closing curly bracket ``'}'``. Expressions in formatted string literals are treated like regular Python expressions surrounded by parentheses, with a few exceptions. @@ -690,6 +693,17 @@ left to right. containing an :keyword:`async for` clause were illegal in the expressions in formatted string literals due to a problem with the implementation. +When the equal sign ``'='`` is provided, the output will have the expression +text, the ``'='`` and the evaluated value. Spaces after the opening brace +``'{'``, within the expression and after the ``'='`` are all retained in the +output. By default, the ``'='`` causes the :func:`repr` of the expression to be +provided, unless there is a format specified. When a format is specified it +defaults to the :func:`str` of the expression unless a conversion ``'!r'`` is +declared. + +.. versionadded:: 3.8 + The equal sign ``'='`` was added in Python 3.8. + If a conversion is specified, the result of evaluating the expression is converted before formatting. Conversion ``'!s'`` calls :func:`str` on the result, ``'!r'`` calls :func:`repr`, and ``'!a'`` calls :func:`ascii`. @@ -724,9 +738,22 @@ Some examples of formatted string literals:: >>> today = datetime(year=2017, month=1, day=27) >>> f"{today:%B %d, %Y}" # using date format specifier 'January 27, 2017' + >>> f"{today=:%B %d, %Y}" # using date format specifier and debugging + 'today=January 27, 2017' >>> number = 1024 >>> f"{number:#0x}" # using integer format specifier '0x400' + >>> foo = "bar" + >>> f"{ foo = }" # preserves whitespace + " foo = 'bar'" + >>> line = "The mill's closed" + >>> f"{line = }" + 'line = "The mill\'s closed"' + >>> f"{line = :20}" + "line = The mill's closed " + >>> f"{line = !r:20}" + 'line = "The mill\'s closed" ' + A consequence of sharing the same syntax as regular string literals is that characters in the replacement fields must not conflict with the diff --git a/Misc/NEWS.d/next/Documentation/2020-07-27-20-46-17.bpo-41045.GFF6Ul.rst b/Misc/NEWS.d/next/Documentation/2020-07-27-20-46-17.bpo-41045.GFF6Ul.rst new file mode 100644 index 0000000000000..dfc9891bb89f2 --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2020-07-27-20-46-17.bpo-41045.GFF6Ul.rst @@ -0,0 +1 @@ +Add documentation for debug feature of f-strings. \ No newline at end of file

1 0

Validate the AST produced by the parser in debug mode (GH-21643)
by Pablo Galindo 27 Jul '20

27 Jul '20

https://github.com/python/cpython/commit/1332226b32da44087a55e1d71990ee6899… commit: 1332226b32da44087a55e1d71990ee6899dfd28a branch: master author: Pablo Galindo <Pablogsal(a)gmail.com> committer: GitHub <noreply(a)github.com> date: 2020-07-27T23:46:59+01:00 summary: Validate the AST produced by the parser in debug mode (GH-21643) This will improve the debug experience if something fails in the produced AST. Previously, errors in the produced AST can be felt much later like in the garbage collector or the compiler, making debugging them much more difficult. files: M Parser/pegen.c diff --git a/Parser/pegen.c b/Parser/pegen.c index e2cbf8ba2461c..f615907f5f571 100644 --- a/Parser/pegen.c +++ b/Parser/pegen.c @@ -4,6 +4,7 @@ #include "pegen.h" #include "string_parser.h" +#include "ast.h" PyObject * _PyPegen_new_type_comment(Parser *p, char *s) @@ -1137,6 +1138,14 @@ _PyPegen_run_parser(Parser *p) return RAISE_SYNTAX_ERROR("multiple statements found while compiling a single statement"); } +#if defined(Py_DEBUG) && defined(Py_BUILD_CORE) + if (p->start_rule == Py_single_input || + p->start_rule == Py_file_input || + p->start_rule == Py_eval_input) + { + assert(PyAST_Validate(res)); + } +#endif return res; }

1 0

bpo-41045: Document debug feature of f-strings ('=') (GH-21509)
by amaajemyfren 27 Jul '20

27 Jul '20

https://github.com/python/cpython/commit/13efaec2e03288d7ff0ee643589c32bde6… commit: 13efaec2e03288d7ff0ee643589c32bde6c6973c branch: master author: amaajemyfren <32741226+amaajemyfren(a)users.noreply.github.com> committer: GitHub <noreply(a)github.com> date: 2020-07-27T15:31:02-07:00 summary: bpo-41045: Document debug feature of f-strings ('=') (GH-21509) Co-Authored-By: Rishi <rishi93dev(a)gmail.com> Automerge-Triggered-By: @gvanrossum files: A Misc/NEWS.d/next/Documentation/2020-07-27-20-46-17.bpo-41045.GFF6Ul.rst M Doc/reference/lexical_analysis.rst diff --git a/Doc/reference/lexical_analysis.rst b/Doc/reference/lexical_analysis.rst index 7d70cbcaa372a..4c0f5688da4d7 100644 --- a/Doc/reference/lexical_analysis.rst +++ b/Doc/reference/lexical_analysis.rst @@ -637,9 +637,11 @@ and formatted string literals may be concatenated with plain string literals. single: string; formatted literal single: string; interpolated literal single: f-string + single: fstring single: {} (curly brackets); in formatted string literal single: ! (exclamation); in formatted string literal single: : (colon); in formatted string literal + single: = (equals); for help in debugging using string literals .. _f-strings: Formatted string literals @@ -659,7 +661,7 @@ for the contents of the string is: .. productionlist:: f_string: (`literal_char` | "{{" | "}}" | `replacement_field`)* - replacement_field: "{" `f_expression` ["!" `conversion`] [":" `format_spec`] "}" + replacement_field: "{" `f_expression` ["="] ["!" `conversion`] [":" `format_spec`] "}" f_expression: (`conditional_expression` | "*" `or_expr`) : ("," `conditional_expression` | "," "*" `or_expr`)* [","] : | `yield_expression` @@ -671,10 +673,11 @@ The parts of the string outside curly braces are treated literally, except that any doubled curly braces ``'{{'`` or ``'}}'`` are replaced with the corresponding single curly brace. A single opening curly bracket ``'{'`` marks a replacement field, which starts with a -Python expression. After the expression, there may be a conversion field, -introduced by an exclamation point ``'!'``. A format specifier may also -be appended, introduced by a colon ``':'``. A replacement field ends -with a closing curly bracket ``'}'``. +Python expression. To display both the expression text and its value after +evaluation, (useful in debugging), an equal sign ``'='`` may be added after the +expression. A conversion field, introduced by an exclamation point ``'!'`` may +follow. A format specifier may also be appended, introduced by a colon ``':'``. +A replacement field ends with a closing curly bracket ``'}'``. Expressions in formatted string literals are treated like regular Python expressions surrounded by parentheses, with a few exceptions. @@ -690,6 +693,17 @@ left to right. containing an :keyword:`async for` clause were illegal in the expressions in formatted string literals due to a problem with the implementation. +When the equal sign ``'='`` is provided, the output will have the expression +text, the ``'='`` and the evaluated value. Spaces after the opening brace +``'{'``, within the expression and after the ``'='`` are all retained in the +output. By default, the ``'='`` causes the :func:`repr` of the expression to be +provided, unless there is a format specified. When a format is specified it +defaults to the :func:`str` of the expression unless a conversion ``'!r'`` is +declared. + +.. versionadded:: 3.8 + The equal sign ``'='`` was added in Python 3.8. + If a conversion is specified, the result of evaluating the expression is converted before formatting. Conversion ``'!s'`` calls :func:`str` on the result, ``'!r'`` calls :func:`repr`, and ``'!a'`` calls :func:`ascii`. @@ -724,9 +738,22 @@ Some examples of formatted string literals:: >>> today = datetime(year=2017, month=1, day=27) >>> f"{today:%B %d, %Y}" # using date format specifier 'January 27, 2017' + >>> f"{today=:%B %d, %Y}" # using date format specifier and debugging + 'today=January 27, 2017' >>> number = 1024 >>> f"{number:#0x}" # using integer format specifier '0x400' + >>> foo = "bar" + >>> f"{ foo = }" # preserves whitespace + " foo = 'bar'" + >>> line = "The mill's closed" + >>> f"{line = }" + 'line = "The mill\'s closed"' + >>> f"{line = :20}" + "line = The mill's closed " + >>> f"{line = !r:20}" + 'line = "The mill\'s closed" ' + A consequence of sharing the same syntax as regular string literals is that characters in the replacement fields must not conflict with the diff --git a/Misc/NEWS.d/next/Documentation/2020-07-27-20-46-17.bpo-41045.GFF6Ul.rst b/Misc/NEWS.d/next/Documentation/2020-07-27-20-46-17.bpo-41045.GFF6Ul.rst new file mode 100644 index 0000000000000..dfc9891bb89f2 --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2020-07-27-20-46-17.bpo-41045.GFF6Ul.rst @@ -0,0 +1 @@ +Add documentation for debug feature of f-strings. \ No newline at end of file

1 0

bpo-40939: Remove even more references to the old parser (GH-21642)
by Lysandros Nikolaou 27 Jul '20

27 Jul '20

https://github.com/python/cpython/commit/b3fbff7289176ba1a322e6899c3d4a0488… commit: b3fbff7289176ba1a322e6899c3d4a04880ed5a7 branch: master author: Lysandros Nikolaou <lisandrosnik(a)gmail.com> committer: GitHub <noreply(a)github.com> date: 2020-07-27T12:52:59-07:00 summary: bpo-40939: Remove even more references to the old parser (GH-21642) Automerge-Triggered-By: @lysnikolaou files: M Include/internal/pycore_interp.h M Makefile.pre.in M Tools/c-analyzer/TODO M Tools/c-analyzer/cpython/_generate.py M Tools/c-analyzer/known.tsv diff --git a/Include/internal/pycore_interp.h b/Include/internal/pycore_interp.h index cfc27470c8041..538aa5a653fbf 100644 --- a/Include/internal/pycore_interp.h +++ b/Include/internal/pycore_interp.h @@ -13,13 +13,6 @@ extern "C" { #include "pycore_gc.h" /* struct _gc_runtime_state */ #include "pycore_warnings.h" /* struct _warnings_runtime_state */ -struct _Py_parser_state { - struct { - int level; - int atbol; - } listnode; -}; - struct _pending_calls { PyThread_type_lock lock; /* Request for running pending calls. */ @@ -240,8 +233,6 @@ struct _is { PyObject *audit_hooks; - struct _Py_parser_state parser; - #if _PY_NSMALLNEGINTS + _PY_NSMALLPOSINTS > 0 /* Small integers are preallocated in this array so that they can be shared. diff --git a/Makefile.pre.in b/Makefile.pre.in index 3428b9842a5a0..6b8f4e086ef35 100644 --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -545,7 +545,6 @@ coverage-lcov: '*/Modules/zlib/*' \ '*/Include/*' \ '*/Modules/xx*.c' \ - '*/Parser/listnode.c' \ '*/Python/pyfpe.c' \ '*/Python/pystrcmp.c' \ '/usr/include/*' \ diff --git a/Tools/c-analyzer/TODO b/Tools/c-analyzer/TODO index 829daba291463..1134055f9a8fd 100644 --- a/Tools/c-analyzer/TODO +++ b/Tools/c-analyzer/TODO @@ -64,8 +64,6 @@ Objects/object.c:_Py_RefTotal Py_ssize_t _Py_ Objects/tupleobject.c:_Py_fast_tuple_allocs Py_ssize_t _Py_fast_tuple_allocs Objects/tupleobject.c:_Py_tuple_zero_allocs Py_ssize_t _Py_tuple_zero_allocs Objects/typeobject.c:next_version_tag static unsigned int next_version_tag -Parser/listnode.c:atbol static int atbol -Parser/listnode.c:level static int level Python/Python-ast.c:init_types():initialized static int initialized Python/bootstrap_hash.c:urandom_cache static struct { int fd; dev_t st_dev; ino_t st_ino; } urandom_cache Python/ceval.c:_Py_CheckRecursionLimit int _Py_CheckRecursionLimit diff --git a/Tools/c-analyzer/cpython/_generate.py b/Tools/c-analyzer/cpython/_generate.py index 4c340acf99e1c..3456604b81470 100644 --- a/Tools/c-analyzer/cpython/_generate.py +++ b/Tools/c-analyzer/cpython/_generate.py @@ -183,9 +183,6 @@ def find_matching_variable(varid, varcache, allfilenames, *, 'float_format': 'static float_format_type ', 'detected_double_format': 'static float_format_type ', 'detected_float_format': 'static float_format_type ', - # Parser/listnode.c - 'level': 'static int ', - 'atbol': 'static int ', # Python/dtoa.c 'private_mem': 'static double private_mem[PRIVATE_mem]', 'pmem_next': 'static double *', diff --git a/Tools/c-analyzer/known.tsv b/Tools/c-analyzer/known.tsv index db44080bc2e31..f92b45e5b4d85 100644 --- a/Tools/c-analyzer/known.tsv +++ b/Tools/c-analyzer/known.tsv @@ -51,7 +51,6 @@ Objects/genobject.c - async_gen_memberlist variable static PyMemberDef async_gen Objects/genobject.c - async_gen_methods variable static PyMethodDef async_gen_methods Python/Python-ast.c - AsyncWith_fields variable static const char *AsyncWith_fields[] Python/Python-ast.c - AsyncWith_type variable static PyTypeObject *AsyncWith_type -Parser/listnode.c - atbol variable static int atbol Modules/atexitmodule.c - atexit_methods variable static PyMethodDef atexit_methods Modules/atexitmodule.c - atexitmodule variable static struct PyModuleDef atexitmodule Modules/atexitmodule.c - atexit_slots variable static PyModuleDef_Slot atexit_slots[] @@ -429,7 +428,6 @@ Modules/_json.c encoder_call kwlist variable static const char *kwlist[] Python/symtable.c - lambda variable static identifier lambda Python/Python-ast.c - Lambda_fields variable static const char *Lambda_fields[] Python/Python-ast.c - Lambda_type variable static PyTypeObject *Lambda_type -Parser/listnode.c - level variable static int level Objects/listobject.c - list_as_mapping variable static PyMappingMethods list_as_mapping Objects/listobject.c - list_as_sequence variable static PySequenceMethods list_as_sequence Python/symtable.c - listcomp variable static identifier listcomp

1 0

bpo-40939: Use the new grammar for the grammar specification documentation (GH-19969)
by Pablo Galindo 27 Jul '20

27 Jul '20

https://github.com/python/cpython/commit/72cabb2aa636272e608285f5a6ba83b62b… commit: 72cabb2aa636272e608285f5a6ba83b62be9be4e branch: master author: Pablo Galindo <Pablogsal(a)gmail.com> committer: GitHub <noreply(a)github.com> date: 2020-07-27T11:20:36-07:00 summary: bpo-40939: Use the new grammar for the grammar specification documentation (GH-19969) (We censor the heck out of actions and some other stuff using a custom "highlighter".) Co-authored-by: Guido van Rossum <guido(a)python.org> files: A Doc/tools/extensions/peg_highlight.py D Grammar/Grammar M Doc/conf.py M Doc/reference/grammar.rst diff --git a/Doc/conf.py b/Doc/conf.py index 12d74ea24ce4a..bfb2a98fc63a2 100644 --- a/Doc/conf.py +++ b/Doc/conf.py @@ -15,7 +15,7 @@ extensions = ['sphinx.ext.coverage', 'sphinx.ext.doctest', 'pyspecific', 'c_annotations', 'escape4chm', - 'asdl_highlight'] + 'asdl_highlight', 'peg_highlight'] doctest_global_setup = ''' diff --git a/Doc/reference/grammar.rst b/Doc/reference/grammar.rst index 83d0f8532c636..acf83765b5796 100644 --- a/Doc/reference/grammar.rst +++ b/Doc/reference/grammar.rst @@ -1,7 +1,19 @@ Full Grammar specification ========================== -This is the full Python grammar, as it is read by the parser generator and used -to parse Python source files: +This is the full Python grammar, derived directly from the grammar +used to generate the CPython parser (see :source:`Grammar/python.gram`). +The version here omits details related to code generation and +error recovery. -.. literalinclude:: ../../Grammar/Grammar +The notation is a mixture of `EBNF +<https://en.wikipedia.org/wiki/Extended_Backus%E2%80%93Naur_form>`_ +and `PEG <https://en.wikipedia.org/wiki/Parsing_expression_grammar>`_. +In particular, ``&`` followed by a symbol, token or parenthesized +group indicates a positive lookahead (i.e., is required to match but +not consumed), while ``!`` indicates a negative lookahead (i.e., is +required _not_ to match). We use the ``|`` separator to mean PEG's +"ordered choice" (written as ``/`` in traditional PEG grammars). + +.. literalinclude:: ../../Grammar/python.gram + :language: peg diff --git a/Doc/tools/extensions/peg_highlight.py b/Doc/tools/extensions/peg_highlight.py new file mode 100644 index 0000000000000..f02515d3919cf --- /dev/null +++ b/Doc/tools/extensions/peg_highlight.py @@ -0,0 +1,75 @@ +from pygments.lexer import RegexLexer, bygroups, include +from pygments.token import Comment, Generic, Keyword, Name, Operator, Punctuation, Text + +from sphinx.highlighting import lexers + + +class PEGLexer(RegexLexer): + """Pygments Lexer for PEG grammar (.gram) files + + This lexer strips the following elements from the grammar: + + - Meta-tags + - Variable assignments + - Actions + - Lookaheads + - Rule types + - Rule options + - Rules named `invalid_*` or `incorrect_*` + """ + + name = "PEG" + aliases = ["peg"] + filenames = ["*.gram"] + _name = r"([^\W\d]\w*)" + _text_ws = r"(\s*)" + + tokens = { + "ws": [(r"\n", Text), (r"\s+", Text), (r"#.*$", Comment.Singleline),], + "lookaheads": [ + (r"(?<=\|\s)(&\w+\s?)", bygroups(None)), + (r"(?<=\|\s)(&'.+'\s?)", bygroups(None)), + (r'(?<=\|\s)(&".+"\s?)', bygroups(None)), + (r"(?<=\|\s)(&$.+$\s?)", bygroups(None)), + ], + "metas": [ + (r"(@\w+ '''(.|\n)+?''')", bygroups(None)), + (r"^(@.*)$", bygroups(None)), + ], + "actions": [(r"{(.|\n)+?}", bygroups(None)),], + "strings": [ + (r"'\w+?'", Keyword), + (r'"\w+?"', Keyword), + (r"'\W+?'", Text), + (r'"\W+?"', Text), + ], + "variables": [(_name + _text_ws + "(=)", bygroups(None, None, None),),], + "invalids": [ + (r"^(\s+\|\s+invalid_\w+\s*\n)", bygroups(None)), + (r"^(\s+\|\s+incorrect_\w+\s*\n)", bygroups(None)), + (r"^(#.*invalid syntax.*(?:.|\n)*)", bygroups(None),), + ], + "root": [ + include("invalids"), + include("ws"), + include("lookaheads"), + include("metas"), + include("actions"), + include("strings"), + include("variables"), + (r"\b(?!(NULL|EXTRA))([A-Z_]+)\b\s*(?!$)", Text,), + ( + r"^\s*" + _name + "\s*" + "(\[.*\])?" + "\s*" + "(\(.+$)?" + "\s*(:)", + bygroups(Name.Function, None, None, Punctuation), + ), + (_name, Name.Function), + (r"[\||\.|\+|\*|\?]", Operator), + (r"{|}|$|$|\[|\]", Punctuation), + (r".", Text), + ], + } + + +def setup(app): + lexers["peg"] = PEGLexer() + return {"version": "1.0", "parallel_read_safe": True} diff --git a/Grammar/Grammar b/Grammar/Grammar deleted file mode 100644 index 61a26243f886b..0000000000000 --- a/Grammar/Grammar +++ /dev/null @@ -1,206 +0,0 @@ -# Grammar for Python - -# NOTE: Editing this file has no effect except on the docs. - -# Start symbols for the grammar: -# single_input is a single interactive statement; -# file_input is a module or sequence of commands read from an input file; -# eval_input is the input for the eval() functions. -# func_type_input is a PEP 484 Python 2 function type comment -# NB: compound_stmt in single_input is followed by extra NEWLINE! -# NB: due to the way TYPE_COMMENT is tokenized it will always be followed by a NEWLINE -single_input: NEWLINE | simple_stmt | compound_stmt NEWLINE -file_input: (NEWLINE | stmt)* ENDMARKER -eval_input: testlist NEWLINE* ENDMARKER - -decorator: '@' namedexpr_test NEWLINE -decorators: decorator+ -decorated: decorators (classdef | funcdef | async_funcdef) - -async_funcdef: ASYNC funcdef -funcdef: 'def' NAME parameters ['->' test] ':' [TYPE_COMMENT] func_body_suite - -parameters: '(' [typedargslist] ')' - -# The following definition for typedarglist is equivalent to this set of rules: -# -# arguments = argument (',' [TYPE_COMMENT] argument)* -# argument = tfpdef ['=' test] -# kwargs = '**' tfpdef [','] [TYPE_COMMENT] -# args = '*' [tfpdef] -# kwonly_kwargs = (',' [TYPE_COMMENT] argument)* (TYPE_COMMENT | [',' [TYPE_COMMENT] [kwargs]]) -# args_kwonly_kwargs = args kwonly_kwargs | kwargs -# poskeyword_args_kwonly_kwargs = arguments ( TYPE_COMMENT | [',' [TYPE_COMMENT] [args_kwonly_kwargs]]) -# typedargslist_no_posonly = poskeyword_args_kwonly_kwargs | args_kwonly_kwargs -# typedarglist = (arguments ',' [TYPE_COMMENT] '/' [',' [[TYPE_COMMENT] typedargslist_no_posonly]])|(typedargslist_no_posonly)" -# -# It needs to be fully expanded to allow our LL(1) parser to work on it. - -typedargslist: ( - (tfpdef ['=' test] (',' [TYPE_COMMENT] tfpdef ['=' test])* ',' [TYPE_COMMENT] '/' [',' [ [TYPE_COMMENT] tfpdef ['=' test] ( - ',' [TYPE_COMMENT] tfpdef ['=' test])* (TYPE_COMMENT | [',' [TYPE_COMMENT] [ - '*' [tfpdef] (',' [TYPE_COMMENT] tfpdef ['=' test])* (TYPE_COMMENT | [',' [TYPE_COMMENT] ['**' tfpdef [','] [TYPE_COMMENT]]]) - | '**' tfpdef [','] [TYPE_COMMENT]]]) - | '*' [tfpdef] (',' [TYPE_COMMENT] tfpdef ['=' test])* (TYPE_COMMENT | [',' [TYPE_COMMENT] ['**' tfpdef [','] [TYPE_COMMENT]]]) - | '**' tfpdef [','] [TYPE_COMMENT]]] ) -| (tfpdef ['=' test] (',' [TYPE_COMMENT] tfpdef ['=' test])* (TYPE_COMMENT | [',' [TYPE_COMMENT] [ - '*' [tfpdef] (',' [TYPE_COMMENT] tfpdef ['=' test])* (TYPE_COMMENT | [',' [TYPE_COMMENT] ['**' tfpdef [','] [TYPE_COMMENT]]]) - | '**' tfpdef [','] [TYPE_COMMENT]]]) - | '*' [tfpdef] (',' [TYPE_COMMENT] tfpdef ['=' test])* (TYPE_COMMENT | [',' [TYPE_COMMENT] ['**' tfpdef [','] [TYPE_COMMENT]]]) - | '**' tfpdef [','] [TYPE_COMMENT]) -) -tfpdef: NAME [':' test] - -# The following definition for varargslist is equivalent to this set of rules: -# -# arguments = argument (',' argument )* -# argument = vfpdef ['=' test] -# kwargs = '**' vfpdef [','] -# args = '*' [vfpdef] -# kwonly_kwargs = (',' argument )* [',' [kwargs]] -# args_kwonly_kwargs = args kwonly_kwargs | kwargs -# poskeyword_args_kwonly_kwargs = arguments [',' [args_kwonly_kwargs]] -# vararglist_no_posonly = poskeyword_args_kwonly_kwargs | args_kwonly_kwargs -# varargslist = arguments ',' '/' [','[(vararglist_no_posonly)]] | (vararglist_no_posonly) -# -# It needs to be fully expanded to allow our LL(1) parser to work on it. - -varargslist: vfpdef ['=' test ](',' vfpdef ['=' test])* ',' '/' [',' [ (vfpdef ['=' test] (',' vfpdef ['=' test])* [',' [ - '*' [vfpdef] (',' vfpdef ['=' test])* [',' ['**' vfpdef [',']]] - | '**' vfpdef [',']]] - | '*' [vfpdef] (',' vfpdef ['=' test])* [',' ['**' vfpdef [',']]] - | '**' vfpdef [',']) ]] | (vfpdef ['=' test] (',' vfpdef ['=' test])* [',' [ - '*' [vfpdef] (',' vfpdef ['=' test])* [',' ['**' vfpdef [',']]] - | '**' vfpdef [',']]] - | '*' [vfpdef] (',' vfpdef ['=' test])* [',' ['**' vfpdef [',']]] - | '**' vfpdef [','] -) -vfpdef: NAME - -stmt: simple_stmt | compound_stmt -simple_stmt: small_stmt (';' small_stmt)* [';'] NEWLINE -small_stmt: (expr_stmt | del_stmt | pass_stmt | flow_stmt | - import_stmt | global_stmt | nonlocal_stmt | assert_stmt) -expr_stmt: testlist_star_expr (annassign | augassign (yield_expr|testlist) | - [('=' (yield_expr|testlist_star_expr))+ [TYPE_COMMENT]] ) -annassign: ':' test ['=' (yield_expr|testlist_star_expr)] -testlist_star_expr: (test|star_expr) (',' (test|star_expr))* [','] -augassign: ('+=' | '-=' | '*=' | '@=' | '/=' | '%=' | '&=' | '|=' | '^=' | - '<<=' | '>>=' | '**=' | '//=') -# For normal and annotated assignments, additional restrictions enforced by the interpreter -del_stmt: 'del' exprlist -pass_stmt: 'pass' -flow_stmt: break_stmt | continue_stmt | return_stmt | raise_stmt | yield_stmt -break_stmt: 'break' -continue_stmt: 'continue' -return_stmt: 'return' [testlist_star_expr] -yield_stmt: yield_expr -raise_stmt: 'raise' [test ['from' test]] -import_stmt: import_name | import_from -import_name: 'import' dotted_as_names -# note below: the ('.' | '...') is necessary because '...' is tokenized as ELLIPSIS -import_from: ('from' (('.' | '...')* dotted_name | ('.' | '...')+) - 'import' ('*' | '(' import_as_names ')' | import_as_names)) -import_as_name: NAME ['as' NAME] -dotted_as_name: dotted_name ['as' NAME] -import_as_names: import_as_name (',' import_as_name)* [','] -dotted_as_names: dotted_as_name (',' dotted_as_name)* -dotted_name: NAME ('.' NAME)* -global_stmt: 'global' NAME (',' NAME)* -nonlocal_stmt: 'nonlocal' NAME (',' NAME)* -assert_stmt: 'assert' test [',' test] - -compound_stmt: if_stmt | while_stmt | for_stmt | try_stmt | with_stmt | funcdef | classdef | decorated | async_stmt -async_stmt: ASYNC (funcdef | with_stmt | for_stmt) -if_stmt: 'if' namedexpr_test ':' suite ('elif' namedexpr_test ':' suite)* ['else' ':' suite] -while_stmt: 'while' namedexpr_test ':' suite ['else' ':' suite] -for_stmt: 'for' exprlist 'in' testlist ':' [TYPE_COMMENT] suite ['else' ':' suite] -try_stmt: ('try' ':' suite - ((except_clause ':' suite)+ - ['else' ':' suite] - ['finally' ':' suite] | - 'finally' ':' suite)) -with_stmt: 'with' with_item (',' with_item)* ':' [TYPE_COMMENT] suite -with_item: test ['as' expr] -# NB compile.c makes sure that the default except clause is last -except_clause: 'except' [test ['as' NAME]] -suite: simple_stmt | NEWLINE INDENT stmt+ DEDENT - -namedexpr_test: test [':=' test] -test: or_test ['if' or_test 'else' test] | lambdef -test_nocond: or_test | lambdef_nocond -lambdef: 'lambda' [varargslist] ':' test -lambdef_nocond: 'lambda' [varargslist] ':' test_nocond -or_test: and_test ('or' and_test)* -and_test: not_test ('and' not_test)* -not_test: 'not' not_test | comparison -comparison: expr (comp_op expr)* -# <> isn't actually a valid comparison operator in Python. It's here for the -# sake of a __future__ import described in PEP 401 (which really works :-) -comp_op: '<'|'>'|'=='|'>='|'<='|'<>'|'!='|'in'|'not' 'in'|'is'|'is' 'not' -star_expr: '*' expr -expr: xor_expr ('|' xor_expr)* -xor_expr: and_expr ('^' and_expr)* -and_expr: shift_expr ('&' shift_expr)* -shift_expr: arith_expr (('<<'|'>>') arith_expr)* -arith_expr: term (('+'|'-') term)* -term: factor (('*'|'@'|'/'|'%'|'//') factor)* -factor: ('+'|'-'|'~') factor | power -power: atom_expr ['**' factor] -atom_expr: [AWAIT] atom trailer* -atom: ('(' [yield_expr|testlist_comp] ')' | - '[' [testlist_comp] ']' | - '{' [dictorsetmaker] '}' | - NAME | NUMBER | STRING+ | '...' | 'None' | 'True' | 'False') -testlist_comp: (namedexpr_test|star_expr) ( comp_for | (',' (namedexpr_test|star_expr))* [','] ) -trailer: '(' [arglist] ')' | '[' subscriptlist ']' | '.' NAME -subscriptlist: subscript (',' subscript)* [','] -subscript: test | [test] ':' [test] [sliceop] -sliceop: ':' [test] -exprlist: (expr|star_expr) (',' (expr|star_expr))* [','] -testlist: test (',' test)* [','] -dictorsetmaker: ( ((test ':' test | '**' expr) - (comp_for | (',' (test ':' test | '**' expr))* [','])) | - ((test | star_expr) - (comp_for | (',' (test | star_expr))* [','])) ) - -classdef: 'class' NAME ['(' [arglist] ')'] ':' suite - -arglist: argument (',' argument)* [','] - -# The reason that keywords are test nodes instead of NAME is that using NAME -# results in an ambiguity. ast.c makes sure it's a NAME. -# "test '=' test" is really "keyword '=' test", but we have no such token. -# These need to be in a single rule to avoid grammar that is ambiguous -# to our LL(1) parser. Even though 'test' includes '*expr' in star_expr, -# we explicitly match '*' here, too, to give it proper precedence. -# Illegal combinations and orderings are blocked in ast.c: -# multiple (test comp_for) arguments are blocked; keyword unpackings -# that precede iterable unpackings are blocked; etc. -argument: ( test [comp_for] | - test ':=' test | - test '=' test | - '**' test | - '*' test ) - -comp_iter: comp_for | comp_if -sync_comp_for: 'for' exprlist 'in' or_test [comp_iter] -comp_for: [ASYNC] sync_comp_for -comp_if: 'if' test_nocond [comp_iter] - -# not used in grammar, but may appear in "node" passed from Parser to Compiler -encoding_decl: NAME - -yield_expr: 'yield' [yield_arg] -yield_arg: 'from' test | testlist_star_expr - -# the TYPE_COMMENT in suites is only parsed for funcdefs, -# but can't go elsewhere due to ambiguity -func_body_suite: simple_stmt | NEWLINE [TYPE_COMMENT NEWLINE] INDENT stmt+ DEDENT - -func_type_input: func_type NEWLINE* ENDMARKER -func_type: '(' [typelist] ')' '->' test -# typelist is a modified typedargslist (see above) -typelist: (test (',' test)* [',' - ['*' [test] (',' test)* [',' '**' test] | '**' test]] - | '*' [test] (',' test)* [',' '**' test] | '**' test)

1 0

bpo-41401: Fix test_fspath_support in test_io. (GH-21640)
by Serhiy Storchaka 27 Jul '20

27 Jul '20

https://github.com/python/cpython/commit/67987acd5dc9776f55f4e139e2b3d9e7a6… commit: 67987acd5dc9776f55f4e139e2b3d9e7a6434d9f branch: master author: Serhiy Storchaka <storchaka(a)gmail.com> committer: GitHub <noreply(a)github.com> date: 2020-07-27T20:58:35+03:00 summary: bpo-41401: Fix test_fspath_support in test_io. (GH-21640) The error is exposed on non-UTF-8 locales. files: M Lib/test/test_io.py diff --git a/Lib/test/test_io.py b/Lib/test/test_io.py index 18b8a7941668f..85fac30e300a6 100644 --- a/Lib/test/test_io.py +++ b/Lib/test/test_io.py @@ -928,7 +928,7 @@ def check_path_succeeds(path): self.assertEqual(f.read(), "egg\n") check_path_succeeds(FakePath(os_helper.TESTFN)) - check_path_succeeds(FakePath(os_helper.TESTFN.encode('utf-8'))) + check_path_succeeds(FakePath(os.fsencode(os_helper.TESTFN))) with self.open(os_helper.TESTFN, "w") as f: bad_path = FakePath(f.fileno())

1 0