https://github.com/python/cpython/commit/0dfb437a32f9b9a8a0ae61d8c43c4741765... commit: 0dfb437a32f9b9a8a0ae61d8c43c47417651f44e branch: 3.13 author: Miss Islington (bot) <31488909+miss-islington@users.noreply.github.com> committer: freakboy3742 <russell@keith-magee.com> date: 2024-06-30T09:10:34+08:00 summary: [3.13] gh-120522: Add a `--with-app-store-compliance` configure option to patch out problematic code (GH-120984) (#121173) gh-120522: Add a `--with-app-store-compliance` configure option to patch out problematic code (GH-120984) * Add --app-store-compliance configuration option. * Added blurb. * Correct tab-vs-spaces formatting issue. * Correct source file name in docs. * Correct source code reference in Mac docs * Only apply the patch forward, and ensure the working directory is correct. * Make patching reslient to multiple builds. * Documentation fixes found during review * Documentation and configure.ac syntax improvements * Regenerate configure script. * Silence the patch echo output. --------- (cherry picked from commit 48cd104b0cf05dad8958efa9cb9666c029ef9201) Co-authored-by: Russell Keith-Magee <russell@keith-magee.com> Co-authored-by: Nice Zombies <nineteendo19d0@gmail.com> Co-authored-by: Alyssa Coghlan <ncoghlan@gmail.com> Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com> files: A Mac/Resources/app-store-compliance.patch A Misc/NEWS.d/next/Build/2024-06-25-15-29-27.gh-issue-120522.5_n515.rst M Doc/library/urllib.parse.rst M Doc/using/configure.rst M Doc/using/ios.rst M Doc/using/mac.rst M Makefile.pre.in M configure M configure.ac diff --git a/Doc/library/urllib.parse.rst b/Doc/library/urllib.parse.rst index 27909b763e9e43..fb5353e1895bf9 100644 --- a/Doc/library/urllib.parse.rst +++ b/Doc/library/urllib.parse.rst @@ -22,11 +22,19 @@ to an absolute URL given a "base URL." The module has been designed to match the internet RFC on Relative Uniform Resource Locators. It supports the following URL schemes: ``file``, ``ftp``, -``gopher``, ``hdl``, ``http``, ``https``, ``imap``, ``mailto``, ``mms``, +``gopher``, ``hdl``, ``http``, ``https``, ``imap``, ``itms-services``, ``mailto``, ``mms``, ``news``, ``nntp``, ``prospero``, ``rsync``, ``rtsp``, ``rtsps``, ``rtspu``, ``sftp``, ``shttp``, ``sip``, ``sips``, ``snews``, ``svn``, ``svn+ssh``, ``telnet``, ``wais``, ``ws``, ``wss``. +.. impl-detail:: + + The inclusion of the ``itms-services`` URL scheme can prevent an app from + passing Apple's App Store review process for the macOS and iOS App Stores. + Handling for the ``itms-services`` scheme is always removed on iOS; on + macOS, it *may* be removed if CPython has been built with the + :option:`--with-app-store-compliance` option. + The :mod:`urllib.parse` module defines functions that fall into two broad categories: URL parsing and URL quoting. These are covered in detail in the following sections. diff --git a/Doc/using/configure.rst b/Doc/using/configure.rst index 428ee5275276a0..2a1f06e2d286ff 100644 --- a/Doc/using/configure.rst +++ b/Doc/using/configure.rst @@ -945,6 +945,17 @@ See :source:`Mac/README.rst`. Specify the name for the python framework on macOS only valid when :option:`--enable-framework` is set (default: ``Python``). +.. option:: --with-app-store-compliance +.. option:: --with-app-store-compliance=PATCH-FILE + + The Python standard library contains strings that are known to trigger + automated inspection tool errors when submitted for distribution by + the macOS and iOS App Stores. If enabled, this option will apply the list of + patches that are known to correct app store compliance. A custom patch + file can also be specified. This option is disabled by default. + + .. versionadded:: 3.13 + iOS Options ----------- diff --git a/Doc/using/ios.rst b/Doc/using/ios.rst index 71fc29c450c8eb..774856e8aec2ac 100644 --- a/Doc/using/ios.rst +++ b/Doc/using/ios.rst @@ -312,3 +312,21 @@ modules in your app, some additional steps will be required: * If you're using a separate folder for third-party packages, ensure that folder is included as part of the ``PYTHONPATH`` configuration in step 10. + +App Store Compliance +==================== + +The only mechanism for distributing apps to third-party iOS devices is to +submit the app to the iOS App Store; apps submitted for distribution must pass +Apple's app review process. This process includes a set of automated validation +rules that inspect the submitted application bundle for problematic code. + +The Python standard library contains some code that is known to violate these +automated rules. While these violations appear to be false positives, Apple's +review rules cannot be challenged; so, it is necessary to modify the Python +standard library for an app to pass App Store review. + +The Python source tree contains +:source:`a patch file <Mac/Resources/app-store-compliance.patch>` that will remove +all code that is known to cause issues with the App Store review process. This +patch is applied automatically when building for iOS. diff --git a/Doc/using/mac.rst b/Doc/using/mac.rst index 31d37aad2a7408..44fb00de3733c5 100644 --- a/Doc/using/mac.rst +++ b/Doc/using/mac.rst @@ -188,6 +188,28 @@ distributable application: * `PyInstaller <https://pyinstaller.org/>`__: A cross-platform packaging tool that creates a single file or folder as a distributable artifact. +App Store Compliance +-------------------- + +Apps submitted for distribution through the macOS App Store must pass Apple's +app review process. This process includes a set of automated validation rules +that inspect the submitted application bundle for problematic code. + +The Python standard library contains some code that is known to violate these +automated rules. While these violations appear to be false positives, Apple's +review rules cannot be challenged. Therefore, it is necessary to modify the +Python standard library for an app to pass App Store review. + +The Python source tree contains +:source:`a patch file <Mac/Resources/app-store-compliance.patch>` that will remove +all code that is known to cause issues with the App Store review process. This +patch is applied automatically when CPython is configured with the +:option:`--with-app-store-compliance` option. + +This patch is not normally required to use CPython on a Mac; nor is it required +if you are distributing an app *outside* the macOS App Store. It is *only* +required if you are using the macOS App Store as a distribution channel. + Other Resources =============== diff --git a/Mac/Resources/app-store-compliance.patch b/Mac/Resources/app-store-compliance.patch new file mode 100644 index 00000000000000..f4b7decc01cf1f --- /dev/null +++ b/Mac/Resources/app-store-compliance.patch @@ -0,0 +1,29 @@ +diff --git a/Lib/test/test_urlparse.py b/Lib/test/test_urlparse.py +index d6c83a75c1c..19ed4e01091 100644 +--- a/Lib/test/test_urlparse.py ++++ b/Lib/test/test_urlparse.py +@@ -237,11 +237,6 @@ def test_roundtrips(self): + '','',''), + ('git+ssh', 'git@github.com','/user/project.git', + '', '')), +- ('itms-services://?action=download-manifest&url=https://example.com/app', +- ('itms-services', '', '', '', +- 'action=download-manifest&url=https://example.com/app', ''), +- ('itms-services', '', '', +- 'action=download-manifest&url=https://example.com/app', '')), + ('+scheme:path/to/file', + ('', '', '+scheme:path/to/file', '', '', ''), + ('', '', '+scheme:path/to/file', '', '')), +diff --git a/Lib/urllib/parse.py b/Lib/urllib/parse.py +index 8f724f907d4..148caf742c9 100644 +--- a/Lib/urllib/parse.py ++++ b/Lib/urllib/parse.py +@@ -59,7 +59,7 @@ + 'imap', 'wais', 'file', 'mms', 'https', 'shttp', + 'snews', 'prospero', 'rtsp', 'rtsps', 'rtspu', 'rsync', + 'svn', 'svn+ssh', 'sftp', 'nfs', 'git', 'git+ssh', +- 'ws', 'wss', 'itms-services'] ++ 'ws', 'wss'] + + uses_params = ['', 'ftp', 'hdl', 'prospero', 'http', 'imap', + 'https', 'shttp', 'rtsp', 'rtsps', 'rtspu', 'sip', diff --git a/Makefile.pre.in b/Makefile.pre.in index 22d8bb96208754..9900560d31255a 100644 --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -178,6 +178,9 @@ EXPORTSFROM= @EXPORTSFROM@ EXE= @EXEEXT@ BUILDEXE= @BUILDEXEEXT@ +# Name of the patch file to apply for app store compliance +APP_STORE_COMPLIANCE_PATCH=@APP_STORE_COMPLIANCE_PATCH@ + # Short name and location for Mac OS X Python framework UNIVERSALSDK=@UNIVERSALSDK@ PYTHONFRAMEWORK= @PYTHONFRAMEWORK@ @@ -691,7 +694,7 @@ list-targets: @grep -E '^[A-Za-z][-A-Za-z0-9]+:' Makefile | awk -F : '{print $$1}' .PHONY: build_all -build_all: check-clean-src $(BUILDPYTHON) platform sharedmods \ +build_all: check-clean-src @APP_STORE_COMPLIANCE_PATCH_TARGET@ $(BUILDPYTHON) platform sharedmods \ gdbhooks Programs/_testembed scripts checksharedmods rundsymutil .PHONY: build_wasm @@ -927,6 +930,18 @@ SRC_GDB_HOOKS=$(srcdir)/Tools/gdb/libpython.py $(BUILDPYTHON)-gdb.py: $(SRC_GDB_HOOKS) $(INSTALL_DATA) $(SRC_GDB_HOOKS) $(BUILDPYTHON)-gdb.py +# Compliance with app stores (such as iOS and macOS) sometimes requires making +# modifications to the Python standard library. If enabled, apply the patch of +# known modifications to the source tree before building. The patch will be +# applied in a dry-run mode (validating, but not applying the patch) on builds +# that *have* a compliance patch, but where compliance has not been enabled. +build/app-store-compliant: + patch @APP_STORE_COMPLIANCE_PATCH_FLAGS@ --forward --strip=1 --directory="$(srcdir)" --input "$(APP_STORE_COMPLIANCE_PATCH)" + @if test "@APP_STORE_COMPLIANCE_PATCH_FLAGS@" == ""; then \ + mkdir -p build ; \ + echo "$(APP_STORE_COMPLIANCE_PATCH)" > build/app-store-compliant ; \ + fi + # This rule is here for OPENSTEP/Rhapsody/MacOSX. It builds a temporary # minimal framework (not including the Lib directory and such) in the current # directory. diff --git a/Misc/NEWS.d/next/Build/2024-06-25-15-29-27.gh-issue-120522.5_n515.rst b/Misc/NEWS.d/next/Build/2024-06-25-15-29-27.gh-issue-120522.5_n515.rst new file mode 100644 index 00000000000000..4a397a445ccc3b --- /dev/null +++ b/Misc/NEWS.d/next/Build/2024-06-25-15-29-27.gh-issue-120522.5_n515.rst @@ -0,0 +1,2 @@ +Added a :option:`--with-app-store-compliance` option to patch out known issues +with macOS/iOS App Store review processes. diff --git a/configure b/configure index c92ec305333c5a..a5578fc9e8bae2 100755 --- a/configure +++ b/configure @@ -981,6 +981,9 @@ IPHONEOS_DEPLOYMENT_TARGET EXPORT_MACOSX_DEPLOYMENT_TARGET CONFIGURE_MACOSX_DEPLOYMENT_TARGET _PYTHON_HOST_PLATFORM +APP_STORE_COMPLIANCE_PATCH_FLAGS +APP_STORE_COMPLIANCE_PATCH_TARGET +APP_STORE_COMPLIANCE_PATCH INSTALLTARGETS FRAMEWORKINSTALLAPPSPREFIX FRAMEWORKUNIXTOOLSPREFIX @@ -1076,6 +1079,7 @@ enable_universalsdk with_universal_archs with_framework_name enable_framework +with_app_store_compliance with_emscripten_target enable_wasm_dynamic_linking enable_wasm_pthreads @@ -1855,6 +1859,10 @@ Optional Packages: specify the name for the python framework on macOS only valid when --enable-framework is set. see Mac/README.rst (default is 'Python') + --with-app-store-compliance=[PATCH-FILE] + Enable any patches required for compiliance with app + stores. Optional PATCH-FILE specifies the custom + patch to apply. --with-emscripten-target=[browser|node] Emscripten platform --with-suffix=SUFFIX set executable suffix to SUFFIX (default is empty, @@ -4430,6 +4438,71 @@ fi printf "%s\n" "#define _PYTHONFRAMEWORK \"${PYTHONFRAMEWORK}\"" >>confdefs.h +{ printf "%s\n" "$as_me:${as_lineno-$LINENO}: checking for --with-app-store-compliance" >&5 +printf %s "checking for --with-app-store-compliance... " >&6; } + +# Check whether --with-app_store_compliance was given. +if test ${with_app_store_compliance+y} +then : + withval=$with_app_store_compliance; + case "$withval" in + yes) + case $ac_sys_system in + Darwin|iOS) + # iOS is able to share the macOS patch + APP_STORE_COMPLIANCE_PATCH="Mac/Resources/app-store-compliance.patch" + APP_STORE_COMPLIANCE_PATCH_TARGET="build/app-store-compliant" + APP_STORE_COMPLIANCE_PATCH_FLAGS= + ;; + *) as_fn_error $? "no default app store compliance patch available for $ac_sys_system" "$LINENO" 5 ;; + esac + { printf "%s\n" "$as_me:${as_lineno-$LINENO}: result: applying default app store compliance patch" >&5 +printf "%s\n" "applying default app store compliance patch" >&6; } + ;; + *) + APP_STORE_COMPLIANCE_PATCH="${withval}" + APP_STORE_COMPLIANCE_PATCH_TARGET="build/app-store-compliant" + APP_STORE_COMPLIANCE_PATCH_FLAGS= + { printf "%s\n" "$as_me:${as_lineno-$LINENO}: result: applying custom app store compliance patch" >&5 +printf "%s\n" "applying custom app store compliance patch" >&6; } + ;; + esac + +else $as_nop + + case $ac_sys_system in + iOS) + # Always apply the compliance patch on iOS; we can use the macOS patch + APP_STORE_COMPLIANCE_PATCH="Mac/Resources/app-store-compliance.patch" + APP_STORE_COMPLIANCE_PATCH_TARGET="build/app-store-compliant" + APP_STORE_COMPLIANCE_PATCH_FLAGS= + { printf "%s\n" "$as_me:${as_lineno-$LINENO}: result: applying default app store compliance patch" >&5 +printf "%s\n" "applying default app store compliance patch" >&6; } + ;; + Darwin) + # Always *check* the compliance patch on macOS + APP_STORE_COMPLIANCE_PATCH="Mac/Resources/app-store-compliance.patch" + APP_STORE_COMPLIANCE_PATCH_TARGET="build/app-store-compliant" + APP_STORE_COMPLIANCE_PATCH_FLAGS="--dry-run" + { printf "%s\n" "$as_me:${as_lineno-$LINENO}: result: checking (not applying) default app store compliance patch" >&5 +printf "%s\n" "checking (not applying) default app store compliance patch" >&6; } + ;; + *) + # No app compliance patching on any other platform + APP_STORE_COMPLIANCE_PATCH= + APP_STORE_COMPLIANCE_PATCH_TARGET= + APP_STORE_COMPLIANCE_PATCH_FLAGS= + { printf "%s\n" "$as_me:${as_lineno-$LINENO}: result: not patching for app store compliance" >&5 +printf "%s\n" "not patching for app store compliance" >&6; } + ;; + esac + +fi + + + + + if test "$cross_compiling" = yes; then case "$host" in diff --git a/configure.ac b/configure.ac index 5fd9b1817347d0..723f36a52a108e 100644 --- a/configure.ac +++ b/configure.ac @@ -695,6 +695,64 @@ AC_SUBST([INSTALLTARGETS]) AC_DEFINE_UNQUOTED([_PYTHONFRAMEWORK], ["${PYTHONFRAMEWORK}"], [framework name]) +dnl quadrigraphs "@<:@" and "@:>@" produce "[" and "]" in the output +AC_MSG_CHECKING([for --with-app-store-compliance]) +AC_ARG_WITH( + [app_store_compliance], + [AS_HELP_STRING( + [--with-app-store-compliance=@<:@PATCH-FILE@:>@], + [Enable any patches required for compiliance with app stores. + Optional PATCH-FILE specifies the custom patch to apply.] + )],[ + case "$withval" in + yes) + case $ac_sys_system in + Darwin|iOS) + # iOS is able to share the macOS patch + APP_STORE_COMPLIANCE_PATCH="Mac/Resources/app-store-compliance.patch" + APP_STORE_COMPLIANCE_PATCH_TARGET="build/app-store-compliant" + APP_STORE_COMPLIANCE_PATCH_FLAGS= + ;; + *) AC_MSG_ERROR([no default app store compliance patch available for $ac_sys_system]) ;; + esac + AC_MSG_RESULT([applying default app store compliance patch]) + ;; + *) + APP_STORE_COMPLIANCE_PATCH="${withval}" + APP_STORE_COMPLIANCE_PATCH_TARGET="build/app-store-compliant" + APP_STORE_COMPLIANCE_PATCH_FLAGS= + AC_MSG_RESULT([applying custom app store compliance patch]) + ;; + esac + ],[ + case $ac_sys_system in + iOS) + # Always apply the compliance patch on iOS; we can use the macOS patch + APP_STORE_COMPLIANCE_PATCH="Mac/Resources/app-store-compliance.patch" + APP_STORE_COMPLIANCE_PATCH_TARGET="build/app-store-compliant" + APP_STORE_COMPLIANCE_PATCH_FLAGS= + AC_MSG_RESULT([applying default app store compliance patch]) + ;; + Darwin) + # Always *check* the compliance patch on macOS + APP_STORE_COMPLIANCE_PATCH="Mac/Resources/app-store-compliance.patch" + APP_STORE_COMPLIANCE_PATCH_TARGET="build/app-store-compliant" + APP_STORE_COMPLIANCE_PATCH_FLAGS="--dry-run" + AC_MSG_RESULT([checking (not applying) default app store compliance patch]) + ;; + *) + # No app compliance patching on any other platform + APP_STORE_COMPLIANCE_PATCH= + APP_STORE_COMPLIANCE_PATCH_TARGET= + APP_STORE_COMPLIANCE_PATCH_FLAGS= + AC_MSG_RESULT([not patching for app store compliance]) + ;; + esac +]) +AC_SUBST([APP_STORE_COMPLIANCE_PATCH]) +AC_SUBST([APP_STORE_COMPLIANCE_PATCH_TARGET]) +AC_SUBST([APP_STORE_COMPLIANCE_PATCH_FLAGS]) + AC_SUBST([_PYTHON_HOST_PLATFORM]) if test "$cross_compiling" = yes; then case "$host" in