Mailman 3 March 2019 - Python-checkins

bpo-36085: Enable better DLL resolution on Windows (GH-12302)
by Steve Dower March 29, 2019

March 29, 2019

https://github.com/python/cpython/commit/2438cdf0e932a341c7613bf4323d06b91a… commit: 2438cdf0e932a341c7613bf4323d06b91ae9f1f1 branch: master author: Steve Dower <steve.dower(a)microsoft.com> committer: GitHub <noreply(a)github.com> date: 2019-03-29T16:37:16-07:00 summary: bpo-36085: Enable better DLL resolution on Windows (GH-12302) files: A Misc/NEWS.d/next/Windows/2019-03-18-11-44-49.bpo-36085.mLfxfc.rst M Doc/library/ctypes.rst M Doc/library/os.rst M Doc/whatsnew/3.8.rst M Lib/… [View More]ctypes/__init__.py M Lib/ctypes/test/test_loading.py M Lib/os.py M Lib/test/test_import/__init__.py M Modules/_ctypes/callproc.c M Modules/clinic/posixmodule.c.h M Modules/posixmodule.c M Python/dynload_win.c diff --git a/Doc/library/ctypes.rst b/Doc/library/ctypes.rst index 500aad8858f2..baab0de8f8ac 100644 --- a/Doc/library/ctypes.rst +++ b/Doc/library/ctypes.rst @@ -1322,14 +1322,14 @@ There are several ways to load shared libraries into the Python process. One way is to instantiate one of the following classes: -.. class:: CDLL(name, mode=DEFAULT_MODE, handle=None, use_errno=False, use_last_error=False) +.. class:: CDLL(name, mode=DEFAULT_MODE, handle=None, use_errno=False, use_last_error=False, winmode=0) Instances of this class represent loaded shared libraries. Functions in these libraries use the standard C calling convention, and are assumed to return :c:type:`int`. -.. class:: OleDLL(name, mode=DEFAULT_MODE, handle=None, use_errno=False, use_last_error=False) +.. class:: OleDLL(name, mode=DEFAULT_MODE, handle=None, use_errno=False, use_last_error=False, winmode=0) Windows only: Instances of this class represent loaded shared libraries, functions in these libraries use the ``stdcall`` calling convention, and are @@ -1342,7 +1342,7 @@ way is to instantiate one of the following classes: :exc:`WindowsError` used to be raised. -.. class:: WinDLL(name, mode=DEFAULT_MODE, handle=None, use_errno=False, use_last_error=False) +.. class:: WinDLL(name, mode=DEFAULT_MODE, handle=None, use_errno=False, use_last_error=False, winmode=0) Windows only: Instances of this class represent loaded shared libraries, functions in these libraries use the ``stdcall`` calling convention, and are @@ -1394,6 +1394,17 @@ the Windows error code which is managed by the :func:`GetLastError` and :func:`ctypes.set_last_error` are used to request and change the ctypes private copy of the windows error code. +The *winmode* parameter is used on Windows to specify how the library is loaded +(since *mode* is ignored). It takes any value that is valid for the Win32 API +``LoadLibraryEx`` flags parameter. When omitted, the default is to use the flags +that result in the most secure DLL load to avoiding issues such as DLL +hijacking. Passing the full path to the DLL is the safest way to ensure the +correct library and dependencies are loaded. + +.. versionchanged:: 3.8 + Added *winmode* parameter. + + .. data:: RTLD_GLOBAL :noindex: diff --git a/Doc/library/os.rst b/Doc/library/os.rst index f8803af95200..85e240a0006a 100644 --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -3079,6 +3079,36 @@ to be ignored. :func:`signal.signal`. +.. function:: add_dll_directory(path) + + Add a path to the DLL search path. + + This search path is used when resolving dependencies for imported + extension modules (the module itself is resolved through sys.path), + and also by :mod:`ctypes`. + + Remove the directory by calling **close()** on the returned object + or using it in a :keyword:`with` statement. + + See the `Microsoft documentation + <https://msdn.microsoft.com/44228cf2-6306-466c-8f16-f513cd3ba8b5>`_ + for more information about how DLLs are loaded. + + .. availability:: Windows. + + .. versionadded:: 3.8 + Previous versions of CPython would resolve DLLs using the default + behavior for the current process. This led to inconsistencies, + such as only sometimes searching :envvar:`PATH` or the current + working directory, and OS functions such as ``AddDllDirectory`` + having no effect. + + In 3.8, the two primary ways DLLs are loaded now explicitly + override the process-wide behavior to ensure consistency. See the + :ref:`porting notes <bpo-36085-whatsnew>` for information on + updating libraries. + + .. function:: execl(path, arg0, arg1, ...) execle(path, arg0, arg1, ..., env) execlp(file, arg0, arg1, ...) diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index 0ffbcab353ec..f0423c376fcd 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -168,6 +168,16 @@ asyncio On Windows, the default event loop is now :class:`~asyncio.ProactorEventLoop`. +ctypes +------ + +On Windows, :class:`~ctypes.CDLL` and subclasses now accept a *winmode* parameter +to specify flags for the underlying ``LoadLibraryEx`` call. The default flags are +set to only load DLL dependencies from trusted locations, including the path +where the DLL is stored (if a full or partial path is used to load the initial +DLL) and paths added by :func:`~os.add_dll_directory`. + + gettext ------- @@ -238,6 +248,13 @@ Added new function, :func:`math.prod`, as analogous function to :func:`sum` that returns the product of a 'start' value (default: 1) times an iterable of numbers. (Contributed by Pablo Galindo in :issue:`35606`) +os +-- + +Added new function :func:`~os.add_dll_directory` on Windows for providing +additional search paths for native dependencies when importing extension +modules or loading DLLs using :mod:`ctypes`. + os.path ------- @@ -727,6 +744,19 @@ Changes in the Python API environment variable and does not use :envvar:`HOME`, which is not normally set for regular user accounts. +.. _bpo-36085-whatsnew: + +* DLL dependencies for extension modules and DLLs loaded with :mod:`ctypes` on + Windows are now resolved more securely. Only the system paths, the directory + containing the DLL or PYD file, and directories added with + :func:`~os.add_dll_directory` are searched for load-time dependencies. + Specifically, :envvar:`PATH` and the current working directory are no longer + used, and modifications to these will no longer have any effect on normal DLL + resolution. If your application relies on these mechanisms, you should check + for :func:`~os.add_dll_directory` and if it exists, use it to add your DLLs + directory while loading your library. + (See :issue:`36085`.) + Changes in the C API -------------------- diff --git a/Lib/ctypes/__init__.py b/Lib/ctypes/__init__.py index 5f78beda5866..4107db3e3972 100644 --- a/Lib/ctypes/__init__.py +++ b/Lib/ctypes/__init__.py @@ -326,7 +326,8 @@ class CDLL(object): def __init__(self, name, mode=DEFAULT_MODE, handle=None, use_errno=False, - use_last_error=False): + use_last_error=False, + winmode=None): self._name = name flags = self._func_flags_ if use_errno: @@ -341,6 +342,15 @@ def __init__(self, name, mode=DEFAULT_MODE, handle=None, """ if name and name.endswith(")") and ".a(" in name: mode |= ( _os.RTLD_MEMBER | _os.RTLD_NOW ) + if _os.name == "nt": + if winmode is not None: + mode = winmode + else: + import nt + mode = nt._LOAD_LIBRARY_SEARCH_DEFAULT_DIRS + if '/' in name or '\\' in name: + self._name = nt._getfullpathname(self._name) + mode |= nt._LOAD_LIBRARY_SEARCH_DLL_LOAD_DIR class _FuncPtr(_CFuncPtr): _flags_ = flags diff --git a/Lib/ctypes/test/test_loading.py b/Lib/ctypes/test/test_loading.py index f3b65b9d6e7e..be367c6fa352 100644 --- a/Lib/ctypes/test/test_loading.py +++ b/Lib/ctypes/test/test_loading.py @@ -1,6 +1,9 @@ from ctypes import * import os +import shutil +import subprocess import sys +import sysconfig import unittest import test.support from ctypes.util import find_library @@ -112,5 +115,65 @@ def test_1703286_B(self): # This is the real test: call the function via 'call_function' self.assertEqual(0, call_function(proc, (None,))) + @unittest.skipUnless(os.name == "nt", + 'test specific to Windows') + def test_load_dll_with_flags(self): + _sqlite3 = test.support.import_module("_sqlite3") + src = _sqlite3.__file__ + if src.lower().endswith("_d.pyd"): + ext = "_d.dll" + else: + ext = ".dll" + + with test.support.temp_dir() as tmp: + # We copy two files and load _sqlite3.dll (formerly .pyd), + # which has a dependency on sqlite3.dll. Then we test + # loading it in subprocesses to avoid it starting in memory + # for each test. + target = os.path.join(tmp, "_sqlite3.dll") + shutil.copy(src, target) + shutil.copy(os.path.join(os.path.dirname(src), "sqlite3" + ext), + os.path.join(tmp, "sqlite3" + ext)) + + def should_pass(command): + with self.subTest(command): + subprocess.check_output( + [sys.executable, "-c", + "from ctypes import *; import nt;" + command], + cwd=tmp + ) + + def should_fail(command): + with self.subTest(command): + with self.assertRaises(subprocess.CalledProcessError): + subprocess.check_output( + [sys.executable, "-c", + "from ctypes import *; import nt;" + command], + cwd=tmp, stderr=subprocess.STDOUT, + ) + + # Default load should not find this in CWD + should_fail("WinDLL('_sqlite3.dll')") + + # Relative path (but not just filename) should succeed + should_pass("WinDLL('./_sqlite3.dll')") + + # Insecure load flags should succeed + should_pass("WinDLL('_sqlite3.dll', winmode=0)") + + # Full path load without DLL_LOAD_DIR shouldn't find dependency + should_fail("WinDLL(nt._getfullpathname('_sqlite3.dll'), " + + "winmode=nt._LOAD_LIBRARY_SEARCH_SYSTEM32)") + + # Full path load with DLL_LOAD_DIR should succeed + should_pass("WinDLL(nt._getfullpathname('_sqlite3.dll'), " + + "winmode=nt._LOAD_LIBRARY_SEARCH_DLL_LOAD_DIR)") + + # User-specified directory should succeed + should_pass("import os; p = os.add_dll_directory(os.getcwd());" + + "WinDLL('_sqlite3.dll'); p.close()") + + + if __name__ == "__main__": unittest.main() diff --git a/Lib/os.py b/Lib/os.py index 7741c7580d0e..79ff7a22d92e 100644 --- a/Lib/os.py +++ b/Lib/os.py @@ -1070,3 +1070,40 @@ def __fspath__(self): @classmethod def __subclasshook__(cls, subclass): return hasattr(subclass, '__fspath__') + + +if name == 'nt': + class _AddedDllDirectory: + def __init__(self, path, cookie, remove_dll_directory): + self.path = path + self._cookie = cookie + self._remove_dll_directory = remove_dll_directory + def close(self): + self._remove_dll_directory(self._cookie) + self.path = None + def __enter__(self): + return self + def __exit__(self, *args): + self.close() + def __repr__(self): + if self.path: + return "<AddedDllDirectory({!r})>".format(self.path) + return "<AddedDllDirectory()>" + + def add_dll_directory(path): + """Add a path to the DLL search path. + + This search path is used when resolving dependencies for imported + extension modules (the module itself is resolved through sys.path), + and also by ctypes. + + Remove the directory by calling close() on the returned object or + using it in a with statement. + """ + import nt + cookie = nt._add_dll_directory(path) + return _AddedDllDirectory( + path, + cookie, + nt._remove_dll_directory + ) diff --git a/Lib/test/test_import/__init__.py b/Lib/test/test_import/__init__.py index 7306e0f7f722..a0bfe1a6c19b 100644 --- a/Lib/test/test_import/__init__.py +++ b/Lib/test/test_import/__init__.py @@ -8,6 +8,8 @@ import platform import py_compile import random +import shutil +import subprocess import stat import sys import threading @@ -17,6 +19,7 @@ import textwrap import errno import contextlib +import glob import test.support from test.support import ( @@ -460,6 +463,51 @@ def run(): finally: del sys.path[0] + @unittest.skipUnless(sys.platform == "win32", "Windows-specific") + def test_dll_dependency_import(self): + from _winapi import GetModuleFileName + dllname = GetModuleFileName(sys.dllhandle) + pydname = importlib.util.find_spec("_sqlite3").origin + depname = os.path.join( + os.path.dirname(pydname), + "sqlite3{}.dll".format("_d" if "_d" in pydname else "")) + + with test.support.temp_dir() as tmp: + tmp2 = os.path.join(tmp, "DLLs") + os.mkdir(tmp2) + + pyexe = os.path.join(tmp, os.path.basename(sys.executable)) + shutil.copy(sys.executable, pyexe) + shutil.copy(dllname, tmp) + for f in glob.glob(os.path.join(sys.prefix, "vcruntime*.dll")): + shutil.copy(f, tmp) + + shutil.copy(pydname, tmp2) + + env = None + env = {k.upper(): os.environ[k] for k in os.environ} + env["PYTHONPATH"] = tmp2 + ";" + os.path.dirname(os.__file__) + + # Test 1: import with added DLL directory + subprocess.check_call([ + pyexe, "-Sc", ";".join([ + "import os", + "p = os.add_dll_directory({!r})".format( + os.path.dirname(depname)), + "import _sqlite3", + "p.close" + ])], + stderr=subprocess.STDOUT, + env=env, + cwd=os.path.dirname(pyexe)) + + # Test 2: import with DLL adjacent to PYD + shutil.copy(depname, tmp2) + subprocess.check_call([pyexe, "-Sc", "import _sqlite3"], + stderr=subprocess.STDOUT, + env=env, + cwd=os.path.dirname(pyexe)) + @skip_if_dont_write_bytecode class FilePermissionTests(unittest.TestCase): diff --git a/Misc/NEWS.d/next/Windows/2019-03-18-11-44-49.bpo-36085.mLfxfc.rst b/Misc/NEWS.d/next/Windows/2019-03-18-11-44-49.bpo-36085.mLfxfc.rst new file mode 100644 index 000000000000..41f23e655652 --- /dev/null +++ b/Misc/NEWS.d/next/Windows/2019-03-18-11-44-49.bpo-36085.mLfxfc.rst @@ -0,0 +1,2 @@ +Enable better DLL resolution on Windows by using safe DLL search paths and +adding :func:`os.add_dll_directory`. diff --git a/Modules/_ctypes/callproc.c b/Modules/_ctypes/callproc.c index 7c25e2e796bf..5a943d3c3708 100644 --- a/Modules/_ctypes/callproc.c +++ b/Modules/_ctypes/callproc.c @@ -1251,19 +1251,21 @@ static PyObject *format_error(PyObject *self, PyObject *args) } static const char load_library_doc[] = -"LoadLibrary(name) -> handle\n\ +"LoadLibrary(name, load_flags) -> handle\n\ \n\ Load an executable (usually a DLL), and return a handle to it.\n\ The handle may be used to locate exported functions in this\n\ -module.\n"; +module. load_flags are as defined for LoadLibraryEx in the\n\ +Windows API.\n"; static PyObject *load_library(PyObject *self, PyObject *args) { const WCHAR *name; PyObject *nameobj; - PyObject *ignored; + int load_flags = 0; HMODULE hMod; + DWORD err; - if (!PyArg_ParseTuple(args, "U|O:LoadLibrary", &nameobj, &ignored)) + if (!PyArg_ParseTuple(args, "U|i:LoadLibrary", &nameobj, &load_flags)) return NULL; name = _PyUnicode_AsUnicode(nameobj); @@ -1271,11 +1273,22 @@ static PyObject *load_library(PyObject *self, PyObject *args) return NULL; Py_BEGIN_ALLOW_THREADS - hMod = LoadLibraryW(name); + /* bpo-36085: Limit DLL search directories to avoid pre-loading + * attacks and enable use of the AddDllDirectory function. + */ + hMod = LoadLibraryExW(name, NULL, (DWORD)load_flags); + err = hMod ? 0 : GetLastError(); Py_END_ALLOW_THREADS - if (!hMod) - return PyErr_SetFromWindowsErr(GetLastError()); + if (err == ERROR_MOD_NOT_FOUND) { + PyErr_Format(PyExc_FileNotFoundError, + ("Could not find module '%.500S'. Try using " + "the full path with constructor syntax."), + nameobj); + return NULL; + } else if (err) { + return PyErr_SetFromWindowsErr(err); + } #ifdef _WIN64 return PyLong_FromVoidPtr(hMod); #else @@ -1291,15 +1304,18 @@ static PyObject *free_library(PyObject *self, PyObject *args) { void *hMod; BOOL result; + DWORD err; if (!PyArg_ParseTuple(args, "O&:FreeLibrary", &_parse_voidp, &hMod)) return NULL; Py_BEGIN_ALLOW_THREADS result = FreeLibrary((HMODULE)hMod); + err = result ? 0 : GetLastError(); Py_END_ALLOW_THREADS - if (!result) - return PyErr_SetFromWindowsErr(GetLastError()); + if (!result) { + return PyErr_SetFromWindowsErr(err); + } Py_RETURN_NONE; } diff --git a/Modules/clinic/posixmodule.c.h b/Modules/clinic/posixmodule.c.h index 55f2cbb91a08..43f8ba6b4e61 100644 --- a/Modules/clinic/posixmodule.c.h +++ b/Modules/clinic/posixmodule.c.h @@ -7961,6 +7961,94 @@ os_getrandom(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObject #endif /* defined(HAVE_GETRANDOM_SYSCALL) */ +#if defined(MS_WINDOWS) + +PyDoc_STRVAR(os__add_dll_directory__doc__, +"_add_dll_directory($module, /, path)\n" +"--\n" +"\n" +"Add a path to the DLL search path.\n" +"\n" +"This search path is used when resolving dependencies for imported\n" +"extension modules (the module itself is resolved through sys.path),\n" +"and also by ctypes.\n" +"\n" +"Returns an opaque value that may be passed to os.remove_dll_directory\n" +"to remove this directory from the search path."); + +#define OS__ADD_DLL_DIRECTORY_METHODDEF \ + {"_add_dll_directory", (PyCFunction)(void(*)(void))os__add_dll_directory, METH_FASTCALL|METH_KEYWORDS, os__add_dll_directory__doc__}, + +static PyObject * +os__add_dll_directory_impl(PyObject *module, path_t *path); + +static PyObject * +os__add_dll_directory(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) +{ + PyObject *return_value = NULL; + static const char * const _keywords[] = {"path", NULL}; + static _PyArg_Parser _parser = {NULL, _keywords, "_add_dll_directory", 0}; + PyObject *argsbuf[1]; + path_t path = PATH_T_INITIALIZE("_add_dll_directory", "path", 0, 0); + + args = _PyArg_UnpackKeywords(args, nargs, NULL, kwnames, &_parser, 1, 1, 0, argsbuf); + if (!args) { + goto exit; + } + if (!path_converter(args[0], &path)) { + goto exit; + } + return_value = os__add_dll_directory_impl(module, &path); + +exit: + /* Cleanup for path */ + path_cleanup(&path); + + return return_value; +} + +#endif /* defined(MS_WINDOWS) */ + +#if defined(MS_WINDOWS) + +PyDoc_STRVAR(os__remove_dll_directory__doc__, +"_remove_dll_directory($module, /, cookie)\n" +"--\n" +"\n" +"Removes a path from the DLL search path.\n" +"\n" +"The parameter is an opaque value that was returned from\n" +"os.add_dll_directory. You can only remove directories that you added\n" +"yourself."); + +#define OS__REMOVE_DLL_DIRECTORY_METHODDEF \ + {"_remove_dll_directory", (PyCFunction)(void(*)(void))os__remove_dll_directory, METH_FASTCALL|METH_KEYWORDS, os__remove_dll_directory__doc__}, + +static PyObject * +os__remove_dll_directory_impl(PyObject *module, PyObject *cookie); + +static PyObject * +os__remove_dll_directory(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) +{ + PyObject *return_value = NULL; + static const char * const _keywords[] = {"cookie", NULL}; + static _PyArg_Parser _parser = {NULL, _keywords, "_remove_dll_directory", 0}; + PyObject *argsbuf[1]; + PyObject *cookie; + + args = _PyArg_UnpackKeywords(args, nargs, NULL, kwnames, &_parser, 1, 1, 0, argsbuf); + if (!args) { + goto exit; + } + cookie = args[0]; + return_value = os__remove_dll_directory_impl(module, cookie); + +exit: + return return_value; +} + +#endif /* defined(MS_WINDOWS) */ + #ifndef OS_TTYNAME_METHODDEF #define OS_TTYNAME_METHODDEF #endif /* !defined(OS_TTYNAME_METHODDEF) */ @@ -8480,4 +8568,12 @@ os_getrandom(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObject #ifndef OS_GETRANDOM_METHODDEF #define OS_GETRANDOM_METHODDEF #endif /* !defined(OS_GETRANDOM_METHODDEF) */ -/*[clinic end generated code: output=1a9c62f5841221ae input=a9049054013a1b77]*/ + +#ifndef OS__ADD_DLL_DIRECTORY_METHODDEF + #define OS__ADD_DLL_DIRECTORY_METHODDEF +#endif /* !defined(OS__ADD_DLL_DIRECTORY_METHODDEF) */ + +#ifndef OS__REMOVE_DLL_DIRECTORY_METHODDEF + #define OS__REMOVE_DLL_DIRECTORY_METHODDEF +#endif /* !defined(OS__REMOVE_DLL_DIRECTORY_METHODDEF) */ +/*[clinic end generated code: output=ab36ec0376a422ae input=a9049054013a1b77]*/ diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c index 3f760183575a..7c4e5f082b5d 100644 --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -1442,17 +1442,23 @@ win32_error(const char* function, const char* filename) } static PyObject * -win32_error_object(const char* function, PyObject* filename) +win32_error_object_err(const char* function, PyObject* filename, DWORD err) { /* XXX - see win32_error for comments on 'function' */ - errno = GetLastError(); if (filename) return PyErr_SetExcFromWindowsErrWithFilenameObject( PyExc_OSError, - errno, + err, filename); else - return PyErr_SetFromWindowsErr(errno); + return PyErr_SetFromWindowsErr(err); +} + +static PyObject * +win32_error_object(const char* function, PyObject* filename) +{ + errno = GetLastError(); + return win32_error_object_err(function, filename, errno); } #endif /* MS_WINDOWS */ @@ -13161,6 +13167,113 @@ os_getrandom_impl(PyObject *module, Py_ssize_t size, int flags) } #endif /* HAVE_GETRANDOM_SYSCALL */ +#ifdef MS_WINDOWS +/* bpo-36085: Helper functions for managing DLL search directories + * on win32 + */ + +typedef DLL_DIRECTORY_COOKIE (WINAPI *PAddDllDirectory)(PCWSTR newDirectory); +typedef BOOL (WINAPI *PRemoveDllDirectory)(DLL_DIRECTORY_COOKIE cookie); + +/*[clinic input] +os._add_dll_directory + + path: path_t + +Add a path to the DLL search path. + +This search path is used when resolving dependencies for imported +extension modules (the module itself is resolved through sys.path), +and also by ctypes. + +Returns an opaque value that may be passed to os.remove_dll_directory +to remove this directory from the search path. +[clinic start generated code]*/ + +static PyObject * +os__add_dll_directory_impl(PyObject *module, path_t *path) +/*[clinic end generated code: output=80b025daebb5d683 input=1de3e6c13a5808c8]*/ +{ + HMODULE hKernel32; + PAddDllDirectory AddDllDirectory; + DLL_DIRECTORY_COOKIE cookie = 0; + DWORD err = 0; + + /* For Windows 7, we have to load this. As this will be a fairly + infrequent operation, just do it each time. Kernel32 is always + loaded. */ + Py_BEGIN_ALLOW_THREADS + if (!(hKernel32 = GetModuleHandleW(L"kernel32")) || + !(AddDllDirectory = (PAddDllDirectory)GetProcAddress( + hKernel32, "AddDllDirectory")) || + !(cookie = (*AddDllDirectory)(path->wide))) { + err = GetLastError(); + } + Py_END_ALLOW_THREADS + + if (err) { + return win32_error_object_err("add_dll_directory", + path->object, err); + } + + return PyCapsule_New(cookie, "DLL directory cookie", NULL); +} + +/*[clinic input] +os._remove_dll_directory + + cookie: object + +Removes a path from the DLL search path. + +The parameter is an opaque value that was returned from +os.add_dll_directory. You can only remove directories that you added +yourself. +[clinic start generated code]*/ + +static PyObject * +os__remove_dll_directory_impl(PyObject *module, PyObject *cookie) +/*[clinic end generated code: output=594350433ae535bc input=c1d16a7e7d9dc5dc]*/ +{ + HMODULE hKernel32; + PRemoveDllDirectory RemoveDllDirectory; + DLL_DIRECTORY_COOKIE cookieValue; + DWORD err = 0; + + if (!PyCapsule_IsValid(cookie, "DLL directory cookie")) { + PyErr_SetString(PyExc_TypeError, + "Provided cookie was not returned from os.add_dll_directory"); + return NULL; + } + + cookieValue = (DLL_DIRECTORY_COOKIE)PyCapsule_GetPointer( + cookie, "DLL directory cookie"); + + /* For Windows 7, we have to load this. As this will be a fairly + infrequent operation, just do it each time. Kernel32 is always + loaded. */ + Py_BEGIN_ALLOW_THREADS + if (!(hKernel32 = GetModuleHandleW(L"kernel32")) || + !(RemoveDllDirectory = (PRemoveDllDirectory)GetProcAddress( + hKernel32, "RemoveDllDirectory")) || + !(*RemoveDllDirectory)(cookieValue)) { + err = GetLastError(); + } + Py_END_ALLOW_THREADS + + if (err) { + return win32_error_object_err("remove_dll_directory", + NULL, err); + } + + if (PyCapsule_SetName(cookie, NULL)) { + return NULL; + } + + Py_RETURN_NONE; +} + +#endif static PyMethodDef posix_methods[] = { @@ -13349,6 +13462,10 @@ static PyMethodDef posix_methods[] = { OS_SCANDIR_METHODDEF OS_FSPATH_METHODDEF OS_GETRANDOM_METHODDEF +#ifdef MS_WINDOWS + OS__ADD_DLL_DIRECTORY_METHODDEF + OS__REMOVE_DLL_DIRECTORY_METHODDEF +#endif {NULL, NULL} /* Sentinel */ }; @@ -13826,6 +13943,14 @@ all_ins(PyObject *m) if (PyModule_AddIntConstant(m, "_COPYFILE_DATA", COPYFILE_DATA)) return -1; #endif +#ifdef MS_WINDOWS + if (PyModule_AddIntConstant(m, "_LOAD_LIBRARY_SEARCH_DEFAULT_DIRS", LOAD_LIBRARY_SEARCH_DEFAULT_DIRS)) return -1; + if (PyModule_AddIntConstant(m, "_LOAD_LIBRARY_SEARCH_APPLICATION_DIR", LOAD_LIBRARY_SEARCH_APPLICATION_DIR)) return -1; + if (PyModule_AddIntConstant(m, "_LOAD_LIBRARY_SEARCH_SYSTEM32", LOAD_LIBRARY_SEARCH_SYSTEM32)) return -1; + if (PyModule_AddIntConstant(m, "_LOAD_LIBRARY_SEARCH_USER_DIRS", LOAD_LIBRARY_SEARCH_USER_DIRS)) return -1; + if (PyModule_AddIntConstant(m, "_LOAD_LIBRARY_SEARCH_DLL_LOAD_DIR", LOAD_LIBRARY_SEARCH_DLL_LOAD_DIR)) return -1; +#endif + return 0; } diff --git a/Python/dynload_win.c b/Python/dynload_win.c index 36918c3579d9..457d47f5eed5 100644 --- a/Python/dynload_win.c +++ b/Python/dynload_win.c @@ -215,12 +215,14 @@ dl_funcptr _PyImport_FindSharedFuncptrWindows(const char *prefix, #if HAVE_SXS cookie = _Py_ActivateActCtx(); #endif - /* We use LoadLibraryEx so Windows looks for dependent DLLs - in directory of pathname first. */ - /* XXX This call doesn't exist in Windows CE */ + /* bpo-36085: We use LoadLibraryEx with restricted search paths + to avoid DLL preloading attacks and enable use of the + AddDllDirectory function. We add SEARCH_DLL_LOAD_DIR to + ensure DLLs adjacent to the PYD are preferred. */ Py_BEGIN_ALLOW_THREADS hDLL = LoadLibraryExW(wpathname, NULL, - LOAD_WITH_ALTERED_SEARCH_PATH); + LOAD_LIBRARY_SEARCH_DEFAULT_DIRS | + LOAD_LIBRARY_SEARCH_DLL_LOAD_DIR); Py_END_ALLOW_THREADS #if HAVE_SXS _Py_DeactivateActCtx(cookie); [View Less]

1 0

bpo-35947: Update Windows to the current version of libffi (GH-11797)
by Steve Dower March 29, 2019

March 29, 2019

https://github.com/python/cpython/commit/32119e10b792ad7ee4e5f951a2d89ddbaf… commit: 32119e10b792ad7ee4e5f951a2d89ddbaf111cc5 branch: master author: Paul Monson <paulmon(a)users.noreply.github.com> committer: Steve Dower <steve.dower(a)microsoft.com> date: 2019-03-29T16:30:10-07:00 summary: bpo-35947: Update Windows to the current version of libffi (GH-11797) We now use a pre-built libffi binary from our binaries repository, and no longer vendor the full implementation. files: A … [View More]Misc/NEWS.d/next/Windows/2019-02-11-14-53-17.bpo-35947.9vI4hP.rst A PCbuild/libffi.props A PCbuild/prepare_libffi.bat D Modules/_ctypes/libffi_msvc/LICENSE D Modules/_ctypes/libffi_msvc/README D Modules/_ctypes/libffi_msvc/README.ctypes D Modules/_ctypes/libffi_msvc/ffi.c D Modules/_ctypes/libffi_msvc/ffi.h D Modules/_ctypes/libffi_msvc/ffi_common.h D Modules/_ctypes/libffi_msvc/fficonfig.h D Modules/_ctypes/libffi_msvc/ffitarget.h D Modules/_ctypes/libffi_msvc/prep_cif.c D Modules/_ctypes/libffi_msvc/types.c D Modules/_ctypes/libffi_msvc/win32.c D Modules/_ctypes/libffi_msvc/win64.asm M Lib/ctypes/test/test_win32.py M Modules/_ctypes/_ctypes.c M Modules/_ctypes/callproc.c M PC/layout/main.py M PCbuild/_ctypes.vcxproj M PCbuild/_ctypes.vcxproj.filters M PCbuild/get_externals.bat M PCbuild/python.props M Tools/msi/lib/lib_files.wxs diff --git a/Lib/ctypes/test/test_win32.py b/Lib/ctypes/test/test_win32.py index a2941f3fe078..e51bdc8ad6b0 100644 --- a/Lib/ctypes/test/test_win32.py +++ b/Lib/ctypes/test/test_win32.py @@ -6,35 +6,6 @@ import _ctypes_test -# Only windows 32-bit has different calling conventions. -(a)unittest.skipUnless(sys.platform == "win32", 'Windows-specific test') -(a)unittest.skipUnless(sizeof(c_void_p) == sizeof(c_int), - "sizeof c_void_p and c_int differ") -class WindowsTestCase(unittest.TestCase): - def test_callconv_1(self): - # Testing stdcall function - - IsWindow = windll.user32.IsWindow - # ValueError: Procedure probably called with not enough arguments - # (4 bytes missing) - self.assertRaises(ValueError, IsWindow) - - # This one should succeed... - self.assertEqual(0, IsWindow(0)) - - # ValueError: Procedure probably called with too many arguments - # (8 bytes in excess) - self.assertRaises(ValueError, IsWindow, 0, 0, 0) - - def test_callconv_2(self): - # Calling stdcall function as cdecl - - IsWindow = cdll.user32.IsWindow - - # ValueError: Procedure called with not enough arguments - # (4 bytes missing) or wrong calling convention - self.assertRaises(ValueError, IsWindow, None) - @unittest.skipUnless(sys.platform == "win32", 'Windows-specific test') class FunctionCallTestCase(unittest.TestCase): @unittest.skipUnless('MSC' in sys.version, "SEH only supported by MSC") diff --git a/Misc/NEWS.d/next/Windows/2019-02-11-14-53-17.bpo-35947.9vI4hP.rst b/Misc/NEWS.d/next/Windows/2019-02-11-14-53-17.bpo-35947.9vI4hP.rst new file mode 100644 index 000000000000..ae3c5a751b46 --- /dev/null +++ b/Misc/NEWS.d/next/Windows/2019-02-11-14-53-17.bpo-35947.9vI4hP.rst @@ -0,0 +1,2 @@ +Added current version of libffi to cpython-source-deps. +Change _ctypes to use current version of libffi on Windows. diff --git a/Modules/_ctypes/_ctypes.c b/Modules/_ctypes/_ctypes.c index 45102f3304ec..7479edaf1110 100644 --- a/Modules/_ctypes/_ctypes.c +++ b/Modules/_ctypes/_ctypes.c @@ -403,24 +403,35 @@ static PyCArgObject * StructUnionType_paramfunc(CDataObject *self) { PyCArgObject *parg; + CDataObject *copied_self; StgDictObject *stgdict; + if (self->b_size > sizeof(void*)) { + void *new_ptr = PyMem_Malloc(self->b_size); + if (new_ptr == NULL) + return NULL; + memcpy(new_ptr, self->b_ptr, self->b_size); + copied_self = (CDataObject *)PyCData_AtAddress( + (PyObject *)Py_TYPE(self), new_ptr); + copied_self->b_needsfree = 1; + } else { + copied_self = self; + Py_INCREF(copied_self); + } + parg = PyCArgObject_new(); - if (parg == NULL) + if (parg == NULL) { + Py_DECREF(copied_self); return NULL; + } parg->tag = 'V'; - stgdict = PyObject_stgdict((PyObject *)self); + stgdict = PyObject_stgdict((PyObject *)copied_self); assert(stgdict); /* Cannot be NULL for structure/union instances */ parg->pffi_type = &stgdict->ffi_type_pointer; - /* For structure parameters (by value), parg->value doesn't contain the structure - data itself, instead parg->value.p *points* to the structure's data - See also _ctypes.c, function _call_function_pointer(). - */ - parg->value.p = self->b_ptr; - parg->size = self->b_size; - Py_INCREF(self); - parg->obj = (PyObject *)self; + parg->value.p = copied_self->b_ptr; + parg->size = copied_self->b_size; + parg->obj = (PyObject *)copied_self; return parg; } diff --git a/Modules/_ctypes/callproc.c b/Modules/_ctypes/callproc.c index d91e84613b2f..7c25e2e796bf 100644 --- a/Modules/_ctypes/callproc.c +++ b/Modules/_ctypes/callproc.c @@ -729,6 +729,23 @@ static int ConvParam(PyObject *obj, Py_ssize_t index, struct argument *pa) } } +#if defined(MS_WIN32) && !defined(_WIN32_WCE) +/* +Per: https://msdn.microsoft.com/en-us/library/7572ztz4.aspx +To be returned by value in RAX, user-defined types must have a length +of 1, 2, 4, 8, 16, 32, or 64 bits +*/ +int can_return_struct_as_int(size_t s) +{ + return s == 1 || s == 2 || s == 4; +} + +int can_return_struct_as_sint64(size_t s) +{ + return s == 8; +} +#endif + ffi_type *_ctypes_get_ffi_type(PyObject *obj) { @@ -778,12 +795,9 @@ static int _call_function_pointer(int flags, int *space; ffi_cif cif; int cc; -#ifdef MS_WIN32 - int delta; -#ifndef DONT_USE_SEH +#if defined(MS_WIN32) && !defined(DONT_USE_SEH) DWORD dwExceptionCode = 0; EXCEPTION_RECORD record; -#endif #endif /* XXX check before here */ if (restype == NULL) { @@ -828,7 +842,6 @@ static int _call_function_pointer(int flags, #ifndef DONT_USE_SEH __try { #endif - delta = #endif ffi_call(&cif, (void *)pProc, resmem, avalues); #ifdef MS_WIN32 @@ -860,35 +873,6 @@ static int _call_function_pointer(int flags, return -1; } #endif -#ifdef MS_WIN64 - if (delta != 0) { - PyErr_Format(PyExc_RuntimeError, - "ffi_call failed with code %d", - delta); - return -1; - } -#else - if (delta < 0) { - if (flags & FUNCFLAG_CDECL) - PyErr_Format(PyExc_ValueError, - "Procedure called with not enough " - "arguments (%d bytes missing) " - "or wrong calling convention", - -delta); - else - PyErr_Format(PyExc_ValueError, - "Procedure probably called with not enough " - "arguments (%d bytes missing)", - -delta); - return -1; - } else if (delta > 0) { - PyErr_Format(PyExc_ValueError, - "Procedure probably called with too many " - "arguments (%d bytes in excess)", - delta); - return -1; - } -#endif #endif if ((flags & FUNCFLAG_PYTHONAPI) && PyErr_Occurred()) return -1; diff --git a/Modules/_ctypes/libffi_msvc/LICENSE b/Modules/_ctypes/libffi_msvc/LICENSE deleted file mode 100644 index f591795152d6..000000000000 --- a/Modules/_ctypes/libffi_msvc/LICENSE +++ /dev/null @@ -1,20 +0,0 @@ -libffi - Copyright (c) 1996-2003 Red Hat, Inc. - -Permission is hereby granted, free of charge, to any person obtaining -a copy of this software and associated documentation files (the -``Software''), to deal in the Software without restriction, including -without limitation the rights to use, copy, modify, merge, publish, -distribute, sublicense, and/or sell copies of the Software, and to -permit persons to whom the Software is furnished to do so, subject to -the following conditions: - -The above copyright notice and this permission notice shall be included -in all copies or substantial portions of the Software. - -THE SOFTWARE IS PROVIDED ``AS IS'', WITHOUT WARRANTY OF ANY KIND, EXPRESS -OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF -MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. -IN NO EVENT SHALL CYGNUS SOLUTIONS BE LIABLE FOR ANY CLAIM, DAMAGES OR -OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, -ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR -OTHER DEALINGS IN THE SOFTWARE. diff --git a/Modules/_ctypes/libffi_msvc/README b/Modules/_ctypes/libffi_msvc/README deleted file mode 100644 index 69e46cbf8a42..000000000000 --- a/Modules/_ctypes/libffi_msvc/README +++ /dev/null @@ -1,500 +0,0 @@ -This directory contains the libffi package, which is not part of GCC but -shipped with GCC as convenience. - -Status -====== - -libffi-2.00 has not been released yet! This is a development snapshot! - -libffi-1.20 was released on October 5, 1998. Check the libffi web -page for updates: <URL:http://sources.redhat.com/libffi/>. - - -What is libffi? -=============== - -Compilers for high level languages generate code that follow certain -conventions. These conventions are necessary, in part, for separate -compilation to work. One such convention is the "calling -convention". The "calling convention" is essentially a set of -assumptions made by the compiler about where function arguments will -be found on entry to a function. A "calling convention" also specifies -where the return value for a function is found. - -Some programs may not know at the time of compilation what arguments -are to be passed to a function. For instance, an interpreter may be -told at run-time about the number and types of arguments used to call -a given function. Libffi can be used in such programs to provide a -bridge from the interpreter program to compiled code. - -The libffi library provides a portable, high level programming -interface to various calling conventions. This allows a programmer to -call any function specified by a call interface description at run -time. - -Ffi stands for Foreign Function Interface. A foreign function -interface is the popular name for the interface that allows code -written in one language to call code written in another language. The -libffi library really only provides the lowest, machine dependent -layer of a fully featured foreign function interface. A layer must -exist above libffi that handles type conversions for values passed -between the two languages. - - -Supported Platforms and Prerequisites -===================================== - -Libffi has been ported to: - - SunOS 4.1.3 & Solaris 2.x (SPARC-V8, SPARC-V9) - - Irix 5.3 & 6.2 (System V/o32 & n32) - - Intel x86 - Linux (System V ABI) - - Alpha - Linux and OSF/1 - - m68k - Linux (System V ABI) - - PowerPC - Linux (System V ABI, Darwin, AIX) - - ARM - Linux (System V ABI) - -Libffi has been tested with the egcs 1.0.2 gcc compiler. Chances are -that other versions will work. Libffi has also been built and tested -with the SGI compiler tools. - -On PowerPC, the tests failed (see the note below). - -You must use GNU make to build libffi. SGI's make will not work. -Sun's probably won't either. - -If you port libffi to another platform, please let me know! I assume -that some will be easy (x86 NetBSD), and others will be more difficult -(HP). - - -Installing libffi -================= - -[Note: before actually performing any of these installation steps, - you may wish to read the "Platform Specific Notes" below.] - -First you must configure the distribution for your particular -system. Go to the directory you wish to build libffi in and run the -"configure" program found in the root directory of the libffi source -distribution. - -You may want to tell configure where to install the libffi library and -header files. To do that, use the --prefix configure switch. Libffi -will install under /usr/local by default. - -If you want to enable extra run-time debugging checks use the the ---enable-debug configure switch. This is useful when your program dies -mysteriously while using libffi. - -Another useful configure switch is --enable-purify-safety. Using this -will add some extra code which will suppress certain warnings when you -are using Purify with libffi. Only use this switch when using -Purify, as it will slow down the library. - -Configure has many other options. Use "configure --help" to see them all. - -Once configure has finished, type "make". Note that you must be using -GNU make. SGI's make will not work. Sun's probably won't either. -You can ftp GNU make from prep.ai.mit.edu:/pub/gnu. - -To ensure that libffi is working as advertised, type "make test". - -To install the library and header files, type "make install". - - -Using libffi -============ - - The Basics - ---------- - -Libffi assumes that you have a pointer to the function you wish to -call and that you know the number and types of arguments to pass it, -as well as the return type of the function. - -The first thing you must do is create an ffi_cif object that matches -the signature of the function you wish to call. The cif in ffi_cif -stands for Call InterFace. To prepare a call interface object, use the -following function: - -ffi_status ffi_prep_cif(ffi_cif *cif, ffi_abi abi, - unsigned int nargs, - ffi_type *rtype, ffi_type **atypes); - - CIF is a pointer to the call interface object you wish - to initialize. - - ABI is an enum that specifies the calling convention - to use for the call. FFI_DEFAULT_ABI defaults - to the system's native calling convention. Other - ABI's may be used with care. They are system - specific. - - NARGS is the number of arguments this function accepts. - libffi does not yet support vararg functions. - - RTYPE is a pointer to an ffi_type structure that represents - the return type of the function. Ffi_type objects - describe the types of values. libffi provides - ffi_type objects for many of the native C types: - signed int, unsigned int, signed char, unsigned char, - etc. There is also a pointer ffi_type object and - a void ffi_type. Use &ffi_type_void for functions that - don't return values. - - ATYPES is a vector of ffi_type pointers. ARGS must be NARGS long. - If NARGS is 0, this is ignored. - - -ffi_prep_cif will return a status code that you are responsible -for checking. It will be one of the following: - - FFI_OK - All is good. - - FFI_BAD_TYPEDEF - One of the ffi_type objects that ffi_prep_cif - came across is bad. - - -Before making the call, the VALUES vector should be initialized -with pointers to the appropriate argument values. - -To call the the function using the initialized ffi_cif, use the -ffi_call function: - -void ffi_call(ffi_cif *cif, void *fn, void *rvalue, void **avalues); - - CIF is a pointer to the ffi_cif initialized specifically - for this function. - - FN is a pointer to the function you want to call. - - RVALUE is a pointer to a chunk of memory that is to hold the - result of the function call. Currently, it must be - at least one word in size (except for the n32 version - under Irix 6.x, which must be a pointer to an 8 byte - aligned value (a long long). It must also be at least - word aligned (depending on the return type, and the - system's alignment requirements). If RTYPE is - &ffi_type_void, this is ignored. If RVALUE is NULL, - the return value is discarded. - - AVALUES is a vector of void* that point to the memory locations - holding the argument values for a call. - If NARGS is 0, this is ignored. - - -If you are expecting a return value from FN it will have been stored -at RVALUE. - - - - An Example - ---------- - -Here is a trivial example that calls puts() a few times. - - #include <stdio.h> - #include <ffi.h> - - int main() - { - ffi_cif cif; - ffi_type *args[1]; - void *values[1]; - char *s; - int rc; - - /* Initialize the argument info vectors */ - args[0] = &ffi_type_uint; - values[0] = &s; - - /* Initialize the cif */ - if (ffi_prep_cif(&cif, FFI_DEFAULT_ABI, 1, - &ffi_type_uint, args) == FFI_OK) - { - s = "Hello World!"; - ffi_call(&cif, puts, &rc, values); - /* rc now holds the result of the call to puts */ - - /* values holds a pointer to the function's arg, so to - call puts() again all we need to do is change the - value of s */ - s = "This is cool!"; - ffi_call(&cif, puts, &rc, values); - } - - return 0; - } - - - - Aggregate Types - --------------- - -Although libffi has no special support for unions or bit-fields, it is -perfectly happy passing structures back and forth. You must first -describe the structure to libffi by creating a new ffi_type object -for it. Here is the definition of ffi_type: - - typedef struct _ffi_type - { - unsigned size; - short alignment; - short type; - struct _ffi_type **elements; - } ffi_type; - -All structures must have type set to FFI_TYPE_STRUCT. You may set -size and alignment to 0. These will be calculated and reset to the -appropriate values by ffi_prep_cif(). - -elements is a NULL terminated array of pointers to ffi_type objects -that describe the type of the structure elements. These may, in turn, -be structure elements. - -The following example initializes a ffi_type object representing the -tm struct from Linux's time.h: - - struct tm { - int tm_sec; - int tm_min; - int tm_hour; - int tm_mday; - int tm_mon; - int tm_year; - int tm_wday; - int tm_yday; - int tm_isdst; - /* Those are for future use. */ - long int __tm_gmtoff__; - __const char *__tm_zone__; - }; - - { - ffi_type tm_type; - ffi_type *tm_type_elements[12]; - int i; - - tm_type.size = tm_type.alignment = 0; - tm_type.elements = &tm_type_elements; - - for (i = 0; i < 9; i++) - tm_type_elements[i] = &ffi_type_sint; - - tm_type_elements[9] = &ffi_type_slong; - tm_type_elements[10] = &ffi_type_pointer; - tm_type_elements[11] = NULL; - - /* tm_type can now be used to represent tm argument types and - return types for ffi_prep_cif() */ - } - - - -Platform Specific Notes -======================= - - Intel x86 - --------- - -There are no known problems with the x86 port. - - Sun SPARC - SunOS 4.1.3 & Solaris 2.x - ------------------------------------- - -You must use GNU Make to build libffi on Sun platforms. - - MIPS - Irix 5.3 & 6.x - --------------------- - -Irix 6.2 and better supports three different calling conventions: o32, -n32 and n64. Currently, libffi only supports both o32 and n32 under -Irix 6.x, but only o32 under Irix 5.3. Libffi will automatically be -configured for whichever calling convention it was built for. - -By default, the configure script will try to build libffi with the GNU -development tools. To build libffi with the SGI development tools, set -the environment variable CC to either "cc -32" or "cc -n32" before -running configure under Irix 6.x (depending on whether you want an o32 -or n32 library), or just "cc" for Irix 5.3. - -With the n32 calling convention, when returning structures smaller -than 16 bytes, be sure to provide an RVALUE that is 8 byte aligned. -Here's one way of forcing this: - - double struct_storage[2]; - my_small_struct *s = (my_small_struct *) struct_storage; - /* Use s for RVALUE */ - -If you don't do this you are liable to get spurious bus errors. - -"long long" values are not supported yet. - -You must use GNU Make to build libffi on SGI platforms. - - ARM - System V ABI - ------------------ - -The ARM port was performed on a NetWinder running ARM Linux ELF -(2.0.31) and gcc 2.8.1. - - - - PowerPC System V ABI - -------------------- - -There are two `System V ABI's which libffi implements for PowerPC. -They differ only in how small structures are returned from functions. - -In the FFI_SYSV version, structures that are 8 bytes or smaller are -returned in registers. This is what GCC does when it is configured -for solaris, and is what the System V ABI I have (dated September -1995) says. - -In the FFI_GCC_SYSV version, all structures are returned the same way: -by passing a pointer as the first argument to the function. This is -what GCC does when it is configured for linux or a generic sysv -target. - -EGCS 1.0.1 (and probably other versions of EGCS/GCC) also has a -inconsistency with the SysV ABI: When a procedure is called with many -floating-point arguments, some of them get put on the stack. They are -all supposed to be stored in double-precision format, even if they are -only single-precision, but EGCS stores single-precision arguments as -single-precision anyway. This causes one test to fail (the `many -arguments' test). - - -What's With The Cryptic Comments? -================================= - -You might notice a number of cryptic comments in the code, delimited -by /*@ and @*/. These are annotations read by the program LCLint, a -tool for statically checking C programs. You can read all about it at -<http://larch-www.lcs.mit.edu:8001/larch/lclint/index.html>. - - -History -======= - -1.20 Oct-5-98 - Raffaele Sena produces ARM port. - -1.19 Oct-5-98 - Fixed x86 long double and long long return support. - m68k bug fixes from Andreas Schwab. - Patch for DU assembler compatibility for the Alpha from Richard - Henderson. - -1.18 Apr-17-98 - Bug fixes and MIPS configuration changes. - -1.17 Feb-24-98 - Bug fixes and m68k port from Andreas Schwab. PowerPC port from - Geoffrey Keating. Various bug x86, Sparc and MIPS bug fixes. - -1.16 Feb-11-98 - Richard Henderson produces Alpha port. - -1.15 Dec-4-97 - Fixed an n32 ABI bug. New libtool, auto* support. - -1.14 May-13-97 - libtool is now used to generate shared and static libraries. - Fixed a minor portability problem reported by Russ McManus - <mcmanr(a)eq.gs.com>. - -1.13 Dec-2-96 - Added --enable-purify-safety to keep Purify from complaining - about certain low level code. - Sparc fix for calling functions with < 6 args. - Linux x86 a.out fix. - -1.12 Nov-22-96 - Added missing ffi_type_void, needed for supporting void return - types. Fixed test case for non MIPS machines. Cygnus Support - is now Cygnus Solutions. - -1.11 Oct-30-96 - Added notes about GNU make. - -1.10 Oct-29-96 - Added configuration fix for non GNU compilers. - -1.09 Oct-29-96 - Added --enable-debug configure switch. Clean-ups based on LCLint - feedback. ffi_mips.h is always installed. Many configuration - fixes. Fixed ffitest.c for sparc builds. - -1.08 Oct-15-96 - Fixed n32 problem. Many clean-ups. - -1.07 Oct-14-96 - Gordon Irlam rewrites v8.S again. Bug fixes. - -1.06 Oct-14-96 - Gordon Irlam improved the sparc port. - -1.05 Oct-14-96 - Interface changes based on feedback. - -1.04 Oct-11-96 - Sparc port complete (modulo struct passing bug). - -1.03 Oct-10-96 - Passing struct args, and returning struct values works for - all architectures/calling conventions. Expanded tests. - -1.02 Oct-9-96 - Added SGI n32 support. Fixed bugs in both o32 and Linux support. - Added "make test". - -1.01 Oct-8-96 - Fixed float passing bug in mips version. Restructured some - of the code. Builds cleanly with SGI tools. - -1.00 Oct-7-96 - First release. No public announcement. - - -Authors & Credits -================= - -libffi was written by Anthony Green <green(a)cygnus.com>. - -Portions of libffi were derived from Gianni Mariani's free gencall -library for Silicon Graphics machines. - -The closure mechanism was designed and implemented by Kresten Krab -Thorup. - -The Sparc port was derived from code contributed by the fine folks at -Visible Decisions Inc <http://www.vdi.com>. Further enhancements were -made by Gordon Irlam at Cygnus Solutions <http://www.cygnus.com>. - -The Alpha port was written by Richard Henderson at Cygnus Solutions. - -Andreas Schwab ported libffi to m68k Linux and provided a number of -bug fixes. - -Geoffrey Keating ported libffi to the PowerPC. - -Raffaele Sena ported libffi to the ARM. - -Jesper Skov and Andrew Haley both did more than their fair share of -stepping through the code and tracking down bugs. - -Thanks also to Tom Tromey for bug fixes and configuration help. - -Thanks to Jim Blandy, who provided some useful feedback on the libffi -interface. - -If you have a problem, or have found a bug, please send a note to -green(a)cygnus.com. diff --git a/Modules/_ctypes/libffi_msvc/README.ctypes b/Modules/_ctypes/libffi_msvc/README.ctypes deleted file mode 100644 index 17e8a40b83f9..000000000000 --- a/Modules/_ctypes/libffi_msvc/README.ctypes +++ /dev/null @@ -1,7 +0,0 @@ -The purpose is to hack the libffi sources so that they can be compiled -with MSVC, and to extend them so that they have the features I need -for ctypes. - -I retrieved the libffi sources from the gcc cvs repository on -2004-01-27. Then I did 'configure' in a 'build' subdirectory on a x86 -linux system, and copied the files I found useful. diff --git a/Modules/_ctypes/libffi_msvc/ffi.c b/Modules/_ctypes/libffi_msvc/ffi.c deleted file mode 100644 index d202b158b079..000000000000 --- a/Modules/_ctypes/libffi_msvc/ffi.c +++ /dev/null @@ -1,530 +0,0 @@ -/* ----------------------------------------------------------------------- - ffi.c - Copyright (c) 1996, 1998, 1999, 2001 Red Hat, Inc. - Copyright (c) 2002 Ranjit Mathew - Copyright (c) 2002 Bo Thorsen - Copyright (c) 2002 Roger Sayle - - x86 Foreign Function Interface - - Permission is hereby granted, free of charge, to any person obtaining - a copy of this software and associated documentation files (the - ``Software''), to deal in the Software without restriction, including - without limitation the rights to use, copy, modify, merge, publish, - distribute, sublicense, and/or sell copies of the Software, and to - permit persons to whom the Software is furnished to do so, subject to - the following conditions: - - The above copyright notice and this permission notice shall be included - in all copies or substantial portions of the Software. - - THE SOFTWARE IS PROVIDED ``AS IS'', WITHOUT WARRANTY OF ANY KIND, EXPRESS - OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF - MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. - IN NO EVENT SHALL CYGNUS SOLUTIONS BE LIABLE FOR ANY CLAIM, DAMAGES OR - OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, - ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR - OTHER DEALINGS IN THE SOFTWARE. - ----------------------------------------------------------------------- */ - -#include <ffi.h> -#include <ffi_common.h> - -#include <stdlib.h> - -/* ffi_prep_args is called by the assembly routine once stack space - has been allocated for the function's arguments */ - -extern void Py_FatalError(const char *msg); - -/*@-exportheader@*/ -void ffi_prep_args(char *stack, extended_cif *ecif) -/*@=exportheader@*/ -{ - register unsigned int i; - register void **p_argv; - register char *argp; - register ffi_type **p_arg; - - argp = stack; - if (ecif->cif->rtype->type == FFI_TYPE_STRUCT) - { - *(void **) argp = ecif->rvalue; - argp += sizeof(void *); - } - - p_argv = ecif->avalue; - - for (i = ecif->cif->nargs, p_arg = ecif->cif->arg_types; - i != 0; - i--, p_arg++) - { - size_t z; - - /* Align if necessary */ - if ((sizeof(void *) - 1) & (size_t) argp) - argp = (char *) ALIGN(argp, sizeof(void *)); - - z = (*p_arg)->size; - if (z < sizeof(intptr_t)) - { - z = sizeof(intptr_t); - switch ((*p_arg)->type) - { - case FFI_TYPE_SINT8: - *(intptr_t *) argp = (intptr_t)*(SINT8 *)(* p_argv); - break; - - case FFI_TYPE_UINT8: - *(uintptr_t *) argp = (uintptr_t)*(UINT8 *)(* p_argv); - break; - - case FFI_TYPE_SINT16: - *(intptr_t *) argp = (intptr_t)*(SINT16 *)(* p_argv); - break; - - case FFI_TYPE_UINT16: - *(uintptr_t *) argp = (uintptr_t)*(UINT16 *)(* p_argv); - break; - - case FFI_TYPE_SINT32: - *(intptr_t *) argp = (intptr_t)*(SINT32 *)(* p_argv); - break; - - case FFI_TYPE_UINT32: - *(uintptr_t *) argp = (uintptr_t)*(UINT32 *)(* p_argv); - break; - - case FFI_TYPE_FLOAT: - *(uintptr_t *) argp = 0; - *(float *) argp = *(float *)(* p_argv); - break; - - // 64-bit value cases should never be used for x86 and AMD64 builds - case FFI_TYPE_SINT64: - *(intptr_t *) argp = (intptr_t)*(SINT64 *)(* p_argv); - break; - - case FFI_TYPE_UINT64: - *(uintptr_t *) argp = (uintptr_t)*(UINT64 *)(* p_argv); - break; - - case FFI_TYPE_STRUCT: - *(uintptr_t *) argp = (uintptr_t)*(UINT32 *)(* p_argv); - break; - - case FFI_TYPE_DOUBLE: - *(uintptr_t *) argp = 0; - *(double *) argp = *(double *)(* p_argv); - break; - - default: - FFI_ASSERT(0); - } - } -#ifdef _WIN64 - else if (z > 8) - { - /* On Win64, if a single argument takes more than 8 bytes, - then it is always passed by reference. */ - *(void **)argp = *p_argv; - z = 8; - } -#endif - else - { - memcpy(argp, *p_argv, z); - } - p_argv++; - argp += z; - } - - if (argp >= stack && (unsigned)(argp - stack) > ecif->cif->bytes) - { - Py_FatalError("FFI BUG: not enough stack space for arguments"); - } - return; -} - -/* -Per: https://msdn.microsoft.com/en-us/library/7572ztz4.aspx -To be returned by value in RAX, user-defined types must have a length -of 1, 2, 4, 8, 16, 32, or 64 bits -*/ -int can_return_struct_as_int(size_t s) -{ - return s == 1 || s == 2 || s == 4; -} - -int can_return_struct_as_sint64(size_t s) -{ - return s == 8; -} - -/* Perform machine dependent cif processing */ -ffi_status ffi_prep_cif_machdep(ffi_cif *cif) -{ - /* Set the return type flag */ - switch (cif->rtype->type) - { - case FFI_TYPE_VOID: - case FFI_TYPE_SINT64: - case FFI_TYPE_FLOAT: - case FFI_TYPE_DOUBLE: - case FFI_TYPE_LONGDOUBLE: - cif->flags = (unsigned) cif->rtype->type; - break; - - case FFI_TYPE_STRUCT: - /* MSVC returns small structures in registers. Put in cif->flags - the value FFI_TYPE_STRUCT only if the structure is big enough; - otherwise, put the 4- or 8-bytes integer type. */ - if (can_return_struct_as_int(cif->rtype->size)) - cif->flags = FFI_TYPE_INT; - else if (can_return_struct_as_sint64(cif->rtype->size)) - cif->flags = FFI_TYPE_SINT64; - else - cif->flags = FFI_TYPE_STRUCT; - break; - - case FFI_TYPE_UINT64: -#ifdef _WIN64 - case FFI_TYPE_POINTER: -#endif - cif->flags = FFI_TYPE_SINT64; - break; - - default: - cif->flags = FFI_TYPE_INT; - break; - } - - return FFI_OK; -} - -#ifdef _WIN32 -extern int -ffi_call_x86(void (*)(char *, extended_cif *), - /*@out@*/ extended_cif *, - unsigned, unsigned, - /*@out@*/ unsigned *, - void (*fn)()); -#endif - -#ifdef _WIN64 -extern int -ffi_call_AMD64(void (*)(char *, extended_cif *), - /*@out@*/ extended_cif *, - unsigned, unsigned, - /*@out@*/ unsigned *, - void (*fn)()); -#endif - -int -ffi_call(/*@dependent@*/ ffi_cif *cif, - void (*fn)(), - /*@out@*/ void *rvalue, - /*@dependent@*/ void **avalue) -{ - extended_cif ecif; - - ecif.cif = cif; - ecif.avalue = avalue; - - /* If the return value is a struct and we don't have a return */ - /* value address then we need to make one */ - - if ((rvalue == NULL) && - (cif->rtype->type == FFI_TYPE_STRUCT)) - { - /*@-sysunrecog@*/ - ecif.rvalue = alloca(cif->rtype->size); - /*@=sysunrecog@*/ - } - else - ecif.rvalue = rvalue; - - - switch (cif->abi) - { -#if !defined(_WIN64) - case FFI_SYSV: - case FFI_STDCALL: - return ffi_call_x86(ffi_prep_args, &ecif, cif->bytes, - cif->flags, ecif.rvalue, fn); - break; -#else - case FFI_SYSV: - /* If a single argument takes more than 8 bytes, - then a copy is passed by reference. */ - for (unsigned i = 0; i < cif->nargs; i++) { - size_t z = cif->arg_types[i]->size; - if (z > 8) { - void *temp = alloca(z); - memcpy(temp, avalue[i], z); - avalue[i] = temp; - } - } - /*@-usedef@*/ - return ffi_call_AMD64(ffi_prep_args, &ecif, cif->bytes, - cif->flags, ecif.rvalue, fn); - /*@=usedef@*/ - break; -#endif - - default: - FFI_ASSERT(0); - break; - } - return -1; /* theller: Hrm. */ -} - - -/** private members **/ - -static void ffi_prep_incoming_args_SYSV (char *stack, void **ret, - void** args, ffi_cif* cif); -/* This function is jumped to by the trampoline */ - -#ifdef _WIN64 -void * -#else -static void __fastcall -#endif -ffi_closure_SYSV (ffi_closure *closure, char *argp) -{ - // this is our return value storage - long double res; - - // our various things... - ffi_cif *cif; - void **arg_area; - unsigned short rtype; - void *resp = (void*)&res; - void *args = argp + sizeof(void*); - - cif = closure->cif; - arg_area = (void**) alloca (cif->nargs * sizeof (void*)); - - /* this call will initialize ARG_AREA, such that each - * element in that array points to the corresponding - * value on the stack; and if the function returns - * a structure, it will re-set RESP to point to the - * structure return address. */ - - ffi_prep_incoming_args_SYSV(args, (void**)&resp, arg_area, cif); - - (closure->fun) (cif, resp, arg_area, closure->user_data); - - rtype = cif->flags; - -#if defined(_WIN32) && !defined(_WIN64) -#ifdef _MSC_VER - /* now, do a generic return based on the value of rtype */ - if (rtype == FFI_TYPE_INT) - { - _asm mov eax, resp ; - _asm mov eax, [eax] ; - } - else if (rtype == FFI_TYPE_FLOAT) - { - _asm mov eax, resp ; - _asm fld DWORD PTR [eax] ; -// asm ("flds (%0)" : : "r" (resp) : "st" ); - } - else if (rtype == FFI_TYPE_DOUBLE) - { - _asm mov eax, resp ; - _asm fld QWORD PTR [eax] ; -// asm ("fldl (%0)" : : "r" (resp) : "st", "st(1)" ); - } - else if (rtype == FFI_TYPE_LONGDOUBLE) - { -// asm ("fldt (%0)" : : "r" (resp) : "st", "st(1)" ); - } - else if (rtype == FFI_TYPE_SINT64) - { - _asm mov edx, resp ; - _asm mov eax, [edx] ; - _asm mov edx, [edx + 4] ; -// asm ("movl 0(%0),%%eax;" -// "movl 4(%0),%%edx" -// : : "r"(resp) -// : "eax", "edx"); - } -#else - /* now, do a generic return based on the value of rtype */ - if (rtype == FFI_TYPE_INT) - { - asm ("movl (%0),%%eax" : : "r" (resp) : "eax"); - } - else if (rtype == FFI_TYPE_FLOAT) - { - asm ("flds (%0)" : : "r" (resp) : "st" ); - } - else if (rtype == FFI_TYPE_DOUBLE) - { - asm ("fldl (%0)" : : "r" (resp) : "st", "st(1)" ); - } - else if (rtype == FFI_TYPE_LONGDOUBLE) - { - asm ("fldt (%0)" : : "r" (resp) : "st", "st(1)" ); - } - else if (rtype == FFI_TYPE_SINT64) - { - asm ("movl 0(%0),%%eax;" - "movl 4(%0),%%edx" - : : "r"(resp) - : "eax", "edx"); - } -#endif -#endif - -#ifdef _WIN64 - /* The result is returned in rax. This does the right thing for - result types except for floats; we have to 'mov xmm0, rax' in the - caller to correct this. - */ - return *(void **)resp; -#endif -} - -/*@-exportheader@*/ -static void -ffi_prep_incoming_args_SYSV(char *stack, void **rvalue, - void **avalue, ffi_cif *cif) -/*@=exportheader@*/ -{ - register unsigned int i; - register void **p_argv; - register char *argp; - register ffi_type **p_arg; - - argp = stack; - - if ( cif->rtype->type == FFI_TYPE_STRUCT ) { - *rvalue = *(void **) argp; - argp += sizeof(void *); - } - - p_argv = avalue; - - for (i = cif->nargs, p_arg = cif->arg_types; (i != 0); i--, p_arg++) - { - size_t z; - - /* Align if necessary */ - if ((sizeof(char *) - 1) & (size_t) argp) { - argp = (char *) ALIGN(argp, sizeof(char*)); - } - - z = (*p_arg)->size; - - /* because we're little endian, this is what it turns into. */ - -#ifdef _WIN64 - if (z > 8) { - /* On Win64, if a single argument takes more than 8 bytes, - * then it is always passed by reference. - */ - *p_argv = *((void**) argp); - z = 8; - } - else -#endif - *p_argv = (void*) argp; - - p_argv++; - argp += z; - } - - return; -} - -/* the cif must already be prep'ed */ -extern void ffi_closure_OUTER(); - -ffi_status -ffi_prep_closure_loc (ffi_closure* closure, - ffi_cif* cif, - void (*fun)(ffi_cif*,void*,void**,void*), - void *user_data, - void *codeloc) -{ - short bytes; - char *tramp; -#ifdef _WIN64 - int mask = 0; -#endif - FFI_ASSERT (cif->abi == FFI_SYSV); - - if (cif->abi == FFI_SYSV) - bytes = 0; -#if !defined(_WIN64) - else if (cif->abi == FFI_STDCALL) - bytes = cif->bytes; -#endif - else - return FFI_BAD_ABI; - - tramp = &closure->tramp[0]; - -#define BYTES(text) memcpy(tramp, text, sizeof(text)), tramp += sizeof(text)-1 -#define POINTER(x) *(void**)tramp = (void*)(x), tramp += sizeof(void*) -#define SHORT(x) *(short*)tramp = x, tramp += sizeof(short) -#define INT(x) *(int*)tramp = x, tramp += sizeof(int) - -#ifdef _WIN64 - if (cif->nargs >= 1 && - (cif->arg_types[0]->type == FFI_TYPE_FLOAT - || cif->arg_types[0]->type == FFI_TYPE_DOUBLE)) - mask |= 1; - if (cif->nargs >= 2 && - (cif->arg_types[1]->type == FFI_TYPE_FLOAT - || cif->arg_types[1]->type == FFI_TYPE_DOUBLE)) - mask |= 2; - if (cif->nargs >= 3 && - (cif->arg_types[2]->type == FFI_TYPE_FLOAT - || cif->arg_types[2]->type == FFI_TYPE_DOUBLE)) - mask |= 4; - if (cif->nargs >= 4 && - (cif->arg_types[3]->type == FFI_TYPE_FLOAT - || cif->arg_types[3]->type == FFI_TYPE_DOUBLE)) - mask |= 8; - - /* 41 BB ---- mov r11d,mask */ - BYTES("\x41\xBB"); INT(mask); - - /* 48 B8 -------- mov rax, closure */ - BYTES("\x48\xB8"); POINTER(closure); - - /* 49 BA -------- mov r10, ffi_closure_OUTER */ - BYTES("\x49\xBA"); POINTER(ffi_closure_OUTER); - - /* 41 FF E2 jmp r10 */ - BYTES("\x41\xFF\xE2"); - -#else - - /* mov ecx, closure */ - BYTES("\xb9"); POINTER(closure); - - /* mov edx, esp */ - BYTES("\x8b\xd4"); - - /* call ffi_closure_SYSV */ - BYTES("\xe8"); POINTER((char*)&ffi_closure_SYSV - (tramp + 4)); - - /* ret bytes */ - BYTES("\xc2"); - SHORT(bytes); - -#endif - - if (tramp - &closure->tramp[0] > FFI_TRAMPOLINE_SIZE) - Py_FatalError("FFI_TRAMPOLINE_SIZE too small in " __FILE__); - - closure->cif = cif; - closure->user_data = user_data; - closure->fun = fun; - return FFI_OK; -} diff --git a/Modules/_ctypes/libffi_msvc/ffi.h b/Modules/_ctypes/libffi_msvc/ffi.h deleted file mode 100644 index ba74202720a6..000000000000 --- a/Modules/_ctypes/libffi_msvc/ffi.h +++ /dev/null @@ -1,322 +0,0 @@ -/* -----------------------------------------------------------------*-C-*- - libffi 2.00-beta - Copyright (c) 1996-2003 Red Hat, Inc. - - Permission is hereby granted, free of charge, to any person obtaining - a copy of this software and associated documentation files (the - ``Software''), to deal in the Software without restriction, including - without limitation the rights to use, copy, modify, merge, publish, - distribute, sublicense, and/or sell copies of the Software, and to - permit persons to whom the Software is furnished to do so, subject to - the following conditions: - - The above copyright notice and this permission notice shall be included - in all copies or substantial portions of the Software. - - THE SOFTWARE IS PROVIDED ``AS IS'', WITHOUT WARRANTY OF ANY KIND, EXPRESS - OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF - MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. - IN NO EVENT SHALL CYGNUS SOLUTIONS BE LIABLE FOR ANY CLAIM, DAMAGES OR - OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, - ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR - OTHER DEALINGS IN THE SOFTWARE. - - ----------------------------------------------------------------------- */ - -/* ------------------------------------------------------------------- - The basic API is described in the README file. - - The raw API is designed to bypass some of the argument packing - and unpacking on architectures for which it can be avoided. - - The closure API allows interpreted functions to be packaged up - inside a C function pointer, so that they can be called as C functions, - with no understanding on the client side that they are interpreted. - It can also be used in other cases in which it is necessary to package - up a user specified parameter and a function pointer as a single - function pointer. - - The closure API must be implemented in order to get its functionality, - e.g. for use by gij. Routines are provided to emulate the raw API - if the underlying platform doesn't allow faster implementation. - - More details on the raw and cloure API can be found in: - - http://gcc.gnu.org/ml/java/1999-q3/msg00138.html - - and - - http://gcc.gnu.org/ml/java/1999-q3/msg00174.html - -------------------------------------------------------------------- */ - -#ifndef LIBFFI_H -#define LIBFFI_H - -#ifdef __cplusplus -extern "C" { -#endif - -/* Specify which architecture libffi is configured for. */ -//XXX #define X86 - -/* ---- System configuration information --------------------------------- */ - -#include <ffitarget.h> - -#ifndef LIBFFI_ASM - -#include <stddef.h> -#include <limits.h> - -/* LONG_LONG_MAX is not always defined (not if STRICT_ANSI, for example). - But we can find it either under the correct ANSI name, or under GNU - C's internal name. */ -#ifdef LONG_LONG_MAX -# define FFI_LONG_LONG_MAX LONG_LONG_MAX -#else -# ifdef LLONG_MAX -# define FFI_LONG_LONG_MAX LLONG_MAX -# else -# ifdef __GNUC__ -# define FFI_LONG_LONG_MAX __LONG_LONG_MAX__ -# endif -# ifdef _MSC_VER -# define FFI_LONG_LONG_MAX _I64_MAX -# endif -# endif -#endif - -#if SCHAR_MAX == 127 -# define ffi_type_uchar ffi_type_uint8 -# define ffi_type_schar ffi_type_sint8 -#else - #error "char size not supported" -#endif - -#if SHRT_MAX == 32767 -# define ffi_type_ushort ffi_type_uint16 -# define ffi_type_sshort ffi_type_sint16 -#elif SHRT_MAX == 2147483647 -# define ffi_type_ushort ffi_type_uint32 -# define ffi_type_sshort ffi_type_sint32 -#else - #error "short size not supported" -#endif - -#if INT_MAX == 32767 -# define ffi_type_uint ffi_type_uint16 -# define ffi_type_sint ffi_type_sint16 -#elif INT_MAX == 2147483647 -# define ffi_type_uint ffi_type_uint32 -# define ffi_type_sint ffi_type_sint32 -#elif INT_MAX == 9223372036854775807 -# define ffi_type_uint ffi_type_uint64 -# define ffi_type_sint ffi_type_sint64 -#else - #error "int size not supported" -#endif - -#define ffi_type_ulong ffi_type_uint64 -#define ffi_type_slong ffi_type_sint64 -#if LONG_MAX == 2147483647 -# if FFI_LONG_LONG_MAX != 9223372036854775807 - #error "no 64-bit data type supported" -# endif -#elif LONG_MAX != 9223372036854775807 - #error "long size not supported" -#endif - -/* The closure code assumes that this works on pointers, i.e. a size_t */ -/* can hold a pointer. */ - -typedef struct _ffi_type -{ - size_t size; - unsigned short alignment; - unsigned short type; - /*@null@*/ struct _ffi_type **elements; -} ffi_type; - -int can_return_struct_as_int(size_t); -int can_return_struct_as_sint64(size_t); - -/* These are defined in types.c */ -extern ffi_type ffi_type_void; -extern ffi_type ffi_type_uint8; -extern ffi_type ffi_type_sint8; -extern ffi_type ffi_type_uint16; -extern ffi_type ffi_type_sint16; -extern ffi_type ffi_type_uint32; -extern ffi_type ffi_type_sint32; -extern ffi_type ffi_type_uint64; -extern ffi_type ffi_type_sint64; -extern ffi_type ffi_type_float; -extern ffi_type ffi_type_double; -extern ffi_type ffi_type_longdouble; -extern ffi_type ffi_type_pointer; - - -typedef enum { - FFI_OK = 0, - FFI_BAD_TYPEDEF, - FFI_BAD_ABI -} ffi_status; - -typedef unsigned FFI_TYPE; - -typedef struct { - ffi_abi abi; - unsigned nargs; - /*@dependent@*/ ffi_type **arg_types; - /*@dependent@*/ ffi_type *rtype; - unsigned bytes; - unsigned flags; -#ifdef FFI_EXTRA_CIF_FIELDS - FFI_EXTRA_CIF_FIELDS; -#endif -} ffi_cif; - -/* ---- Definitions for the raw API -------------------------------------- */ - -#ifdef _WIN64 -#define FFI_SIZEOF_ARG 8 -#else -#define FFI_SIZEOF_ARG 4 -#endif - -typedef union { - ffi_sarg sint; - ffi_arg uint; - float flt; - char data[FFI_SIZEOF_ARG]; - void* ptr; -} ffi_raw; - -void ffi_raw_call (/*@dependent@*/ ffi_cif *cif, - void (*fn)(), - /*@out@*/ void *rvalue, - /*@dependent@*/ ffi_raw *avalue); - -void ffi_ptrarray_to_raw (ffi_cif *cif, void **args, ffi_raw *raw); -void ffi_raw_to_ptrarray (ffi_cif *cif, ffi_raw *raw, void **args); -size_t ffi_raw_size (ffi_cif *cif); - -/* This is analogous to the raw API, except it uses Java parameter */ -/* packing, even on 64-bit machines. I.e. on 64-bit machines */ -/* longs and doubles are followed by an empty 64-bit word. */ - -void ffi_java_raw_call (/*@dependent@*/ ffi_cif *cif, - void (*fn)(), - /*@out@*/ void *rvalue, - /*@dependent@*/ ffi_raw *avalue); - -void ffi_java_ptrarray_to_raw (ffi_cif *cif, void **args, ffi_raw *raw); -void ffi_java_raw_to_ptrarray (ffi_cif *cif, ffi_raw *raw, void **args); -size_t ffi_java_raw_size (ffi_cif *cif); - -/* ---- Definitions for closures ----------------------------------------- */ - -#if FFI_CLOSURES - -typedef struct { - char tramp[FFI_TRAMPOLINE_SIZE]; - ffi_cif *cif; - void (*fun)(ffi_cif*,void*,void**,void*); - void *user_data; -} ffi_closure; - -void ffi_closure_free(void *); -void *ffi_closure_alloc (size_t size, void **code); - -ffi_status -ffi_prep_closure_loc (ffi_closure*, - ffi_cif *, - void (*fun)(ffi_cif*,void*,void**,void*), - void *user_data, - void *codeloc); - -typedef struct { - char tramp[FFI_TRAMPOLINE_SIZE]; - - ffi_cif *cif; - -#if !FFI_NATIVE_RAW_API - - /* if this is enabled, then a raw closure has the same layout - as a regular closure. We use this to install an intermediate - handler to do the transaltion, void** -> ffi_raw*. */ - - void (*translate_args)(ffi_cif*,void*,void**,void*); - void *this_closure; - -#endif - - void (*fun)(ffi_cif*,void*,ffi_raw*,void*); - void *user_data; - -} ffi_raw_closure; - -ffi_status -ffi_prep_raw_closure (ffi_raw_closure*, - ffi_cif *cif, - void (*fun)(ffi_cif*,void*,ffi_raw*,void*), - void *user_data); - -ffi_status -ffi_prep_java_raw_closure (ffi_raw_closure*, - ffi_cif *cif, - void (*fun)(ffi_cif*,void*,ffi_raw*,void*), - void *user_data); - -#endif /* FFI_CLOSURES */ - -/* ---- Public interface definition -------------------------------------- */ - -ffi_status ffi_prep_cif(/*@out@*/ /*@partial@*/ ffi_cif *cif, - ffi_abi abi, - unsigned int nargs, - /*@dependent@*/ /*@out@*/ /*@partial@*/ ffi_type *rtype, - /*@dependent@*/ ffi_type **atypes); - -int -ffi_call(/*@dependent@*/ ffi_cif *cif, - void (*fn)(), - /*@out@*/ void *rvalue, - /*@dependent@*/ void **avalue); - -/* Useful for eliminating compiler warnings */ -#define FFI_FN(f) ((void (*)())f) - -/* ---- Definitions shared with assembly code ---------------------------- */ - -#endif - -/* If these change, update src/mips/ffitarget.h. */ -#define FFI_TYPE_VOID 0 -#define FFI_TYPE_INT 1 -#define FFI_TYPE_FLOAT 2 -#define FFI_TYPE_DOUBLE 3 -#if 1 -#define FFI_TYPE_LONGDOUBLE 4 -#else -#define FFI_TYPE_LONGDOUBLE FFI_TYPE_DOUBLE -#endif -#define FFI_TYPE_UINT8 5 -#define FFI_TYPE_SINT8 6 -#define FFI_TYPE_UINT16 7 -#define FFI_TYPE_SINT16 8 -#define FFI_TYPE_UINT32 9 -#define FFI_TYPE_SINT32 10 -#define FFI_TYPE_UINT64 11 -#define FFI_TYPE_SINT64 12 -#define FFI_TYPE_STRUCT 13 -#define FFI_TYPE_POINTER 14 - -/* This should always refer to the last type code (for sanity checks) */ -#define FFI_TYPE_LAST FFI_TYPE_POINTER - -#ifdef __cplusplus -} -#endif - -#endif - diff --git a/Modules/_ctypes/libffi_msvc/ffi_common.h b/Modules/_ctypes/libffi_msvc/ffi_common.h deleted file mode 100644 index 43fb83b48105..000000000000 --- a/Modules/_ctypes/libffi_msvc/ffi_common.h +++ /dev/null @@ -1,77 +0,0 @@ -/* ----------------------------------------------------------------------- - ffi_common.h - Copyright (c) 1996 Red Hat, Inc. - - Common internal definitions and macros. Only necessary for building - libffi. - ----------------------------------------------------------------------- */ - -#ifndef FFI_COMMON_H -#define FFI_COMMON_H - -#ifdef __cplusplus -extern "C" { -#endif - -#include <fficonfig.h> -#include <malloc.h> - -/* Check for the existence of memcpy. */ -#if STDC_HEADERS -# include <string.h> -#else -# ifndef HAVE_MEMCPY -# define memcpy(d, s, n) bcopy ((s), (d), (n)) -# endif -#endif - -#if defined(FFI_DEBUG) -#include <stdio.h> -#endif - -#ifdef FFI_DEBUG -/*@exits@*/ void ffi_assert(/*@temp@*/ char *expr, /*@temp@*/ char *file, int line); -void ffi_stop_here(void); -void ffi_type_test(/*@temp@*/ /*@out@*/ ffi_type *a, /*@temp@*/ char *file, int line); - -#define FFI_ASSERT(x) ((x) ? (void)0 : ffi_assert(#x, __FILE__,__LINE__)) -#define FFI_ASSERT_AT(x, f, l) ((x) ? 0 : ffi_assert(#x, (f), (l))) -#define FFI_ASSERT_VALID_TYPE(x) ffi_type_test (x, __FILE__, __LINE__) -#else -#define FFI_ASSERT(x) -#define FFI_ASSERT_AT(x, f, l) -#define FFI_ASSERT_VALID_TYPE(x) -#endif - -#define ALIGN(v, a) (((((size_t) (v))-1) | ((a)-1))+1) - -/* Perform machine dependent cif processing */ -ffi_status ffi_prep_cif_machdep(ffi_cif *cif); - -/* Extended cif, used in callback from assembly routine */ -typedef struct -{ - /*@dependent@*/ ffi_cif *cif; - /*@dependent@*/ void *rvalue; - /*@dependent@*/ void **avalue; -} extended_cif; - -/* Terse sized type definitions. */ -typedef unsigned int UINT8 __attribute__((__mode__(__QI__))); -typedef signed int SINT8 __attribute__((__mode__(__QI__))); -typedef unsigned int UINT16 __attribute__((__mode__(__HI__))); -typedef signed int SINT16 __attribute__((__mode__(__HI__))); -typedef unsigned int UINT32 __attribute__((__mode__(__SI__))); -typedef signed int SINT32 __attribute__((__mode__(__SI__))); -typedef unsigned int UINT64 __attribute__((__mode__(__DI__))); -typedef signed int SINT64 __attribute__((__mode__(__DI__))); - -typedef float FLOAT32; - - -#ifdef __cplusplus -} -#endif - -#endif - - diff --git a/Modules/_ctypes/libffi_msvc/fficonfig.h b/Modules/_ctypes/libffi_msvc/fficonfig.h deleted file mode 100644 index c14f653ec894..000000000000 --- a/Modules/_ctypes/libffi_msvc/fficonfig.h +++ /dev/null @@ -1,96 +0,0 @@ -/* fficonfig.h. Originally created by configure, now hand_maintained for MSVC. */ - -/* fficonfig.h. Generated automatically by configure. */ -/* fficonfig.h.in. Generated automatically from configure.in by autoheader. */ - -/* Define this for MSVC, but not for mingw32! */ -#ifdef _MSC_VER -#define __attribute__(x) /* */ -#endif -#define alloca _alloca - -/*----------------------------------------------------------------*/ - -/* Define if using alloca.c. */ -/* #undef C_ALLOCA */ - -/* Define to one of _getb67, GETB67, getb67 for Cray-2 and Cray-YMP systems. - This function is required for alloca.c support on those systems. */ -/* #undef CRAY_STACKSEG_END */ - -/* Define if you have alloca, as a function or macro. */ -#define HAVE_ALLOCA 1 - -/* Define if you have <alloca.h> and it should be used (not on Ultrix). */ -/* #define HAVE_ALLOCA_H 1 */ - -/* If using the C implementation of alloca, define if you know the - direction of stack growth for your system; otherwise it will be - automatically deduced at run-time. - STACK_DIRECTION > 0 => grows toward higher addresses - STACK_DIRECTION < 0 => grows toward lower addresses - STACK_DIRECTION = 0 => direction of growth unknown - */ -/* #undef STACK_DIRECTION */ - -/* Define if you have the ANSI C header files. */ -#define STDC_HEADERS 1 - -/* Define if you have the memcpy function. */ -#define HAVE_MEMCPY 1 - -/* Define if read-only mmap of a plain file works. */ -//#define HAVE_MMAP_FILE 1 - -/* Define if mmap of /dev/zero works. */ -//#define HAVE_MMAP_DEV_ZERO 1 - -/* Define if mmap with MAP_ANON(YMOUS) works. */ -//#define HAVE_MMAP_ANON 1 - -/* The number of bytes in type double */ -#define SIZEOF_DOUBLE 8 - -/* The number of bytes in type long double */ -#define SIZEOF_LONG_DOUBLE 12 - -/* Define if you have the long double type and it is bigger than a double */ -#define HAVE_LONG_DOUBLE 1 - -/* whether byteorder is bigendian */ -/* #undef WORDS_BIGENDIAN */ - -/* Define if the host machine stores words of multi-word integers in - big-endian order. */ -/* #undef HOST_WORDS_BIG_ENDIAN */ - -/* 1234 = LIL_ENDIAN, 4321 = BIGENDIAN */ -#define BYTEORDER 1234 - -/* Define if your assembler and linker support unaligned PC relative relocs. */ -/* #undef HAVE_AS_SPARC_UA_PCREL */ - -/* Define if your assembler supports .register. */ -/* #undef HAVE_AS_REGISTER_PSEUDO_OP */ - -/* Define if .eh_frame sections should be read-only. */ -/* #undef HAVE_RO_EH_FRAME */ - -/* Define to the flags needed for the .section .eh_frame directive. */ -/* #define EH_FRAME_FLAGS "aw" */ - -/* Define to the flags needed for the .section .eh_frame directive. */ -/* #define EH_FRAME_FLAGS "aw" */ - -/* Define this if you want extra debugging. */ -/* #undef FFI_DEBUG */ - -/* Define this is you do not want support for aggregate types. */ -/* #undef FFI_NO_STRUCTS */ - -/* Define this is you do not want support for the raw API. */ -/* #undef FFI_NO_RAW_API */ - -/* Define this if you are using Purify and want to suppress spurious messages. */ -/* #undef USING_PURIFY */ - diff --git a/Modules/_ctypes/libffi_msvc/ffitarget.h b/Modules/_ctypes/libffi_msvc/ffitarget.h deleted file mode 100644 index 85f5ee81bbe6..000000000000 --- a/Modules/_ctypes/libffi_msvc/ffitarget.h +++ /dev/null @@ -1,85 +0,0 @@ -/* -----------------------------------------------------------------*-C-*- - ffitarget.h - Copyright (c) 1996-2003 Red Hat, Inc. - Target configuration macros for x86 and x86-64. - - Permission is hereby granted, free of charge, to any person obtaining - a copy of this software and associated documentation files (the - ``Software''), to deal in the Software without restriction, including - without limitation the rights to use, copy, modify, merge, publish, - distribute, sublicense, and/or sell copies of the Software, and to - permit persons to whom the Software is furnished to do so, subject to - the following conditions: - - The above copyright notice and this permission notice shall be included - in all copies or substantial portions of the Software. - - THE SOFTWARE IS PROVIDED ``AS IS'', WITHOUT WARRANTY OF ANY KIND, EXPRESS - OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF - MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. - IN NO EVENT SHALL CYGNUS SOLUTIONS BE LIABLE FOR ANY CLAIM, DAMAGES OR - OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, - ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR - OTHER DEALINGS IN THE SOFTWARE. - - ----------------------------------------------------------------------- */ - -#ifndef LIBFFI_TARGET_H -#define LIBFFI_TARGET_H - -/* ---- System specific configurations ----------------------------------- */ - -#if defined (X86_64) && defined (__i386__) -#undef X86_64 -#define X86 -#endif - -/* ---- Generic type definitions ----------------------------------------- */ - -#ifndef LIBFFI_ASM -#ifndef _WIN64 -typedef unsigned long ffi_arg; -#else -typedef unsigned __int64 ffi_arg; -#endif -typedef signed long ffi_sarg; - -typedef enum ffi_abi { - FFI_FIRST_ABI = 0, - - /* ---- Intel x86 Win32 ---------- */ - FFI_SYSV, -#ifndef _WIN64 - FFI_STDCALL, -#endif - /* TODO: Add fastcall support for the sake of completeness */ - FFI_DEFAULT_ABI = FFI_SYSV, - - /* ---- Intel x86 and AMD x86-64 - */ -/* #if !defined(X86_WIN32) && (defined(__i386__) || defined(__x86_64__)) */ -/* FFI_SYSV, */ -/* FFI_UNIX64,*/ /* Unix variants all use the same ABI for x86-64 */ -/* #ifdef __i386__ */ -/* FFI_DEFAULT_ABI = FFI_SYSV, */ -/* #else */ -/* FFI_DEFAULT_ABI = FFI_UNIX64, */ -/* #endif */ -/* #endif */ - - FFI_LAST_ABI = FFI_DEFAULT_ABI + 1 -} ffi_abi; -#endif - -/* ---- Definitions for closures ----------------------------------------- */ - -#define FFI_CLOSURES 1 - -#ifdef _WIN64 -#define FFI_TRAMPOLINE_SIZE 29 -#define FFI_NATIVE_RAW_API 0 -#else -#define FFI_TRAMPOLINE_SIZE 15 -#define FFI_NATIVE_RAW_API 1 /* x86 has native raw api support */ -#endif - -#endif - diff --git a/Modules/_ctypes/libffi_msvc/prep_cif.c b/Modules/_ctypes/libffi_msvc/prep_cif.c deleted file mode 100644 index 022435e53fcd..000000000000 --- a/Modules/_ctypes/libffi_msvc/prep_cif.c +++ /dev/null @@ -1,188 +0,0 @@ -/* ----------------------------------------------------------------------- - prep_cif.c - Copyright (c) 1996, 1998 Red Hat, Inc. - - Permission is hereby granted, free of charge, to any person obtaining - a copy of this software and associated documentation files (the - ``Software''), to deal in the Software without restriction, including - without limitation the rights to use, copy, modify, merge, publish, - distribute, sublicense, and/or sell copies of the Software, and to - permit persons to whom the Software is furnished to do so, subject to - the following conditions: - - The above copyright notice and this permission notice shall be included - in all copies or substantial portions of the Software. - - THE SOFTWARE IS PROVIDED ``AS IS'', WITHOUT WARRANTY OF ANY KIND, EXPRESS - OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF - MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. - IN NO EVENT SHALL CYGNUS SOLUTIONS BE LIABLE FOR ANY CLAIM, DAMAGES OR - OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, - ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR - OTHER DEALINGS IN THE SOFTWARE. - ----------------------------------------------------------------------- */ - -#include <ffi.h> -#include <ffi_common.h> -#include <stdlib.h> - - -/* Round up to FFI_SIZEOF_ARG. */ - -#define STACK_ARG_SIZE(x) ALIGN(x, FFI_SIZEOF_ARG) - -/* Perform machine independent initialization of aggregate type - specifications. */ - -static ffi_status initialize_aggregate(/*@out@*/ ffi_type *arg) -{ - ffi_type **ptr; - - FFI_ASSERT(arg != NULL); - - /*@-usedef@*/ - - FFI_ASSERT(arg->elements != NULL); - FFI_ASSERT(arg->size == 0); - FFI_ASSERT(arg->alignment == 0); - - ptr = &(arg->elements[0]); - - while ((*ptr) != NULL) - { - if (((*ptr)->size == 0) && (initialize_aggregate((*ptr)) != FFI_OK)) - return FFI_BAD_TYPEDEF; - - /* Perform a sanity check on the argument type */ - FFI_ASSERT_VALID_TYPE(*ptr); - - arg->size = ALIGN(arg->size, (*ptr)->alignment); - arg->size += (*ptr)->size; - - arg->alignment = (arg->alignment > (*ptr)->alignment) ? - arg->alignment : (*ptr)->alignment; - - ptr++; - } - - /* Structure size includes tail padding. This is important for - structures that fit in one register on ABIs like the PowerPC64 - Linux ABI that right justify small structs in a register. - It's also needed for nested structure layout, for example - struct A { long a; char b; }; struct B { struct A x; char y; }; - should find y at an offset of 2*sizeof(long) and result in a - total size of 3*sizeof(long). */ - arg->size = ALIGN (arg->size, arg->alignment); - - if (arg->size == 0) - return FFI_BAD_TYPEDEF; - else - return FFI_OK; - - /*@=usedef@*/ -} - -/* Perform machine independent ffi_cif preparation, then call - machine dependent routine. */ - -ffi_status ffi_prep_cif(/*@out@*/ /*@partial@*/ ffi_cif *cif, - ffi_abi abi, unsigned int nargs, - /*@dependent@*/ /*@out@*/ /*@partial@*/ ffi_type *rtype, - /*@dependent@*/ ffi_type **atypes) -{ - unsigned bytes = 0; - unsigned int i; - ffi_type **ptr; - - FFI_ASSERT(cif != NULL); - FFI_ASSERT((abi > FFI_FIRST_ABI) && (abi <= FFI_DEFAULT_ABI)); - - cif->abi = abi; - cif->arg_types = atypes; - cif->nargs = nargs; - cif->rtype = rtype; - - cif->flags = 0; - - /* Initialize the return type if necessary */ - /*@-usedef@*/ - if ((cif->rtype->size == 0) && (initialize_aggregate(cif->rtype) != FFI_OK)) - return FFI_BAD_TYPEDEF; - /*@=usedef@*/ - - /* Perform a sanity check on the return type */ - FFI_ASSERT_VALID_TYPE(cif->rtype); - - /* x86-64 and s390 stack space allocation is handled in prep_machdep. */ -#if !defined M68K && !defined __x86_64__ && !defined S390 - /* Make space for the return structure pointer */ - if (cif->rtype->type == FFI_TYPE_STRUCT -#ifdef _WIN32 - && !can_return_struct_as_int(cif->rtype->size) /* MSVC returns small structs in registers */ - && !can_return_struct_as_sint64(cif->rtype->size) -#endif -#ifdef SPARC - && (cif->abi != FFI_V9 || cif->rtype->size > 32) -#endif - ) - bytes = STACK_ARG_SIZE(sizeof(void*)); -#endif - - for (ptr = cif->arg_types, i = cif->nargs; i > 0; i--, ptr++) - { - - /* Initialize any uninitialized aggregate type definitions */ - if (((*ptr)->size == 0) && (initialize_aggregate((*ptr)) != FFI_OK)) - return FFI_BAD_TYPEDEF; - - /* Perform a sanity check on the argument type, do this - check after the initialization. */ - FFI_ASSERT_VALID_TYPE(*ptr); - -#if !defined __x86_64__ && !defined S390 -#ifdef SPARC - if (((*ptr)->type == FFI_TYPE_STRUCT - && ((*ptr)->size > 16 || cif->abi != FFI_V9)) - || ((*ptr)->type == FFI_TYPE_LONGDOUBLE - && cif->abi != FFI_V9)) - bytes += sizeof(void*); - else -#elif defined (_WIN64) - if ((*ptr)->type == FFI_TYPE_STRUCT && - !can_return_struct_as_int((*ptr)->size) && - !can_return_struct_as_sint64((*ptr)->size)) - bytes += sizeof(void*); - else -#endif - { -#if !defined(_MSC_VER) && !defined(__MINGW32__) - /* Don't know if this is a libffi bug or not. At least on - Windows with MSVC, function call parameters are *not* - aligned in the same way as structure fields are, they are - only aligned in integer boundaries. - - This doesn't do any harm for cdecl functions and closures, - since the caller cleans up the stack, but it is wrong for - stdcall functions where the callee cleans. - */ - - /* Add any padding if necessary */ - if (((*ptr)->alignment - 1) & bytes) - bytes = ALIGN(bytes, (*ptr)->alignment); - -#endif - bytes += STACK_ARG_SIZE((*ptr)->size); - } -#endif - } - -#ifdef _WIN64 - /* Function call needs at least 40 bytes stack size, on win64 AMD64 */ - if (bytes < 40) - bytes = 40; -#endif - - cif->bytes = bytes; - - /* Perform machine dependent cif processing */ - return ffi_prep_cif_machdep(cif); -} diff --git a/Modules/_ctypes/libffi_msvc/types.c b/Modules/_ctypes/libffi_msvc/types.c deleted file mode 100644 index 4433ac28c8c8..000000000000 --- a/Modules/_ctypes/libffi_msvc/types.c +++ /dev/null @@ -1,104 +0,0 @@ -/* ----------------------------------------------------------------------- - types.c - Copyright (c) 1996, 1998 Red Hat, Inc. - - Predefined ffi_types needed by libffi. - - Permission is hereby granted, free of charge, to any person obtaining - a copy of this software and associated documentation files (the - ``Software''), to deal in the Software without restriction, including - without limitation the rights to use, copy, modify, merge, publish, - distribute, sublicense, and/or sell copies of the Software, and to - permit persons to whom the Software is furnished to do so, subject to - the following conditions: - - The above copyright notice and this permission notice shall be included - in all copies or substantial portions of the Software. - - THE SOFTWARE IS PROVIDED ``AS IS'', WITHOUT WARRANTY OF ANY KIND, EXPRESS - OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF - MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. - IN NO EVENT SHALL CYGNUS SOLUTIONS BE LIABLE FOR ANY CLAIM, DAMAGES OR - OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, - ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR - OTHER DEALINGS IN THE SOFTWARE. - ----------------------------------------------------------------------- */ - -#include <ffi.h> -#include <ffi_common.h> - -/* Type definitions */ - -#define FFI_INTEGRAL_TYPEDEF(n, s, a, t) ffi_type ffi_type_##n = { s, a, t, NULL } -#define FFI_AGGREGATE_TYPEDEF(n, e) ffi_type ffi_type_##n = { 0, 0, FFI_TYPE_STRUCT, e } - -/* Size and alignment are fake here. They must not be 0. */ -FFI_INTEGRAL_TYPEDEF(void, 1, 1, FFI_TYPE_VOID); - -FFI_INTEGRAL_TYPEDEF(uint8, 1, 1, FFI_TYPE_UINT8); -FFI_INTEGRAL_TYPEDEF(sint8, 1, 1, FFI_TYPE_SINT8); -FFI_INTEGRAL_TYPEDEF(uint16, 2, 2, FFI_TYPE_UINT16); -FFI_INTEGRAL_TYPEDEF(sint16, 2, 2, FFI_TYPE_SINT16); -FFI_INTEGRAL_TYPEDEF(uint32, 4, 4, FFI_TYPE_UINT32); -FFI_INTEGRAL_TYPEDEF(sint32, 4, 4, FFI_TYPE_SINT32); -FFI_INTEGRAL_TYPEDEF(float, 4, 4, FFI_TYPE_FLOAT); - -#if defined ALPHA || defined SPARC64 || defined X86_64 || defined S390X \ - || defined IA64 || defined _WIN64 - -FFI_INTEGRAL_TYPEDEF(pointer, 8, 8, FFI_TYPE_POINTER); - -#else - -FFI_INTEGRAL_TYPEDEF(pointer, 4, 4, FFI_TYPE_POINTER); - -#endif - -#if defined X86 || defined X86_WIN32 || defined ARM || defined M68K - -FFI_INTEGRAL_TYPEDEF(uint64, 8, 4, FFI_TYPE_UINT64); -FFI_INTEGRAL_TYPEDEF(sint64, 8, 4, FFI_TYPE_SINT64); - -#elif defined SH - -FFI_INTEGRAL_TYPEDEF(uint64, 8, 4, FFI_TYPE_UINT64); -FFI_INTEGRAL_TYPEDEF(sint64, 8, 4, FFI_TYPE_SINT64); - -#else - -FFI_INTEGRAL_TYPEDEF(uint64, 8, 8, FFI_TYPE_UINT64); -FFI_INTEGRAL_TYPEDEF(sint64, 8, 8, FFI_TYPE_SINT64); - -#endif - - -#if defined X86 || defined X86_WIN32 || defined M68K - -FFI_INTEGRAL_TYPEDEF(double, 8, 4, FFI_TYPE_DOUBLE); -FFI_INTEGRAL_TYPEDEF(longdouble, 12, 4, FFI_TYPE_LONGDOUBLE); - -#elif defined ARM || defined SH || defined POWERPC_AIX || defined POWERPC_DARWIN - -FFI_INTEGRAL_TYPEDEF(double, 8, 4, FFI_TYPE_DOUBLE); -FFI_INTEGRAL_TYPEDEF(longdouble, 8, 4, FFI_TYPE_LONGDOUBLE); - -#elif defined SPARC - -FFI_INTEGRAL_TYPEDEF(double, 8, 8, FFI_TYPE_DOUBLE); -#ifdef SPARC64 -FFI_INTEGRAL_TYPEDEF(longdouble, 16, 16, FFI_TYPE_LONGDOUBLE); -#else -FFI_INTEGRAL_TYPEDEF(longdouble, 16, 8, FFI_TYPE_LONGDOUBLE); -#endif - -#elif defined X86_64 - -FFI_INTEGRAL_TYPEDEF(double, 8, 8, FFI_TYPE_DOUBLE); -FFI_INTEGRAL_TYPEDEF(longdouble, 16, 16, FFI_TYPE_LONGDOUBLE); - -#else - -FFI_INTEGRAL_TYPEDEF(double, 8, 8, FFI_TYPE_DOUBLE); -FFI_INTEGRAL_TYPEDEF(longdouble, 8, 8, FFI_TYPE_LONGDOUBLE); - -#endif - diff --git a/Modules/_ctypes/libffi_msvc/win32.c b/Modules/_ctypes/libffi_msvc/win32.c deleted file mode 100644 index f44a5fe3697c..000000000000 --- a/Modules/_ctypes/libffi_msvc/win32.c +++ /dev/null @@ -1,162 +0,0 @@ -/* ----------------------------------------------------------------------- - win32.S - Copyright (c) 1996, 1998, 2001, 2002 Red Hat, Inc. - Copyright (c) 2001 John Beniton - Copyright (c) 2002 Ranjit Mathew - - - X86 Foreign Function Interface - - Permission is hereby granted, free of charge, to any person obtaining - a copy of this software and associated documentation files (the - ``Software''), to deal in the Software without restriction, including - without limitation the rights to use, copy, modify, merge, publish, - distribute, sublicense, and/or sell copies of the Software, and to - permit persons to whom the Software is furnished to do so, subject to - the following conditions: - - The above copyright notice and this permission notice shall be included - in all copies or substantial portions of the Software. - - THE SOFTWARE IS PROVIDED ``AS IS'', WITHOUT WARRANTY OF ANY KIND, EXPRESS - OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF - MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. - IN NO EVENT SHALL CYGNUS SOLUTIONS BE LIABLE FOR ANY CLAIM, DAMAGES OR - OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, - ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR - OTHER DEALINGS IN THE SOFTWARE. - ----------------------------------------------------------------------- */ - -/* theller: almost verbatim translation from gas syntax to MSVC inline - assembler code. */ - -/* theller: ffi_call_x86 now returns an integer - the difference of the stack - pointer before and after the function call. If everything is ok, zero is - returned. If stdcall functions are passed the wrong number of arguments, - the difference will be nonzero. */ - -#include <ffi.h> -#include <ffi_common.h> - -__declspec(naked) int -ffi_call_x86(void (* prepfunc)(char *, extended_cif *), /* 8 */ - extended_cif *ecif, /* 12 */ - unsigned bytes, /* 16 */ - unsigned flags, /* 20 */ - unsigned *rvalue, /* 24 */ - void (*fn)()) /* 28 */ -{ - _asm { - push ebp - mov ebp, esp - - push esi // NEW: this register must be preserved across function calls -// XXX SAVE ESP NOW! - mov esi, esp // save stack pointer before the call - -// Make room for all of the new args. - mov ecx, [ebp+16] - sub esp, ecx // sub esp, bytes - - mov eax, esp - -// Place all of the ffi_prep_args in position - push [ebp + 12] // ecif - push eax - call [ebp + 8] // prepfunc - -// Return stack to previous state and call the function - add esp, 8 -// FIXME: Align the stack to a 128-bit boundary to avoid -// potential performance hits. - call [ebp + 28] - -// Load ecif->cif->abi - mov ecx, [ebp + 12] - mov ecx, [ecx]ecif.cif - mov ecx, [ecx]ecif.cif.abi - - cmp ecx, FFI_STDCALL - je noclean -// STDCALL: Remove the space we pushed for the args - mov ecx, [ebp + 16] - add esp, ecx -// CDECL: Caller has already cleaned the stack -noclean: -// Check that esp has the same value as before! - sub esi, esp - -// Load %ecx with the return type code - mov ecx, [ebp + 20] - -// If the return value pointer is NULL, assume no return value. -/* - Intel asm is weird. We have to explicitly specify 'DWORD PTR' in the next instruction, - otherwise only one BYTE will be compared (instead of a DWORD)! - */ - cmp DWORD PTR [ebp + 24], 0 - jne sc_retint - -// Even if there is no space for the return value, we are -// obliged to handle floating-point values. - cmp ecx, FFI_TYPE_FLOAT - jne sc_noretval -// fstp %st(0) - fstp st(0) - - jmp sc_epilogue - -sc_retint: - cmp ecx, FFI_TYPE_INT - jne sc_retfloat -// # Load %ecx with the pointer to storage for the return value - mov ecx, [ebp + 24] - mov [ecx + 0], eax - jmp sc_epilogue - -sc_retfloat: - cmp ecx, FFI_TYPE_FLOAT - jne sc_retdouble -// Load %ecx with the pointer to storage for the return value - mov ecx, [ebp+24] -// fstps (%ecx) - fstp DWORD PTR [ecx] - jmp sc_epilogue - -sc_retdouble: - cmp ecx, FFI_TYPE_DOUBLE - jne sc_retlongdouble -// movl 24(%ebp),%ecx - mov ecx, [ebp+24] - fstp QWORD PTR [ecx] - jmp sc_epilogue - - jmp sc_retlongdouble // avoid warning about unused label -sc_retlongdouble: - cmp ecx, FFI_TYPE_LONGDOUBLE - jne sc_retint64 -// Load %ecx with the pointer to storage for the return value - mov ecx, [ebp+24] -// fstpt (%ecx) - fstp QWORD PTR [ecx] /* XXX ??? */ - jmp sc_epilogue - -sc_retint64: - cmp ecx, FFI_TYPE_SINT64 - jne sc_retstruct -// Load %ecx with the pointer to storage for the return value - mov ecx, [ebp+24] - mov [ecx+0], eax - mov [ecx+4], edx - -sc_retstruct: -// Nothing to do! - -sc_noretval: -sc_epilogue: - mov eax, esi - pop esi // NEW restore: must be preserved across function calls - mov esp, ebp - pop ebp - ret - } -} diff --git a/Modules/_ctypes/libffi_msvc/win64.asm b/Modules/_ctypes/libffi_msvc/win64.asm deleted file mode 100644 index 301188bc9c17..000000000000 --- a/Modules/_ctypes/libffi_msvc/win64.asm +++ /dev/null @@ -1,156 +0,0 @@ -PUBLIC ffi_call_AMD64 - -EXTRN __chkstk:NEAR -EXTRN ffi_closure_SYSV:NEAR - -_TEXT SEGMENT - -;;; ffi_closure_OUTER will be called with these registers set: -;;; rax points to 'closure' -;;; r11 contains a bit mask that specifies which of the -;;; first four parameters are float or double -;;; -;;; It must move the parameters passed in registers to their stack location, -;;; call ffi_closure_SYSV for the actual work, then return the result. -;;; -ffi_closure_OUTER PROC FRAME - ;; save actual arguments to their stack space. - test r11, 1 - jne first_is_float - mov QWORD PTR [rsp+8], rcx - jmp second -first_is_float: - movlpd QWORD PTR [rsp+8], xmm0 - -second: - test r11, 2 - jne second_is_float - mov QWORD PTR [rsp+16], rdx - jmp third -second_is_float: - movlpd QWORD PTR [rsp+16], xmm1 - -third: - test r11, 4 - jne third_is_float - mov QWORD PTR [rsp+24], r8 - jmp forth -third_is_float: - movlpd QWORD PTR [rsp+24], xmm2 - -forth: - test r11, 8 - jne forth_is_float - mov QWORD PTR [rsp+32], r9 - jmp done -forth_is_float: - movlpd QWORD PTR [rsp+32], xmm3 - -done: -.ALLOCSTACK 40 - sub rsp, 40 -.ENDPROLOG - mov rcx, rax ; context is first parameter - mov rdx, rsp ; stack is second parameter - add rdx, 40 ; correct our own area - mov rax, ffi_closure_SYSV - call rax ; call the real closure function - ;; Here, code is missing that handles float return values - add rsp, 40 - movd xmm0, rax ; In case the closure returned a float. - ret 0 -ffi_closure_OUTER ENDP - - -;;; ffi_call_AMD64 - -stack$ = 0 -prepfunc$ = 32 -ecif$ = 40 -bytes$ = 48 -flags$ = 56 -rvalue$ = 64 -fn$ = 72 - -ffi_call_AMD64 PROC FRAME - - mov QWORD PTR [rsp+32], r9 - mov QWORD PTR [rsp+24], r8 - mov QWORD PTR [rsp+16], rdx - mov QWORD PTR [rsp+8], rcx -.PUSHREG rbp - push rbp -.ALLOCSTACK 48 - sub rsp, 48 ; 00000030H -.SETFRAME rbp, 32 - lea rbp, QWORD PTR [rsp+32] -.ENDPROLOG - - mov eax, DWORD PTR bytes$[rbp] - add rax, 15 - and rax, -16 - call __chkstk - sub rsp, rax - lea rax, QWORD PTR [rsp+32] - mov QWORD PTR stack$[rbp], rax - - mov rdx, QWORD PTR ecif$[rbp] - mov rcx, QWORD PTR stack$[rbp] - call QWORD PTR prepfunc$[rbp] - - mov rsp, QWORD PTR stack$[rbp] - - movlpd xmm3, QWORD PTR [rsp+24] - movd r9, xmm3 - - movlpd xmm2, QWORD PTR [rsp+16] - movd r8, xmm2 - - movlpd xmm1, QWORD PTR [rsp+8] - movd rdx, xmm1 - - movlpd xmm0, QWORD PTR [rsp] - movd rcx, xmm0 - - call QWORD PTR fn$[rbp] -ret_int$: - cmp DWORD PTR flags$[rbp], 1 ; FFI_TYPE_INT - jne ret_float$ - - mov rcx, QWORD PTR rvalue$[rbp] - mov DWORD PTR [rcx], eax - jmp SHORT ret_nothing$ - -ret_float$: - cmp DWORD PTR flags$[rbp], 2 ; FFI_TYPE_FLOAT - jne SHORT ret_double$ - - mov rax, QWORD PTR rvalue$[rbp] - movlpd QWORD PTR [rax], xmm0 - jmp SHORT ret_nothing$ - -ret_double$: - cmp DWORD PTR flags$[rbp], 3 ; FFI_TYPE_DOUBLE - jne SHORT ret_int64$ - - mov rax, QWORD PTR rvalue$[rbp] - movlpd QWORD PTR [rax], xmm0 - jmp SHORT ret_nothing$ - -ret_int64$: - cmp DWORD PTR flags$[rbp], 12 ; FFI_TYPE_SINT64 - jne ret_nothing$ - - mov rcx, QWORD PTR rvalue$[rbp] - mov QWORD PTR [rcx], rax - jmp SHORT ret_nothing$ - -ret_nothing$: - xor eax, eax - - lea rsp, QWORD PTR [rbp+16] - pop rbp - ret 0 -ffi_call_AMD64 ENDP -_TEXT ENDS -END diff --git a/PC/layout/main.py b/PC/layout/main.py index 910085c01bb2..185e6498e1bc 100644 --- a/PC/layout/main.py +++ b/PC/layout/main.py @@ -52,7 +52,7 @@ EXCLUDE_FROM_COMPILE = FileNameSet("badsyntax_*", "bad_*") EXCLUDE_FROM_CATALOG = FileSuffixSet(".exe", ".pyd", ".dll") -REQUIRED_DLLS = FileStemSet("libcrypto*", "libssl*") +REQUIRED_DLLS = FileStemSet("libcrypto*", "libssl*", "libffi*") LIB2TO3_GRAMMAR_FILES = FileNameSet("Grammar.txt", "PatternGrammar.txt") diff --git a/PCbuild/_ctypes.vcxproj b/PCbuild/_ctypes.vcxproj index d4c9f87b6fe7..a265427a6568 100644 --- a/PCbuild/_ctypes.vcxproj +++ b/PCbuild/_ctypes.vcxproj @@ -70,6 +70,7 @@ <ImportGroup Label="PropertySheets"> <Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" /> <Import Project="pyproject.props" /> + <Import Project="libffi.props" /> </ImportGroup> <PropertyGroup Label="UserMacros" /> <PropertyGroup> @@ -77,7 +78,7 @@ </PropertyGroup> <ItemDefinitionGroup> <ClCompile> - <AdditionalIncludeDirectories>..\Modules\_ctypes\libffi_msvc;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories> + <PreprocessorDefinitions>FFI_BUILDING;%(PreprocessorDefinitions)</PreprocessorDefinitions> </ClCompile> <Link> <AdditionalOptions>/EXPORT:DllGetClassObject,PRIVATE /EXPORT:DllCanUnloadNow,PRIVATE %(AdditionalOptions)</AdditionalOptions> @@ -86,32 +87,14 @@ <ItemGroup> <ClInclude Include="..\Modules\_ctypes\ctypes.h" /> <ClInclude Include="..\Modules\_ctypes\ctypes_dlfcn.h" /> - <ClInclude Include="..\Modules\_ctypes\libffi_msvc\ffi.h" /> - <ClInclude Include="..\Modules\_ctypes\libffi_msvc\ffi_common.h" /> - <ClInclude Include="..\Modules\_ctypes\libffi_msvc\fficonfig.h" /> - <ClInclude Include="..\Modules\_ctypes\libffi_msvc\ffitarget.h" /> </ItemGroup> <ItemGroup> <ClCompile Include="..\Modules\_ctypes\_ctypes.c" /> <ClCompile Include="..\Modules\_ctypes\callbacks.c" /> <ClCompile Include="..\Modules\_ctypes\callproc.c" /> <ClCompile Include="..\Modules\_ctypes\cfield.c" /> - <ClCompile Include="..\Modules\_ctypes\libffi_msvc\ffi.c" /> <ClCompile Include="..\Modules\_ctypes\malloc_closure.c" /> - <ClCompile Include="..\Modules\_ctypes\libffi_msvc\prep_cif.c"> - <DisableSpecificWarnings Condition="'$(Platform)'=='x64'">4267;%(DisableSpecificWarnings)</DisableSpecificWarnings> - </ClCompile> <ClCompile Include="..\Modules\_ctypes\stgdict.c" /> - <ClCompile Include="..\Modules\_ctypes\libffi_msvc\win32.c"> - <ExcludedFromBuild Condition="'$(Platform)'=='x64'">true</ExcludedFromBuild> - </ClCompile> - </ItemGroup> - <ItemGroup> - <CustomBuild Include="..\Modules\_ctypes\libffi_msvc\win64.asm"> - <ExcludedFromBuild Condition="'$(Platform)'=='Win32'">true</ExcludedFromBuild> - <Command>ml64 /nologo /c /Zi /Fo "$(IntDir)win64.obj" "%(FullPath)"</Command> - <Outputs>$(IntDir)win64.obj;%(Outputs)</Outputs> - </CustomBuild> </ItemGroup> <ItemGroup> <ResourceCompile Include="..\PC\python_nt.rc" /> @@ -122,4 +105,4 @@ </ProjectReference> </ItemGroup> <Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" /> -</Project> \ No newline at end of file +</Project> diff --git a/PCbuild/_ctypes.vcxproj.filters b/PCbuild/_ctypes.vcxproj.filters index 83d7a7b67ab9..3123286347ae 100644 --- a/PCbuild/_ctypes.vcxproj.filters +++ b/PCbuild/_ctypes.vcxproj.filters @@ -15,18 +15,6 @@ <ClInclude Include="..\Modules\_ctypes\ctypes_dlfcn.h"> <Filter>Header Files</Filter> </ClInclude> - <ClInclude Include="..\Modules\_ctypes\libffi_msvc\ffi.h"> - <Filter>Header Files</Filter> - </ClInclude> - <ClInclude Include="..\Modules\_ctypes\libffi_msvc\ffi_common.h"> - <Filter>Header Files</Filter> - </ClInclude> - <ClInclude Include="..\Modules\_ctypes\libffi_msvc\fficonfig.h"> - <Filter>Header Files</Filter> - </ClInclude> - <ClInclude Include="..\Modules\_ctypes\libffi_msvc\ffitarget.h"> - <Filter>Header Files</Filter> - </ClInclude> </ItemGroup> <ItemGroup> <ClCompile Include="..\Modules\_ctypes\_ctypes.c"> @@ -41,25 +29,14 @@ <ClCompile Include="..\Modules\_ctypes\cfield.c"> <Filter>Source Files</Filter> </ClCompile> - <ClCompile Include="..\Modules\_ctypes\libffi_msvc\ffi.c"> - <Filter>Source Files</Filter> - </ClCompile> <ClCompile Include="..\Modules\_ctypes\malloc_closure.c"> <Filter>Source Files</Filter> </ClCompile> - <ClCompile Include="..\Modules\_ctypes\libffi_msvc\prep_cif.c"> - <Filter>Source Files</Filter> - </ClCompile> <ClCompile Include="..\Modules\_ctypes\stgdict.c"> <Filter>Source Files</Filter> </ClCompile> - <ClCompile Include="..\Modules\_ctypes\libffi_msvc\win32.c"> - <Filter>Source Files</Filter> - </ClCompile> </ItemGroup> <ItemGroup> - <CustomBuild Include="..\Modules\_ctypes\libffi_msvc\win64.asm"> - <Filter>Source Files</Filter> - </CustomBuild> + <ResourceCompile Include="..\PC\python_nt.rc" /> </ItemGroup> </Project> \ No newline at end of file diff --git a/PCbuild/get_externals.bat b/PCbuild/get_externals.bat index 887fdc941171..b82b6e6588e6 100644 --- a/PCbuild/get_externals.bat +++ b/PCbuild/get_externals.bat @@ -7,14 +7,17 @@ if NOT DEFINED EXTERNALS_DIR (set EXTERNALS_DIR=%PCBUILD%\..\externals) set DO_FETCH=true set DO_CLEAN=false +set IncludeLibffiSrc=false set IncludeTkinterSrc=false set IncludeSSLSrc=false :CheckOpts if "%~1"=="--no-tkinter" (set IncludeTkinter=false) & shift & goto CheckOpts if "%~1"=="--no-openssl" (set IncludeSSL=false) & shift & goto CheckOpts +if "%~1"=="--no-libffi" (set IncludeLibffi=false) & shift & goto CheckOpts if "%~1"=="--tkinter-src" (set IncludeTkinterSrc=true) & shift & goto CheckOpts if "%~1"=="--openssl-src" (set IncludeSSLSrc=true) & shift & goto CheckOpts +if "%~1"=="--libffi-src" (set IncludeLibffiSrc=true) & shift & goto CheckOpts if "%~1"=="--python" (set PYTHON=%2) & shift & shift & goto CheckOpts if "%~1"=="--organization" (set ORG=%2) & shift & shift & goto CheckOpts if "%~1"=="-c" (set DO_CLEAN=true) & shift & goto CheckOpts @@ -49,6 +52,7 @@ echo.Fetching external libraries... set libraries= set libraries=%libraries% bzip2-1.0.6 +if NOT "%IncludeLibffiSrc%"=="false" set libraries=%libraries% libffi-3.3.0-rc0-r1 if NOT "%IncludeSSLSrc%"=="false" set libraries=%libraries% openssl-1.1.0j set libraries=%libraries% sqlite-3.21.0.0 if NOT "%IncludeTkinterSrc%"=="false" set libraries=%libraries% tcl-core-8.6.9.0 @@ -72,6 +76,7 @@ for %%e in (%libraries%) do ( echo.Fetching external binaries... set binaries= +if NOT "%IncludeLibffi%"=="false" set binaries=%binaries% libffi if NOT "%IncludeSSL%"=="false" set binaries=%binaries% openssl-bin-1.1.0j if NOT "%IncludeTkinter%"=="false" set binaries=%binaries% tcltk-8.6.9.0 if NOT "%IncludeSSLSrc%"=="false" set binaries=%binaries% nasm-2.11.06 diff --git a/PCbuild/libffi.props b/PCbuild/libffi.props new file mode 100644 index 000000000000..975c4a0d355f --- /dev/null +++ b/PCbuild/libffi.props @@ -0,0 +1,21 @@ +<?xml version="1.0" encoding="utf-8"?> +<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003"> + <ItemDefinitionGroup> + <ClCompile> + <AdditionalIncludeDirectories>$(libffiIncludeDir);%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories> + </ClCompile> + <Link> + <AdditionalLibraryDirectories>$(libffiOutDir);%(AdditionalLibraryDirectories)</AdditionalLibraryDirectories> + <AdditionalDependencies>libffi-7.lib;%(AdditionalDependencies)</AdditionalDependencies> + </Link> + </ItemDefinitionGroup> + <ItemGroup> + <_LIBFFIDLL Include="$(libffiOutDir)\libffi-7.dll" /> + </ItemGroup> + <Target Name="_CopyLIBFFIDLL" Inputs="@(_LIBFFIDLL)" Outputs="@(_LIBFFIDLL->'$(OutDir)%(Filename)%(Extension)')" AfterTargets="Build"> + <Copy SourceFiles="@(_LIBFFIDLL)" DestinationFolder="$(OutDir)" /> + </Target> + <Target Name="_CleanLIBFFIDLL" BeforeTargets="Clean"> + <Delete Files="@(_LIBFFIDLL->'$(OutDir)%(Filename)%(Extension)')" TreatErrorsAsWarnings="true" /> + </Target> +</Project> \ No newline at end of file diff --git a/PCbuild/prepare_libffi.bat b/PCbuild/prepare_libffi.bat new file mode 100644 index 000000000000..3df85130f48a --- /dev/null +++ b/PCbuild/prepare_libffi.bat @@ -0,0 +1,169 @@ +@echo off +goto :Run + +:Usage +echo. +echo Before running prepare_libffi.bat +echo LIBFFI_SOURCE environment variable must be set to the location of +echo of python-source-deps clone of libffi branch +echo VCVARSALL must be set to location of vcvarsall.bat +echo cygwin must be installed (see below) +echo SH environment variable must be set to the location of sh.exe +echo. +echo Tested with cygwin-x86 from https://www.cygwin.com/install.html +echo Select http://mirrors.kernel.org as the download site +echo Include the following cygwin packages in cygwin configuration: +echo make, autoconf, automake, libtool, dejagnu +echo. +echo NOTE: dejagnu is only required for running tests. +echo set LIBFFI_TEST=1 to run tests (optional) +echo. +echo Based on https://github.com/libffi/libffi/blob/master/.appveyor.yml +echo. +echo. +echo.Available flags: +echo. -x64 build for x64 +echo. -x86 build for x86 +echo. -? this help +echo. --install-cygwin install cygwin to c:\cygwin +exit /b 127 + +:Run + +set BUILD_X64= +set BUILD_X86= +set INSTALL_CYGWIN= + +:CheckOpts +if "%1"=="" goto :CheckOptsDone +if /I "%1"=="-x64" (set BUILD_X64=1) & shift & goto :CheckOpts +if /I "%1"=="-x86" (set BUILD_X86=1) & shift & goto :CheckOpts +if /I "%1"=="-?" goto :Usage +if /I "%1"=="--install-cygwin" (set INSTALL_CYGWIN=1) & shift & goto :CheckOpts +goto :Usage + +:CheckOptsDone + +if NOT DEFINED BUILD_X64 if NOT DEFINED BUILD_X86 if NOT DEFINED BUILD_ARM32 ( + set BUILD_X64=1 + set BUILD_X86=1 +) + +if "%INSTALL_CYGWIN%"=="1" call :InstallCygwin + +setlocal +if NOT DEFINED SH if exist c:\cygwin\bin\sh.exe set SH=c:\cygwin\bin\sh.exe + +if NOT DEFINED VCVARSALL ( + if exist "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Auxiliary\Build\vcvarsall.bat" ( + set VCVARSALL="C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Auxiliary\Build\vcvarsall.bat" + ) +) +if ^%VCVARSALL:~0,1% NEQ ^" SET VCVARSALL="%VCVARSALL%" + +if NOT DEFINED LIBFFI_SOURCE echo.&&echo ERROR LIBFFI_SOURCE environment variable not set && goto :Usage +if NOT DEFINED SH echo ERROR SH environment variable not set && goto :Usage + +if not exist %SH% echo ERROR %SH% does not exist && goto :Usage +if not exist %LIBFFI_SOURCE% echo ERROR %LIBFFI_SOURCE% does not exist && goto :Usage + +set OLDPWD=%LIBFFI_SOURCE% +pushd %LIBFFI_SOURCE% + +%SH% --login -lc "cygcheck -dc cygwin" +set GET_MSVCC=%SH% -lc "cd $OLDPWD; export MSVCC=`/usr/bin/find $PWD -name msvcc.sh`; echo ${MSVCC};" +FOR /F "usebackq delims==" %%i IN (`%GET_MSVCC%`) do @set MSVCC=%%i + +echo. +echo VCVARSALL : %VCVARSALL% +echo SH : %SH% +echo LIBFFI_SOURCE: %LIBFFI_SOURCE% +echo MSVCC : %MSVCC% +echo. + +if not exist Makefile.in (%SH% -lc "(cd $LIBFFI_SOURCE; ./autogen.sh;)") + +call :BuildOne x86 i686-pc-cygwin i686-pc-cygwin +call :BuildOne x64 x86_64-w64-cygwin x86_64-w64-cygwin + +popd +endlocal +exit /b 0 +REM all done + + +REM this subroutine is called once for each architecture +:BuildOne + +setlocal + +REM Initialize variables +set VCVARS_PLATFORM=%1 +set BUILD=%2 +set HOST=%3 +set ASSEMBLER= +set SRC_ARCHITECTURE=x86 + +if NOT DEFINED VCVARS_PLATFORM echo ERROR bad VCVARS_PLATFORM&&exit /b 123 + +if /I "%VCVARS_PLATFORM%" EQU "x64" ( + set ARCH=amd64 + set ARTIFACTS=%LIBFFI_SOURCE%\x86_64-w64-cygwin + set ASSEMBLER=-m64 + set SRC_ARCHITECTURE=x86 +) +if /I "%VCVARS_PLATFORM%" EQU "x86" ( + set ARCH=win32 + set ARTIFACTS=%LIBFFI_SOURCE%\i686-pc-cygwin + set ASSEMBLER= + set SRC_ARCHITECTURE=x86 +) + +if NOT DEFINED LIBFFI_OUT set LIBFFI_OUT=%~dp0\..\externals\libffi +set _LIBFFI_OUT=%LIBFFI_OUT%\%ARCH% + +echo get VS build environment +call %VCVARSALL% %VCVARS_PLATFORM% + +echo clean %_LIBFFI_OUT% +if exist %_LIBFFI_OUT% (rd %_LIBFFI_OUT% /s/q) + +echo Configure the build to generate fficonfig.h and ffi.h +%SH% -lc "(cd $OLDPWD; ./configure CC='%MSVCC% %ASSEMBLER%' CXX='%MSVCC% %ASSEMBLER%' LD='link' CPP='cl -nologo -EP' CXXCPP='cl -nologo -EP' CPPFLAGS='-DFFI_BUILDING_DLL' NM='dumpbin -symbols' STRIP=':' --build=$BUILD --host=$HOST;)" + +echo Building libffi +%SH% -lc "(cd $OLDPWD; export PATH=/usr/bin:$PATH; cp src/%SRC_ARCHITECTURE%/ffitarget.h include; make; find .;)" + +REM Tests are not needed to produce artifacts +if "%LIBFFI_TEST%" EQU "1" ( + echo "Running tests..." + %SH% -lc "(cd $OLDPWD; export PATH=/usr/bin:$PATH; cp `find $PWD -name 'libffi-?.dll'` $HOST/testsuite/; make check; cat `find ./ -name libffi.log`)" +) else ( + echo "Not running tests" +) + + +echo copying files to %_LIBFFI_OUT% +if not exist %_LIBFFI_OUT%\include (md %_LIBFFI_OUT%\include) +copy %ARTIFACTS%\.libs\libffi-7.dll %_LIBFFI_OUT% +copy %ARTIFACTS%\.libs\libffi-7.lib %_LIBFFI_OUT% +copy %ARTIFACTS%\fficonfig.h %_LIBFFI_OUT%\include +copy %ARTIFACTS%\include\*.h %_LIBFFI_OUT%\include + +endlocal +exit /b + +:InstallCygwin +setlocal + +if NOT DEFINED CYG_ROOT (set CYG_ROOT=c:/cygwin) +if NOT DEFINED CYG_CACHE (set CYG_CACHE=C:/cygwin/var/cache/setup) +if NOT DEFINED CYG_MIRROR (set CYG_MIRROR=http://mirrors.kernel.org/sourceware/cygwin/) + +powershell -c "md $env:CYG_ROOT -ErrorAction SilentlyContinue" +powershell -c "$setup = $env:CYG_ROOT+'/setup.exe'; if (!(Test-Path $setup)){invoke-webrequest https://cygwin.com/setup-x86.exe -outfile $setup} +%CYG_ROOT%/setup.exe -qnNdO -R "%CYG_ROOT%" -s "%CYG_MIRROR%" -l "%CYG_CACHE%" -P make -P autoconf -P automake -P libtool -P dejagnu + +endlocal +exit /b + diff --git a/PCbuild/python.props b/PCbuild/python.props index 3a0ddceda664..52bc99e0560c 100644 --- a/PCbuild/python.props +++ b/PCbuild/python.props @@ -53,6 +53,9 @@ <sqlite3Dir>$(ExternalsDir)sqlite-3.21.0.0\</sqlite3Dir> <bz2Dir>$(ExternalsDir)bzip2-1.0.6\</bz2Dir> <lzmaDir>$(ExternalsDir)xz-5.2.2\</lzmaDir> + <libffiDir>$(ExternalsDir)libffi\</libffiDir> + <libffiOutDir>$(ExternalsDir)libffi\$(ArchName)\</libffiOutDir> + <libffiIncludeDir>$(libffiOutDir)include</libffiIncludeDir> <opensslDir>$(ExternalsDir)openssl-1.1.0j\</opensslDir> <opensslOutDir>$(ExternalsDir)openssl-bin-1.1.0j\$(ArchName)\</opensslOutDir> <opensslIncludeDir>$(opensslOutDir)include</opensslIncludeDir> diff --git a/Tools/msi/lib/lib_files.wxs b/Tools/msi/lib/lib_files.wxs index 251f9b1aeb86..472bacf20dba 100644 --- a/Tools/msi/lib/lib_files.wxs +++ b/Tools/msi/lib/lib_files.wxs @@ -22,6 +22,9 @@ <Component Id="libssl.dll" Directory="DLLs" Guid="*"> <File Name="libssl$(var.ssltag).dll" KeyPath="yes" /> </Component> + <Component Id="libffi.dll" Directory="DLLs" Guid="*"> + <File Name="libffi-7.dll" KeyPath="yes" /> + </Component> <Component Id="venvlauncher.exe" Directory="Lib_venv_scripts_nt" Guid="*"> <File Name="python.exe" Source="venvlauncher.exe" KeyPath="yes" /> </Component> @@ -59,6 +62,9 @@ <Component Id="libssl.pdb" Directory="DLLs" Guid="*"> <File Name="libssl$(var.ssltag).pdb" KeyPath="yes" /> </Component> + <Component Id="libffi.pdb" Directory="DLLs" Guid="*"> + <File Name="libffi-7.pdb" KeyPath="yes" /> + </Component> <Component Id="venvlauncher.pdb" Directory="Lib_venv_scripts_nt__pdbs" Guid="*"> <File Name="python.pdb" Source="venvlauncher.pdb" KeyPath="yes" /> </Component> [View Less]

1 0

bpo-36448: mention 'make regen-all' in error message (GH-12585)
by Steve Dower March 29, 2019

March 29, 2019

https://github.com/python/cpython/commit/3396d1e0ca858065c5bb7e5a9737be6ffc… commit: 3396d1e0ca858065c5bb7e5a9737be6ffc4028f7 branch: master author: Jeroen Demeyer <J.Demeyer(a)UGent.be> committer: Steve Dower <steve.dower(a)microsoft.com> date: 2019-03-29T11:37:22-07:00 summary: bpo-36448: mention 'make regen-all' in error message (GH-12585) files: M PCbuild/_freeze_importlib.vcxproj diff --git a/PCbuild/_freeze_importlib.vcxproj b/PCbuild/_freeze_importlib.vcxproj index … [View More]

1 0

bpo-36471: Add _Py_RunMain() (GH-12618)
by Victor Stinner March 29, 2019

March 29, 2019

https://github.com/python/cpython/commit/2f54908afc5665937d763510b4430f10cf… commit: 2f54908afc5665937d763510b4430f10cf764641 branch: master author: Victor Stinner <vstinner(a)redhat.com> committer: GitHub <noreply(a)github.com> date: 2019-03-29T15:13:46+01:00 summary: bpo-36471: Add _Py_RunMain() (GH-12618) * Add config_read_cmdline() subfunction. Remove _PyCmdline structure. * _PyCoreConfig_Read() now also parses config->argv command line arguments files: M Include/cpython/… [View More]pylifecycle.h M Lib/test/test_embed.py M Modules/main.c M Programs/_testembed.c M Python/coreconfig.c diff --git a/Include/cpython/pylifecycle.h b/Include/cpython/pylifecycle.h index e293b04e3f19..2366c774e7fa 100644 --- a/Include/cpython/pylifecycle.h +++ b/Include/cpython/pylifecycle.h @@ -41,6 +41,9 @@ PyAPI_FUNC(_PyInitError) _Py_InitializeFromWideArgs( int argc, wchar_t **argv); +PyAPI_FUNC(int) _Py_RunMain(void); + + PyAPI_FUNC(void) _Py_NO_RETURN _Py_ExitInitError(_PyInitError err); /* Py_PyAtExit is for the atexit module, Py_AtExit is for low-level diff --git a/Lib/test/test_embed.py b/Lib/test/test_embed.py index 164527a664af..cd6027205a92 100644 --- a/Lib/test/test_embed.py +++ b/Lib/test/test_embed.py @@ -53,7 +53,12 @@ def run_embedded_interpreter(self, *args, env=None): stderr=subprocess.PIPE, universal_newlines=True, env=env) - (out, err) = p.communicate() + try: + (out, err) = p.communicate() + except: + p.terminate() + p.wait() + raise if p.returncode != 0 and support.verbose: print(f"--- {cmd} failed ---") print(f"stdout:\n{out}") @@ -254,6 +259,11 @@ def test_initialize_pymain(self): self.assertEqual(out.rstrip(), "Py_Main() after Py_Initialize: sys.argv=['-c', 'arg2']") self.assertEqual(err, '') + def test_run_main(self): + out, err = self.run_embedded_interpreter("run_main") + self.assertEqual(out.rstrip(), "_Py_RunMain(): sys.argv=['-c', 'arg2']") + self.assertEqual(err, '') + class InitConfigTests(EmbeddingTestsMixin, unittest.TestCase): maxDiff = 4096 @@ -549,10 +559,11 @@ def test_init_from_config(self): 'pycache_prefix': 'conf_pycache_prefix', 'program_name': './conf_program_name', - 'argv': ['-c', 'pass'], + 'argv': ['-c', 'arg2'], 'program': 'conf_program', 'xoptions': ['core_xoption1=3', 'core_xoption2=', 'core_xoption3'], 'warnoptions': ['error::ResourceWarning', 'default::BytesWarning'], + 'run_command': 'pass\n', 'site_import': 0, 'bytes_warning': 1, diff --git a/Modules/main.c b/Modules/main.c index 766576939db1..42d2c3c2aeec 100644 --- a/Modules/main.c +++ b/Modules/main.c @@ -567,20 +567,22 @@ exit_sigint(void) } -static int -pymain_main(_PyArgv *args) +static void _Py_NO_RETURN +pymain_exit_error(_PyInitError err) { - _PyInitError err; + pymain_free(); + _Py_ExitInitError(err); +} - err = pymain_init(args); - if (_Py_INIT_FAILED(err)) { - goto exit_init_error; - } +int +_Py_RunMain(void) +{ int exitcode = 0; - err = pymain_run_python(&exitcode); + + _PyInitError err = pymain_run_python(&exitcode); if (_Py_INIT_FAILED(err)) { - goto exit_init_error; + pymain_exit_error(err); } if (Py_FinalizeEx() < 0) { @@ -596,10 +598,18 @@ pymain_main(_PyArgv *args) } return exitcode; +} -exit_init_error: - pymain_free(); - _Py_ExitInitError(err); + +static int +pymain_main(_PyArgv *args) +{ + _PyInitError err = pymain_init(args); + if (_Py_INIT_FAILED(err)) { + pymain_exit_error(err); + } + + return _Py_RunMain(); } diff --git a/Programs/_testembed.c b/Programs/_testembed.c index d8e12cf3ffe5..7d71a961602d 100644 --- a/Programs/_testembed.c +++ b/Programs/_testembed.c @@ -447,9 +447,11 @@ static int test_init_from_config(void) Py_SetProgramName(L"./globalvar"); config.program_name = L"./conf_program_name"; - static wchar_t* argv[2] = { + static wchar_t* argv[] = { + L"python3", L"-c", L"pass", + L"arg2", }; config.argv.length = Py_ARRAY_LENGTH(argv); config.argv.items = argv; @@ -528,7 +530,6 @@ static int test_init_from_config(void) config._frozen = 1; err = _Py_InitializeFromConfig(&config); - /* Don't call _PyCoreConfig_Clear() since all strings are static */ if (_Py_INIT_FAILED(err)) { _Py_ExitInitError(err); } @@ -712,6 +713,27 @@ static int test_init_dev_mode(void) } +static int test_run_main(void) +{ + _PyCoreConfig config = _PyCoreConfig_INIT; + + wchar_t *argv[] = {L"python3", L"-c", + (L"import sys; " + L"print(f'_Py_RunMain(): sys.argv={sys.argv}')"), + L"arg2"}; + config.argv.length = Py_ARRAY_LENGTH(argv); + config.argv.items = argv; + config.program_name = L"./python3"; + + _PyInitError err = _Py_InitializeFromConfig(&config); + if (_Py_INIT_FAILED(err)) { + _Py_ExitInitError(err); + } + + return _Py_RunMain(); +} + + /* ********************************************************* * List of test cases and the function that implements it. * @@ -748,6 +770,7 @@ static struct TestCase TestCases[] = { { "init_isolated", test_init_isolated }, { "preinit_isolated1", test_preinit_isolated1 }, { "preinit_isolated2", test_preinit_isolated2 }, + { "run_main", test_run_main }, { NULL, NULL } }; diff --git a/Python/coreconfig.c b/Python/coreconfig.c index 13aa2272011f..7b55b06b7a6e 100644 --- a/Python/coreconfig.c +++ b/Python/coreconfig.c @@ -1618,33 +1618,34 @@ _PyCoreConfig_Write(const _PyCoreConfig *config) } -/* --- _PyCmdline ------------------------------------------------- */ - -typedef struct { - _PyWstrList cmdline_warnoptions; /* Command line -W options */ - _PyWstrList env_warnoptions; /* PYTHONWARNINGS environment variables */ - int print_help; /* -h, -? options */ - int print_version; /* -V option */ - int need_usage; -} _PyCmdline; - +/* --- _PyCoreConfig command line parser -------------------------- */ static void -cmdline_clear(_PyCmdline *cmdline) +config_usage(int error, const wchar_t* program) { - _PyWstrList_Clear(&cmdline->cmdline_warnoptions); - _PyWstrList_Clear(&cmdline->env_warnoptions); -} + FILE *f = error ? stderr : stdout; + fprintf(f, usage_line, program); + if (error) + fprintf(f, "Try `python -h' for more information.\n"); + else { + fputs(usage_1, f); + fputs(usage_2, f); + fputs(usage_3, f); + fprintf(f, usage_4, (wint_t)DELIM); + fprintf(f, usage_5, (wint_t)DELIM, PYTHONHOMEHELP); + fputs(usage_6, f); + } +} -/* --- _PyCoreConfig command line parser -------------------------- */ /* Parse the command line arguments */ static _PyInitError -config_parse_cmdline(_PyCoreConfig *config, _PyCmdline *cmdline, - _PyPreCmdline *precmdline) +config_parse_cmdline(_PyCoreConfig *config, _PyPreCmdline *precmdline, + _PyWstrList *warnoptions) { const _PyWstrList *argv = &precmdline->argv; + int print_version = 0; _PyOS_ResetGetOpt(); do { @@ -1698,8 +1699,8 @@ config_parse_cmdline(_PyCoreConfig *config, _PyCmdline *cmdline, } else { fprintf(stderr, "--check-hash-based-pycs must be one of " "'default', 'always', or 'never'\n"); - cmdline->need_usage = 1; - return _Py_INIT_OK(); + config_usage(1, config->program); + return _Py_INIT_EXIT(2); } break; @@ -1758,15 +1759,15 @@ config_parse_cmdline(_PyCoreConfig *config, _PyCmdline *cmdline, case 'h': case '?': - cmdline->print_help++; - break; + config_usage(0, config->program); + return _Py_INIT_EXIT(0); case 'V': - cmdline->print_version++; + print_version++; break; case 'W': - if (_PyWstrList_Append(&cmdline->cmdline_warnoptions, _PyOS_optarg) < 0) { + if (_PyWstrList_Append(warnoptions, _PyOS_optarg) < 0) { return _Py_INIT_NO_MEMORY(); } break; @@ -1783,11 +1784,17 @@ config_parse_cmdline(_PyCoreConfig *config, _PyCmdline *cmdline, default: /* unknown argument: parsing failed */ - cmdline->need_usage = 1; - return _Py_INIT_OK(); + config_usage(1, config->program); + return _Py_INIT_EXIT(2); } } while (1); + if (print_version) { + printf("Python %s\n", + (print_version >= 2) ? Py_GetVersion() : PY_VERSION); + return _Py_INIT_EXIT(0); + } + if (config->run_command == NULL && config->run_module == NULL && _PyOS_optind < argv->length && wcscmp(argv->items[_PyOS_optind], L"-") != 0 @@ -1819,7 +1826,7 @@ config_parse_cmdline(_PyCoreConfig *config, _PyCmdline *cmdline, /* Get warning options from PYTHONWARNINGS environment variable. */ static _PyInitError -cmdline_init_env_warnoptions(_PyCmdline *cmdline, const _PyCoreConfig *config) +config_init_env_warnoptions(const _PyCoreConfig *config, _PyWstrList *warnoptions) { wchar_t *env; int res = _PyCoreConfig_GetEnvDup(config, &env, @@ -1838,7 +1845,7 @@ cmdline_init_env_warnoptions(_PyCmdline *cmdline, const _PyCoreConfig *config) warning != NULL; warning = WCSTOK(NULL, L",", &context)) { - if (_PyWstrList_Append(&cmdline->env_warnoptions, warning) < 0) { + if (_PyWstrList_Append(warnoptions, warning) < 0) { PyMem_RawFree(env); return _Py_INIT_NO_MEMORY(); } @@ -1883,7 +1890,9 @@ config_add_warnoption(_PyCoreConfig *config, const wchar_t *option) static _PyInitError -config_init_warnoptions(_PyCoreConfig *config, const _PyCmdline *cmdline) +config_init_warnoptions(_PyCoreConfig *config, + const _PyWstrList *cmdline_warnoptions, + const _PyWstrList *env_warnoptions) { /* The priority order for warnings configuration is (highest precedence * first): @@ -1909,14 +1918,14 @@ config_init_warnoptions(_PyCoreConfig *config, const _PyCmdline *cmdline) Py_ssize_t i; const _PyWstrList *options; - options = &cmdline->env_warnoptions; + options = env_warnoptions; for (i = 0; i < options->length; i++) { if (config_add_warnoption(config, options->items[i]) < 0) { return _Py_INIT_NO_MEMORY(); } } - options = &cmdline->cmdline_warnoptions; + options = cmdline_warnoptions; for (i = 0; i < options->length; i++) { if (config_add_warnoption(config, options->items[i]) < 0) { return _Py_INIT_NO_MEMORY(); @@ -1992,36 +2001,14 @@ config_init_argv(_PyCoreConfig *config, const _PyPreCmdline *cmdline) } -static void -config_usage(int error, const wchar_t* program) -{ - FILE *f = error ? stderr : stdout; - - fprintf(f, usage_line, program); - if (error) - fprintf(f, "Try `python -h' for more information.\n"); - else { - fputs(usage_1, f); - fputs(usage_2, f); - fputs(usage_3, f); - fprintf(f, usage_4, (wint_t)DELIM); - fprintf(f, usage_5, (wint_t)DELIM, PYTHONHOMEHELP); - fputs(usage_6, f); - } -} - - static _PyInitError core_read_precmdline(_PyCoreConfig *config, const _PyArgv *args, _PyPreCmdline *precmdline) { _PyInitError err; - if (args) { - err = _PyPreCmdline_SetArgv(precmdline, args); - if (_Py_INIT_FAILED(err)) { - return err; - } + if (_PyWstrList_Copy(&precmdline->argv, &config->argv) < 0) { + return _Py_INIT_NO_MEMORY(); } _PyPreConfig preconfig = _PyPreConfig_INIT; @@ -2040,83 +2027,88 @@ core_read_precmdline(_PyCoreConfig *config, const _PyArgv *args, } -/* Read the configuration into _PyCoreConfig from: - - * Command line arguments - * Environment variables - * Py_xxx global configuration variables */ -_PyInitError -_PyCoreConfig_Read(_PyCoreConfig *config, const _PyArgv *args) +static _PyInitError +config_read_cmdline(_PyCoreConfig *config, _PyPreCmdline *precmdline) { _PyInitError err; + _PyWstrList cmdline_warnoptions = _PyWstrList_INIT; + _PyWstrList env_warnoptions = _PyWstrList_INIT; - err = _Py_PreInitializeFromCoreConfig(config); + err = config_parse_cmdline(config, precmdline, &cmdline_warnoptions); if (_Py_INIT_FAILED(err)) { - return err; + goto done; } - _PyCoreConfig_GetGlobalConfig(config); + err = config_init_argv(config, precmdline); + if (_Py_INIT_FAILED(err)) { + goto done; + } - _PyPreCmdline precmdline = _PyPreCmdline_INIT; - err = core_read_precmdline(config, args, &precmdline); + err = config_read(config, precmdline); if (_Py_INIT_FAILED(err)) { goto done; } - if (config->program == NULL) { - err = config_init_program(config, &precmdline); + if (config->use_environment) { + err = config_init_env_warnoptions(config, &env_warnoptions); if (_Py_INIT_FAILED(err)) { goto done; } } - _PyCmdline cmdline; - memset(&cmdline, 0, sizeof(cmdline)); + err = config_init_warnoptions(config, + &cmdline_warnoptions, &env_warnoptions); + if (_Py_INIT_FAILED(err)) { + goto done; + } - if (args) { - err = config_parse_cmdline(config, &cmdline, &precmdline); - if (_Py_INIT_FAILED(err)) { - goto done; - } + err = _Py_INIT_OK(); - if (cmdline.need_usage) { - config_usage(1, config->program); - err = _Py_INIT_EXIT(2); - goto done; - } +done: + _PyWstrList_Clear(&cmdline_warnoptions); + _PyWstrList_Clear(&env_warnoptions); + return err; +} - if (cmdline.print_help) { - config_usage(0, config->program); - err = _Py_INIT_EXIT(0); - goto done; - } - if (cmdline.print_version) { - printf("Python %s\n", - (cmdline.print_version >= 2) ? Py_GetVersion() : PY_VERSION); - err = _Py_INIT_EXIT(0); - goto done; - } +/* Read the configuration into _PyCoreConfig from: + + * Command line arguments + * Environment variables + * Py_xxx global configuration variables */ +_PyInitError +_PyCoreConfig_Read(_PyCoreConfig *config, const _PyArgv *args) +{ + _PyInitError err; - err = config_init_argv(config, &precmdline); + if (args) { + err = _PyArgv_AsWstrList(args, &config->argv); if (_Py_INIT_FAILED(err)) { - goto done; + return err; } } - err = config_read(config, &precmdline); + err = _Py_PreInitializeFromCoreConfig(config); + if (_Py_INIT_FAILED(err)) { + return err; + } + + _PyCoreConfig_GetGlobalConfig(config); + + _PyPreCmdline precmdline = _PyPreCmdline_INIT; + err = core_read_precmdline(config, args, &precmdline); if (_Py_INIT_FAILED(err)) { goto done; } - if (config->use_environment) { - err = cmdline_init_env_warnoptions(&cmdline, config); + if (config->program == NULL) { + err = config_init_program(config, &precmdline); if (_Py_INIT_FAILED(err)) { goto done; } } - err = config_init_warnoptions(config, &cmdline); + err = config_read_cmdline(config, &precmdline); if (_Py_INIT_FAILED(err)) { goto done; } @@ -2176,7 +2168,6 @@ _PyCoreConfig_Read(_PyCoreConfig *config, const _PyArgv *args) err = _Py_INIT_OK(); done: - cmdline_clear(&cmdline); _PyPreCmdline_Clear(&precmdline); return err; } [View Less]

1 0

bpo-35194: cjkcodec: check the encoded value is not truncated (GH-10432)
by Inada Naoki March 29, 2019

March 29, 2019

https://github.com/python/cpython/commit/5f45979b63300338b68709bfe817ddae56… commit: 5f45979b63300338b68709bfe817ddae563b93fd branch: master author: Alexey Izbyshev <izbyshev(a)ispras.ru> committer: Inada Naoki <songofacandy(a)gmail.com> date: 2019-03-29T16:48:47+09:00 summary: bpo-35194: cjkcodec: check the encoded value is not truncated (GH-10432) files: M Modules/cjkcodecs/cjkcodecs.h diff --git a/Modules/cjkcodecs/cjkcodecs.h b/Modules/cjkcodecs/cjkcodecs.h index 2ae28ecbe207.… [View More].b67f3482faf8 100644 --- a/Modules/cjkcodecs/cjkcodecs.h +++ b/Modules/cjkcodecs/cjkcodecs.h @@ -149,40 +149,42 @@ static const struct dbcs_map *mapping_list; writer->pos += 2; \ } while (0) -#define OUTBYTE1(c) \ - do { ((*outbuf)[0]) = (c); } while (0) -#define OUTBYTE2(c) \ - do { ((*outbuf)[1]) = (c); } while (0) -#define OUTBYTE3(c) \ - do { ((*outbuf)[2]) = (c); } while (0) -#define OUTBYTE4(c) \ - do { ((*outbuf)[3]) = (c); } while (0) +#define OUTBYTEI(c, i) \ + do { \ + assert((unsigned char)(c) == (c)); \ + ((*outbuf)[i]) = (c); \ + } while (0) + +#define OUTBYTE1(c) OUTBYTEI(c, 0) +#define OUTBYTE2(c) OUTBYTEI(c, 1) +#define OUTBYTE3(c) OUTBYTEI(c, 2) +#define OUTBYTE4(c) OUTBYTEI(c, 3) #define WRITEBYTE1(c1) \ do { \ REQUIRE_OUTBUF(1); \ - (*outbuf)[0] = (c1); \ + OUTBYTE1(c1); \ } while (0) #define WRITEBYTE2(c1, c2) \ do { \ REQUIRE_OUTBUF(2); \ - (*outbuf)[0] = (c1); \ - (*outbuf)[1] = (c2); \ + OUTBYTE1(c1); \ + OUTBYTE2(c2); \ } while (0) #define WRITEBYTE3(c1, c2, c3) \ do { \ REQUIRE_OUTBUF(3); \ - (*outbuf)[0] = (c1); \ - (*outbuf)[1] = (c2); \ - (*outbuf)[2] = (c3); \ + OUTBYTE1(c1); \ + OUTBYTE2(c2); \ + OUTBYTE3(c3); \ } while (0) #define WRITEBYTE4(c1, c2, c3, c4) \ do { \ REQUIRE_OUTBUF(4); \ - (*outbuf)[0] = (c1); \ - (*outbuf)[1] = (c2); \ - (*outbuf)[2] = (c3); \ - (*outbuf)[3] = (c4); \ + OUTBYTE1(c1); \ + OUTBYTE2(c2); \ + OUTBYTE3(c3); \ + OUTBYTE4(c4); \ } while (0) #define _TRYMAP_ENC(m, assi, val) \ [View Less]

1 0

bpo-36064: Clarify allowed data types for urllib.request.Request. (GH-11990)
by Carol Willing March 29, 2019

March 29, 2019

https://github.com/python/cpython/commit/9e30fbac019d9c0a31d444a711e845c7e8… commit: 9e30fbac019d9c0a31d444a711e845c7e883caf5 branch: master author: Julien Palard <julien(a)palard.fr> committer: Carol Willing <carolcode(a)willingconsulting.com> date: 2019-03-28T19:15:34-07:00 summary: bpo-36064: Clarify allowed data types for urllib.request.Request. (GH-11990) files: M Doc/library/urllib.request.rst diff --git a/Doc/library/urllib.request.rst b/Doc/library/urllib.request.rst … [View More]

1 0

Fixed capital letters missing and missing . (GH-12584)
by Miss Islington (bot) March 29, 2019

March 29, 2019

https://github.com/python/cpython/commit/40ee9a3640d702bce127e9877c82a99ce8… commit: 40ee9a3640d702bce127e9877c82a99ce817f0d1 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: GitHub <noreply(a)github.com> date: 2019-03-28T19:11:06-07:00 summary: Fixed capital letters missing and missing . (GH-12584) No `bpo` for minor doc fix (cherry picked from commit 3d78c4a6e5ae91eaf337b6f5cc6e8bb01af7c7b1) Co-authored-by: Jules Lasne (… [View More]jlasne) <jlasne(a)student.42.fr> files: M Doc/library/unittest.rst diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst index 3e9e38c074a2..13230daf85f0 100644 --- a/Doc/library/unittest.rst +++ b/Doc/library/unittest.rst @@ -1000,7 +1000,7 @@ Test cases int('XYZ') .. versionadded:: 3.1 - under the name ``assertRaisesRegexp``. + Added under the name ``assertRaisesRegexp``. .. versionchanged:: 3.2 Renamed to :meth:`assertRaisesRegex`. @@ -1145,7 +1145,7 @@ Test cases +---------------------------------------+--------------------------------+--------------+ | :meth:`assertCountEqual(a, b) | *a* and *b* have the same | 3.2 | | <TestCase.assertCountEqual>` | elements in the same number, | | - | | regardless of their order | | + | | regardless of their order. | | +---------------------------------------+--------------------------------+--------------+ @@ -1193,7 +1193,7 @@ Test cases expression suitable for use by :func:`re.search`. .. versionadded:: 3.1 - under the name ``assertRegexpMatches``. + Added under the name ``assertRegexpMatches``. .. versionchanged:: 3.2 The method ``assertRegexpMatches()`` has been renamed to :meth:`.assertRegex`. @@ -1483,14 +1483,14 @@ along with their deprecated aliases: ============================== ====================== ======================= .. deprecated:: 3.1 - the fail* aliases listed in the second column. + The fail* aliases listed in the second column have been deprecated. .. deprecated:: 3.2 - the assert* aliases listed in the third column. + The assert* aliases listed in the third column have been deprecated. .. deprecated:: 3.2 ``assertRegexpMatches`` and ``assertRaisesRegexp`` have been renamed to :meth:`.assertRegex` and :meth:`.assertRaisesRegex`. .. deprecated:: 3.5 - the ``assertNotRegexpMatches`` name in favor of :meth:`.assertNotRegex`. + The ``assertNotRegexpMatches`` name is deprecated in favor of :meth:`.assertNotRegex`. .. _testsuite-objects: [View Less]

1 0

bpo-33043: Add a Contributing to Docs link and Update the Found a Bug Page (#12006)
by Carol Willing March 29, 2019

March 29, 2019

https://github.com/python/cpython/commit/081158e3ba20dfa95d09cd652a44e271b9… commit: 081158e3ba20dfa95d09cd652a44e271b95eb14c branch: master author: Susan Su <susansu.software(a)gmail.com> committer: Carol Willing <carolcode(a)willingconsulting.com> date: 2019-03-28T18:55:24-07:00 summary: bpo-33043: Add a Contributing to Docs link and Update the Found a Bug Page (#12006) * changes to html file -> added contributing to docs link at the end of the page * revisions to the … [View More]dealing with bugs page. added more links in the documentation bugs section * 📜🤖 Added by blurb_it. * Update Doc/bugs.rst Updated Doc/bugs.rst in accordance with willingc and JulienPalard suggestions. Co-Authored-By: suhearsawho <susansu.software(a)gmail.com> files: A Misc/NEWS.d/next/Documentation/2019-02-24-03-15-10.bpo-33043.8knWTS.rst M Doc/bugs.rst M Doc/tools/templates/indexcontent.html diff --git a/Doc/bugs.rst b/Doc/bugs.rst index c449ba2e7192..1e044ad2033d 100644 --- a/Doc/bugs.rst +++ b/Doc/bugs.rst @@ -25,7 +25,15 @@ docs(a)python.org (behavioral bugs can be sent to python-list(a)python.org). though it may take a while to be processed. .. seealso:: - `Documentation bugs`_ on the Python issue tracker + + `Documentation bugs`_ + A list of documentation bugs that have been submitted to the Python issue tracker. + + `Issue Tracking <https://devguide.python.org/tracker/>`_ + Overview of the process involved in reporting an improvement on the tracker. + + `Helping with Documentation <https://devguide.python.org/docquality/#helping-with-documentation>`_ + Comprehensive guide for individuals that are interested in contributing to Python documentation. .. _using-the-tracker: diff --git a/Doc/tools/templates/indexcontent.html b/Doc/tools/templates/indexcontent.html index d795c0a5586b..152162ab0dd2 100644 --- a/Doc/tools/templates/indexcontent.html +++ b/Doc/tools/templates/indexcontent.html @@ -57,6 +57,7 @@ <h1>{{ docstitle|e }}</h1> <table class="contentstable" align="center"><tr> <td width="50%"> <p class="biglink"><a class="biglink" href="{{ pathto("bugs") }}">{% trans %}Reporting bugs{% endtrans %}</a></p> + <p class="biglink"><a class="biglink" href="https://devguide.python.org/docquality/#helping-with-documentation">{% trans %}Contributing to Docs{% endtrans %}</a></p> <p class="biglink"><a class="biglink" href="{{ pathto("about") }}">{% trans %}About the documentation{% endtrans %}</a></p> </td><td width="50%"> <p class="biglink"><a class="biglink" href="{{ pathto("license") }}">{% trans %}History and License of Python{% endtrans %}</a></p> diff --git a/Misc/NEWS.d/next/Documentation/2019-02-24-03-15-10.bpo-33043.8knWTS.rst b/Misc/NEWS.d/next/Documentation/2019-02-24-03-15-10.bpo-33043.8knWTS.rst new file mode 100644 index 000000000000..124aa5e027ab --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2019-02-24-03-15-10.bpo-33043.8knWTS.rst @@ -0,0 +1 @@ +Updates the docs.python.org page with the addition of a 'Contributing to Docs' link at the end of the page (between 'Reporting Bugs' and 'About Documentation'). Updates the 'Found a Bug' page with additional links and information in the Documentation Bugs section. \ No newline at end of file [View Less]

1 0

bpo-30427: eliminate redundant type checks in os.path.normcase() (GH-1712)
by Miss Islington (bot) March 28, 2019

March 28, 2019

https://github.com/python/cpython/commit/74510e2a57f6d4b51ac1ab4f778cd7a4c5… commit: 74510e2a57f6d4b51ac1ab4f778cd7a4c54b541e branch: master author: Wolfgang Maier <wolfgang.maier(a)biologie.uni-freiburg.de> committer: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> date: 2019-03-28T14:47:18-07:00 summary: bpo-30427: eliminate redundant type checks in os.path.normcase() (GH-1712) https://bugs.python.org/issue30427 files: A Misc/NEWS.d/next/Library/… [View More]

1 0

bpo-36366: Return None on stopping unstarted patch object (GH-12472)
by Miss Islington (bot) March 28, 2019

March 28, 2019

https://github.com/python/cpython/commit/02b84cb1b4f5407309c81c8b1ae0397355… commit: 02b84cb1b4f5407309c81c8b1ae0397355d6e568 branch: master author: Xtreak <tir.karthi(a)gmail.com> committer: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> date: 2019-03-28T14:08:43-07:00 summary: bpo-36366: Return None on stopping unstarted patch object (GH-12472) Return None after calling unittest.mock.patch.object.stop() regardless of whether the object was started. … [View More]This makes the method idempotent. https://bugs.python.org/issue36366 files: A Misc/NEWS.d/next/Library/2019-03-20-15-13-18.bpo-36366.n0eav_.rst M Lib/unittest/mock.py M Lib/unittest/test/testmock/testpatch.py diff --git a/Lib/unittest/mock.py b/Lib/unittest/mock.py index fdde16be03a3..8684f1dfa572 100644 --- a/Lib/unittest/mock.py +++ b/Lib/unittest/mock.py @@ -1398,7 +1398,7 @@ def __enter__(self): def __exit__(self, *exc_info): """Undo the patch.""" if not _is_started(self): - raise RuntimeError('stop called on unstarted patcher') + return if self.is_local and self.temp_original is not DEFAULT: setattr(self.target, self.attribute, self.temp_original) diff --git a/Lib/unittest/test/testmock/testpatch.py b/Lib/unittest/test/testmock/testpatch.py index c484adb60508..2c14360b2df5 100644 --- a/Lib/unittest/test/testmock/testpatch.py +++ b/Lib/unittest/test/testmock/testpatch.py @@ -772,10 +772,18 @@ def test_patch_start_stop(self): def test_stop_without_start(self): + # bpo-36366: calling stop without start will return None. patcher = patch(foo_name, 'bar', 3) + self.assertIsNone(patcher.stop()) - # calling stop without start used to produce a very obscure error - self.assertRaises(RuntimeError, patcher.stop) + + def test_stop_idempotent(self): + # bpo-36366: calling stop on an already stopped patch will return None. + patcher = patch(foo_name, 'bar', 3) + + patcher.start() + patcher.stop() + self.assertIsNone(patcher.stop()) def test_patchobject_start_stop(self): diff --git a/Misc/NEWS.d/next/Library/2019-03-20-15-13-18.bpo-36366.n0eav_.rst b/Misc/NEWS.d/next/Library/2019-03-20-15-13-18.bpo-36366.n0eav_.rst new file mode 100644 index 000000000000..a43504839c6f --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-03-20-15-13-18.bpo-36366.n0eav_.rst @@ -0,0 +1,4 @@ +Calling ``stop()`` on an unstarted or stopped :func:`unittest.mock.patch` +object will now return `None` instead of raising :exc:`RuntimeError`, +making the method idempotent. +Patch byKarthikeyan Singaravelan. [View Less]

1 0