Mailman 3 August 2018 - Python-Dev

Re: [Python-Dev] Let's change to C API!
by Jeroen Demeyer Aug. 23, 2018

Aug. 23, 2018

On 2018-08-23 07:36, Antoine Pitrou wrote: > - the bootstrap problem (Cython self-compiles with CPython) This is not a big problem: we can make sure that all stdlib dependencies of Cython either have PEP 399 pure Python implementations or we keep them in pure C. > - the dependency / versioning problem (Cython is a large quick-evolving > third-party package that we can't decently vendor) Is that a real problem? You're sort of doing the same thing with pip already. > - the maintenance problem (how do ensure we can change small things in > the C API, especially semi-private ones, without having to submit PRs > to Cython as well) Why don't you want to submit PRs to Cython? If you're saying "I don't want to wait for the next stable release of Cython", you could use development versions of Cython for development versions of CPython. > - the debugging problem (Cython's generated C code is unreadable, > unlike Argument Clinic's, which can make debugging annoying) Luckily, you don't need to read the C code most of the time. And it's also a matter of experience: I can read Cython-generated C code just fine. Jeroen.

3 2

Issue?
by 楼晓峰 Aug. 22, 2018

Aug. 22, 2018

Why compare twice? class Student(object): def __init__(self, name, score): self.name = name self.score = score def __str__(self): return '(%s: %s)' % (self.name, self.score) __repr__ = __str__ def __lt__(self, s): #print(self, '--', s) if(self.score<s.score): print(self, '<', s) return True if(self.score>s.score): print(self, '>', s) return False if(self.score==s.score): if(self.name>s.name): print(self, '>', s) return False if(self.name<s.name): print(self, '<', s) return True if(self.name==s.name): print(self, '==', s) return False def __eq__(self, s): return (self.score == s.score) and (self.name == s.name) def __gt__(self, s): return not ((self == s) or (self < s)) def __le__(self, s): return ((self == s) or (self < s)) def __ge__(self, s): return ((self == s) or (self > s)) def __nq__(self, s): return not (self == s) L = [Student('Tim', 22), Student('Bob', 33), Student('Kevin', 11), Student('Alice', 11)] print(sorted(L)) Output: (Bob: 33) > (Tim: 22) (Kevin: 11) < (Bob: 33) (Kevin: 11) < (Bob: 33) (Kevin: 11) < (Tim: 22) (Alice: 11) < (Tim: 22) (Alice: 11) < (Kevin: 11) [(Alice: 11), (Kevin: 11), (Tim: 22), (Bob: 33)] Best regards, Xiaofeng

4 3

Re: [Python-Dev] Starting to use gcc-8 on upstream Python project CI
by Miro Hrončok Aug. 22, 2018

Aug. 22, 2018

On 20.8.2018 20:02, Jun Aruga wrote: > Dear Python sig. > > Someone can you help to promote for the upstream Python project to use > gcc-8 on the Travis CI test? > Right now the project has 4 test cases [1] including defaut gcc > version 4.8 cases on Travis CI. > > However technically it is possible to use gcc-N (4.8, 5, 6, 7, 8, and > etc) on Travis CI. > I think that using the latest version gcc-8 on the upstream project is > quite beneficial for us. > Because maybe we Fedora people are working to fix new version gcc's issues. > When the python project start to use gcc-8, it is easy to share the > situation publicly outside of Fedora, and of course they can help to > check the issues. > As I checked the Python project's .travis.yml, I had no idea about how > to add gcc-8 case. ;( > > I can show you 2 cases to use the technique as an example. [2][3] > > > [1] Python > https://travis-ci.org/python/cpython > https://github.com/python/cpython/blob/master/.travis.yml > > [2] Ruby > https://travis-ci.org/junaruga/ruby/builds/418242410 > https://github.com/junaruga/ruby/blob/feature/ci-new-gcc/.travis.yml > https://github.com/ruby/ruby/pull/1937 > > [3] A project I am working as a hobby. > https://travis-ci.org/trinityrnaseq/trinityrnaseq > https://github.com/trinityrnaseq/trinityrnaseq/blob/master/.travis.yml I'm taking this to python-dev(a)python.org which is more appropriate place to discuss this. I think Victor is involved in the CIs, is that right? -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

4 3

Winding down 3.4
by Larry Hastings Aug. 21, 2018

Aug. 21, 2018

We of the core dev community commit to supporting Python releases for five years. Releases get eighteen months of active bug fixes, followed by three and a half years of security fixes. Python 3.4 turns 5 next March--at which point we'll stop supporting it, and I'll retire as 3.4 release manager. My plan is to make one final release on or around its fifth birthday containing the last round of security fixes. That's about seven months from now. Nothing has been merged since the releases of 3.4.9 and 3.5.6 last week, and there are no open PRs against either of those releases. But! There are still a couple languishing "critical" bugs: "shutil copy* unsafe on POSIX - they preserve setuid/setgit bits" https://bugs.python.org/issue17180 "XML vulnerabilities in Python" https://bugs.python.org/issue17239 "fflush called on pointer to potentially closed file" (Windows only) https://bugs.python.org/issue19050 It'd be nice to resolve all those issues, one way or another, before we retire 3.4. See you next March, //arry/

5 5

unsolicited removal request
by Abdur-Rahmaan Janhangeer Aug. 20, 2018

Aug. 20, 2018

just notifying that 199.103.2.101 was trying to remove me from this list. might try to remove others yours, Abdur-Rahmaan Janhangeer https://github.com/Abdur-rahmaanJ Mauritius

4 5

improvement on shutil.make_archive
by Zhao Lee Aug. 20, 2018

Aug. 20, 2018

The current behavior of shutil.make_archive caused many issues , the problem is mainly on the extracted archive directory hierarchy. These are the proofs: https://stackoverflow.com/questions/51914467/directory-hierarchy-issue-when… https://stackoverflow.com/questions/32640053/compressing-directory-using-sh… https://stackoverflow.com/questions/41624800/shutil-make-archive-issue-dont… https://stackoverflow.com/questions/50156657/unexpected-file-using-shutil-m… For example , if I want to create a zip archive of the pip package (path specified by pip.__path__[0])), and need a directory named pip to hold the files and folders which originally reside in the pip package when unpacking the archive, then I need give root_dirparameter of shutil.make_archive the parent directory of the pippackage path (root_dir=Path(pip.__path__[0]).parent), and then the base_dir parameter the final path component of the pip package path(base_dir=Path(pip.__path__[0]).name) , so it is os.path.join(root_dir, base_dir) that specified the directory to archive , so weird !!! I suggest to change shutil.make_archive(base_name, format[, root_dir[, base_dir]]) to shutil.make_archive(base_name, format[, archived_dir[, archive_prfix]]) where archived_dirdenotes the path to be archived and archive_prfix denotes the common prefix of all files and directories in the archive (it is just a path component and we shouldn’t assume the existence of it on the file system). If the current behavior of shutil.make_archive won’t be changed , I’d suggest improve its doc, because so many people couldn’t grasp the use of shutil.make_archive even consulting the doc , these are the proofs: https://stackoverflow.com/questions/45245079/python-how-to-use-shutil-make-… https://stackoverflow.com/questions/30049201/how-to-compress-a-file-with-sh…

3 2

Summary of Python tracker Issues
by Python tracker Aug. 17, 2018

Aug. 17, 2018

ACTIVITY SUMMARY (2018-08-10 - 2018-08-17) Python tracker at https://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 6775 (+19) closed 39396 (+29) total 46171 (+48) Open issues with patches: 2697 Issues opened (31) ================== #34217: windows: cross compilation fails due to headers with uppercase https://bugs.python.org/issue34217 reopened by erikjanss #34373: test_time errors on AIX https://bugs.python.org/issue34373 opened by Michael.Felt #34374: Shouldn't shutil.copyfile replace link in dst with follow_syml https://bugs.python.org/issue34374 opened by theger #34381: Make tests with outbound connection optional https://bugs.python.org/issue34381 opened by michael-o #34382: test_os.test_mode fails when directory base directory has g+s https://bugs.python.org/issue34382 opened by Michael.Felt #34383: asyncio passes SSL certificate error to loop.call_exception_ha https://bugs.python.org/issue34383 opened by dandrei #34384: os.readlink does not accept pathlib.Path on Windows https://bugs.python.org/issue34384 opened by girtsf #34388: collect_gdb fails for test.pythoninfo in several AMD64 FreeBSD https://bugs.python.org/issue34388 opened by pablogsal #34389: CPython may fail to build in the presence of a ~/.pydistutils. https://bugs.python.org/issue34389 opened by Antony.Lee #34391: test_ftplib is failing with TLS 1.3 https://bugs.python.org/issue34391 opened by christian.heimes #34392: Add sys.isinterned() https://bugs.python.org/issue34392 opened by serhiy.storchaka #34394: Descriptors HowTo doesn't mention __set_name__ https://bugs.python.org/issue34394 opened by MarSoft #34395: memory leaks in error handling in csv and pickle modules https://bugs.python.org/issue34395 opened by sir-sigurd #34396: Certain methods that heap allocated subtypes inherit suffer a https://bugs.python.org/issue34396 opened by bup #34397: remove redundant overflow checks in tuple and list implementat https://bugs.python.org/issue34397 opened by sir-sigurd #34398: Docs search does not index glossary items https://bugs.python.org/issue34398 opened by jfine2358 #34401: [SOLUTION] Make test_gdb work on HP-UX https://bugs.python.org/issue34401 opened by michael-o #34402: [SOLUTION] strftime fails on HP-UX https://bugs.python.org/issue34402 opened by michael-o #34403: test_utf8_mode.test_cmd_line() fails on HP-UX due to false ass https://bugs.python.org/issue34403 opened by michael-o #34404: test_time incorrectly defined https://bugs.python.org/issue34404 opened by michael-o #34405: Upgrade to OpenSSL 1.1.0i / 1.0.2p https://bugs.python.org/issue34405 opened by christian.heimes #34406: Typo in documentation https://bugs.python.org/issue34406 opened by Alisue Lambda #34407: datetime.time.isoformat function has inconsistent behavior wit https://bugs.python.org/issue34407 opened by Maksym Shalenyi (Enkidulan) #34408: possible null pointer dereference in pystate.c https://bugs.python.org/issue34408 opened by pablogsal #34409: Add a way to customize iteration over fields in asdict() for t https://bugs.python.org/issue34409 opened by mkurnikov #34410: Segfault/TimeoutError: itertools.tee of multiprocessing.pool.i https://bugs.python.org/issue34410 opened by carlorosati #34411: ProactorEventLoop should use implement GetAddrInfo, GetNameInf https://bugs.python.org/issue34411 opened by ysangkok+launchpad #34412: strsignal(3) does not exist on HP-UX https://bugs.python.org/issue34412 opened by michael-o #34413: Porting section of 3.5 "What's New" doc does not mention bytec https://bugs.python.org/issue34413 opened by eric.snow #34415: Typo in logging.Formatter docstring https://bugs.python.org/issue34415 opened by MarSoft #34416: PyCapsule_Import seems to release the GIL without acquiring it https://bugs.python.org/issue34416 opened by p-ganssle Most recent 15 issues with no replies (15) ========================================== #34415: Typo in logging.Formatter docstring https://bugs.python.org/issue34415 #34408: possible null pointer dereference in pystate.c https://bugs.python.org/issue34408 #34407: datetime.time.isoformat function has inconsistent behavior wit https://bugs.python.org/issue34407 #34403: test_utf8_mode.test_cmd_line() fails on HP-UX due to false ass https://bugs.python.org/issue34403 #34401: [SOLUTION] Make test_gdb work on HP-UX https://bugs.python.org/issue34401 #34389: CPython may fail to build in the presence of a ~/.pydistutils. https://bugs.python.org/issue34389 #34383: asyncio passes SSL certificate error to loop.call_exception_ha https://bugs.python.org/issue34383 #34382: test_os.test_mode fails when directory base directory has g+s https://bugs.python.org/issue34382 #34373: test_time errors on AIX https://bugs.python.org/issue34373 #34368: ftplib __init__ function can't handle 120 or 4xy reply when co https://bugs.python.org/issue34368 #34367: AsyncResult.get() only notifies one thread https://bugs.python.org/issue34367 #34366: _uuid module fails to compile on FreeBSD when libuuid is insta https://bugs.python.org/issue34366 #34365: datetime's documentation refers to "comparison [...] falling b https://bugs.python.org/issue34365 #34354: Memory leak on _testCongestion https://bugs.python.org/issue34354 #34345: Add tests for PEP 468 and PEP 520 https://bugs.python.org/issue34345 Most recent 15 issues waiting for review (15) ============================================= #34412: strsignal(3) does not exist on HP-UX https://bugs.python.org/issue34412 #34408: possible null pointer dereference in pystate.c https://bugs.python.org/issue34408 #34405: Upgrade to OpenSSL 1.1.0i / 1.0.2p https://bugs.python.org/issue34405 #34401: [SOLUTION] Make test_gdb work on HP-UX https://bugs.python.org/issue34401 #34398: Docs search does not index glossary items https://bugs.python.org/issue34398 #34397: remove redundant overflow checks in tuple and list implementat https://bugs.python.org/issue34397 #34395: memory leaks in error handling in csv and pickle modules https://bugs.python.org/issue34395 #34392: Add sys.isinterned() https://bugs.python.org/issue34392 #34391: test_ftplib is failing with TLS 1.3 https://bugs.python.org/issue34391 #34384: os.readlink does not accept pathlib.Path on Windows https://bugs.python.org/issue34384 #34382: test_os.test_mode fails when directory base directory has g+s https://bugs.python.org/issue34382 #34381: Make tests with outbound connection optional https://bugs.python.org/issue34381 #34373: test_time errors on AIX https://bugs.python.org/issue34373 #34366: _uuid module fails to compile on FreeBSD when libuuid is insta https://bugs.python.org/issue34366 #34363: dataclasses.asdict() mishandles dataclass instance attributes https://bugs.python.org/issue34363 Top 10 most discussed issues (10) ================================= #34217: windows: cross compilation fails due to headers with uppercase https://bugs.python.org/issue34217 13 msgs #34398: Docs search does not index glossary items https://bugs.python.org/issue34398 7 msgs #34410: Segfault/TimeoutError: itertools.tee of multiprocessing.pool.i https://bugs.python.org/issue34410 7 msgs #17180: shutil copy* unsafe on POSIX - they preserve setuid/setgit bit https://bugs.python.org/issue17180 6 msgs #34384: os.readlink does not accept pathlib.Path on Windows https://bugs.python.org/issue34384 6 msgs #34395: memory leaks in error handling in csv and pickle modules https://bugs.python.org/issue34395 6 msgs #34363: dataclasses.asdict() mishandles dataclass instance attributes https://bugs.python.org/issue34363 5 msgs #34391: test_ftplib is failing with TLS 1.3 https://bugs.python.org/issue34391 5 msgs #34411: ProactorEventLoop should use implement GetAddrInfo, GetNameInf https://bugs.python.org/issue34411 5 msgs #34369: kqueue.control() documentation and implementation mismatch https://bugs.python.org/issue34369 4 msgs Issues closed (29) ================== #8478: tokenize.untokenize first token missing failure case https://bugs.python.org/issue8478 closed by berker.peksag #9372: pulldom.DOMEventStream.__getitem__ is broken https://bugs.python.org/issue9372 closed by berker.peksag #13837: test_shutil fails with symlinks enabled under Windows https://bugs.python.org/issue13837 closed by berker.peksag #15258: argparse documentation: Improve optparse section regarding all https://bugs.python.org/issue15258 closed by berker.peksag #24111: Valgrind suppression file should be updated https://bugs.python.org/issue24111 closed by benjamin.peterson #26818: trace CLI doesn't respect -s option https://bugs.python.org/issue26818 closed by berker.peksag #27026: async/await keywords are missing from reference docs https://bugs.python.org/issue27026 closed by berker.peksag #33451: Start pyc file lock the file https://bugs.python.org/issue33451 closed by berker.peksag #34151: use malloc() for better performance of some list operations https://bugs.python.org/issue34151 closed by xiang.zhang #34333: Path.with_suffix() raises TypeError when doing %-formatting https://bugs.python.org/issue34333 closed by berker.peksag #34356: Add support for args and kwargs in logging.conf https://bugs.python.org/issue34356 closed by vinay.sajip #34357: situation where urllib3 works, but urllib does not work https://bugs.python.org/issue34357 closed by deivid #34375: Subtests (unittest) https://bugs.python.org/issue34375 closed by r.david.murray #34376: Improve accuracy of math.hypot() and math.dist() https://bugs.python.org/issue34376 closed by rhettinger #34377: Update valgrind suppressions https://bugs.python.org/issue34377 closed by benjamin.peterson #34378: Documentation 3.5+ https://bugs.python.org/issue34378 closed by Mariatta #34379: Move note about repeated calls to json.dump using the same fp https://bugs.python.org/issue34379 closed by inada.naoki #34380: Relax Restrictions on await? https://bugs.python.org/issue34380 closed by yselivanov #34385: Failed to build these modules: _ctypes on Solaris 10 https://bugs.python.org/issue34385 closed by LarBob Doomer #34386: Expose a dictionary of interned strings in sys module https://bugs.python.org/issue34386 closed by serhiy.storchaka #34387: Deletion of attributes in dataclass is buggy https://bugs.python.org/issue34387 closed by eric.smith #34390: arparse.ArgumentParser misparses list arguments followed by un https://bugs.python.org/issue34390 closed by rhettinger #34393: json.dumps - allow compression https://bugs.python.org/issue34393 closed by eric.smith #34399: [TLS] Update test keys to >= 2048bit https://bugs.python.org/issue34399 closed by christian.heimes #34400: Undefined behavior in Parser/parsetok.c https://bugs.python.org/issue34400 closed by benjamin.peterson #34414: Absolute imports conflict with local files https://bugs.python.org/issue34414 closed by ronaldoussoren #34418: Arguments missing from documentation for HTTPErrorProcessor.ht https://bugs.python.org/issue34418 closed by berker.peksag #34419: selectmodule.c does not compile on HP-UX due to bpo-31938/6dc5 https://bugs.python.org/issue34419 closed by taleinat #34420: Complete your registration to Python tracker -- keysnSwaZe6Pti https://bugs.python.org/issue34420 closed by berker.peksag

1 0

TLS 1.3 support
by Christian Heimes Aug. 16, 2018

Aug. 16, 2018

Hi, I have some exciting news. RFC 8446 [1] for TLS 1.3 was finalized a couple of days ago. The new TLS standard comes with several major improvements, but also with major changes. First some good news: Python's ssl module not compiles with OpenSSL 1.1.1-pre. It's also able to negotiate and establish TLS 1.3 connections successfully. Almost all unit tests are passing, too. Yesterday I pushed some patches to fix 2.7, 3.6, 3.7, and master. Of course, there is a catch. While TLS 1.0 to 1.2 were just gradual improvements over SSLv3, TLS 1.3 behaves differently on several levels. The article [2] from Nick Sullivan explains the biggest changes well. Most improvements like 1-RTT handshake, enforced perfect forward secrecy and improved cryptography are not visible to applications. However some changes and new features are going to need additional work and new APIs. I'm open to discuss the matter at the core dev sprints in Seattle next month. I'm still in the process of reviewing and assessing changes. OpenSSL 1.1.1's APIs are not finalized yet, too. I'm working closely with Red Hat's crypto team and OpenSSL devs on the topic. I'll walk you through some changes and possibly new APIs that I have identified so far. New cipher suites ================= Old ciphers suites in TLS 1.2 and earlier looked like TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 The parameters are key agreement/key exchange (ECDHE), authentication (RSA), bulk encryption (AES-256) with block and/or authenticated encryption mode (GCM) and finally a pseudo random function for MAC. In TLS 1.3, the cipher suites only specify bulk encryption and PRF. The parameters for key agreement and authentication are configured differently. TLS 1.3 uses dedicated TLS extensions for that, too. This allows more fine grained control to specific supported EC curves, signature algorithms (RSA PKCS#1.5, RSASSA-PSS) and so on. Some settings also apply to TLS 1.2 already. Further more, OpenSSL has introduced new APIs to configure TLS 1.3 cipher suites. SSLContext.set_ciphers() cannot be used to disable TLS 1.3 suites. For now, that is fine. All default algorithms are secure and state-of-the-art. For 3.8, I'm planning to make some of the settings configurable. I might need to backport the new APIs to 3.7 and 3.6, though. post handshake authentication ============================= TLS 1.3 introduced the concept of post handshake authentication (PHA). A server can either request a client to provide client cert authentication during the handshake, directly after the handshake, or at any later point. It makes optional authentication for HTTP much more sane. For example a server may only require authentication for POST requests and GET requests to /secure/* path. In TLS 1.2, optional client cert auth requires a complete renegotiation of the handshake. OpenSSL 1.1.1-pre9 will most likely disable PHA by default [3], because PHA breaks some assumptions of TLS 1.2 clients and servers. But PHA might be required for path specific authentication. I need to wait and see how mod_ssl implements the feature. I might have to add SSL_VERIFY_POST_HANDSHAKE, SSL_verify_client_post_handshake(), and SSL_CTX_set_post_handshake_auth() from [3][4] to Python 2.7, 3.6, 3.7 and master. session handling ================ TLS 1.3 uses a different approach to handle sessions. Simple speaking, session data is exchanged after the handshake, there can be more than one ticket, tickets must not be reused, and session resumption involved DHE. ssl.SSLSession [5] feature only applies to TLS 1.2 and earlier. I need to come up with a different approach here. For the time being, TLS 1.3 won't be compatible with manual session control. It's not a big deal since it is an advanced feature any way. The new session system also seems to break FTP over TLS. FTP uses two different TCP connections (control channel, data channel). For FTP over TLS, the data channel must reuse a session of the control channel. For now, I'm plannung to restrict FTP to TLSv1.2. 0-RTT resumption ================= 0 round trip time handshake on session resumption is an optional and advanced feature. I might expose the feature in Python 3.8. Regards, Christian [1] https://tools.ietf.org/html/rfc8446 [2] https://blog.cloudflare.com/rfc-8446-aka-tls-1-3/ [3] https://github.com/openssl/openssl/issues/6933 [4] https://www.openssl.org/docs/man1.1.1/man3/SSL_CTX_set_verify.html [5] https://docs.python.org/3/library/ssl.html#ssl-session

3 3

Re: [Python-Dev] [issue17180] shutil copy* unsafe on POSIX - they preserve setuid/setgit bits
by Michael Felt Aug. 16, 2018

Aug. 16, 2018

I want to believe this can be resolved - without breakage on POSIX. Clarification: while Mac/OS falls under "posix" in python terms - maybe "breakage" will need to be accepted, or, for "back-ports" Mac/OS will be "as if root or super-user" and use an additional (optional) argument in 3.8 and beyond to keep backwards compatibility. Short text: to proceed I think we should start with getting some additional tests into test_shutil.py asap so that we can see how systems respond without any changes. My experience is that cp -r[pP] behaves the same as shutil.copy*() when the EUID==0, aka superuser, but strips special bits from files and cannot copy the UID/GID owner bits of the inode. I would appreciate someone helping me writing more extensive testing. We need to test: * root * not-root, but owner * not-root, not owner, but in group * not-root, "other", other "read" access exists * if the test does not already exist - also check behavior when directories have/do not have "search" (x-bit) enabled. I am working on a patch to address these different conditions. Ideally, the "not-owner" and "not-group" tests can be run by creating the "copy.me" area as root, setting perms, etc. and then using su -c to run the shutil.copy*() call and back as root make the verification. ±±± Perspective ±±± If this is too much discussion, please reply with suggestions - privately - on what I could do better to not waste your time. The issue seems unchanged since original posting. The original report states: hen copying the mode of a file with copy, copy2, copymode, copystat or copytree, all permission bits are copied (including setuid and setgit), but the owner of the file is not. This can be used for privilege escalation. ...snip... The behaviour of copymode/copystat in this case is the same as `chmod --reference', and there can be some expectation of unsafety, but copy/copy2/copytree's behaviour differs from that of `cp -p', and this is a non-obvious difference. For clarity: GNU chmod states: --reference=RFILE use RFILE's mode instead of MODE values Additionally, the chmod man page reminds us the "special bit" masking behavior is different for files and directries. Specifically, SUID, SGID and SVTX should not be cleared unless specifically requested by a chmod "u-s,g-s" specification. "... a directory's unmentioned set user and group ID bits are not affected" Additional comments discuss: short window of opportunity (files are copied first, then mode bits copied) breakage with the past (copytree used as "backup", regardless of version) And the comment/opinion that shutil.copy() should emulate cp (implies emulate "cp -r", so neither -p nor -P) it seems shutil.copy2() is adding the -p (or -P if follow_symlinks=false) There was a modification to test_shutil.py suggested as part of a patch. I added that to verify the issue is still current. ±±± diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py index 7e0a3292e0..7ceefd1ebc 100644 --- a/Lib/test/test_shutil.py +++ b/Lib/test/test_shutil.py @@ -1471,6 +1471,24 @@ class TestShutil(unittest.TestCase): rv = shutil.copytree(src_dir, dst_dir) self.assertEqual(['foo'], os.listdir(rv)) + @unittest.skipUnless((os.name == "posix" and os.geteuid() != 0), "Requires POSIX compatible OS and non-root userid") + def test_copy_remove_setuid(self): + src_dir = self.mkdtemp() + src_file = os.path.join(src_dir, 'foo') + write_file(src_file, 'foo') + dst_file = os.path.join(src_dir, 'bar') + harmful_mode = stat.S_IRUSR | stat.S_IXUSR | stat.S_ISUID + harmless_mode = stat.S_IRUSR | stat.S_IXUSR + + # set mode and verify + os.chmod(src_file, harmful_mode) + mode = stat.S_IMODE(os.stat(src_file).st_mode) + self.assertTrue(oct(mode), oct(harmful_mode)) + + # check that copy does not preserve harmful bits + shutil.copy(src_file, dst_file) + mode = stat.S_IMODE(os.stat(dst_file).st_mode) + self.assertEqual(oct(mode), oct(harmless_mode)) class TestWhich(unittest.TestCase): ±±± The result is: root@x066:[/data/prj/python/python3-3.8]./python -m test -v test_shutil == CPython 3.8.0a0 (heads/master:cca4eec3c0, Aug 13 2018, 04:53:15) [C] == AIX-1-00C291F54C00-powerpc-32bit big-endian == cwd: /data/prj/python/python3-3.8/build/test_python_10944516 == CPU count: 8 == encodings: locale=ISO8859-1, FS=iso8859-1 Run tests sequentially ... test_copy_remove_setuid (test.test_shutil.TestShutil) ... FAIL ... ====================================================================== FAIL: test_copy_remove_setuid (test.test_shutil.TestShutil) ---------------------------------------------------------------------- Traceback (most recent call last): File "/data/prj/python/git/python3-3.8/Lib/test/test_shutil.py", line 1491, in test_copy_remove_setuid self.assertEqual(oct(mode), oct(harmless_mode)) AssertionError: '0o4500' != '0o500' - 0o4500 ? - + 0o500 ---------------------------------------------------------------------- On 8/15/2018 1:01 PM, Michael Felt wrote: > Michael Felt <aixtools(a)felt.demon.nl> added the comment: > > I am looking at this.

1 0

Pre- and post-conditions
by Marko Ristin-Kaufmann Aug. 15, 2018

Aug. 15, 2018

Hi python devs, I would be very interested to bring design-by-contract into python 3. I looked at some of the packages I found on pypi and also we rolled our own solution (https://github.com/Parquery/icontract/). I also looked into https://www.python.org/dev/peps/pep-0316/. However, all these solutions seem quite clunky to me. The decorators involve an unnecessary computational overhead and the implementation of icontract became quite tricky once we wanted to get the default values of the decorated function. What do you think about the following solution as an extension to python compiler / interpreter? * We specify pre- and post-conditions in the docstring. * Python interpreter parses these conditions and adapts the corresponding byte codes at the begining and the end of the function body. I'm very grateful for any feedback on this! Marko

2 2