Mailman 3 September 2016 - Python-Dev

Possibly inconsistent behavior in re groupdict
by Gordon R. Burgess Sept. 28, 2016

Sept. 28, 2016

I've been lurking for a couple of months, working up the confidence to ask the list about this behavior - I've searched through the PEPs but couldn't find any specific reference to it. In a nutshell, in the Python 3.5 library re patterns and search buffers both need to be either unicode or byte strings - but the keys in the groupdict are always returned as str in either case. I don't know whether or not this is by design, but it would make more sense to me if when searching a bytes object with a bytes pattern the keys returned in the groupdict were bytes as well. I reworked the example a little just now so it would run it on 2.7 as well; on 2.7 the keys in the dictionary correspond to the mode of the pattern as expected (and bytes and unicode are interconverted silently) - code and output are inline below. Thanks for your time, Gordon [Code] import sys import re from datetime import datetime data = (u"first string (unicode)", b"second string (bytes)") pattern = [re.compile(u"(?P<ordinal>\\w+) .*\$(?P<type>\\w+)\$"), re.compile(b"(?P<ordinal>\\w+) .*\$(?P<type>\\w+)\$")] print("*** re consistency check ***\nRun: %s\nVersion: Python %s\n" % (datetime.now(), sys.version)) for p in pattern: for d in data: try: result = "groupdict: %s" % (p.match(d) and p.match(d).groupdict()) except Exception as e: result = "error: %s" % e.args[0] print("mode: %s\npattern: %s\ndata: %s\n%s\n" % (type(p.pattern).__name__, p.pattern, d, result)) [Output] gordon@w540:~/workspace/regex_demo$ python3 regex_demo.py *** re consistency check *** Run: 2016-09-25 20:06:29.472332 Version: Python 3.5.2+ (default, Sep 10 2016, 10:24:58) [GCC 6.2.0 20160901] mode: str pattern: (?P<ordinal>\w+) .*$(?P<type>\w+)$ data: first string (unicode) groupdict: {'ordinal': 'first', 'type': 'unicode'} mode: str pattern: (?P<ordinal>\w+) .*$(?P<type>\w+)$ data: b'second string (bytes)' error: cannot use a string pattern on a bytes-like object mode: bytes pattern: b'(?P<ordinal>\\w+) .*\$(?P<type>\\w+)\$' data: first string (unicode) error: cannot use a bytes pattern on a string-like object mode: bytes pattern: b'(?P<ordinal>\\w+) .*\$(?P<type>\\w+)\$' data: b'second string (bytes)' groupdict: {'ordinal': b'second', 'type': b'bytes'} gordon@w540:~/workspace/regex_demo$ python regex_demo.py *** re consistency check *** Run: 2016-09-25 20:06:23.375322 Version: Python 2.7.12+ (default, Sep 1 2016, 20:27:38) [GCC 6.2.0 20160822] mode: unicode pattern: (?P<ordinal>\w+) .*$(?P<type>\w+)$ data: first string (unicode) groupdict: {u'ordinal': u'first', u'type': u'unicode'} mode: unicode pattern: (?P<ordinal>\w+) .*$(?P<type>\w+)$ data: second string (bytes) groupdict: {u'ordinal': 'second', u'type': 'bytes'} mode: str pattern: (?P<ordinal>\w+) .*$(?P<type>\w+)$ data: first string (unicode) groupdict: {'ordinal': u'first', 'type': u'unicode'} mode: str pattern: (?P<ordinal>\w+) .*$(?P<type>\w+)$ data: second string (bytes) groupdict: {'ordinal': 'second', 'type': 'bytes'}

2 2

3.6.0 Beta Phase Development
by Ned Deily Sept. 27, 2016

Sept. 27, 2016

Wow! What a busy and productive couple of weeks it has been leading up to 3.6.0b1 and feature code freeze! Congratulations and thanks to all of you who've contributed to the amazing number of PEPs, features, bug fixes, and doc changes that have gone into 3.6.0b1! Now that feature development for 3.6 is over, the challenge is to put the finishing touches on the features and documentation, squash bugs, and test test test. The next preview release will be 3.6.0b2 scheduled for 2016-10-03. In the cpython repo, there is now a 3.6 branch. Starting now, all changes for 3.6.0 should get pushed to the 3.6 branch and then merged to default for 3.7. New features nay continue to be pushed to the default branch for release in 3.7; no new features are now permitted in 3.6 (unless you have contacted me and we have agreed on an extension). Bug fixes appropriate for 3.5.x should get pushed to the 3.5 branch and then merged to 3.6 and then to default. I've updated the Developer's Guide to reflect the now current workflow. Let me know if you find any bugs in it. Likewise, please contact me if you have any questions about the workflow or about whether a change is appropriate for 3.6 beta. To recap: 2016-09-12 3.6 branch open for 3.6.0; 3.7.0 feature development begins 2016-09-12 to 2016-12-04: 3.6.0 beta phase (no new features) - push code for 3.6.0 (bug/regression/doc fixes) to the new 3.6 branch - push code for new features to the default branch for release in 3.7 2016-10-03: 3.6.0 beta 2 2016-12-04 3.6.0 release candidate 1 (3.6.0 code freeze) 2016-12-16 3.6.0 release (3.6.0rc1 plus, if necessary, any dire emergency fixes) 2018-06 3.7.0 release (3.6.0 release + 18 months, details TBD) Thank you all again for your great efforts so far on 3.6! --Ned http://cpython-devguide.readthedocs.io/en/latest/ https://www.python.org/dev/peps/pep-0494/ -- Ned Deily nad(a)python.org -- []

3 3

TextIO seek and tell cookies
by Ben Leslie Sept. 26, 2016

Sept. 26, 2016

Hi all, I recently shot myself in the foot by assuming that TextIO.tell returned integers rather than opaque cookies. Specifically I was adding an offset to the value returned by TextIO.tell. In retrospect this doesn't make sense/ Now, I don't want to drive change simply because I failed to read the documentation carefully, but I think the current API is very easy to misuse. Most of the time TextIO.tell returns a cookie that is actually an integer and adding an offset to it and seek-ing works fine. The only indication you get that you are mis-using the API is that sometimes tell returns a cookie that when you add an integer offset to it will cause seek() to fail with an OverflowError. Would it be possible to change the API to return something more opaque? E.g.: rather than converting the C cookie structure to a long, could it instead be converted to a bytes() object. (I.e.: Change textiowrapper_build_cookie to use PyBytes_FromStringAndSize rather than _PyLong_FromByteArray and equivalent for textiowrapper_parse_cookie). This would ensure the return value is never mis-used and is probably also faster using bytes objects than converting to/from an integer. Are there any downsides to this? I've made some progress developing a patch to change this functionality. Is it worth polishing and submitting? Cheers, Ben

7 10

Re: [Python-Dev] cpython (3.6): replace usage of Py_VA_COPY with the (C99) standard va_copy
by Victor Stinner Sept. 25, 2016

Sept. 25, 2016

I see that the old macro is now an alias to va_copy(). A similar change was done for Py_MEMCPY(). Would it make sense to put these old macros in a new backward_compat.h header, so maybe one day we can remove them? :-) Maybe we need at least a comment mentionning when (python version) the macro became an alias. Victor

7 12

Code quality report
by Christian Heimes Sept. 24, 2016

Sept. 24, 2016

Hi, here is a short code quality report. Overall we are in a good shape for Python 3.6.0. I'm a bit worried about the amount of security bugs, though. Some haven't progressed in more than a year. Coverity Scan ------------- 3.6.0b1 added a bunch of new defects, most of them were false positives. Python is down again to zero open defects (default branch on Linux X86_64). total defects: 1,115 outstanding defects: 0 dismissed: 169 fixed: 946 https://scan.coverity.com/projects/python C code coverage --------------- I have updated my LCOV report (GCC on Linux X86_64). Our test coverage is quite good. line coverage: 81.9 % function coverage: 92.5 % https://tiran.bitbucket.io/python-lcov/ security bugs ------------- I'm seeing 46 open security bugs on our bug tracker, http://bit.ly/2cYWZy0 . configure / compile warnings ---------------------------- Python configures and compiles without warnings with GCC on Linux X86_64. Clang emits four warnings for unreachable code. All warnings are harmless. On i686 I'm still getting four warnings in the KeccakCodePackage (sha3), https://bugs.python.org/issue28117. Regards, Christian

2 1

Summary of Python tracker Issues
by Python tracker Sept. 23, 2016

Sept. 23, 2016

ACTIVITY SUMMARY (2016-09-16 - 2016-09-23) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 5530 (+23) closed 34485 (+53) total 40015 (+76) Open issues with patches: 2381 Issues opened (63) ================== #16700: Document that bytes OS API can returns unusable results on Win http://bugs.python.org/issue16700 reopened by serhiy.storchaka #27922: Make IDLE tests less flashy http://bugs.python.org/issue27922 reopened by terry.reedy #28162: WindowsConsoleIO readall() fails if first line starts with Ctr http://bugs.python.org/issue28162 reopened by eryksun #28164: _PyIO_get_console_type fails for various paths http://bugs.python.org/issue28164 reopened by eryksun #28165: The 'subprocess' module leaks memory when called in certain wa http://bugs.python.org/issue28165 reopened by Xavion #28176: Fix callbacks race in asyncio.SelectorLoop.sock_connect http://bugs.python.org/issue28176 reopened by haypo #28183: Clean up and speed up dict iteration http://bugs.python.org/issue28183 opened by serhiy.storchaka #28185: Tabs in C source code http://bugs.python.org/issue28185 opened by franciscouzo #28186: Autogenerated tabs / trailing whitespace http://bugs.python.org/issue28186 opened by franciscouzo #28188: os.putenv should support bytes arguments on Windows http://bugs.python.org/issue28188 opened by eryksun #28190: Cross-build _curses failed if host ncurses headers and target http://bugs.python.org/issue28190 opened by Chi Hsuan Yen #28191: Support RFC4985 SRVName in SAN name http://bugs.python.org/issue28191 opened by christian.heimes #28194: Clean up some checks in dict implementation http://bugs.python.org/issue28194 opened by xiang.zhang #28196: ssl.match_hostname() should check for SRV-ID and URI-ID http://bugs.python.org/issue28196 opened by christian.heimes #28197: range.index mismatch with documentation http://bugs.python.org/issue28197 opened by veky #28199: Compact dict resizing is doing too much work http://bugs.python.org/issue28199 opened by rhettinger #28201: dict: perturb shift should be done when first conflict http://bugs.python.org/issue28201 opened by inada.naoki #28202: Python 3.5.1 C API, the global variable is not destroyed when http://bugs.python.org/issue28202 opened by Jack Liu #28203: complex() gives wrong error when the second argument has an in http://bugs.python.org/issue28203 opened by manishearth #28206: signal.Signals not documented http://bugs.python.org/issue28206 opened by Samuel Colvin #28207: SQLite headers are not searched in custom locations http://bugs.python.org/issue28207 opened by Santiago Castro #28208: update sqlite to 3.14.2 http://bugs.python.org/issue28208 opened by Big Stone #28209: Exe or MSI unable to find Py3.5 http://bugs.python.org/issue28209 opened by jcrmatos #28210: argparse with subcommands difference in python 2.7 / 3.5 http://bugs.python.org/issue28210 opened by stephan #28211: Wrong return value type in the doc of PyMapping_Keys/Values/It http://bugs.python.org/issue28211 opened by xiang.zhang #28212: Closing server in asyncio is not efficient http://bugs.python.org/issue28212 opened by Константин Волков #28213: asyncio SSLProtocol _app_transport is private http://bugs.python.org/issue28213 opened by Константин Волков #28214: Improve exception reporting for problematic __set_name__ attri http://bugs.python.org/issue28214 opened by Tim.Graham #28215: PyModule_AddIntConstant() wraps >=2^31 values when long is 4 b http://bugs.python.org/issue28215 opened by altendky #28217: Add interactive console tests http://bugs.python.org/issue28217 opened by steve.dower #28218: Windows docs have wrong versionadded description http://bugs.python.org/issue28218 opened by steve.dower #28219: Is order of argparse --help output officially defined? http://bugs.python.org/issue28219 opened by barry #28221: Unused indata in test_ssl.ThreadedTests.test_asyncore_server http://bugs.python.org/issue28221 opened by martin.panter #28222: test_distutils fails http://bugs.python.org/issue28222 opened by xiang.zhang #28223: test_tools fails with timeout on AMD64 Snow Leop 3.x buildbot http://bugs.python.org/issue28223 opened by haypo #28224: Compilation warnings on Windows: export 'PyInit_xx' specified http://bugs.python.org/issue28224 opened by haypo #28225: bz2 does not support pathlib http://bugs.python.org/issue28225 opened by ethan.furman #28226: compileall does not support pathlib http://bugs.python.org/issue28226 opened by ethan.furman #28227: gzip does not support pathlib http://bugs.python.org/issue28227 opened by ethan.furman #28228: imghdr does not support pathlib http://bugs.python.org/issue28228 opened by ethan.furman #28229: lzma does not support pathlib http://bugs.python.org/issue28229 opened by ethan.furman #28230: tarfile does not support pathlib http://bugs.python.org/issue28230 opened by ethan.furman #28231: zipfile does not support pathlib http://bugs.python.org/issue28231 opened by ethan.furman #28232: asyncio: wrap_future() doesn't handle cancellation correctly http://bugs.python.org/issue28232 opened by haypo #28234: In xml.etree.ElementTree docs there are many absent Element cl http://bugs.python.org/issue28234 opened by py.user #28235: In xml.etree.ElementTree docs there is no parser argument in f http://bugs.python.org/issue28235 opened by py.user #28236: In xml.etree.ElementTree Element can be created with empty and http://bugs.python.org/issue28236 opened by py.user #28237: In xml.etree.ElementTree bytes tag or attributes raises on ser http://bugs.python.org/issue28237 opened by py.user #28238: In xml.etree.ElementTree findall() can't search all elements i http://bugs.python.org/issue28238 opened by py.user #28240: Enhance the timeit module: display average +- std dev instead http://bugs.python.org/issue28240 opened by haypo #28243: Performance regression in functools.partial() http://bugs.python.org/issue28243 opened by serhiy.storchaka #28247: Add an option to zipapp to produce a Windows executable http://bugs.python.org/issue28247 opened by paul.moore #28248: Upgrade installers to OpenSSL 1.0.2i http://bugs.python.org/issue28248 opened by alex #28249: doctest.DocTestFinder reports incorrect line numbers with excl http://bugs.python.org/issue28249 opened by cpitclaudel #28250: typing.NamedTuple instances are not picklable Two http://bugs.python.org/issue28250 opened by Kurt #28251: Help manuals do not appear in Windows search http://bugs.python.org/issue28251 opened by steve.dower #28252: Tuples used before introduction to tuple in tutorial http://bugs.python.org/issue28252 opened by Eswar Yaganti #28253: calendar.prcal(9999) output has a problem http://bugs.python.org/issue28253 opened by jiangping.li #28254: Add C API for gc.enable, gc.disable, and gc.isenabled http://bugs.python.org/issue28254 opened by llllllllll #28255: TextCalendar.prweek/month/year outputs an extra whitespace cha http://bugs.python.org/issue28255 opened by xiang.zhang #28256: Cleanup Modules/_math.c http://bugs.python.org/issue28256 opened by haypo #28257: Regression for star argument parameter error messages http://bugs.python.org/issue28257 opened by kayhayen #28258: Broken python-config generated with Estonian locale http://bugs.python.org/issue28258 opened by Arfrever Most recent 15 issues with no replies (15) ========================================== #28254: Add C API for gc.enable, gc.disable, and gc.isenabled http://bugs.python.org/issue28254 #28252: Tuples used before introduction to tuple in tutorial http://bugs.python.org/issue28252 #28250: typing.NamedTuple instances are not picklable Two http://bugs.python.org/issue28250 #28249: doctest.DocTestFinder reports incorrect line numbers with excl http://bugs.python.org/issue28249 #28238: In xml.etree.ElementTree findall() can't search all elements i http://bugs.python.org/issue28238 #28237: In xml.etree.ElementTree bytes tag or attributes raises on ser http://bugs.python.org/issue28237 #28236: In xml.etree.ElementTree Element can be created with empty and http://bugs.python.org/issue28236 #28232: asyncio: wrap_future() doesn't handle cancellation correctly http://bugs.python.org/issue28232 #28231: zipfile does not support pathlib http://bugs.python.org/issue28231 #28230: tarfile does not support pathlib http://bugs.python.org/issue28230 #28229: lzma does not support pathlib http://bugs.python.org/issue28229 #28227: gzip does not support pathlib http://bugs.python.org/issue28227 #28226: compileall does not support pathlib http://bugs.python.org/issue28226 #28225: bz2 does not support pathlib http://bugs.python.org/issue28225 #28219: Is order of argparse --help output officially defined? http://bugs.python.org/issue28219 Most recent 15 issues waiting for review (15) ============================================= #28258: Broken python-config generated with Estonian locale http://bugs.python.org/issue28258 #28256: Cleanup Modules/_math.c http://bugs.python.org/issue28256 #28255: TextCalendar.prweek/month/year outputs an extra whitespace cha http://bugs.python.org/issue28255 #28254: Add C API for gc.enable, gc.disable, and gc.isenabled http://bugs.python.org/issue28254 #28253: calendar.prcal(9999) output has a problem http://bugs.python.org/issue28253 #28240: Enhance the timeit module: display average +- std dev instead http://bugs.python.org/issue28240 #28235: In xml.etree.ElementTree docs there is no parser argument in f http://bugs.python.org/issue28235 #28234: In xml.etree.ElementTree docs there are many absent Element cl http://bugs.python.org/issue28234 #28231: zipfile does not support pathlib http://bugs.python.org/issue28231 #28230: tarfile does not support pathlib http://bugs.python.org/issue28230 #28229: lzma does not support pathlib http://bugs.python.org/issue28229 #28228: imghdr does not support pathlib http://bugs.python.org/issue28228 #28227: gzip does not support pathlib http://bugs.python.org/issue28227 #28226: compileall does not support pathlib http://bugs.python.org/issue28226 #28225: bz2 does not support pathlib http://bugs.python.org/issue28225 Top 10 most discussed issues (10) ================================= #28165: The 'subprocess' module leaks memory when called in certain wa http://bugs.python.org/issue28165 14 msgs #28183: Clean up and speed up dict iteration http://bugs.python.org/issue28183 14 msgs #28240: Enhance the timeit module: display average +- std dev instead http://bugs.python.org/issue28240 14 msgs #27761: Private _nth_root function loses accuracy http://bugs.python.org/issue27761 12 msgs #28202: Python 3.5.1 C API, the global variable is not destroyed when http://bugs.python.org/issue28202 12 msgs #28203: complex() gives wrong error when the second argument has an in http://bugs.python.org/issue28203 11 msgs #28182: Expose OpenSSL verification results in SSLError http://bugs.python.org/issue28182 10 msgs #26351: Occasionally check for Ctrl-C in long-running operations like http://bugs.python.org/issue26351 9 msgs #28197: range.index mismatch with documentation http://bugs.python.org/issue28197 9 msgs #28214: Improve exception reporting for problematic __set_name__ attri http://bugs.python.org/issue28214 9 msgs Issues closed (53) ================== #16293: curses.ungetch raises OverflowError when given -1 http://bugs.python.org/issue16293 closed by berker.peksag #20173: Derby #4: Convert 53 sites to Argument Clinic across 5 files http://bugs.python.org/issue20173 closed by zach.ware #21516: pathlib.Path(...).is_dir() crashes on some directories (Window http://bugs.python.org/issue21516 closed by berker.peksag #23372: defaultdict.fromkeys should accept a callable factory http://bugs.python.org/issue23372 closed by rhettinger #24022: Python heap corruption issue http://bugs.python.org/issue24022 closed by python-dev #25400: robotparser doesn't return crawl delay for default entry http://bugs.python.org/issue25400 closed by berker.peksag #25470: Random Malloc error raised http://bugs.python.org/issue25470 closed by haypo #25651: Confusing output for TestCase.subTest(0) http://bugs.python.org/issue25651 closed by berker.peksag #26384: UnboundLocalError in socket._sendfile_use_sendfile http://bugs.python.org/issue26384 closed by berker.peksag #26661: python fails to locate system libffi http://bugs.python.org/issue26661 closed by christian.heimes #27111: redundant variables in long_add and long_sub http://bugs.python.org/issue27111 closed by mark.dickinson #27213: Rework CALL_FUNCTION* opcodes http://bugs.python.org/issue27213 closed by serhiy.storchaka #27222: redundant checks and a weird use of goto statements in long_rs http://bugs.python.org/issue27222 closed by mark.dickinson #27282: Raise BlockingIOError in os.urandom if kernel is not ready http://bugs.python.org/issue27282 closed by haypo #27348: traceback (and threading) drops exception message http://bugs.python.org/issue27348 closed by martin.panter #27441: redundant assignments to ob_size of new ints that _PyLong_New http://bugs.python.org/issue27441 closed by mark.dickinson #27482: heap-buffer-overflow on address 0x6250000078ff http://bugs.python.org/issue27482 closed by benjamin.peterson #27806: 2.7 32-bit builds fail on macOS 10.12 Sierra due to dependency http://bugs.python.org/issue27806 closed by ned.deily #27932: platform.win32_ver() leaks in 2.7.12 http://bugs.python.org/issue27932 closed by steve.dower #27950: Superfluous messages when running make http://bugs.python.org/issue27950 closed by martin.panter #27955: getrandom() syscall returning EPERM make the system unusable. http://bugs.python.org/issue27955 closed by haypo #27979: Remove bundled libffi http://bugs.python.org/issue27979 closed by python-dev #27990: Provide a way to enable getrandom on Linux even when build sys http://bugs.python.org/issue27990 closed by ncoghlan #28042: Coverity Scan defects in new dict code http://bugs.python.org/issue28042 closed by christian.heimes #28075: os.stat fails when access is denied http://bugs.python.org/issue28075 closed by berker.peksag #28086: test.test_getargs2.TupleSubclass test failure http://bugs.python.org/issue28086 closed by serhiy.storchaka #28110: launcher.msi has different product codes between 32 and 64-bit http://bugs.python.org/issue28110 closed by steve.dower #28137: Windows sys.path file should be renamed http://bugs.python.org/issue28137 closed by steve.dower #28138: Windows _sys.path file should allow import site http://bugs.python.org/issue28138 closed by steve.dower #28145: Fix whitespace in C source code http://bugs.python.org/issue28145 closed by franciscouzo #28151: testPythonOrg() of test_robotparser fails on validating python http://bugs.python.org/issue28151 closed by berker.peksag #28161: Opening CON for write access fails http://bugs.python.org/issue28161 closed by steve.dower #28163: WindowsConsoleIO fileno() passes wrong flags to _open_osfhandl http://bugs.python.org/issue28163 closed by steve.dower #28178: allow to cache_clear(some_key) in lru_cache http://bugs.python.org/issue28178 closed by rhettinger #28184: Trailing whitespace in C source code http://bugs.python.org/issue28184 closed by python-dev #28187: Check return value of _PyBytes_Resize http://bugs.python.org/issue28187 closed by berker.peksag #28189: dictitems_contains swallows compare errors http://bugs.python.org/issue28189 closed by rhettinger #28192: Don't import readline in isolated mode http://bugs.python.org/issue28192 closed by steve.dower #28193: Consider using lru_cache for the re.py caches http://bugs.python.org/issue28193 closed by rhettinger #28195: test_huntrleaks_fd_leak fails on Windows http://bugs.python.org/issue28195 closed by haypo #28198: heap-buffer-overflow in tok_nextc (Parser/tokenizer.c:954) http://bugs.python.org/issue28198 closed by berker.peksag #28200: Windows: path_converter() leaks memory for Unicode filenames http://bugs.python.org/issue28200 closed by haypo #28204: Spam http://bugs.python.org/issue28204 closed by xiang.zhang #28205: Add optional suffix to str.join http://bugs.python.org/issue28205 closed by rhettinger #28216: micro optimization for import_all_from http://bugs.python.org/issue28216 closed by xiang.zhang #28220: argparse's add_mutually_exclusive_group() should accept title http://bugs.python.org/issue28220 closed by berker.peksag #28233: PyUnicode_FromFormatV can leak PyUnicodeWriter http://bugs.python.org/issue28233 closed by haypo #28239: Implement functools.lru_cache() using ordered dict http://bugs.python.org/issue28239 closed by serhiy.storchaka #28241: Nested fuctions Unexpected behaviour when stored in a list and http://bugs.python.org/issue28241 closed by zach.ware #28242: os.environ.get documentation missing http://bugs.python.org/issue28242 closed by ned.deily #28244: Incorrect Example in itertools.product description http://bugs.python.org/issue28244 closed by rhettinger #28245: Embeddable Python does not use PYTHONPATH. http://bugs.python.org/issue28245 closed by steve.dower #28246: Unable to read simple text file http://bugs.python.org/issue28246 closed by eryksun

1 0

Re: [Python-Dev] cpython (3.6): replace usage of Py_VA_COPY with the (C99) standard va_copy
by Christian Heimes Sept. 22, 2016

Sept. 22, 2016

On 2016-09-21 05:39, benjamin.peterson wrote: > https://hg.python.org/cpython/rev/278b21d8e86e > changeset: 103977:278b21d8e86e > branch: 3.6 > parent: 103975:d31b4de433b7 > user: Benjamin Peterson <benjamin(a)python.org> > date: Tue Sep 20 20:39:33 2016 -0700 > summary: > replace usage of Py_VA_COPY with the (C99) standard va_copy Thanks! Coverity has been complaining about Py_VA_COPY() for a long time. Your change may cause a memory leak on some platforms. You must va_end() a va_copy() region: Each invocation of va_copy() must be matched by a corresponding invocation of va_end() in the same function. https://linux.die.net/man/3/va_copy

2 1

Python 3.6 dict becomes compact and gets a private version; and keywords become ordered
by Victor Stinner Sept. 21, 2016

Sept. 21, 2016

Hi, I pushed INADA Naoki's implementation of the "compact dict". The hash table now stores indices pointing to a new second table which contains keys and values: it adds one new level of indirection. The table of indices is "compact": use 1, 2, 4 or 8 bytes per indice depending on the size of the dictionary. Moreover, the keys/values table is also more compact: its size is 2/3 of the indices table. A nice "side effect" of compact dict is that the dictionary now preserves the insertion order. It means that keyword arguments can now be iterated by their creation order: Python 3.5.1 (default, Jun 20 2016, 14:48:22) >>> def func(**kw): print(kw.keys()) ... >>> func(a=1, b=2, c=3, d=4, e=5) dict_keys(['c', 'd', 'e', 'b', 'a']) # random order vs Python 3.6.0a4+ (default:d43f819caea7, Sep 8 2016, 13:05:34) >>> def func(**kw): print(kw.keys()) ... >>> func(a=1, b=2, c=3, d=4, e=5) dict_keys(['a', 'b', 'c', 'd', 'e']) # expected order It means that the main goal of the PEP 468 "Preserving the order of **kwargs in a function" is now implemented in Python 3.6: https://www.python.org/dev/peps/pep-0468/ But Eric Snow still wants to rephrase the PEP 468 to replace "OrderedDict" with "ordered mapping". For more information on compact dict, see: * http://bugs.python.org/issue27350 * https://mail.python.org/pipermail/python-dev/2016-June/145299.html * https://morepypy.blogspot.jp/2015/01/faster-more-memory-efficient-and-more.… *https://mail.python.org/pipermail/python-dev/2012-December/123028.html PyPy also implements the "compact dict", but it uses further "tricks" to preserve the order even if items are removed and then others are added. We might also implement these tricks in CPython, so dict will be ordered as well! -- Moreover, since Guido approved the PEP 509 "Add a private version to dict", I just pushed the implementation. The PEP 509 provides a C API (a dict version field) to implement efficient caches on namespaces. It might be used to implement a cache on builtins in Python 3.6 using Yury's opcode cache (stay tuned!). Victor

28 77

Re: [Python-Dev] Python 3.6 dict becomes compact and gets a private version; and keywords become ordered
by INADA Naoki Sept. 21, 2016

Sept. 21, 2016

On Tue, Sep 20, 2016 at 7:02 PM, INADA Naoki <songofacandy(a)gmail.com> wrote: > On Tue, Sep 20, 2016 at 6:56 PM, Dima Tisnek <dimaqq(a)gmail.com> wrote: >> Totally random thought: >> >> Can lru_cache be simplified to use an ordered dict instead of dict + >> linked list? >> > > I think so. > See also: http://bugs.python.org/issue28199#msg276938 > FYI, current dict implementation is not optimized for removing first item like this: ``` // When hit max_size Py_ssize_t pos; PyObject *key; if (PyDict_Next(d, &pos, &key, NULL)) { if (PyDict_DelItem(key) < 0) { // error. } } ``` So, before changing lru_cache implementation, I (or someone else) should rewrite OrderedDict which has O(1) "remove first item" method. (At least max_size is not None). But both of OrderedDict and lru_cache improvements can't be in 3.6 since 3.6 is beta now. I'll try it after 3.6rc1. -- INADA Naoki <songofacandy(a)gmail.com>

3 2

Re: [Python-Dev] [Python-checkins] cpython (2.7): properly handle the single null-byte file (closes #24022)
by Eric V. Smith Sept. 20, 2016

Sept. 20, 2016

Shouldn't there be a test added for this? -- Eric. > On Sep 19, 2016, at 2:44 AM, benjamin.peterson <python-checkins(a)python.org> wrote: > > https://hg.python.org/cpython/rev/c6438a3df7a4 > changeset: 103950:c6438a3df7a4 > branch: 2.7 > parent: 103927:a8771f230c06 > user: Benjamin Peterson <benjamin(a)python.org> > date: Sun Sep 18 23:41:11 2016 -0700 > summary: > properly handle the single null-byte file (closes #24022) > > files: > Parser/tokenizer.c | 2 +- > 1 files changed, 1 insertions(+), 1 deletions(-) > > > diff --git a/Parser/tokenizer.c b/Parser/tokenizer.c > --- a/Parser/tokenizer.c > +++ b/Parser/tokenizer.c > @@ -951,7 +951,7 @@ > else { > tok->done = E_OK; > tok->inp = strchr(tok->buf, '\0'); > - done = tok->inp[-1] == '\n'; > + done = tok->inp == tok->buf || tok->inp[-1] == '\n'; > } > } > else { > > -- > Repository URL: https://hg.python.org/cpython > _______________________________________________ > Python-checkins mailing list > Python-checkins(a)python.org > https://mail.python.org/mailman/listinfo/python-checkins

2 1