Python-Dev October 2016

python-dev@python.org

62 participants
37 discussions

by Python tracker

ACTIVITY SUMMARY (2016-10-21 - 2016-10-28) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 5525 ( +1) closed 34779 (+51) total 40304 (+52) Open issues with patches: 2398 Issues opened (36) ================== #5830: heapq item comparison problematic with sched's events http://bugs.python.org/issue5830 reopened by serhiy.storchaka #5996: abstract class instantiable when subclassing dict http://bugs.python.org/issue5996 reopened by gvanrossum #25002: Deprecate asyncore/asynchat http://bugs.python.org/issue25002 reopened by serhiy.storchaka #25152: venv documentation doesn't tell you how to specify a particula http://bugs.python.org/issue25152 reopened by serhiy.storchaka #28498: tk busy command http://bugs.python.org/issue28498 opened by tkinter #28499: Logging module documentation needs a rework. http://bugs.python.org/issue28499 opened by Pierre Bousquie #28501: [Patch] Make os.umask() optional http://bugs.python.org/issue28501 opened by EdSchouten #28502: [Patch] Make os.chdir() optional http://bugs.python.org/issue28502 opened by EdSchouten #28503: [Patch] '_crypt' module: fix implicit declaration of crypt(), http://bugs.python.org/issue28503 opened by EdSchouten #28506: Multiprocessing Pool starmap - struct.error: 'i' format requir http://bugs.python.org/issue28506 opened by Justin Ting #28512: PyErr_SyntaxLocationEx() and PyErr_SyntaxLocationObject() alwa http://bugs.python.org/issue28512 opened by serhiy.storchaka #28513: Document zipfile CLI http://bugs.python.org/issue28513 opened by serhiy.storchaka #28516: contextlib.ExitStack.__enter__ has trivial but undocumented be http://bugs.python.org/issue28516 opened by walker.hale.iv #28518: execute("begin immediate") throwing OperationalError http://bugs.python.org/issue28518 opened by fschulze #28519: Update pydoc tool to support generic types http://bugs.python.org/issue28519 opened by levkivskyi #28521: _PyEval_RequestCodeExtraIndex should return a globally valid i http://bugs.python.org/issue28521 opened by jpe #28522: can't make IDLEX work with python._pth and python-3.6.0b2 http://bugs.python.org/issue28522 opened by Big Stone #28523: Idlelib.configdialog: use 'color' insteadof 'colour' http://bugs.python.org/issue28523 opened by terry.reedy #28524: Set default argument of logging.disable() to logging.CRITICAL http://bugs.python.org/issue28524 opened by Al.Sweigart #28528: Pdb.checkline() http://bugs.python.org/issue28528 opened by takluyver #28530: Howto detect if an object is of type os.DirEntry http://bugs.python.org/issue28530 opened by stephan #28531: Improve utf7 encoder memory usage http://bugs.python.org/issue28531 opened by xiang.zhang #28532: Show sys.version when -V option is supplied twice. http://bugs.python.org/issue28532 opened by inada.naoki #28533: Replace asyncore http://bugs.python.org/issue28533 opened by Mariatta #28534: Replace asynchat http://bugs.python.org/issue28534 opened by Mariatta #28536: Show the qualified name when a call fails http://bugs.python.org/issue28536 opened by Ryan.Gonzalez #28538: _socket module cross-compilation error on android-24 http://bugs.python.org/issue28538 opened by xdegaye #28539: httplib/http.client HTTPConnection._set_hostport() regression http://bugs.python.org/issue28539 opened by cfs-pure #28540: math.degrees(sys.float_info.max) should throw an OverflowError http://bugs.python.org/issue28540 opened by franciscouzo #28541: Improve test coverage for json library - loading bytes http://bugs.python.org/issue28541 opened by Eric Appelt #28542: document cross compilation http://bugs.python.org/issue28542 opened by xdegaye #28543: Incomplete fast path codecs aliases in codecs doc http://bugs.python.org/issue28543 opened by xiang.zhang #28544: Implement asyncio.Task in C http://bugs.python.org/issue28544 opened by yselivanov #28546: Better explain setting pdb breakpoints http://bugs.python.org/issue28546 opened by iank #28547: Python to use Windows Certificate Store http://bugs.python.org/issue28547 opened by Jean-Philippe Landry #28548: http.server parse_request() bug and error reporting http://bugs.python.org/issue28548 opened by barry Most recent 15 issues with no replies (15) ========================================== #28548: http.server parse_request() bug and error reporting http://bugs.python.org/issue28548 #28546: Better explain setting pdb breakpoints http://bugs.python.org/issue28546 #28543: Incomplete fast path codecs aliases in codecs doc http://bugs.python.org/issue28543 #28542: document cross compilation http://bugs.python.org/issue28542 #28538: _socket module cross-compilation error on android-24 http://bugs.python.org/issue28538 #28534: Replace asynchat http://bugs.python.org/issue28534 #28533: Replace asyncore http://bugs.python.org/issue28533 #28532: Show sys.version when -V option is supplied twice. http://bugs.python.org/issue28532 #28528: Pdb.checkline() http://bugs.python.org/issue28528 #28521: _PyEval_RequestCodeExtraIndex should return a globally valid i http://bugs.python.org/issue28521 #28519: Update pydoc tool to support generic types http://bugs.python.org/issue28519 #28512: PyErr_SyntaxLocationEx() and PyErr_SyntaxLocationObject() alwa http://bugs.python.org/issue28512 #28503: [Patch] '_crypt' module: fix implicit declaration of crypt(), http://bugs.python.org/issue28503 #28502: [Patch] Make os.chdir() optional http://bugs.python.org/issue28502 #28501: [Patch] Make os.umask() optional http://bugs.python.org/issue28501 Most recent 15 issues waiting for review (15) ============================================= #28546: Better explain setting pdb breakpoints http://bugs.python.org/issue28546 #28544: Implement asyncio.Task in C http://bugs.python.org/issue28544 #28543: Incomplete fast path codecs aliases in codecs doc http://bugs.python.org/issue28543 #28542: document cross compilation http://bugs.python.org/issue28542 #28541: Improve test coverage for json library - loading bytes http://bugs.python.org/issue28541 #28540: math.degrees(sys.float_info.max) should throw an OverflowError http://bugs.python.org/issue28540 #28539: httplib/http.client HTTPConnection._set_hostport() regression http://bugs.python.org/issue28539 #28536: Show the qualified name when a call fails http://bugs.python.org/issue28536 #28532: Show sys.version when -V option is supplied twice. http://bugs.python.org/issue28532 #28531: Improve utf7 encoder memory usage http://bugs.python.org/issue28531 #28528: Pdb.checkline() http://bugs.python.org/issue28528 #28524: Set default argument of logging.disable() to logging.CRITICAL http://bugs.python.org/issue28524 #28523: Idlelib.configdialog: use 'color' insteadof 'colour' http://bugs.python.org/issue28523 #28516: contextlib.ExitStack.__enter__ has trivial but undocumented be http://bugs.python.org/issue28516 #28513: Document zipfile CLI http://bugs.python.org/issue28513 Top 10 most discussed issues (10) ================================= #28498: tk busy command http://bugs.python.org/issue28498 36 msgs #28522: can't make IDLEX work with python._pth and python-3.6.0b2 http://bugs.python.org/issue28522 18 msgs #25002: Deprecate asyncore/asynchat http://bugs.python.org/issue25002 16 msgs #23262: webbrowser module broken with Firefox 36+ http://bugs.python.org/issue23262 14 msgs #28544: Implement asyncio.Task in C http://bugs.python.org/issue28544 12 msgs #28539: httplib/http.client HTTPConnection._set_hostport() regression http://bugs.python.org/issue28539 9 msgs #28199: Compact dict resizing is doing too much work http://bugs.python.org/issue28199 7 msgs #28524: Set default argument of logging.disable() to logging.CRITICAL http://bugs.python.org/issue28524 7 msgs #27275: KeyError thrown by optimised collections.OrderedDict.popitem() http://bugs.python.org/issue27275 6 msgs #5996: abstract class instantiable when subclassing dict http://bugs.python.org/issue5996 5 msgs Issues closed (53) ================== #3015: tkinter with wantobjects=False has been broken for some time http://bugs.python.org/issue3015 closed by serhiy.storchaka #17711: Persistent id in pickle with protocol version 0 http://bugs.python.org/issue17711 closed by serhiy.storchaka #20357: Mention buildbots in the core dev section of the devguide http://bugs.python.org/issue20357 closed by berker.peksag #20491: textwrap: Non-breaking space not honored http://bugs.python.org/issue20491 closed by serhiy.storchaka #22757: TclStackFree: incorrect freePtr. Call out of sequence? http://bugs.python.org/issue22757 closed by serhiy.storchaka #22949: fnmatch.translate doesn't add ^ at the beginning http://bugs.python.org/issue22949 closed by serhiy.storchaka #23214: BufferedReader.read1(size) signature incompatible with Buffere http://bugs.python.org/issue23214 closed by martin.panter #25464: Tix HList header_exists should be "exist" http://bugs.python.org/issue25464 closed by serhiy.storchaka #25953: re fails to identify invalid numeric group references in repla http://bugs.python.org/issue25953 closed by serhiy.storchaka #26240: Docstring of the subprocess module should be cleaned up http://bugs.python.org/issue26240 closed by martin.panter #26682: Ttk Notebook tabs do not show with 1-2 char names http://bugs.python.org/issue26682 closed by ned.deily #26796: BaseEventLoop.run_in_executor shouldn't specify max_workers fo http://bugs.python.org/issue26796 closed by yselivanov #26923: asyncio.gather drops cancellation http://bugs.python.org/issue26923 closed by yselivanov #27025: More human readable generated widget names http://bugs.python.org/issue27025 closed by serhiy.storchaka #27989: incomplete signature with help function using typing http://bugs.python.org/issue27989 closed by gvanrossum #28107: Update typing module documentation for NamedTuple http://bugs.python.org/issue28107 closed by gvanrossum #28213: asyncio SSLProtocol _app_transport is private http://bugs.python.org/issue28213 closed by yselivanov #28255: TextCalendar.prweek/month/year outputs an extra whitespace cha http://bugs.python.org/issue28255 closed by serhiy.storchaka #28314: ElementTree: Element.getiterator(tag) broken in 3.6 http://bugs.python.org/issue28314 closed by serhiy.storchaka #28330: Use simpler and faster sched.Event definition http://bugs.python.org/issue28330 closed by rhettinger #28333: input() with Unicode prompt produces mojibake on Windows http://bugs.python.org/issue28333 closed by steve.dower #28353: os.fwalk() unhandled exception when error occurs accessing sym http://bugs.python.org/issue28353 closed by serhiy.storchaka #28396: Remove *.pyo references from man page http://bugs.python.org/issue28396 closed by brett.cannon #28408: Fix redundant code and memory leak in _PyUnicodeWriter_Finish http://bugs.python.org/issue28408 closed by serhiy.storchaka #28426: PyUnicode_AsDecodedObject can only return unicode now http://bugs.python.org/issue28426 closed by serhiy.storchaka #28430: asyncio: C implemeted Future cause Tornado test fail http://bugs.python.org/issue28430 closed by inada.naoki #28435: test_urllib2_localnet.ProxyAuthTests fails with no_proxy and N http://bugs.python.org/issue28435 closed by martin.panter #28439: Remove redundant checks in PyUnicode_EncodeLocale and PyUnicod http://bugs.python.org/issue28439 closed by serhiy.storchaka #28465: python 3.5 magic number http://bugs.python.org/issue28465 closed by brett.cannon #28469: timeit: use powers of 2 in autorange(), instead of powers of 1 http://bugs.python.org/issue28469 closed by serhiy.storchaka #28482: test_typing fails if asyncio unavailable http://bugs.python.org/issue28482 closed by gvanrossum #28488: shutil.make_archive (xxx, zip, root_dir) is adding './' entry http://bugs.python.org/issue28488 closed by serhiy.storchaka #28496: Mark up constants 0, 1, -1 in C API docs http://bugs.python.org/issue28496 closed by serhiy.storchaka #28497: future in tkinter http://bugs.python.org/issue28497 closed by serhiy.storchaka #28500: pep 525/asyncio: Handle when async generators are GCed from an http://bugs.python.org/issue28500 closed by yselivanov #28504: Cleanup unicode_decode_call_errorhandler_wchar/writer http://bugs.python.org/issue28504 closed by serhiy.storchaka #28505: pip installation issues with default path on Windows http://bugs.python.org/issue28505 closed by eryksun #28507: Regenerate ./configure on the default branch http://bugs.python.org/issue28507 closed by ned.deily #28508: Need way to expose incremental size of key sharing dicts http://bugs.python.org/issue28508 closed by rhettinger #28509: dict.update allocates too much http://bugs.python.org/issue28509 closed by inada.naoki #28510: PyUnicodeDecodeError_GetObject always return bytes http://bugs.python.org/issue28510 closed by serhiy.storchaka #28511: Use the "U" format for parsing Unicode object arg in PyArg_Par http://bugs.python.org/issue28511 closed by xiang.zhang #28514: Python (IDLE?) freezes on file save on Windows http://bugs.python.org/issue28514 closed by SilentGhost #28515: Py3k warnings in Python 2.7 tests http://bugs.python.org/issue28515 closed by serhiy.storchaka #28517: Dead code in wordcode http://bugs.python.org/issue28517 closed by serhiy.storchaka #28520: Failed to install Python 3.3.5 on OSX 10.11.6 http://bugs.python.org/issue28520 closed by ned.deily #28525: Incorrect documented parameter for gc.collect http://bugs.python.org/issue28525 closed by python-dev #28526: Use PyUnicode_AsEncodedString instead of PyUnicode_AsEncodedOb http://bugs.python.org/issue28526 closed by serhiy.storchaka #28527: Hack in `genericpath.commonprefix()` crashes if `m` argument i http://bugs.python.org/issue28527 closed by christian.heimes #28529: 0 ** 0 should raise ArithmeticError http://bugs.python.org/issue28529 closed by tim.peters #28535: round seems to provide floor, not proper rounding http://bugs.python.org/issue28535 closed by r.david.murray #28537: abc module fails to reject instantiation of some multiply-inhe http://bugs.python.org/issue28537 closed by r.david.murray #28545: socket.bind to AF_PACKET should use passed interface name http://bugs.python.org/issue28545 closed by mturon

4 years, 7 months

Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor

by INADA Naoki

Hi. While there were no reply to my previous post on Python-ideas ML, Now I'm sure about bytes.frombuffer() is worth enough. Let's describe why I think it's important. Background ========= >From Python 3.4, bytearray is good solution for I/O buffer, thanks to #19087 [1]. Actually, asyncio uses bytearray as I/O buffer often. When bytearray is used for read buffer, we can parse received data on bytearray directly, and consume it. For example, read until '\r\n' is easier than io.BytesIO(). Sample code: def read_line(buf: bytearray) -> bytes: try: n = buf.index(b'\r\n') except ValueError: return b'' line = bytes(buf)[:n] # bytearray -> bytes -> bytes del buf[:n+2] return line buf = bytearray(b'foo\r\nbar\r\nbaz\r\n') while True: line = read_line(buf) if not line: break print(line) As you can see, redundant temporary bytes is used. This is not ideal for performance and memory efficiency. Since code like this is typically in lower level code (e.g. asyncio), performance and efficiency is important. [1] https://bugs.python.org/issue19087 (Off topic: bytearray is nice for write buffer too. written = s.send(buf); del buf[:written];) Memoryview problem ================= To avoid redundant copy of `line = bytes(buf)[:n]`, current solution is using memoryview. First code I wrote is: `line = bytes(memoryview(buf)[:n])`. On CPython, it works fine. But `del buff[:n+2]` in next line may fail on other Python implementations. Changing bytearray size is inhibited while memoryview is alive. So right code is: with memoryview(buf) as m: line = bytes(m[:n]) The problem of memoryview approach is: * Overhead: creating temporary memoryview, __enter__, and __exit__. (see below) * It isn't "one obvious way": Developers including me may forget to use context manager. And since it works on CPython, it's hard to point it out. Quick benchmark: (temporary bytes) $ python3 -m perf timeit -s 'buf = bytearray(b"foo\r\nbar\r\nbaz\r\n")' -- 'bytes(buf)[:3]' .................... Median +- std dev: 652 ns +- 19 ns (temporary memoryview without "with" $ python3 -m perf timeit -s 'buf = bytearray(b"foo\r\nbar\r\nbaz\r\n")' -- 'bytes(memoryview(buf)[:3])' .................... Median +- std dev: 886 ns +- 26 ns (temporary memoryview with "with") $ python3 -m perf timeit -s 'buf = bytearray(b"foo\r\nbar\r\nbaz\r\n")' -- ' with memoryview(buf) as m: bytes(m[:3]) ' .................... Median +- std dev: 1.11 us +- 0.03 us Proposed solution =============== Adding one more constructor to bytes: # when length=-1 (default), use until end of *byteslike*. bytes.frombuffer(byteslike, length=-1, offset=0) With ths API with memoryview(buf) as m: line = bytes(m[:n]) becomes line = bytes.frombuffer(buf, n) Thanks, -- INADA Naoki <songofacandy(a)gmail.com>

4 years, 7 months

Preference for of patch submission?

by Barry Scott

I am over due providing a patch for a doc issue that was discussed on the ideas list. What is the preferred way to provide cpython with a patch these days? bug report + patch? pull request on github? something else? Barry

4 years, 7 months

Have I got my hg dependencies correct?

by Skip Montanaro

I've recently run into a problem building the math and cmath modules for 2.7. (I don't rebuild very often, so this problem might have been around for awhile.) My hg repos look like this: * My cpython repo pulls from https://hg.python.org/cpython * My 2.7 repo (and other non-tip repos) pulls from my cpython repo I think this setup was recommended way back in the day when hg was new to the Python toolchain to avoid unnecessary network bandwidth. So, if I execute hg pull hg update in first cpython, then 2.7 repos I should be up-to-date, correct? However, rebuilding in my 2.7 repo fails to build math and cmath. The compiler complains that Modules/_math.o doesn't exist. If I manually execute make Modules/_math.o make after the failure, then the math and cmath modules build. Looking on bugs.python.org I saw this closed issue: http://bugs.python.org/issue24421 which seems related. Is it possible that the fix wasn't propagated to the 2.7 branch? Or perhaps I've fouled up my hg repo relationships? My other repos which depend on cpython (3.5, 3.4, 3.3, and 3.2) all build the math module just fine. I'm running on an ancient MacBook Pro with OS X 10.11.6 (El Capitan) and XCode 8.0 installed. Any suggestions? Skip

4 years, 7 months

Summary of Python tracker Issues

by Python tracker

ACTIVITY SUMMARY (2016-10-14 - 2016-10-21) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 5524 ( -3) closed 34728 (+55) total 40252 (+52) Open issues with patches: 2398 Issues opened (28) ================== #28404: Logging SyslogHandler not appending '\n' to the end http://bugs.python.org/issue28404 reopened by elelement #28437: Documentation for handling of non-type metaclass hints is uncl http://bugs.python.org/issue28437 reopened by ncoghlan #28445: Wrong documentation for GzipFile.peek http://bugs.python.org/issue28445 opened by abacabadabacaba #28446: pyvenv generates malformed hashbangs for scripts http://bugs.python.org/issue28446 opened by alexreg #28449: tarfile.open(mode = 'r:*', ignore_zeros = True) has 50% chance http://bugs.python.org/issue28449 opened by Silver Fox #28450: Misleading/inaccurate documentation about unknown escape seque http://bugs.python.org/issue28450 opened by lelit #28451: pydoc.safeimport() raises ErrorDuringImport() if __builtin__._ http://bugs.python.org/issue28451 opened by segfault87 #28453: SSLObject.selected_alpn_protocol() not documented http://bugs.python.org/issue28453 opened by alex.gronholm #28457: Make public the current private known hash functions in the C- http://bugs.python.org/issue28457 opened by rhettinger #28459: _pyio module broken on Cygwin / setmode not usable http://bugs.python.org/issue28459 opened by erik.bray #28460: Minidom, order of attributes, datachars http://bugs.python.org/issue28460 opened by Petr Pulc #28462: subprocess pipe can't see EOF from a child in case of a few ch http://bugs.python.org/issue28462 opened by Vyacheslav Grigoryev #28463: Email long headers parsing/serialization http://bugs.python.org/issue28463 opened by Константин Волков #28464: BaseEventLoop.close should shutdown executor before marking it http://bugs.python.org/issue28464 opened by cmeyer #28465: python 3.5 magic number http://bugs.python.org/issue28465 opened by 曹忠 #28469: timeit: use powers of 2 in autorange(), instead of powers of 1 http://bugs.python.org/issue28469 opened by haypo #28470: configure.ac -g debug compiler option when not Py_DEBUG http://bugs.python.org/issue28470 opened by Chris Byers #28474: WinError(): Python int too large to convert to C long http://bugs.python.org/issue28474 opened by Kelvin You #28475: Misleading error on random.sample when k < 0 http://bugs.python.org/issue28475 opened by franciscouzo #28477: Add optional user argument to pathlib.Path.home() http://bugs.python.org/issue28477 opened by josh.r #28478: Built-in module 'time' does not enable functions if -Werror sp http://bugs.python.org/issue28478 opened by toast12 #28482: test_typing fails if asyncio unavailable http://bugs.python.org/issue28482 opened by martin.panter #28485: compileall.compile_dir(workers=<negative>) does not raise Valu http://bugs.python.org/issue28485 opened by martin.panter #28488: shutil.make_archive (xxx, zip, root_dir) is adding './' entry http://bugs.python.org/issue28488 opened by bialix #28489: Fix comment in tokenizer.c http://bugs.python.org/issue28489 opened by Ryan.Gonzalez #28491: Remove bundled libffi for OSX http://bugs.python.org/issue28491 opened by zach.ware #28494: is_zipfile false positives http://bugs.python.org/issue28494 opened by Thomas.Waldmann #28496: Mark up constants 0, 1, -1 in C API docs http://bugs.python.org/issue28496 opened by serhiy.storchaka Most recent 15 issues with no replies (15) ========================================== #28485: compileall.compile_dir(workers=<negative>) does not raise Valu http://bugs.python.org/issue28485 #28470: configure.ac -g debug compiler option when not Py_DEBUG http://bugs.python.org/issue28470 #28464: BaseEventLoop.close should shutdown executor before marking it http://bugs.python.org/issue28464 #28460: Minidom, order of attributes, datachars http://bugs.python.org/issue28460 #28457: Make public the current private known hash functions in the C- http://bugs.python.org/issue28457 #28446: pyvenv generates malformed hashbangs for scripts http://bugs.python.org/issue28446 #28439: Remove redundant checks in PyUnicode_EncodeLocale and PyUnicod http://bugs.python.org/issue28439 #28429: ctypes fails to import with grsecurity's TPE http://bugs.python.org/issue28429 #28422: multiprocessing Manager mutable type member access failure http://bugs.python.org/issue28422 #28416: defining persistent_id in _pickle.Pickler subclass causes refe http://bugs.python.org/issue28416 #28412: os.path.splitdrive documentation out of date http://bugs.python.org/issue28412 #28408: Fix redundant code and memory leak in _PyUnicodeWriter_Finish http://bugs.python.org/issue28408 #28407: Improve coverage of email.utils.make_msgid() http://bugs.python.org/issue28407 #28401: Don't support the PEP384 stable ABI in pydebug builds http://bugs.python.org/issue28401 #28396: Remove *.pyo references from man page http://bugs.python.org/issue28396 Most recent 15 issues waiting for review (15) ============================================= #28496: Mark up constants 0, 1, -1 in C API docs http://bugs.python.org/issue28496 #28494: is_zipfile false positives http://bugs.python.org/issue28494 #28491: Remove bundled libffi for OSX http://bugs.python.org/issue28491 #28489: Fix comment in tokenizer.c http://bugs.python.org/issue28489 #28488: shutil.make_archive (xxx, zip, root_dir) is adding './' entry http://bugs.python.org/issue28488 #28485: compileall.compile_dir(workers=<negative>) does not raise Valu http://bugs.python.org/issue28485 #28482: test_typing fails if asyncio unavailable http://bugs.python.org/issue28482 #28475: Misleading error on random.sample when k < 0 http://bugs.python.org/issue28475 #28469: timeit: use powers of 2 in autorange(), instead of powers of 1 http://bugs.python.org/issue28469 #28459: _pyio module broken on Cygwin / setmode not usable http://bugs.python.org/issue28459 #28451: pydoc.safeimport() raises ErrorDuringImport() if __builtin__._ http://bugs.python.org/issue28451 #28449: tarfile.open(mode = 'r:*', ignore_zeros = True) has 50% chance http://bugs.python.org/issue28449 #28444: Missing extensions modules when cross compiling python 3.5.2 f http://bugs.python.org/issue28444 #28443: Logger methods never use kwargs http://bugs.python.org/issue28443 #28441: Change sys.executable to include executable suffix http://bugs.python.org/issue28441 Top 10 most discussed issues (10) ================================= #28444: Missing extensions modules when cross compiling python 3.5.2 f http://bugs.python.org/issue28444 25 msgs #28437: Documentation for handling of non-type metaclass hints is uncl http://bugs.python.org/issue28437 12 msgs #19795: Formatting of True/False/None in docs http://bugs.python.org/issue19795 10 msgs #26240: Docstring of the subprocess module should be cleaned up http://bugs.python.org/issue26240 8 msgs #28463: Email long headers parsing/serialization http://bugs.python.org/issue28463 6 msgs #28489: Fix comment in tokenizer.c http://bugs.python.org/issue28489 6 msgs #28491: Remove bundled libffi for OSX http://bugs.python.org/issue28491 6 msgs #23214: BufferedReader.read1(size) signature incompatible with Buffere http://bugs.python.org/issue23214 5 msgs #28474: WinError(): Python int too large to convert to C long http://bugs.python.org/issue28474 5 msgs #14991: Option for regex groupdict() to show only matching names http://bugs.python.org/issue14991 4 msgs Issues closed (54) ================== #16007: Improved Error message for failing re expressions http://bugs.python.org/issue16007 closed by serhiy.storchaka #18219: csv.DictWriter is slow when writing files with large number of http://bugs.python.org/issue18219 closed by inada.naoki #21955: ceval.c: implement fast path for integers with a single digit http://bugs.python.org/issue21955 closed by haypo #23231: Fix codecs.iterencode/decode() by allowing data parameter to b http://bugs.python.org/issue23231 closed by martin.panter #23782: Leak in _PyTraceback_Add http://bugs.python.org/issue23782 closed by serhiy.storchaka #24381: Got warning when compiling ffi.c on Mac http://bugs.python.org/issue24381 closed by ned.deily #25550: RecursionError in re with '(' * 500 http://bugs.python.org/issue25550 closed by serhiy.storchaka #26010: document CO_* constants http://bugs.python.org/issue26010 closed by matrixise #26436: Add the regex-dna benchmark http://bugs.python.org/issue26436 closed by haypo #26942: android: test_ctypes crashes on armv7 and aarch64 http://bugs.python.org/issue26942 closed by xdegaye #26944: test_posix: Android 'id -G' is entirely wrong or missing the e http://bugs.python.org/issue26944 closed by xdegaye #27471: sre_constants.error: bad escape \d http://bugs.python.org/issue27471 closed by serhiy.storchaka #27627: clang fails to build ctypes on Android armv7 http://bugs.python.org/issue27627 closed by xdegaye #27800: Regular expressions with multiple repeat codes http://bugs.python.org/issue27800 closed by martin.panter #27844: Python-3.6a4 build messages to stderr (on AIX and xlc compiler http://bugs.python.org/issue27844 closed by martin.panter #27896: Allow passing sphinx options to Doc/Makefile http://bugs.python.org/issue27896 closed by haypo #28092: Build failure for 3.6 on Centos 5.11 http://bugs.python.org/issue28092 closed by benjamin.peterson #28099: Drop Mac OS X Tiger support in Python 3.6 http://bugs.python.org/issue28099 closed by ned.deily #28214: Improve exception reporting for problematic __set_name__ attri http://bugs.python.org/issue28214 closed by serhiy.storchaka #28240: Enhance the timeit module: display average +- std dev instead http://bugs.python.org/issue28240 closed by haypo #28256: Cleanup Modules/_math.c http://bugs.python.org/issue28256 closed by haypo #28409: test.regrtest does not support multiple -x flags http://bugs.python.org/issue28409 closed by haypo #28410: Add convenient C API for "raise ... from ..." http://bugs.python.org/issue28410 closed by serhiy.storchaka #28425: Python3 ignores __init__.py that are links to /dev/null http://bugs.python.org/issue28425 closed by brett.cannon #28428: Rename _futures module to _asyncio http://bugs.python.org/issue28428 closed by inada.naoki #28432: Fix doc of PyUnicode_EncodeLocale http://bugs.python.org/issue28432 closed by berker.peksag #28442: tuple(a list subclass) does not iterate through the list http://bugs.python.org/issue28442 closed by siming85 #28447: socket.getpeername() failure on broken TCP/IP connection http://bugs.python.org/issue28447 closed by martin.panter #28448: C implemented Future doesn't work on Windows http://bugs.python.org/issue28448 closed by inada.naoki #28452: Remove _asyncio._init_module function http://bugs.python.org/issue28452 closed by inada.naoki #28454: Spurious arguments to PyErr_Format in unicodeobject.c http://bugs.python.org/issue28454 closed by python-dev #28455: argparse: convert_arg_line_to_args does not actually expect se http://bugs.python.org/issue28455 closed by berker.peksag #28456: Test failures under macOS 10.12 Sierra http://bugs.python.org/issue28456 closed by ned.deily #28458: from __future__ import print_function does not emulate the flu http://bugs.python.org/issue28458 closed by berker.peksag #28461: Replacement of re with the regex package http://bugs.python.org/issue28461 closed by serhiy.storchaka #28466: SIGALRM fails to interrupt time.sleep() call on Python 3.5 http://bugs.python.org/issue28466 closed by martin.panter #28467: Installer should be a 64-bit executable http://bugs.python.org/issue28467 closed by steve.dower #28468: Add platform.linux_os_release() http://bugs.python.org/issue28468 closed by christian.heimes #28471: Python 3.6b2 crashes with "Python memory allocator called with http://bugs.python.org/issue28471 closed by yselivanov #28472: SystemTap usage examples in docs are incorrect http://bugs.python.org/issue28472 closed by python-dev #28473: mailbox.MH crashes on certain Claws Mail .mh_sequences files http://bugs.python.org/issue28473 closed by r.david.murray #28476: Remove redundant definition of factorial on test_random http://bugs.python.org/issue28476 closed by python-dev #28479: Missing indentation in using/windows.rst http://bugs.python.org/issue28479 closed by python-dev #28480: Compile error on Modules/socketmodule.c http://bugs.python.org/issue28480 closed by martin.panter #28481: Weird string comparison bug http://bugs.python.org/issue28481 closed by eryksun #28483: python --version prints output to stderr http://bugs.python.org/issue28483 closed by SilentGhost #28484: test_capi, test_regrtest fail when multithreading disabled http://bugs.python.org/issue28484 closed by martin.panter #28486: Wrong end index and subgroup for group match http://bugs.python.org/issue28486 closed by serhiy.storchaka #28487: missing _math.o target in 2.7 Makefile http://bugs.python.org/issue28487 closed by haypo #28490: inappropriate OS.Error "Invalid cross-device link" http://bugs.python.org/issue28490 closed by steve.newcomb #28492: C implementation of asyncio.Future doesn't set value of StopIt http://bugs.python.org/issue28492 closed by yselivanov #28493: Typo in _asynciomodule.c http://bugs.python.org/issue28493 closed by yselivanov #28495: MagickMock created by patсh annotation refuses to apply funct http://bugs.python.org/issue28495 closed by r.david.murray #1520879: make install change: Allow $DESTDIR to be relative http://bugs.python.org/issue1520879 closed by dgreiman

4 years, 7 months

Re: [Python-Dev] Python-Dev Digest, Vol 159, Issue 27

by Wang, Peter Xihong

Hi Victor, Thanks for the great contribution to the unified benchmark development! In addition to the OutReachy program that we are currently supporting, let us know how else we could help out in this effort. Other than micros and benchmarking ideas, we'd also like to hear suggestions from the community on workload development around real world use cases, especially in the enterprise world, cloud computing, data analytics, machine learning, high performance computing, etc. Thanks, Peter -----Original Message----- From: Python-Dev [mailto:python-dev-bounces+peter.xihong.wang=intel.com@python.org] On Behalf Of python-dev-request(a)python.org Sent: Thursday, October 20, 2016 9:00 AM To: python-dev(a)python.org Subject: Python-Dev Digest, Vol 159, Issue 27 Send Python-Dev mailing list submissions to python-dev(a)python.org To subscribe or unsubscribe via the World Wide Web, visit https://mail.python.org/mailman/listinfo/python-dev or, via email, send a message with subject or body 'help' to python-dev-request(a)python.org You can reach the person managing the list at python-dev-owner(a)python.org When replying, please edit your Subject line so it is more specific than "Re: Contents of Python-Dev digest..." Today's Topics: 1. Benchmarking Python and micro-optimizations (Victor Stinner) 2. Have I got my hg dependencies correct? (Skip Montanaro) 3. Re: Have I got my hg dependencies correct? (Skip Montanaro) 4. Re: Have I got my hg dependencies correct? (Victor Stinner) 5. Re: Benchmarking Python and micro-optimizations (Maciej Fijalkowski) 6. Re: Benchmarking Python and micro-optimizations (Eric Snow) ---------------------------------------------------------------------- Message: 1 Date: Thu, 20 Oct 2016 12:56:06 +0200 From: Victor Stinner <victor.stinner(a)gmail.com> To: Python Dev <Python-Dev(a)python.org> Subject: [Python-Dev] Benchmarking Python and micro-optimizations Message-ID: <CAMpsgwYsAXJgHfJH_hDH6pj13pHacMWYbi_LExLoXdcoUo6F-Q(a)mail.gmail.com> Content-Type: text/plain; charset=UTF-8 Hi, Last months, I worked a lot on benchmarks. I ran benchmarks, analyzed results in depth (up to the hardware and kernel drivers!), I wrote new tools and enhanced existing tools. * I wrote a new perf module which runs benchmarks in a reliable way and contains a LOT of features: collect metadata, JSON file format, commands to compare, render an histogram, etc. * I rewrote the Python benchmark suite: the old benchmarks Mercurial repository moved to a new performance GitHub project which uses my perf module and contains more benchmarks. * I also made minor enhancements to timeit in Python 3.7 -- some dev don't want major changes to not "break the backward compatibility". For timeit, I suggest to use my perf tool which includes a reliable timeit command and has much more features like --duplicate (repeat the statements to reduce the cost of the outer loop) and --compare-to (compare two versions of Python), but also all builtin perf features (JSON output, statistics, histogram, etc.). I added benchmarks from PyPy and Pyston benchmark suites to performance: performance 0.3.1 contains 51 benchmark scripts which run a total of 121 benchmarks. Example of tested Python modules: * SQLAlchemy * Dulwich (full Git implementation in Python) * Mercurial (currently only the startup time) * html5lib * pyaes (AES crypto cipher in pure Python) * sympy * Tornado (HTTP client and server) * Django (sadly, only the template engine right now, Pyston contains HTTP benchmarks) * pathlib * spambayes More benchmarks will be added later. It would be nice to add benchmarks on numpy for example, numpy is important for a large part of our community. All these (new or updated) tools can now be used to take smarter decisions on optimizations. Please don't push any optimization anymore without providing reliable benchmark results! My first major action was to close the latest attempt to micro-optimize int+int in Python/ceval.c, http://bugs.python.org/issue21955 : I closed the issue as rejected, because there is no significant speedup on benchmarks other than two (tiny) microbenchmarks. To make sure that no one looses its time on trying to micro-optimize int+int, I even added a comment to Python/ceval.c :-) https://hg.python.org/cpython/rev/61fcb12a9873 "Please don't try to micro-optimize int+int" The perf and performance are now well tested: Travis CI runs tests on the new commits and pull requests, and the "tox" command can be used locally to test different Python versions, pep8, doc, ... in a single command. Next steps: * Run performance 0.3.1 on speed.python.org: the benchmark runner is currently stopped (and still uses the old benchmarks project). The website part may be updated to allow to download full JSON files which includes *all* information (all timings, metadata and more). * I plan to run performance on CPython 2.7, CPython 3.7, PyPy and PyPy 3. Maybe also CPython 3.5 and CPython 3.6 if they don't take too much resources. * Later, we can consider adding more implementations of Python: Jython, IronPython, MicroPython, Pyston, Pyjion, etc. All benchmarks should be run on the same hardware to be comparable. * Later, we might also allow other projects to upload their own benchmark results, but we should find a solution to groups benchmark results per benchmark runner (ex: at least by the hostname, perf JSON contains the hostname) to not compare two results from two different hardware * We should continue to add more benchmarks to the performance benchmark suite, especially benchmarks more representative of real applications (we have enough microbenchmarks!) Links: * perf: http://perf.readthedocs.io/ * performance: https://github.com/python/performance * Python Speed mailing list: https://mail.python.org/mailman/listinfo/speed * https://speed.python.org/ (currently outdated, and don't use performance yet) See https://pypi.python.org/pypi/performance which contains even more links to Python benchmarks (PyPy, Pyston, Numba, Pythran, etc.) Victor ------------------------------ Message: 2 Date: Thu, 20 Oct 2016 06:47:42 -0500 From: Skip Montanaro <skip.montanaro(a)gmail.com> To: python-dev Dev <python-dev(a)python.org> Subject: [Python-Dev] Have I got my hg dependencies correct? Message-ID: <CANc-5UwuWxYkN5LgOdDPEO5-7xYDCLK5Xv6dFP=sLBaxbH1K9g(a)mail.gmail.com> Content-Type: text/plain; charset=UTF-8 I've recently run into a problem building the math and cmath modules for 2.7. (I don't rebuild very often, so this problem might have been around for awhile.) My hg repos look like this: * My cpython repo pulls from https://hg.python.org/cpython * My 2.7 repo (and other non-tip repos) pulls from my cpython repo I think this setup was recommended way back in the day when hg was new to the Python toolchain to avoid unnecessary network bandwidth. So, if I execute hg pull hg update in first cpython, then 2.7 repos I should be up-to-date, correct? However, rebuilding in my 2.7 repo fails to build math and cmath. The compiler complains that Modules/_math.o doesn't exist. If I manually execute make Modules/_math.o make after the failure, then the math and cmath modules build. Looking on bugs.python.org I saw this closed issue: http://bugs.python.org/issue24421 which seems related. Is it possible that the fix wasn't propagated to the 2.7 branch? Or perhaps I've fouled up my hg repo relationships? My other repos which depend on cpython (3.5, 3.4, 3.3, and 3.2) all build the math module just fine. I'm running on an ancient MacBook Pro with OS X 10.11.6 (El Capitan) and XCode 8.0 installed. Any suggestions? Skip ------------------------------ Message: 3 Date: Thu, 20 Oct 2016 07:13:47 -0500 From: Skip Montanaro <skip.montanaro(a)gmail.com> To: python-dev Dev <python-dev(a)python.org> Subject: Re: [Python-Dev] Have I got my hg dependencies correct? Message-ID: <CANc-5Uy+xE3OWaVDNy+EOchCPBhgiHmbetsBQxP9TLVGQo_8ZA(a)mail.gmail.com> Content-Type: text/plain; charset=UTF-8 On Thu, Oct 20, 2016 at 6:47 AM, Skip Montanaro <skip.montanaro(a)gmail.com> wrote: > Is it possible that the fix wasn't propagated to the 2.7 branch? Or > perhaps I've fouled up my hg repo relationships? Either way, I went ahead and opened a ticket: http://bugs.python.org/issue28487 S ------------------------------ Message: 4 Date: Thu, 20 Oct 2016 14:35:10 +0200 From: Victor Stinner <victor.stinner(a)gmail.com> To: Skip Montanaro <skip.montanaro(a)gmail.com> Cc: python-dev Dev <python-dev(a)python.org> Subject: Re: [Python-Dev] Have I got my hg dependencies correct? Message-ID: <CAMpsgwa0eK0nMa2PZk7PjRJGTVu1c5yBuZP0ROH_dSaWH4PyMw(a)mail.gmail.com> Content-Type: text/plain; charset=UTF-8 Are you on the 2.7 branch or the default branch? You might try to cleanup your checkout: hg up -C -r 2.7 make distclean hg purge # WARNING! it removes *all* files not tracked by Mercurial ./configure && make You should also paste the full error message. Victor 2016-10-20 13:47 GMT+02:00 Skip Montanaro <skip.montanaro(a)gmail.com>: > I've recently run into a problem building the math and cmath modules > for 2.7. (I don't rebuild very often, so this problem might have been > around for awhile.) My hg repos look like this: > > * My cpython repo pulls from https://hg.python.org/cpython > > * My 2.7 repo (and other non-tip repos) pulls from my cpython repo > > I think this setup was recommended way back in the day when hg was new > to the Python toolchain to avoid unnecessary network bandwidth. > > So, if I execute > > hg pull > hg update > > in first cpython, then 2.7 repos I should be up-to-date, correct? > However, rebuilding in my 2.7 repo fails to build math and cmath. The > compiler complains that Modules/_math.o doesn't exist. If I manually > execute > > make Modules/_math.o > make > > after the failure, then the math and cmath modules build. > > Looking on bugs.python.org I saw this closed issue: > > http://bugs.python.org/issue24421 > > which seems related. Is it possible that the fix wasn't propagated to > the 2.7 branch? Or perhaps I've fouled up my hg repo relationships? My > other repos which depend on cpython (3.5, 3.4, 3.3, and 3.2) all build > the math module just fine. > > I'm running on an ancient MacBook Pro with OS X 10.11.6 (El Capitan) > and XCode 8.0 installed. > > Any suggestions? > > Skip > _______________________________________________ > Python-Dev mailing list > Python-Dev(a)python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/victor.stinner%40gm > ail.com ------------------------------ Message: 5 Date: Thu, 20 Oct 2016 15:15:43 +0200 From: Maciej Fijalkowski <fijall(a)gmail.com> To: Victor Stinner <victor.stinner(a)gmail.com> Cc: Python Dev <Python-Dev(a)python.org> Subject: Re: [Python-Dev] Benchmarking Python and micro-optimizations Message-ID: <CAK5idxRK1Muf5X8OWot2pkUCvvAG2SgU6D+LDKnnaeUH3P7erA(a)mail.gmail.com> Content-Type: text/plain; charset=UTF-8 Hi Victor Despite the fact that I was not able to find time to run your stuff yet, thanks for all the awesome work! On Thu, Oct 20, 2016 at 12:56 PM, Victor Stinner <victor.stinner(a)gmail.com> wrote: > Hi, > > Last months, I worked a lot on benchmarks. I ran benchmarks, analyzed > results in depth (up to the hardware and kernel drivers!), I wrote new > tools and enhanced existing tools. > > * I wrote a new perf module which runs benchmarks in a reliable way > and contains a LOT of features: collect metadata, JSON file format, > commands to compare, render an histogram, etc. > > * I rewrote the Python benchmark suite: the old benchmarks Mercurial > repository moved to a new performance GitHub project which uses my > perf module and contains more benchmarks. > > * I also made minor enhancements to timeit in Python 3.7 -- some dev > don't want major changes to not "break the backward compatibility". > > For timeit, I suggest to use my perf tool which includes a reliable > timeit command and has much more features like --duplicate (repeat the > statements to reduce the cost of the outer loop) and --compare-to > (compare two versions of Python), but also all builtin perf features > (JSON output, statistics, histogram, etc.). > > I added benchmarks from PyPy and Pyston benchmark suites to > performance: performance 0.3.1 contains 51 benchmark scripts which run > a total of 121 benchmarks. Example of tested Python modules: > > * SQLAlchemy > * Dulwich (full Git implementation in Python) > * Mercurial (currently only the startup time) > * html5lib > * pyaes (AES crypto cipher in pure Python) > * sympy > * Tornado (HTTP client and server) > * Django (sadly, only the template engine right now, Pyston contains > HTTP benchmarks) > * pathlib > * spambayes > > More benchmarks will be added later. It would be nice to add > benchmarks on numpy for example, numpy is important for a large part > of our community. > > All these (new or updated) tools can now be used to take smarter > decisions on optimizations. Please don't push any optimization anymore > without providing reliable benchmark results! > > > My first major action was to close the latest attempt to > micro-optimize int+int in Python/ceval.c, > http://bugs.python.org/issue21955 : I closed the issue as rejected, > because there is no significant speedup on benchmarks other than two > (tiny) microbenchmarks. To make sure that no one looses its time on > trying to micro-optimize int+int, I even added a comment to > Python/ceval.c :-) > > https://hg.python.org/cpython/rev/61fcb12a9873 > "Please don't try to micro-optimize int+int" > > > The perf and performance are now well tested: Travis CI runs tests on > the new commits and pull requests, and the "tox" command can be used > locally to test different Python versions, pep8, doc, ... in a single > command. > > > Next steps: > > * Run performance 0.3.1 on speed.python.org: the benchmark runner is > currently stopped (and still uses the old benchmarks project). The > website part may be updated to allow to download full JSON files which > includes *all* information (all timings, metadata and more). > > * I plan to run performance on CPython 2.7, CPython 3.7, PyPy and PyPy > 3. Maybe also CPython 3.5 and CPython 3.6 if they don't take too much > resources. > > * Later, we can consider adding more implementations of Python: > Jython, IronPython, MicroPython, Pyston, Pyjion, etc. All benchmarks > should be run on the same hardware to be comparable. > > * Later, we might also allow other projects to upload their own > benchmark results, but we should find a solution to groups benchmark > results per benchmark runner (ex: at least by the hostname, perf JSON > contains the hostname) to not compare two results from two different > hardware > > * We should continue to add more benchmarks to the performance > benchmark suite, especially benchmarks more representative of real > applications (we have enough microbenchmarks!) > > > Links: > > * perf: http://perf.readthedocs.io/ > * performance: https://github.com/python/performance > * Python Speed mailing list: > https://mail.python.org/mailman/listinfo/speed > * https://speed.python.org/ (currently outdated, and don't use > performance yet) > > See https://pypi.python.org/pypi/performance which contains even more > links to Python benchmarks (PyPy, Pyston, Numba, Pythran, etc.) > > Victor > _______________________________________________ > Python-Dev mailing list > Python-Dev(a)python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/fijall%40gmail.com ------------------------------ Message: 6 Date: Thu, 20 Oct 2016 09:31:42 -0600 From: Eric Snow <ericsnowcurrently(a)gmail.com> To: Victor Stinner <victor.stinner(a)gmail.com> Cc: Python Dev <Python-Dev(a)python.org> Subject: Re: [Python-Dev] Benchmarking Python and micro-optimizations Message-ID: <CALFfu7AZsqrazOvfSDMMruqqTumUvxELWrijV08uAsgnbRnwfw(a)mail.gmail.com> Content-Type: text/plain; charset=UTF-8 On Thu, Oct 20, 2016 at 4:56 AM, Victor Stinner <victor.stinner(a)gmail.com> wrote: > Hi, > > Last months, I worked a lot on benchmarks. I ran benchmarks, analyzed > results in depth (up to the hardware and kernel drivers!), I wrote new > tools and enhanced existing tools. This is a massive contribution. Thanks! > All these (new or updated) tools can now be used to take smarter > decisions on optimizations. Please don't push any optimization anymore > without providing reliable benchmark results! +1 -eric ------------------------------ Subject: Digest Footer _______________________________________________ Python-Dev mailing list Python-Dev(a)python.org https://mail.python.org/mailman/listinfo/python-dev ------------------------------ End of Python-Dev Digest, Vol 159, Issue 27 *******************************************

4 years, 7 months

typing.py doesn't export Pattern and Match; is that intentional

by Ryan Gonzalez

E.g.: import typing print(typing.Pattern, typing.Match) # Works. from typing import * print(Pattern, Match) # NameError: name 'Pattern' is not defined A quick look shows that typing.py doesn't have Pattern and Match in __all__. Was this intentional, or just an oversight? -- Ryan [ERROR]: Your autotools build scripts are 200 lines longer than your program. Something’s wrong. http://kirbyfan64.github.io/

4 years, 8 months

O(1) deletes from the front of bytearray (was: Re: Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor)

by Nathaniel Smith

On Tue, Oct 11, 2016 at 10:53 PM, Serhiy Storchaka <storchaka(a)gmail.com> wrote: > On 12.10.16 08:03, Nathaniel Smith wrote: >> >> On Tue, Oct 11, 2016 at 9:08 PM, INADA Naoki <songofacandy(a)gmail.com> >> wrote: >>> >>> From Python 3.4, bytearray is good solution for I/O buffer, thanks to >>> #19087 [1]. >>> Actually, asyncio uses bytearray as I/O buffer often. >> >> >> Whoa what?! This is awesome, I had no idea that bytearray had O(1) >> deletes at the front. I literally reimplemented this myself on type of >> bytearray for some 3.5-only code recently because I assumed bytearray >> had the same asymptotics as list, and AFAICT this is totally >> undocumented. Shouldn't we write this down somewhere? Maybe here? -> >> https://docs.python.org/3/library/functions.html#bytearray > > > I afraid this is CPython implementation detail (like string concatenation > optimization). Other implementations can have O(N) deletes at the front of > bytearray. Well, it shouldn't be :-). The problem with the string concatenation optimization is that to work, it requires both the use of refcounting GC and that you get lucky with how the underlying malloc implementation happened to lay things out in memory. Obviously it shouldn't be part of the language spec. But amortized O(1) deletes from the front of bytearray are totally different, and more like amortized O(1) appends to list: there are important use cases[1] that simply cannot be implemented without some feature like this, and putting the implementation inside bytearray is straightforward, deterministic, and more efficiently than hacking together something on top. Python should just guarantee it, IMO. -n [1] My use case is parsing HTTP out of a receive buffer. If deleting the first k bytes of an N byte buffer is O(N), then not only does parsing becomes O(N^2) in the worst case, but it's the sort of O(N^2) that random untrusted network clients can trigger at will to DoS your server. -- Nathaniel J. Smith -- https://vorpus.org

4 years, 8 months

Re: [Python-Dev] [Python-checkins] cpython (3.5): remove extra PyErr_Format arguments (closes #28454)

by Jeremy Kloth

On Sun, Oct 16, 2016 at 4:42 PM, benjamin.peterson <python-checkins(a)python.org> wrote: > --- a/Objects/unicodeobject.c > +++ b/Objects/unicodeobject.c > @@ -3009,7 +3009,7 @@ > "'%.400s' decoder returned '%.400s' instead of 'str'; " > "use codecs.decode() to decode to arbitrary types", > encoding, > - Py_TYPE(unicode)->tp_name, Py_TYPE(unicode)->tp_name); > + Py_TYPE(unicode)->tp_name); > Py_DECREF(unicode); > goto onError; > } Um, isn't that now an error? There are 2 format sequences, but only 1 argument. -- Jeremy Kloth

4 years, 8 months

Re: [Python-Dev] single dispatch for instance methods

by Aric Coady

On Oct 14, 2016, at 8:49 AM, python-dev-request(a)python.org wrote: > Date: Fri, 14 Oct 2016 14:11:18 +1300 > From: Tim Mitchell <tim.mitchell(a)leapfrog3d.com> > It would be nice if the @singledispatch decorator worked on instance > methods. > I have a reference implementation on pypi that does this ( > https://pypi.python.org/pypi/methoddispatch). You might be interested in this thread from my multimethod library: https://bitbucket.org/coady/multimethod/issues/2/ <https://bitbucket.org/coady/multimethod/issues/2/>. It was a feature request for multiple dispatch to work on instance methods. I implemented a variation where the initial self argument still contributed to the dispatch. I think the same reasoning would apply in the single dispatch case: that it’s not obvious the “dispatch” would occur once on the self argument to find the method, and then again on the “first” argument. Especially since Py3 did away with bound methods, so you can call a method through its class and explicitly supply self as the first argument.

4 years, 8 months

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

Python-Dev October 2016