Mailman 3 October 2015 - Python-Dev

Re: [Python-Dev] Generated Bytecode ...
by Terry Reedy 27 Oct '15

27 Oct '15

On 10/23/2015 4:23 AM, Victor Stinner wrote: > Hi, > > 2015-10-22 19:02 GMT+02:00 Brett Cannon <brett(a)python.org>: >> It's not specified anywhere; it's just what the peepholer decides to remove. >> The exact code can be found at >> https://hg.python.org/cpython/file/default/Python/peephole.c . There has >> been talk in the past for adding a -X flag to disable the peepholer, but it >> never went any farther beyond that. > > Yeah, I remember that I had the same issue than Stéphane when I worked > on my astoptimizer project :-) I wanted to completly disable the > peephole optimizer because I wanted to reimplement the optimizations > on the AST instead of rewriting the bytecode. > > I would be nice to have a "-X noopt" command line option for that. How about -x nopeep to specifically skip the peephole optimizer? -- Terry Jan Reedy

2 2

(no subject)
by Maciej Szulik 26 Oct '15

26 Oct '15

Thanks to Nick Coghlan and Barry Warsaw we've setup a new SIG dedicated to discussing python features from different distributions point of view. Here is Nick's reasoning: > With the Python 3 migration, and the growth in interest in user level > package management for development purposes, what do you think of the idea > of setting up a new Linux SIG to have those discussions? I know it's a case > of "yet another mailing list", but I think it will be worthwhile to have a > clear point of collaboration within the Python ecosystem, rather than > expecting Pythonistas to know how to reach out to (other) distros directly. The list is available @ https://mail.python.org/mailman/listinfo/linux-sig Maciej

1 0

Where is defined the grammar of Python?
by Stéphane Wirtel 26 Oct '15

26 Oct '15

Hi all, Just to understand, we have the Parser/Python.asdl and Grammar/Grammar files. Which one is used for the AST ? I would like to understand this part of Python, could you help me? Thank you Stéphane -- Stéphane Wirtel - http://wirtel.be - @matrixise

4 3

PEP 484 -- proposal to allow @overload in non-stub files
by Guido van Rossum 25 Oct '15

25 Oct '15

PEP 484 (Type Hinting) currently disallows @overload outside stub files. The hope was that a PEP for multi-dispatch would emerge, but that's been slow coming. Meanwhile, over in https://github.com/ambv/typehinting/issues/72 a proposal has emerged to allow @overload in regular modules, as long as it is followed by a non-overloaded definition that serves as the default/fallback. A motivating example is __getitem__, which is often overloaded for item access and slicing. In stubs, you can use: class Foo(Generic[T]): @overload def __getitem__(self, i: int) -> T: ... @overload def __getitem__(self, s: slice) -> Foo[T]: ... (Note that the '...' are part of the actual code -- an ellipsis is how you represent the body of all functions in stub files.) However, in source files the best you can do is: class Foo(Generic[T]): def __getitem__(self, i: Union[int, slice]) -> Union[T, List[T]]: ... which will require unacceptable casts at every call site. You can work around it by having a stub file but that's cumbersome if this is the only reason to have one. The proposal is to allow this to be written as follows in implementation (non-stub) modules: class Foo(Generic[T]): @overload def __getitem__(self, i: int) -> T: ... @overload def __getitem__(self, s: slice) -> Foo[T]: ... def __getitem__(self, x): <actual implementation goes here> The actual implementation must be last, so at run time it will override the definition. It has to use isinstance() to distinguish the cases. A type checker would have to recognize this as a special case (so as not to complain about the non-overloaded version). Jukka thinks it would be about a day's work to implement in mypy; the work in typing.py would be a few minutes. Thoughts? -- --Guido van Rossum (python.org/~guido)

6 9

Generated Bytecode ...
by Stéphane Wirtel 25 Oct '15

25 Oct '15

Hi all, When we compile a python script # test.py if 0: x = 1 python -mdis test.py There is no byte code for the condition. So my question is, the byte code generator removes the unused functions, variables etc…, is it right? What are the cases where the generator does not generate the byte codes ? Thank you, Stéphane -- Stéphane Wirtel - http://wirtel.be - @matrixise

5 6

Summary of Python tracker Issues
by Python tracker 23 Oct '15

23 Oct '15

ACTIVITY SUMMARY (2015-10-16 - 2015-10-23) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 5181 (+29) closed 32043 (+17) total 37224 (+46) Open issues with patches: 2287 Issues opened (40) ================== #24379: operator.subscript http://bugs.python.org/issue24379 reopened by rhettinger #25154: Drop the pyvenv script http://bugs.python.org/issue25154 reopened by brett.cannon #25234: test_eintr.test_os_open hangs under Xcode 7 http://bugs.python.org/issue25234 reopened by brett.cannon #25421: Make __sizeof__ for builtin types more subclass friendly http://bugs.python.org/issue25421 opened by serhiy.storchaka #25423: Deprecate benchmarks that execute too quickly http://bugs.python.org/issue25423 opened by brett.cannon #25425: white-spaces encountered in 3.4 http://bugs.python.org/issue25425 opened by pavan kumar Dharmavarapu #25426: Deprecate the regex_compile benchmark http://bugs.python.org/issue25426 opened by brett.cannon #25427: Remove the pyvenv script in Python 3.8 http://bugs.python.org/issue25427 opened by brett.cannon #25430: speed up ipaddress __contain__ method http://bugs.python.org/issue25430 opened by gescheit #25431: implement address in network in ipaddress module http://bugs.python.org/issue25431 opened by gescheit #25432: isinstance documentation doesn't explain what happens when typ http://bugs.python.org/issue25432 opened by Michael Crouch #25433: whitespace in strip()/lstrip()/rstrip() http://bugs.python.org/issue25433 opened by Dimitri Papadopoulos Orfanos #25435: Wrong function calls and referring to not removed concepts in http://bugs.python.org/issue25435 opened by David Becher #25436: argparse.ArgumentError missing error message in __repr__ http://bugs.python.org/issue25436 opened by dfortunov #25437: Issue with ftplib.FTP_TLS and server forcing SSL connection re http://bugs.python.org/issue25437 opened by dwaites #25438: document what codec PyMemberDef T_STRING decodes the char * as http://bugs.python.org/issue25438 opened by gregory.p.smith #25439: Add type checks to urllib.request.Request http://bugs.python.org/issue25439 opened by ezio.melotti #25442: Shelve consistency issues http://bugs.python.org/issue25442 opened by Yanyan Jiang #25443: Add a count of how many benchmarks are left to run http://bugs.python.org/issue25443 opened by brett.cannon #25444: Py Launch Icon http://bugs.python.org/issue25444 opened by Nils-Hero #25446: smtplib.py AUTH LOGIN code messed up sending login and passwor http://bugs.python.org/issue25446 opened by merkel #25447: TypeError invoking deepcopy on lru_cache http://bugs.python.org/issue25447 opened by jason.coombs #25449: Test OrderedDict subclass http://bugs.python.org/issue25449 opened by serhiy.storchaka #25450: Python 3.5 starts in C:\Windows\system32 as current directory http://bugs.python.org/issue25450 opened by Maja Tomic #25451: tkinter: PhotoImage transparency methods http://bugs.python.org/issue25451 opened by None Becoming #25452: Add __bool__() method to subprocess.CompletedProcess http://bugs.python.org/issue25452 opened by conqp #25453: Arithmetics with complex infinities is inconsistent with C/C++ http://bugs.python.org/issue25453 opened by Francesco Biscani #25454: operator.methodcaller should accept additional arguments durin http://bugs.python.org/issue25454 opened by abacabadabacaba #25455: Some repr implementations don't check for self-referential str http://bugs.python.org/issue25455 opened by abacabadabacaba #25456: 3.4 _tkinter build requires undocumented manual steps to be us http://bugs.python.org/issue25456 opened by terry.reedy #25457: json dump fails for mixed-type keys when sort_keys is specifie http://bugs.python.org/issue25457 opened by tanzer(a)swing.co.at #25458: ftplib: command response shift - mismatch http://bugs.python.org/issue25458 opened by peterpan #25459: EAGAIN errors in Python logging module http://bugs.python.org/issue25459 opened by Henrique Andrade #25460: Misc/.gdbinit uses preprocessor macro http://bugs.python.org/issue25460 opened by philtweir #25461: Unclear language (the word ineffective) in the documentation f http://bugs.python.org/issue25461 opened by Bernt.Røskar.Brenna #25462: Avoid repeated hash calculation in C implementation of Ordered http://bugs.python.org/issue25462 opened by serhiy.storchaka #25463: 2.7.10 glibc double free detected http://bugs.python.org/issue25463 opened by wcolburnnrao #25464: Tix HList header_exists should be "exist" http://bugs.python.org/issue25464 opened by rtw #25465: Pickle uses O(n) memory overhead http://bugs.python.org/issue25465 opened by prinsherbert #25466: offer "from __future__ import" option for "raise... from" http://bugs.python.org/issue25466 opened by alanf Most recent 15 issues with no replies (15) ========================================== #25466: offer "from __future__ import" option for "raise... from" http://bugs.python.org/issue25466 #25462: Avoid repeated hash calculation in C implementation of Ordered http://bugs.python.org/issue25462 #25458: ftplib: command response shift - mismatch http://bugs.python.org/issue25458 #25455: Some repr implementations don't check for self-referential str http://bugs.python.org/issue25455 #25454: operator.methodcaller should accept additional arguments durin http://bugs.python.org/issue25454 #25451: tkinter: PhotoImage transparency methods http://bugs.python.org/issue25451 #25443: Add a count of how many benchmarks are left to run http://bugs.python.org/issue25443 #25433: whitespace in strip()/lstrip()/rstrip() http://bugs.python.org/issue25433 #25431: implement address in network in ipaddress module http://bugs.python.org/issue25431 #25430: speed up ipaddress __contain__ method http://bugs.python.org/issue25430 #25427: Remove the pyvenv script in Python 3.8 http://bugs.python.org/issue25427 #25421: Make __sizeof__ for builtin types more subclass friendly http://bugs.python.org/issue25421 #25416: Add encoding aliases from the (HTML5) Encoding Standard http://bugs.python.org/issue25416 #25413: ctypes (libffi) fails to compile on Solaris X86 http://bugs.python.org/issue25413 #25397: improve ac_cv_have_long_long_format GCC fallback http://bugs.python.org/issue25397 Most recent 15 issues waiting for review (15) ============================================= #25464: Tix HList header_exists should be "exist" http://bugs.python.org/issue25464 #25462: Avoid repeated hash calculation in C implementation of Ordered http://bugs.python.org/issue25462 #25461: Unclear language (the word ineffective) in the documentation f http://bugs.python.org/issue25461 #25460: Misc/.gdbinit uses preprocessor macro http://bugs.python.org/issue25460 #25449: Test OrderedDict subclass http://bugs.python.org/issue25449 #25447: TypeError invoking deepcopy on lru_cache http://bugs.python.org/issue25447 #25446: smtplib.py AUTH LOGIN code messed up sending login and passwor http://bugs.python.org/issue25446 #25439: Add type checks to urllib.request.Request http://bugs.python.org/issue25439 #25436: argparse.ArgumentError missing error message in __repr__ http://bugs.python.org/issue25436 #25435: Wrong function calls and referring to not removed concepts in http://bugs.python.org/issue25435 #25431: implement address in network in ipaddress module http://bugs.python.org/issue25431 #25430: speed up ipaddress __contain__ method http://bugs.python.org/issue25430 #25421: Make __sizeof__ for builtin types more subclass friendly http://bugs.python.org/issue25421 #25420: "import random" blocks on entropy collection on Linux with low http://bugs.python.org/issue25420 #25419: Readline completion of module names in import statements http://bugs.python.org/issue25419 Top 10 most discussed issues (10) ================================= #25410: Clean up and fix OrderedDict http://bugs.python.org/issue25410 22 msgs #25453: Arithmetics with complex infinities is inconsistent with C/C++ http://bugs.python.org/issue25453 14 msgs #25461: Unclear language (the word ineffective) in the documentation f http://bugs.python.org/issue25461 12 msgs #25420: "import random" blocks on entropy collection on Linux with low http://bugs.python.org/issue25420 11 msgs #25446: smtplib.py AUTH LOGIN code messed up sending login and passwor http://bugs.python.org/issue25446 8 msgs #25395: SIGSEGV using json.tool: highly nested OrderedDict http://bugs.python.org/issue25395 7 msgs #23735: Readline not adjusting width after resize with 6.3 http://bugs.python.org/issue23735 6 msgs #25154: Drop the pyvenv script http://bugs.python.org/issue25154 6 msgs #25447: TypeError invoking deepcopy on lru_cache http://bugs.python.org/issue25447 6 msgs #25439: Add type checks to urllib.request.Request http://bugs.python.org/issue25439 5 msgs Issues closed (18) ================== #13195: subprocess: args with shell=True is not documented on Windows http://bugs.python.org/issue13195 closed by martin.panter #23981: Update test_unicodedata.py to use script_helpers http://bugs.python.org/issue23981 closed by berker.peksag #24885: StreamReaderProtocol docs recommend using private API http://bugs.python.org/issue24885 closed by gvanrossum #25390: Can't define a typing.Union containing a typing.re.Pattern http://bugs.python.org/issue25390 closed by gvanrossum #25407: Update PEP 4 to keep modules in Python 3 http://bugs.python.org/issue25407 closed by brett.cannon #25408: Consider dropping html5lib and spambayes from the default benc http://bugs.python.org/issue25408 closed by brett.cannon #25411: SMTPHandler in the logging module fails with unicode strings http://bugs.python.org/issue25411 closed by python-dev #25414: Bypass unnecessary size limit test from deques on builds with http://bugs.python.org/issue25414 closed by rhettinger #25417: Minor typo in Path.samefile docstring http://bugs.python.org/issue25417 closed by berker.peksag #25422: tokenize: add tests for multi-line strings http://bugs.python.org/issue25422 closed by eric.smith #25424: Deprecate older versions of benchmarks http://bugs.python.org/issue25424 closed by brett.cannon #25428: Have `datetime` understand integer arguments for timezones http://bugs.python.org/issue25428 closed by r.david.murray #25429: Can segfault Python with itertools.chain.from_iterable http://bugs.python.org/issue25429 closed by rhettinger #25434: Fix typo in whatsnew/3.5 http://bugs.python.org/issue25434 closed by berker.peksag #25440: python3.4-config --extension-suffix not expanded http://bugs.python.org/issue25440 closed by doko #25441: StreamWriter.drain() unreliably reports closed sockets http://bugs.python.org/issue25441 closed by gvanrossum #25445: type xterm in python http://bugs.python.org/issue25445 closed by r.david.murray #25448: Exception ABC doesn't work in Python 3 (but does in Python 2.7 http://bugs.python.org/issue25448 closed by eryksun

1 0

PEP-8 wart... it recommends short names because of DOS
by Gregory P. Smith 22 Oct '15

22 Oct '15

https://www.python.org/dev/peps/pep-0008/#names-to-avoid *"Since module names are mapped to file names, and some file systems are case insensitive and truncate long names, it is important that module names be chosen to be fairly short -- this won't be a problem on Unix, but it may be a problem when the code is transported to older Mac or Windows versions, or DOS."* There haven't been computers with less than 80 character file or path name element length limits in wide use in decades... ;) -gps

6 7

Re: [Python-Dev] [Python-checkins] Daily reference leaks (d7e490db8d54): sum=61494
by Antoine Pitrou 21 Oct '15

21 Oct '15

> These leaks have been here a while. Anyone know the cause? > > On Tue, 20 Oct 2015 at 01:47 <solipsis at pitrou.net> wrote: > >> results for d7e490db8d54 on branch "default" >> -------------------------------------------- >> >> test_capi leaked [5411, 5411, 5411] references, sum=16233 >> test_capi leaked [1421, 1423, 1423] memory blocks, sum=4267 >> test_functools leaked [0, 2, 2] memory blocks, sum=4 >> test_threading leaked [10820, 10820, 10820] references, sum=32460 >> test_threading leaked [2842, 2844, 2844] memory blocks, sum=8530 Bisection shows they were probably introduced by: changeset: 97413:dccc4e63aef5 user: Raymond Hettinger <python(a)rcn.com> date: Sun Aug 16 19:43:34 2015 -0700 files: Doc/library/operator.rst Doc/whatsnew/3.6.rst Lib/operator.py Lib/test/test_operator.py description: Issue #24379: Add operator.subscript() as a convenience for building slices. If you comment out `(a)object.__new__` on line 411 in operator.py, or if you remove the __slots__ assignment (which is a bit worrying), the leak seems suppressed. You can reproduce using: $ ./python -m test -m SubinterpreterTest -R 3:3 test_capi [1/1] test_capi beginning 6 repetitions 123456 ...... test_capi leaked [5443, 5443, 5443] references, sum=16329 test_capi leaked [1432, 1434, 1434] memory blocks, sum=4300 1 test failed: test_capi Regards Antoine.

11 10

Re: [Python-Dev] [Python-checkins] Daily reference leaks (d7e490db8d54): sum=61494
by Brett Cannon 20 Oct '15

20 Oct '15

These leaks have been here a while. Anyone know the cause? On Tue, 20 Oct 2015 at 01:47 <solipsis(a)pitrou.net> wrote: > results for d7e490db8d54 on branch "default" > -------------------------------------------- > > test_capi leaked [5411, 5411, 5411] references, sum=16233 > test_capi leaked [1421, 1423, 1423] memory blocks, sum=4267 > test_functools leaked [0, 2, 2] memory blocks, sum=4 > test_threading leaked [10820, 10820, 10820] references, sum=32460 > test_threading leaked [2842, 2844, 2844] memory blocks, sum=8530 > > > Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', > '3:3:/home/psf-users/antoine/refleaks/reflogyrNnBL', '--timeout', '7200'] > _______________________________________________ > Python-checkins mailing list > Python-checkins(a)python.org > https://mail.python.org/mailman/listinfo/python-checkins >

1 0

compatibility for C-accelerated types
by Eric Snow 20 Oct '15

20 Oct '15

A recent discussion in a tracker issue [1] brought up the matter of compatibility between the pure Python implementation of OrderedDict and the new C implementation. In working on that port I stuck as closely as possible to the Python implementation. This meant some parts of the code are bit more complex than they would be otherwise. (Serhiy has been kind enough to do some cleanup.) Compatibility was one of the fundamental goals of the porting effort. Not only does compatibility make sense but it's also specifically required by PEP 399 [2]: Any new accelerated code must act as a drop-in replacement as close to the pure Python implementation as reasonable. Technical details of the VM providing the accelerated code are allowed to differ as necessary, e.g., a class being a type when implemented in C. For the most part I have questions about what is "reasonable", specifically in relation to OrderedDict. I've already opened up a separate thread related to my main question: type(obj) vs. obj.__class__. [3] In the tracker issue, Serhiy pointed out: There is no a difference. io, pickle, ElementTree, bz2, virtually all accelerator classes was created as replacements of pure Python implementations. All C implementations use Py_TYPE(self) for repr() and pickling. I think this deviation is common and acceptable. In a review comment on the associated patch he said: Isn't type(self) is always the same as self.__class__ for pure Python class? If right, then this change doesn't have any effect. To which he later replied: It is the same if you assigned the __class__ attribute, but can be different if set __class__ in the subclass declaration. So it isn't clear if that is a compatibility break or how much so it might be. Serhiy also noted that, as of 3.5 [4], you can no longer assign to obj.__class__ for instances of subclasses of builtin (non-heap) types. So that is another place where the two OrderedDict implementations differ. I expect there are a few others in dark corner cases. On the tracker he notes another OrderedDict compatibility break: Backward compatibility related to __class__ assignment was already broken in C implementation. In 3.4 following code works: >>> from collections import * >>> class foo(OrderedDict): ... def bark(self): return "spam" ... >>> class bar(OrderedDict): ... pass ... >>> od = bar() >>> od.__class__ = foo >>> od.bark() 'spam' In 3.5 it doesn't. As PEP 399 says, we should go as far "as reasonable" in the pursuit of compatibility. At the same time, I feel not insignificant responsibility for *any* incompatibility that comes from the C implementation of OrderedDict. The corner cases impacted by the above compatibility concerns are borderline enough that I wanted to get some feedback. Thanks! -eric [1] http://bugs.python.org/issue25410 [2] https://www.python.org/dev/peps/pep-0399/ [3] https://mail.python.org/pipermail/python-dev/2015-October/141953.html [4] http://bugs.python.org/issue24912

4 9