Python-Dev May 2015

python-dev@python.org

112 participants
85 discussions

by Larry Hastings

Right now we have eight online buildbots for Python trunk. Of those, currently *six* are reporting errors in either the compile or test phases. http://buildbot.python.org/all/waterfall?category=3.x.stable There's one platform ("AMD64 Snow Leop") where the failures are sporadic "stack overflow" errors encountered during the test suite. But the other five platforms have consistent failures, build after build. Those platforms: AMD64 OpenIndiana AMD64 Windows7 SP1 x86 Windows Server 2003 [SB] x86 Windows7 x86 XP-4 That includes *all* of the Windows buildbots. Gosh, it sure would be nice if Beta 1 didn't fail on Windows, wouldn't it? Could some Windows core devs take a look at the failures and see about cleaning them up? Naturally the OpenIndiana and OS X devs are invited to fix the errors on those platforms too, //arry/ p.s. My apologies for not bringing attention to this sooner. But, well, we still have a day left, right?

5 years, 10 months

PEP 484 (Type Hints) announcement

by Mark Shannon

Hello all, I am pleased to announce that I am accepting PEP 484 (Type Hints). Given the proximity of the beta release I thought I would get this announcement out now, even though there are some (very) minor details to iron out. (If you want to know the details, it's all at https://github.com/ambv/typehinting) I hope that PEP 484 will be a benefit to all users of Python. I think the proposed annotation semantics and accompanying module are technically sound and I hope that they are socially acceptable to the Python community. I have long been aware that as well as a powerful, sophisticated and "production quality" language, Python is also used by many casual programmers, and as a language to introduce children to programming. I also realise that this PEP does not look like it will be any help to the part-time programmer or beginner. However, I am convinced that it will enable significant improvements to IDEs (hopefully including IDLE), static checkers and other tools. These tools will then help us all, beginners included. This PEP has been a huge amount of work, involving a lot of people. So thank you to everyone involved. If I were to list names I would inevitably miss someone out. You know who you are. Finally, if you are worried that this will make Python ugly and turn it into some sort of inferior Java, then I share you concerns, but I would like to remind you of another potential ugliness; operator overloading. C++, Perl and Haskell have operator overloading and it gets abused something rotten to produce "concise" (a.k.a. line noise) code. Python also has operator overloading and it is used sensibly, as it should be. Why? It's a cultural issue; readability matters. Python is your language, please use type-hints responsibly :) Cheers, Mark.

5 years, 10 months

Hello, I am André Freitas :)

by André Freitas

Hi there, My name is André Freitas, I'm 22 years old and I live in Portugal (Porto). I'm currently finishing my Masters in Informatics and Computer Science with a thesis in Mining Software Repositories, where I am able to predict defects in Software components: https://github.com/andrefreitas/schwa I'm a Python developer with 4 years of experience and as a Speaker, did a lot of Python workshops in Engineering Universities. I'm always learning new things and I really love Python (it's my religion)! I have skills in Security, Tests and Quality, Devops, Software Architecture and Engineering, UI/UX and Compilers. I am reading your guidelines and just checking around to see how this mailing list works. Hope to send some patches soon and share some ideas. Feel free to follow me on Github https://github.com/andrefreitas Best regards, André Freitas.

5 years, 10 months

Accepting PEP 489 (Multi-phase extension module initialization)

by Eric Snow

Hi all, After extended discussion over the last several months on import-sig, the resulting proposal for multi-phase (PEP 451) extension module initialization has finalized. The resulting PEP provides a clean, straight-forward, and backward-compatible way to import extension modules using ModuleSpecs. With that in mind and given the improvement it provides, PEP 489 is now accepted. I want to thank Petr, Nick, and Stefan for the time, thought, and effort they put into the proposal (and implementation). It was a disappointment to me when, at the time, we couldn't find a good way to apply PEP 451 to builtins and extension modules. So thanks for easing my anxiety! -eric

5 years, 10 months

How shall we conduct the Python 3.5 beta and rc periods? (Please vote!)

by Larry Hastings

Python 3.5 beta 1 is coming up soon. After beta is rc; after rc is 3.5.0 final. During the beta and rc periods the Python developer workflow changes a little--what sorts of checkins are permissible, and how to get something accepted and merged generally becomes more complicated. I was the release manager for Python 3.4, and the way I ran the rc period for 3.4 was miserable. Everyone hated it--and that includes me. I'm absolutely not doing it that way for 3.5. But that leads one to the question: okay, how *am* I going to run it? I have three ideas for workflows for the beta and rc periods for 3.5, below. But first let's talk about what I/we hope to achieve. Traditionally during the beta and rc periods, development of new features stops completely. Obviously, after feature freeze no new features can go into the beta / rc releases. But why can't people develop new features for the *next* release? The reason given is that we're trying to guide the core dev community in the right direction--they should concentrate on fixing bugs in the new release. I suspect the real reason for this is that back in the bad old days of Subversion (and CVS!) branching and merging were painful. This social engineering policy is a justification after-the-fact. But Mercurial makes branching and merging nearly effortless. Mercurial also makes it painless to develop new features in private. So consider: anyone who wants to work on new features during beta and rc can easily do so. All our "no new features during beta and rc" policy really does is drive such development away out of public view. I think it's time we experimented with lifting that policy. The trick is finding a good place for us to check in the work. You see, during the 3.5 rc period, we arguably want to accept checkins for *three* different revisions: 3.5.0 3.5.1 3.6 3.5.1? Yep. During the rc period for 3.4--and particularly after the last rc was tagged--there were a lot of minor fixes that were desirable, but I didn't want to accept into 3.4 just to avoid destabilizing the release. I still wanted those checkins, I just didn't want them in 3.4.0. So the way it worked was, developers would check those bugfixes in to trunk, then I'd cherry-pick the revisions I wanted into the rc branch. In other words, during the rc period for 3.4, trunk effectively represented 3.4.1. This was valuable and I absolutely want to do it again. So here are the workflows. Workflow 0 most resembles what we've done in the past. Workflow 1 is my favorite and the most ambitious. All three give us a public place to put revisions for 3.5.0 and 3.5.1; Workflow 1 also gives us a place to check in work for 3.6 during the beta and rc periods for 3.5. Workflow 2 is a less ambitious compromise. Workflow 0 ========== When I ship beta 1, trunk remains 3.5. When I ship rc 1, trunk becomes 3.5.1. I create a publically visible read-only repo that represents 3.5.0, and any checkins that I want to accept into 3.5.0 I must manually cherry-pick from trunk. When I ship Python 3.5.0 final, we branch 3.5, and trunk becomes 3.6. Workflow 1 ========== When I ship beta 1, we create the 3.5 branch. trunk become 3.6. When I ship rc 1, the 3.5 branch becomes 3.5.1. I maintain a publically visible repo /on bitbucket/ for 3.5.0, and we use bitbucket "pull requests" for cherry-picks from 3.5.1 into 3.5.0. This gives us a pilot project to try out a web GUI for merging. It makes my workflow easier, as I can push a button to accept cherry-picks. (If they don't apply cleanly I can tell the author to go fix the conflict and resubmit it.) The downside: it requires core devs to have and use bitbucket accounts. Workflow 2 ========== When I ship beta 1, trunk remains 3.5. When I ship rc 1, we create the 3.5 branch. The 3.5 branch is 3.5.0 and trunk is 3.5.1. Only blessed stuff gets cherry-picked from 3.5.1 back into 3.5.0. What do you think? My votes are as follows: Workflow 0: -0.5 Workflow 1: +1 Workflow 2: +0.5 Please cast your votes, //arry/

5 years, 10 months

Reminder: Python 3.5 beta 1 will be tagged tomorrow

by Larry Hastings

Howdy howdy. It's-a me, Larry, your friendly neighborhood Python 3.5 Release Manager. Somewhere around 2 or 3pm tomorrow I expect to tag Python 3.5 beta 1. We'll actually release beta 1 on Sunday, once the binary installers are all built. Beta 1 is also feature-freeze, meaning no new features may be added to 3.5 without my permission. Since it seems useful to have a specific cutoff time, please stop adding features at ** 8pm Saturday UTC **. (That's 1pm Pacific Daylight Time. It's also almost exactly 24 hours from... now.) I remind you that this time we're trying something new: we're going to create the 3.5 branch when we release beta 1, allowing feature development (for 3.6) to continue in trunk. At the point that I check in and push beta 1, I'll also merge all checkins from trunk back into the 3.5 branch. After that it'll be responsibility of the person checking in to check their bug fixes in to the appropriate place. So please keep in mind: once the 3.5 branch becomes generally available on Sunday, the usual rules for a release branch will apply: bug fixes for 3.5 should be checked in to the 3.5 branch and get merged forward into trunk. If you have new features you want to ship with Python 3.5, please check them in as soon as possible! Thank you for helping to make Python better, //arry/

5 years, 10 months

Re: [Python-Dev] Status of PEP 484 and the typing module

by Guido van Rossum

Hi Mark, We're down to the last few items here. I'm CC'ing python-dev so folks can see how close we are. I'll answer point by point. On Thu, May 21, 2015 at 6:24 AM, Mark Shannon <mark(a)hotpy.org> wrote: > Hi, > > The PEP itself is looking fairly good. > I hope you'll accept it at least provisionally so we can iterate over the finer points while a prototype of typing.py in in beta 1. > However, I don't think that typing.py is ready yet, for a number of > reasons: > > 1. > As I've said before, there needs to be a distinction between classes and > types. > They is no need for Any, Generic, Generic's subtypes, or Union to subclass > builtins.type. > I strongly disagree. They can appear in many positions where real classes are acceptable, in particular annotations can have classes (e.g. int) or types (e.g. Union[int, str]). > Playing around with typing.py, it has also become clear to me that it > is also important to distinguish type constructors from types. > > What do I mean by a type constructor? > A type constructor makes types. > "List" is an example of a type constructor. It constructs types such as > List[T] and List[int]. > Saying that something is a List (as opposed to a list) should be rejected. > The PEP actually says that plain List (etc.) is equivalent to List[Any]. (Well, at least that's the intention; it's implied by the section about the equivalence between Node() and Node[Any](). > 2. > Usability of typing as it stands: > > Let's try to make a class that implements a mutable mapping. > > >>> import typing as tp > #Make some variables. > >>> T = tp.TypeVar('T') > >>> K = tp.TypeVar('K') > >>> V = tp.TypeVar('V') > > #Then make our class: > > >>> class MM(tp.MutableMapping): pass > ... > #Oh that worked, but it shouldn't. MutableMapping is a type constructor. > It means MutableMapping[Any]. > #Let's make one > >>> MM() > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > File "/home/mark/repositories/typehinting/prototyping/typing.py", line > 1095, in __new__ > if _gorg(c) is Generic: > File "/home/mark/repositories/typehinting/prototyping/typing.py", line > 887, in _gorg > while a.__origin__ is not None: > AttributeError: type object 'Sized' has no attribute '__origin__' > > # ??? > Sorry, that's a bug I introduced in literally the last change to typing.py. I will fix it. The expected behavior is TypeError: Can't instantiate abstract class MM with abstract methods __len__ > #Well let's try using type variables. > class MM2(tp.MutableMapping[K, V]): pass > ... > >>> MM2() > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > File "/home/mark/repositories/typehinting/prototyping/typing.py", line > 1095, in __new__ > if _gorg(c) is Generic: > File "/home/mark/repositories/typehinting/prototyping/typing.py", line > 887, in _gorg > while a.__origin__ is not None: > AttributeError: type object 'Sized' has no attribute '__origin__' > Ditto, and sorry. > > At this point, we have to resort to using 'Dict', which forces us to > subclass 'dict' which may not be what we want as it may cause metaclass > conflicts. > > 3. > Memory consumption is also a worry. There is no caching, which means every > time I use "List[int]" as an annotation, a new class object is created. > Each class may only be a few KB, but collectively this could easily add up > to several MBs. > This should be easy to fix. > I can work on this after the beta-1 release. Until then, type aliases can be used to avoid redundant type creation (and often they are clearer anyway :-). > 4. > PY2, etc. really need to go. > Assuming that this code type checks OK: > > if typing.PY2: > type_safe_under_py2_only() > else: > type_safe_under_py3_only() > > Is the checker supposed to pass this: > > if sys.hexversion < 0x03000000: > type_safe_under_py2_only() > else: > type_safe_under_py3_only() > > If it should pass, then why have PY2, etc. at all. > If it should fail, well that is just stupid and annoying. > > Pylint already understands version checks, as does our (Semmle's) checker. > I suspect most IDEs do as well. > I have to negotiate this with Jukka but I think he'll agree. > 5. > Removing isinstance() support: > > As I said before, this is the job of a checker not typing.py. > > It also introduces some strange situations: > D = tp.Dict[str,int] > d = {} > assert isinstance(d, D) > d["x"] = None > assert isinstance(d, D) > > In the above case the first check passes, and the second fails. > But d is either of type D or it isn't. It can't be both, as types > are static properties of programs, unlike classes. > Well, isinstance() is a dynamic function. The type checker has no authority over its behavior beyond its signature. > And it's broken anyway: > >>> D = tp.Dict[str,'D'] > >>> d = {"x": {}} > >>> isinstance(d, D) > False > That's because _ForwardRef doesn't implement __instancheck__ or __subclasscheck__. It's easily fixed. > > Realistically, I don't see typing.py being ready in time for 3.5. > I'd be happy to be proved wrong. > > Cheers, > Mark. > > > P.S. > I am worried by the lack of formal specification. It all seems a bit > hand-waving. A formal spec reduces the likelihood of some unforeseen corner > case being a permanent wart. > Formal specs are not my cup of tea. :-( (I'm not proud of this, but it just is a fact -- see how terrible a job I've done of the Python reference manual.) The best I could come up with is PEP 483. > Take the recursive type above. There is no mention of recursive types in > the PEP and they are clearly possible. Are they allowed? > They should be allowed. I imagine you could create one for which a naive isinstance() imeplementation ends up in an infinite loop. That can be fixed too (we fixed this for printing self-referential lists and dicts). > I'm guessing that Jukka's thesis should cover a lot of this. > Has it been published yet? > Hopefully Jukka can answer that. :-) -- --Guido van Rossum (python.org/~guido)

5 years, 10 months

Summary of Python tracker Issues

by Python tracker

ACTIVITY SUMMARY (2015-05-15 - 2015-05-22) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 4833 ( -7) closed 31194 (+71) total 36027 (+64) Open issues with patches: 2219 Issues opened (40) ================== #24147: Dialect class defaults are not documented. http://bugs.python.org/issue24147 reopened by r.david.murray #24203: Depreciate threading.Thread.isDaemon etc http://bugs.python.org/issue24203 opened by anon #24204: string.strip() documentation is misleading http://bugs.python.org/issue24204 opened by PhoenixofMT #24206: Issues with equality of inspect objects http://bugs.python.org/issue24206 opened by serhiy.storchaka #24207: Argument Clinic doesn't mangle conflicting names http://bugs.python.org/issue24207 opened by serhiy.storchaka #24209: Allow IPv6 bind in http.server http://bugs.python.org/issue24209 opened by Link Mauve #24212: Idle, 2.7, backport idlelib.__main__, enable py -m idlelib http://bugs.python.org/issue24212 opened by terry.reedy #24214: Exception with utf-8, surrogatepass and incremental decoding http://bugs.python.org/issue24214 opened by RalfM #24215: test_trace uses test_pprint http://bugs.python.org/issue24215 opened by serhiy.storchaka #24217: O_RDWR undefined in mmapmodule.c http://bugs.python.org/issue24217 opened by Jeffrey.Armstrong #24219: Repeated integer in Lexical analysis/Integer literals section http://bugs.python.org/issue24219 opened by vlth #24224: test_msilib is inadequate http://bugs.python.org/issue24224 opened by zach.ware #24225: Idlelib: changing file names http://bugs.python.org/issue24225 opened by Al.Sweigart #24228: Interpreter triggers segmentation fault at the starting http://bugs.python.org/issue24228 opened by mdootb #24229: pathlib.Path should have a copy() method http://bugs.python.org/issue24229 opened by jshholland #24230: tempfile.mkdtemp() doesn't work with bytes paths http://bugs.python.org/issue24230 opened by durin42 #24231: os.makedirs('/', exist_ok=True) fails on Darwin http://bugs.python.org/issue24231 opened by mew #24234: Should we define complex.__complex__ and bytes.__bytes__? http://bugs.python.org/issue24234 opened by gvanrossum #24235: ABCs don't fail metaclass instantiation http://bugs.python.org/issue24235 opened by Devin Jeanpierre #24238: Avoid entity expansion attacks in Element Tree http://bugs.python.org/issue24238 opened by vadmium #24239: Allow to configure which gpg to use in distutils upload http://bugs.python.org/issue24239 opened by ced #24241: webbrowser default browser detection and/or public API for _tr http://bugs.python.org/issue24241 opened by daves #24243: behavior for finding an empty string is inconsistent with docu http://bugs.python.org/issue24243 opened by swanson #24244: Python exception on strftime with %f on Python 3 and Python 2 http://bugs.python.org/issue24244 opened by MajeedArni #24247: "unittest discover" does modify sys.path http://bugs.python.org/issue24247 opened by redixin #24249: unittest API for detecting test failure in cleanup/teardown http://bugs.python.org/issue24249 opened by r.david.murray #24251: Different behavior for argparse between 2.7.8 and 2.7.9 when a http://bugs.python.org/issue24251 opened by hhuang #24252: IDLE removes elements from tracebacks. http://bugs.python.org/issue24252 opened by ppperry #24253: pydoc for namespace packages indicates FILE as built-in http://bugs.python.org/issue24253 opened by Antony.Lee #24254: Make class definition namespace ordered by default http://bugs.python.org/issue24254 opened by eric.snow #24255: Replace debuglevel-related logic with logging http://bugs.python.org/issue24255 opened by demian.brecht #24256: threading.Timer is not a class http://bugs.python.org/issue24256 opened by jrunyon #24258: BZ2File objects do not have name attribute http://bugs.python.org/issue24258 opened by jojko.sivek #24259: tar.extractall() does not recognize unexpected EOF http://bugs.python.org/issue24259 opened by Thomas Güttler #24260: TabError behavior doesn't match documentation http://bugs.python.org/issue24260 opened by abacabadabacaba #24261: Add a command line flag to suppress default signal handlers http://bugs.python.org/issue24261 opened by abacabadabacaba #24263: Why VALID_MODULE_NAME in unittest/loader.py is r'[_a-z]\w*\.py http://bugs.python.org/issue24263 opened by sih4sing5hong5 #24264: imageop Unsafe Arithmetic http://bugs.python.org/issue24264 opened by JohnLeitch #24265: IDLE produces error message when run with both -s and -c. http://bugs.python.org/issue24265 opened by ppperry #24266: raw_input function (with readline): Ctrl+C (during search mode http://bugs.python.org/issue24266 opened by sping Most recent 15 issues with no replies (15) ========================================== #24266: raw_input function (with readline): Ctrl+C (during search mode http://bugs.python.org/issue24266 #24265: IDLE produces error message when run with both -s and -c. http://bugs.python.org/issue24265 #24264: imageop Unsafe Arithmetic http://bugs.python.org/issue24264 #24263: Why VALID_MODULE_NAME in unittest/loader.py is r'[_a-z]\w*\.py http://bugs.python.org/issue24263 #24260: TabError behavior doesn't match documentation http://bugs.python.org/issue24260 #24259: tar.extractall() does not recognize unexpected EOF http://bugs.python.org/issue24259 #24258: BZ2File objects do not have name attribute http://bugs.python.org/issue24258 #24253: pydoc for namespace packages indicates FILE as built-in http://bugs.python.org/issue24253 #24247: "unittest discover" does modify sys.path http://bugs.python.org/issue24247 #24239: Allow to configure which gpg to use in distutils upload http://bugs.python.org/issue24239 #24238: Avoid entity expansion attacks in Element Tree http://bugs.python.org/issue24238 #24235: ABCs don't fail metaclass instantiation http://bugs.python.org/issue24235 #24234: Should we define complex.__complex__ and bytes.__bytes__? http://bugs.python.org/issue24234 #24224: test_msilib is inadequate http://bugs.python.org/issue24224 #24214: Exception with utf-8, surrogatepass and incremental decoding http://bugs.python.org/issue24214 Most recent 15 issues waiting for review (15) ============================================= #24254: Make class definition namespace ordered by default http://bugs.python.org/issue24254 #24244: Python exception on strftime with %f on Python 3 and Python 2 http://bugs.python.org/issue24244 #24238: Avoid entity expansion attacks in Element Tree http://bugs.python.org/issue24238 #24230: tempfile.mkdtemp() doesn't work with bytes paths http://bugs.python.org/issue24230 #24225: Idlelib: changing file names http://bugs.python.org/issue24225 #24219: Repeated integer in Lexical analysis/Integer literals section http://bugs.python.org/issue24219 #24217: O_RDWR undefined in mmapmodule.c http://bugs.python.org/issue24217 #24215: test_trace uses test_pprint http://bugs.python.org/issue24215 #24209: Allow IPv6 bind in http.server http://bugs.python.org/issue24209 #24206: Issues with equality of inspect objects http://bugs.python.org/issue24206 #24204: string.strip() documentation is misleading http://bugs.python.org/issue24204 #24198: please align the platform tag for windows http://bugs.python.org/issue24198 #24195: Add `Executor.filter` to concurrent.futures http://bugs.python.org/issue24195 #24165: Free list for single-digits ints http://bugs.python.org/issue24165 #24164: Support pickling objects with __new__ with keyword arguments w http://bugs.python.org/issue24164 Top 10 most discussed issues (10) ================================= #16991: Add OrderedDict written in C http://bugs.python.org/issue16991 22 msgs #4709: Mingw-w64 and python on windows x64 http://bugs.python.org/issue4709 18 msgs #24230: tempfile.mkdtemp() doesn't work with bytes paths http://bugs.python.org/issue24230 16 msgs #24244: Python exception on strftime with %f on Python 3 and Python 2 http://bugs.python.org/issue24244 12 msgs #24195: Add `Executor.filter` to concurrent.futures http://bugs.python.org/issue24195 10 msgs #12319: [http.client] HTTPConnection.request not support "chunked" Tra http://bugs.python.org/issue12319 9 msgs #24225: Idlelib: changing file names http://bugs.python.org/issue24225 9 msgs #23699: Add a macro to ease writing rich comparisons http://bugs.python.org/issue23699 8 msgs #24215: test_trace uses test_pprint http://bugs.python.org/issue24215 8 msgs #6598: calling email.utils.make_msgid frequently has a non-trivial pr http://bugs.python.org/issue6598 7 msgs Issues closed (61) ================== #4254: _cursesmodule.c callable update_lines_cols() http://bugs.python.org/issue4254 closed by r.david.murray #9858: Python and C implementations of io are out of sync http://bugs.python.org/issue9858 closed by pitrou #10170: Relationship between turtle speed setting and actual speed is http://bugs.python.org/issue10170 closed by terry.reedy #15267: tempfile.TemporaryFile and httplib incompatibility http://bugs.python.org/issue15267 closed by serhiy.storchaka #15836: unittest assertRaises should verify excClass is actually a Bas http://bugs.python.org/issue15836 closed by serhiy.storchaka #16261: Fix bare excepts in various places in std lib http://bugs.python.org/issue16261 closed by serhiy.storchaka #18682: [PATCH] remove bogus codepath from pprint._safe_repr http://bugs.python.org/issue18682 closed by serhiy.storchaka #18986: Add a case-insensitive case-preserving dict http://bugs.python.org/issue18986 closed by serhiy.storchaka #20098: email policy needs a mangle_from setting http://bugs.python.org/issue20098 closed by r.david.murray #20438: inspect: Deprecate getfullargspec? http://bugs.python.org/issue20438 closed by yselivanov #20596: Support for alternate wcstok syntax for Windows compilers http://bugs.python.org/issue20596 closed by Jeffrey.Armstrong #20691: inspect.signature: Consider exposing 'follow_wrapper_chains' o http://bugs.python.org/issue20691 closed by yselivanov #21083: Add get_content_disposition() to email.message.Message http://bugs.python.org/issue21083 closed by r.david.murray #21800: Implement RFC 6855 (IMAP Support for UTF-8) in imaplib. http://bugs.python.org/issue21800 closed by r.david.murray #21804: Implement thr UTF8 command (RFC 6856) in poplib. http://bugs.python.org/issue21804 closed by r.david.murray #21931: Nonsense errors reported by msilib.FCICreate for bad argument http://bugs.python.org/issue21931 closed by python-dev #22027: RFC 6531 (SMTPUTF8) support in smtplib http://bugs.python.org/issue22027 closed by r.david.murray #22107: tempfile module misinterprets access denied error on Windows http://bugs.python.org/issue22107 closed by serhiy.storchaka #22155: Out of date code example for tkinter's createfilehandler http://bugs.python.org/issue22155 closed by terry.reedy #22804: Can't run Idle in Windows 8 or windows 64 http://bugs.python.org/issue22804 closed by terry.reedy #23184: Unused imports, variables, file in IDLE http://bugs.python.org/issue23184 closed by terry.reedy #23780: Surprising behaviour when passing list to os.path.join. http://bugs.python.org/issue23780 closed by serhiy.storchaka #23889: Speedup inspect.Signature.bind http://bugs.python.org/issue23889 closed by yselivanov #23898: inspect() changes in Python3.4 are not compatible with objects http://bugs.python.org/issue23898 closed by yselivanov #23964: Update README documentation for IDLE tests. http://bugs.python.org/issue23964 closed by terry.reedy #23985: Crash when deleting slices from duplicated bytearray http://bugs.python.org/issue23985 closed by pitrou #24004: avoid explicit generator type check in asyncio http://bugs.python.org/issue24004 closed by yselivanov #24091: Use after free in Element.extend (1) http://bugs.python.org/issue24091 closed by serhiy.storchaka #24102: Multiple type confusions in unicode error handlers http://bugs.python.org/issue24102 closed by serhiy.storchaka #24127: Fatal error in launcher: Job information querying failed http://bugs.python.org/issue24127 closed by paul.moore #24162: [2.7 regression] test_asynchat test failure on i586-linux-gnu http://bugs.python.org/issue24162 closed by python-dev #24176: Incorrect parsing of unpacked expressions in call http://bugs.python.org/issue24176 closed by python-dev #24180: PEP 492: Documentation http://bugs.python.org/issue24180 closed by yselivanov #24190: BoundArguments facility to inject defaults http://bugs.python.org/issue24190 closed by yselivanov #24192: unexpected system error with pep420 style namespace packages http://bugs.python.org/issue24192 closed by eric.snow #24200: Redundant id in informative reprs http://bugs.python.org/issue24200 closed by yselivanov #24205: signature.bind error messages are sub-optimal http://bugs.python.org/issue24205 closed by yselivanov #24208: test_inspect leaks temporary directory http://bugs.python.org/issue24208 closed by serhiy.storchaka #24210: Tests failed with -Werror http://bugs.python.org/issue24210 closed by berker.peksag #24211: Add RFC 6532 support to the email library http://bugs.python.org/issue24211 closed by r.david.murray #24213: ProcessPoolExecutor().map() fails following an identical map() http://bugs.python.org/issue24213 closed by ned.deily #24216: Typo in bytes.join/bytearray.join documentation http://bugs.python.org/issue24216 closed by r.david.murray #24218: Also support SMTPUTF8 in smtplib's send_message method. http://bugs.python.org/issue24218 closed by r.david.murray #24220: ast.Call signature changed http://bugs.python.org/issue24220 closed by willingc #24221: Clean-up and optimization for heapq siftup() and siftdown() http://bugs.python.org/issue24221 closed by rhettinger #24222: Idle 2.7 -c, -r compile with print as function. http://bugs.python.org/issue24222 closed by terry.reedy #24223: argparse parsing (mingling --option and optional positional ar http://bugs.python.org/issue24223 closed by vadmium #24226: [3.5 Regression] unable to byte-compile the attached IN.py http://bugs.python.org/issue24226 closed by yselivanov #24227: IndentationError caused by async / await changes in parser http://bugs.python.org/issue24227 closed by zach.ware #24232: Speling fixes http://bugs.python.org/issue24232 closed by berker.peksag #24233: Link to getfqdn rather than "see above" http://bugs.python.org/issue24233 closed by berker.peksag #24236: TestNG results to Junit results conversion http://bugs.python.org/issue24236 closed by tusharm #24237: PEP 479: missing DeprecationWarning when generator_stop is not http://bugs.python.org/issue24237 closed by yselivanov #24240: PEP 448: Update the language reference http://bugs.python.org/issue24240 closed by benjamin.peterson #24242: property performance regression http://bugs.python.org/issue24242 closed by rhettinger #24245: Eliminate do-nothing exception handlers http://bugs.python.org/issue24245 closed by serhiy.storchaka #24246: mimetypes.guess_extension returns different values when init() http://bugs.python.org/issue24246 closed by r.david.murray #24248: Deprecate inspect.Signature.from_function and from_builtin http://bugs.python.org/issue24248 closed by yselivanov #24250: Optimization for strcpy(..., "") in file 'install.c' http://bugs.python.org/issue24250 closed by benjamin.peterson #24257: Incorrect use of PyObject_IsInstance http://bugs.python.org/issue24257 closed by serhiy.storchaka #24262: logging.FileHandler.close() is not thread-safe http://bugs.python.org/issue24262 closed by dubglan

5 years, 10 months

PEP 484 (Type Hints) -- penultimate(?) draft

by Guido van Rossum

I'm happy to present a much updated PEP 484 for your review. It will (hopefully) appear on python.org within the next hour. I'm also working on an implementation ( https://github.com/ambv/typehinting/tree/master/prototyping) which I hope will be good enough to include in beta 1, assuming the BDFL-Delegate (Mark Shannon) approves. --Guido PEP: 484 Title: Type Hints Version: $Revision$ Last-Modified: $Date$ Author: Guido van Rossum <guido(a)python.org>, Jukka Lehtosalo < jukka.lehtosalo(a)iki.fi>, Łukasz Langa <lukasz(a)langa.pl> BDFL-Delegate: Mark Shannon Discussions-To: Python-Dev <python-dev(a)python.org> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 29-Sep-2014 Post-History: 16-Jan-2015,20-Mar-2015,17-Apr-2015,20-May-2015 Resolution: Abstract ======== PEP 3107 introduced syntax for function annotations, but the semantics were deliberately left undefined. There has now been enough 3rd party usage for static type analysis that the community would benefit from a standard vocabulary and baseline tools within the standard library. This PEP introduces a provisional module to provide these standard definitions and tools, along with some conventions for situations where annotations are not available. Note that this PEP still explicitly does NOT prevent other uses of annotations, nor does it require (or forbid) any particular processing of annotations, even when they conform to this specification. It simply enables better coordination, as PEP 333 did for web frameworks. For example, here is a simple function whose argument and return type are declared in the annotations:: def greeting(name: str) -> str: return 'Hello ' + name While these annotations are available at runtime through the usual ``__annotations__`` attribute, *no type checking happens at runtime*. Instead, the proposal assumes the existence of a separate off-line type checker which users can run over their source code voluntarily. Essentially, such a type checker acts as a very powerful linter. (While it would of course be possible for individual users to employ a similar checker at run time for Design By Contract enforcement or JIT optimization, those tools are not yet as mature.) The proposal is strongly inspired by mypy [mypy]_. For example, the type "sequence of integers" can be written as ``Sequence[int]``. The square brackets mean that no new syntax needs to be added to the language. The example here uses a custom type ``Sequence``, imported from a pure-Python module ``typing``. The ``Sequence[int]`` notation works at runtime by implementing ``__getitem__()`` in the metaclass (but its significance is primarily to an offline type checker). The type system supports unions, generic types, and a special type named ``Any`` which is consistent with (i.e. assignable to and from) all types. This latter feature is taken from the idea of gradual typing. Gradual typing and the full type system are explained in PEP 483. Other approaches from which we have borrowed or to which ours can be compared and contrasted are described in PEP 482. Rationale and Goals =================== PEP 3107 added support for arbitrary annotations on parts of a function definition. Although no meaning was assigned to annotations then, there has always been an implicit goal to use them for type hinting [gvr-artima]_, which is listed as the first possible use case in said PEP. This PEP aims to provide a standard syntax for type annotations, opening up Python code to easier static analysis and refactoring, potential runtime type checking, and (perhaps, in some contexts) code generation utilizing type information. Of these goals, static analysis is the most important. This includes support for off-line type checkers such as mypy, as well as providing a standard notation that can be used by IDEs for code completion and refactoring. Non-goals --------- While the proposed typing module will contain some building blocks for runtime type checking -- in particular a useful ``isinstance()`` implementation -- third party packages would have to be developed to implement specific runtime type checking functionality, for example using decorators or metaclasses. Using type hints for performance optimizations is left as an exercise for the reader. It should also be emphasized that **Python will remain a dynamically typed language, and the authors have no desire to ever make type hints mandatory, even by convention.** The meaning of annotations ========================== Any function without annotations should be treated as having the most general type possible, or ignored, by any type checker. Functions with the ``@no_type_check`` decorator or with a ``# type: ignore`` comment should be treated as having no annotations. It is recommended but not required that checked functions have annotations for all arguments and the return type. For a checked function, the default annotation for arguments and for the return type is ``Any``. An exception is that the first argument of instance and class methods does not need to be annotated; it is assumed to have the type of the containing class for instance methods, and ``type`` for class methods. (Note that the return type of ``__init__`` ought to be annotated with ``-> None``. The reason for this is subtle. If ``__init__`` assumed a return annotation of ``-> None``, would that mean that an argument-less, un-annotated ``__init__`` method should still be type-checked? Rather than leaving this ambiguous or introducing an exception to the exception, we simply say that ``__init__`` ought to have a return annotation; the default behavior is thus the same as for other methods.) A type checker is expected to check the body of a checked function for consistency with the given annotations. The annotations may also used to check correctness of calls appearing in other checked functions. Type checkers are expected to attempt to infer as much information as necessary. The minimum requirement is to handle the builtin decorators ``@property``, ``@staticmethod`` and ``@classmethod``. Type Definition Syntax ====================== The syntax leverages PEP 3107-style annotations with a number of extensions described in sections below. In its basic form, type hinting is used by filling function annotation slots with classes:: def greeting(name: str) -> str: return 'Hello ' + name This states that the expected type of the ``name`` argument is ``str``. Analogically, the expected return type is ``str``. Expressions whose type is a subtype of a specific argument type are also accepted for that argument. Acceptable type hints --------------------- Type hints may be built-in classes (including those defined in standard library or third-party extension modules), abstract base classes, types available in the ``types`` module, and user-defined classes (including those defined in the standard library or third-party modules). While annotations are normally the best format for type hints, there are times when it is more appropriate to represent them by a special comment, or in a separately distributed interface file. (See below for examples.) Annotations must be valid expressions that evaluate without raising exceptions at the time the function is defined (but see below for forward references). Annotations should be kept simple or static analysis tools may not be able to interpret the values. For example, dynamically computed types are unlikely to be understood. (This is an intentionally somewhat vague requirement, specific inclusions and exclusions may be added to future versions of this PEP as warranted by the discussion.) In addition to the above, the following special constructs defined below may be used: ``None``, ``Any``, ``Union``, ``Tuple``, ``Callable``, all ABCs and stand-ins for concrete classes exported from ``typing`` (e.g. ``Sequence`` and ``Dict``), type variables, and type aliases. All newly introduced names used to support features described in following sections (such as ``Any`` and ``Union``) are available in the ``typing`` module. Using None ---------- When used in a type hint, the expression ``None`` is considered equivalent to ``type(None)``. Type aliases ------------ Type aliases are defined by simple variable assignments:: Url = str def retry(url: Url, retry_count: int) -> None: ... Note that we recommend capitalizing alias names, since they represent user-defined types, which (like user-defined classes) are typically spelled that way. Type aliases may be as complex as type hints in annotations -- anything that is acceptable as a type hint is acceptable in a type alias:: from typing import TypeVar, Iterable, Tuple T = TypeVar('T', int, float, complex) Vector = Iterable[Tuple[T, T]] def inproduct(v: Vector) -> T: return sum(x*y for x, y in v) This is equivalent to:: from typing import TypeVar, Iterable, Tuple T = TypeVar('T', int, float, complex) def inproduct(v: Iterable[Tuple[T, T]]) -> T: return sum(x*y for x, y in v) Callable -------- Frameworks expecting callback functions of specific signatures might be type hinted using ``Callable[[Arg1Type, Arg2Type], ReturnType]``. Examples:: from typing import Callable def feeder(get_next_item: Callable[[], str]) -> None: # Body def async_query(on_success: Callable[[int], None], on_error: Callable[[int, Exception], None]) -> None: # Body It is possible to declare the return type of a callable without specifying the call signature by substituting a literal ellipsis (three dots) for the list of arguments:: def partial(func: Callable[..., str], *args) -> Callable[..., str]: # Body Note that there are no square brackets around the ellipsis. The arguments of the callback are completely unconstrained in this case (and keyword arguments are acceptable). Since using callbacks with keyword arguments is not perceived as a common use case, there is currently no support for specifying keyword arguments with ``Callable``. Similarly, there is no support for specifying callback signatures with a variable number of argument of a specific type. Generics -------- Since type information about objects kept in containers cannot be statically inferred in a generic way, abstract base classes have been extended to support subscription to denote expected types for container elements. Example:: from typing import Mapping, Set def notify_by_email(employees: Set[Employee], overrides: Mapping[str, str]) -> None: ... Generics can be parametrized by using a new factory available in ``typing`` called ``TypeVar``. Example:: from typing import Sequence, TypeVar T = TypeVar('T') # Declare type variable def first(l: Sequence[T]) -> T: # Generic function return l[0] In this case the contract is that the returned value is consistent with the elements held by the collection. ``TypeVar`` supports constraining parametric types to a fixed set of possible types. For example, we can define a type variable that ranges over just ``str`` and ``bytes``. By default, a type variable ranges over all possible types. Example of constraining a type variable:: from typing import TypeVar AnyStr = TypeVar('AnyStr', str, bytes) def concat(x: AnyStr, y: AnyStr) -> AnyStr: return x + y The function ``concat`` can be called with either two ``str`` arguments or two ``bytes`` arguments, but not with a mix of ``str`` and ``bytes`` arguments. There should be at least two constraints, if any; specifying a single constraint is disallowed. Subtypes of types constrained by a type variable should be treated as their respective explicitly listed base types in the context of the type variable. Consider this example:: class MyStr(str): ... x = concat(MyStr('apple'), MyStr('pie')) The call is valid but the type variable ``AnyStr`` will be set to ``str`` and not ``MyStr``. In effect, the inferred type of the return value assigned to ``x`` will also be ``str``. Additionally, ``Any`` is a valid value for every type variable. Consider the following:: def count_truthy(elements: List[Any]) -> int: return sum(1 for elem in elements if element) This is equivalent to omitting the generic notation and just saying ``elements: List``. User-defined generic types -------------------------- You can include a ``Generic`` base class to define a user-defined class as generic. Example:: from typing import TypeVar, Generic T = TypeVar('T') class LoggedVar(Generic[T]): def __init__(self, value: T, name: str, logger: Logger) -> None: self.name = name self.logger = logger self.value = value def set(self, new: T) -> None: self.log('Set ' + repr(self.value)) self.value = new def get(self) -> T: self.log('Get ' + repr(self.value)) return self.value def log(self, message: str) -> None: self.logger.info('{}: {}'.format(self.name message)) ``Generic[T]`` as a base class defines that the class ``LoggedVar`` takes a single type parameter ``T``. This also makes ``T`` valid as a type within the class body. The ``Generic`` base class uses a metaclass that defines ``__getitem__`` so that ``LoggedVar[t]`` is valid as a type:: from typing import Iterable def zero_all_vars(vars: Iterable[LoggedVar[int]]) -> None: for var in vars: var.set(0) A generic type can have any number of type variables, and type variables may be constrained. This is valid:: from typing import TypeVar, Generic ... T = TypeVar('T') S = TypeVar('S') class Pair(Generic[T, S]): ... Each type variable argument to ``Generic`` must be distinct. This is thus invalid:: from typing import TypeVar, Generic ... T = TypeVar('T') class Pair(Generic[T, T]): # INVALID ... You can use multiple inheritance with ``Generic``:: from typing import TypeVar, Generic, Sized T = TypeVar('T') class LinkedList(Sized, Generic[T]): ... Subclassing a generic class without specifying type parameters assumes ``Any`` for each position. In the following example, ``MyIterable`` is not generic but implicitly inherits from ``Iterable[Any]``: from typing import Iterable class MyIterable(Iterable): # Same as Iterable[Any] ... Instantiating generic classes and type erasure ---------------------------------------------- Generic types like ``List`` or ``Sequence`` cannot be instantiated. However, user-defined classes derived from them can be instantiated. Given a generic class ``Node[T]`` there are three forms of instantiation: * ``x = Node()`` -- the type of x is ``Node[Any]``. * ``x = Node[T]()`` -- the type of x is ``Node[T]``. * ``x = Node[int]()`` -- the type of x is ``Node[int]``. At runtime the type is not preserved, and the observable type of x is just ``Node``. This is type erasure and common practice in languages with generics (e.g. Java, Typescript). Arbitrary generic types as base classes --------------------------------------- ``Generic[T]`` is only valid as a base class -- it's not a proper type. However, user-defined generic types such as ``LinkedList[T]`` from the above example and built-in generic types and ABCs such as ``List[T]`` and ``Iterable[T]`` are valid both as types and as base classes. For example, we can define a subclass of ``Dict`` that specializes type arguments:: from typing import Dict, List, Optional class Node: ... class SymbolTable(Dict[str, List[Node]]): def push(self, name: str, node: Node) -> None: self.setdefault(name, []).append(node) def pop(self, name: str) -> Node: return self[name].pop() def lookup(self, name: str) -> Optional[Node]: nodes = self.get(name) if nodes: return nodes[-1] return None ``SymbolTable`` is a subclass of ``dict`` and a subtype of ``Dict[str, List[Node]]``. If a generic base class has a type variable as a type argument, this makes the defined class generic. For example, we can define a generic ``LinkedList`` class that is iterable and a container:: from typing import TypeVar, Iterable, Container T = TypeVar('T') class LinkedList(Iterable[T], Container[T]): ... Now ``LinkedList[int]`` is a valid type. Note that we can use ``T`` multiple times in the base class list, as long as we don't use the same type variable ``T`` multiple times within ``Generic[...]``. Also consider the following example:: from typing import TypeVar, Mapping T = TypeVar('T') class MyDict(Mapping[str, T]): ... In this case MyDict has a single parameter, T. Abstract generic types ---------------------- The metaclass used by ``Generic`` is a subclass of ``abc.ABCMeta``. A generic class can be an ABC by including abstract methods or properties, and generic classes can also have ABCs as base classes without a metaclass conflict. Type variables with an upper bound ---------------------------------- A type variable may specify an upper bound using ``bound=<type>``. This means that an actual type substituted (explicitly or implictly) for the type variable must be a subclass of the boundary type. A common example is the definition of a Comparable type that works well enough to catch the most common errors:: from typing import TypeVar class Comparable(metaclass=ABCMeta): @abstractmethod def __lt__(self, other: Any) -> bool: ... ... # __gt__ etc. as well CT = TypeVar('CT', bound=Comparable) def min(x: CT, y: CT) -> CT: if x < y: return x else: return y min(1, 2) # ok, return type int min('x', 'y') # ok, return type str (Note that this is not ideal -- for example ``min('x', 1)`` is invalid at runtime but a type checker would simply infer the return type ``Comparable``. Unfortunately, addressing this would require introducing a much more powerful and also much more complicated concept, F-bounded polymorphism. We may revisit this in the future.) An upper bound cannot be combined with type constraints (as in used ``AnyStr``, see the example earlier); type constraints cause the inferred type to be _exactly_ one of the constraint types, while an upper bound just requires that the actual type is a subclass of the boundary type. Covariance and contravariance ----------------------------- Consider a class ``Employee`` with a subclass ``Manager``. Now suppose we have a function with an argument annotated with ``List[Employee]``. Should we be allowed to call this function with a variable of type ``List[Manager]`` as its argument? Many people would answer "yes, of course" without even considering the consequences. But unless we know more about the function, a type checker should reject such a call: the function might append an ``Employee`` instance to the list, which would violate the variable's type in the caller. It turns out such an argument acts _contravariantly_, whereas the intuitive answer (which is correct in case the function doesn't mutate its argument!) requires the argument to act _covariantly_. A longer introduction to these concepts can be found on Wikipedia [wiki-variance]_; here we just show how to control a type checker's behavior. By default type variables are considered _invariant_, which means that arguments for arguments annotated with types like ``List[Employee]`` must exactly match the type annotation -- no subclasses or superclasses of the type parameter (in this example ``Employee``) are allowed. To facilitate the declaration of container types where covariant type checking is acceptable, a type variable can be declared using ``covariant=True``. For the (rare) case where contravariant behavior is desirable, pass ``contravariant=True``. At most one of these may be passed. A typical example involves defining an immutable container class:: from typing import TypeVar T = TypeVar('T', covariant=True) class ImmutableList(Generic[T]): def append(self, T): ... ... class Employee: ... class Manager(Employee): ... def dump_employees(emps: ImmutableList[Employee]) -> None: ... mgrs = ... # type: ImmutableList[Mananger] mgrs.append(Manager()) dump_employees(mgrs) # OK The immutable collection classes in ``typing`` are all defined using a covariant type variable (e.g. ``Mapping`` and ``Sequence``). The mutable collection classes (e.g. ``MutableMapping`` and ``MutableSequence``) are defined using regular invariant type variables. The one example of a contravariant type variable is the ``Generator`` type, which is contravariant in the ``send()`` argument type (see below). Note: variance affects type parameters for generic types -- it does not affect regular parameters. For example, the following example is fine:: from typing import TypeVar class Employee: ... class Manager(Employee): ... E = TypeVar('E', bound=Employee) # Invariant def dump_employee(e: E) -> None: ... dump_employee(Manager()) # OK The numeric tower ----------------- PEP 3141 defines Python's numeric tower, and the stdlib module ``numbers`` implements the corresponding ABCs (``Number``, ``Complex``, ``Real``, ``Rational`` and ``Integral``). There are some issues with these ABCs, but the built-in concrete numeric classes ``complex``, ``float`` and ``int`` are ubiquitous (especially the latter two :-). Rather than requiring that users write ``import numbers`` and then use ``numbers.Float`` etc., this PEP proposes a straightforward shortcut that is almost as effective: when an argument is annotated as having type ``float``, an argument of type ``int`` is acceptable; similar, for an argument annotated as having type ``complex``, arguments of type ``float`` or ``int`` are acceptable. This does not handle classes implementing the corresponding ABCs or the ``fractions.Fraction`` class, but we believe those use cases are exceedingly rare. The bytes types --------------- There are three different builtin classes used for arrays of bytes (not counting the classes available in the ``array`` module): ``bytes``, ``bytearray`` and ``memoryview``. Of these, ``bytes`` and ``bytearray`` have many behaviors in common (though not all -- ``bytearray`` is mutable). While there is an ABC ``ByteString`` defined in ``collections.abc`` and a corresponding type in ``typing``, functions accepting bytes (of some form) are so common that it would be cumbersome to have to write ``typing.ByteString`` everywhere. So, as a shortcut similar to that for the builtin numeric classes, when an argument is annotated as having type ``bytes``, arguments of type ``bytearray`` or ``memoryview`` are acceptable. (Again, there are situations where this isn't sound, but we believe those are exceedingly rare in practice.) Forward references ------------------ When a type hint contains names that have not been defined yet, that definition may be expressed as a string literal, to be resolved later. A situation where this occurs commonly is the definition of a container class, where the class being defined occurs in the signature of some of the methods. For example, the following code (the start of a simple binary tree implementation) does not work:: class Tree: def __init__(self, left: Tree, right: Tree): self.left = left self.right = right To address this, we write:: class Tree: def __init__(self, left: 'Tree', right: 'Tree'): self.left = left self.right = right The string literal should contain a valid Python expression (i.e., ``compile(lit, '', 'expr')`` should be a valid code object) and it should evaluate without errors once the module has been fully loaded. The local and global namespace in which it is evaluated should be the same namespaces in which default arguments to the same function would be evaluated. Moreover, the expression should be parseable as a valid type hint, i.e., it is constrained by the rules from the section `Acceptable type hints`_ above. It is allowable to use string literals as *part* of a type hint, for example:: class Tree: ... def leaves(self) -> List['Tree']: ... A common use for forward references is when e.g. Django models are needed in the signatures. Typically, each model is in a separate file, and has methods that arguments whose type involves other models. Because of the way circular imports work in Python, it is often not possible to import all the needed models directly:: # File models/a.py from models.b import B class A(Model): def foo(self, b: B): ... # File models/b.py from models.a import A class B(Model): def bar(self, a: A): ... # File main.py from a import A from b import B Assuming main is imported first, this will fail with an ImportError at the line ``from models.a import A`` in models/b.py, which is being imported from models/a.py before a has defined class A. The solution is to switch to module-only imports and reference the models by their _module_._class_ name:: # File models/a.py from models import b class A(Model): def foo(self, b: 'b.B'): ... # File models/b.py from models import a class B(Model): def bar(self, a: 'a.A'): ... # File main.py from a import A from b import B Union types ----------- Since accepting a small, limited set of expected types for a single argument is common, there is a new special factory called ``Union``. Example:: from typing import Union def handle_employees(e: Union[Employee, Sequence[Employee]]) -> None: if isinstance(e, Employee): e = [e] ... A type factored by ``Union[T1, T2, ...]`` responds ``True`` to ``issubclass`` checks for ``T1`` and any of its subtypes, ``T2`` and any of its subtypes, and so on. One common case of union types are *optional* types. By default, ``None`` is an invalid value for any type, unless a default value of ``None`` has been provided in the function definition. Examples:: def handle_employee(e: Union[Employee, None]) -> None: ... As a shorthand for ``Union[T1, None]`` you can write ``Optional[T1]``; for example, the above is equivalent to:: from typing import Optional def handle_employee(e: Optional[Employee]) -> None: ... An optional type is also automatically assumed when the default value is ``None``, for example:: def handle_employee(e: Employee = None): ... This is equivalent to:: def handle_employee(e: Optional[Employee] = None) -> None: ... The ``Any`` type ---------------- A special kind of type is ``Any``. Every type is a subtype of ``Any``. This is also true for the builtin type ``object``. However, to the static type checker these are completely different. When the type of a value is ``object``, the type checker will reject almost all operations on it, and assigning it to a variable (or using it as a return value) of a more specialized type is a type error. On the other hand, when a value has type ``Any``, the type checker will allow all operations on it, and a value of type ``Any`` can be assigned to a variable (or used as a return value) of a more constrained type. Predefined constants -------------------- Some predefined Boolean constants are defined in the ``typing`` module to enable platform-specific type definitions and such:: from typing import PY2, PY3, WINDOWS, POSIX if PY2: text = unicode else: text = str def f() -> text: ... if WINDOWS: loop = ProactorEventLoop else: loop = UnixSelectorEventLoop It is up to the type checker implementation to define their values, as long as ``PY2 == not PY3`` and ``WINDOWS == not POSIX``. When the program is being executed these always reflect the current platform, and this is also the suggested default when the program is being type-checked. Default argument values ----------------------- In stubs it may be useful to declare an argument as having a default without specifying the actual default value. For example:: def foo(x: AnyStr, y: AnyStr = ...) -> AnyStr: ... What should the default value look like? Any of the options ``""``, ``b""`` or ``None`` fails to satisfy the type constraint (actually, ``None`` will *modify* the type to become ``Optional[AnyStr]``). In such cases the default value may be specified as a literal ellipsis, i.e. the above example is literally what you would write. Compatibility with other uses of function annotations ===================================================== A number of existing or potential use cases for function annotations exist, which are incompatible with type hinting. These may confuse a static type checker. However, since type hinting annotations have no runtime behavior (other than evaluation of the annotation expression and storing annotations in the ``__annotations__`` attribute of the function object), this does not make the program incorrect -- it just may cause a type checker to emit spurious warnings or errors. To mark portions of the program that should not be covered by type hinting, you can use one or more of the following: * a ``# type: ignore`` comment; * a ``@no_type_check`` decorator on a class or function; * a custom class or function decorator marked with ``@no_type_check_decorator``. For more details see later sections. In order for maximal compatibility with offline type checking it may eventually be a good idea to change interfaces that rely on annotations to switch to a different mechanism, for example a decorator. In Python 3.5 there is no pressure to do this, however. See also the longer discussion under `Rejected alternatives`_ below. Type comments ============= No first-class syntax support for explicitly marking variables as being of a specific type is added by this PEP. To help with type inference in complex cases, a comment of the following format may be used:: x = [] # type: List[Employee] x, y, z = [], [], [] # type: List[int], List[int], List[str] x, y, z = [], [], [] # type: (List[int], List[int], List[str]) x = [ 1, 2, ] # type: List[int] Type comments should be put on the last line of the statement that contains the variable definition. They can also be placed on ``with`` statements and ``for`` statements, right after the colon. Examples of type comments on ``with`` and ``for`` statements:: with frobnicate() as foo: # type: int # Here foo is an int ... for x, y in points: # type: float, float # Here x and y are floats ... In stubs it may be useful to declare the existence of a variable without giving it an initial value. This can be done using a literal ellipsis:: from typing import IO stream = ... # type: IO[str] In non-stub code, there is a similar special case: from typing import IO stream = None # type: IO[str] Type checkers should not complain about this (despite the value ``None`` not matching the given type), nor should they change the inferred type to ``Optional[...]`` (despite the rule that does this for annotated arguments with a default value of ``None``). The assumption here is that other code will ensure that the variable is given a value of the proper type, and all uses can assume that the variable has the given type. The ``# type: ignore`` comment should be put on the line that the error refers to:: import http.client errors = { 'not_found': http.client.NOT_FOUND # type: ignore } A ``# type: ignore`` comment on a line by itself disables all type checking for the rest of the file. If type hinting proves useful in general, a syntax for typing variables may be provided in a future Python version. Casts ===== Occasionally the type checker may need a different kind of hint: the programmer may know that an expression is of a more constrained type than a type checker may be able to infer. For example:: from typing import List, cast def find_first_str(a: List[object]) -> str: index = next(i for i, x in enumerate(a) if isinstance(x, str)) # We only get here if there's at least one string in a return cast(str, a[index]) Some type checkers may not be able to infers that the type of ``a[index]`` is ``str`` and only infer ``object`` or ``Any``", but we know that (if the code gets to that point) it must be a string. The ``cast(t, x)`` call tells the type checker that we are confident that the type of ``x`` is ``t``. At runtime a cast always returns the expression unchanged -- it does not check the type, and it does not convert or coerce the value. Casts differ from type comments (see the previous section). When using a type comment, the type checker should still verify that the inferred type is consistent with the stated type. When using a cast, the type checker should blindly believe the programmer. Also, casts can be used in expressions, while type comments only apply to assignments. Stub Files ========== Stub files are files containing type hints that are only for use by the type checker, not at runtime. There are several use cases for stub files: * Extension modules * Third-party modules whose authors have not yet added type hints * Standard library modules for which type hints have not yet been written * Modules that must be compatible with Python 2 and 3 * Modules that use annotations for other purposes Stub files have the same syntax as regular Python modules. There is one feature of the ``typing`` module that may only be used in stub files: the ``@overload`` decorator described below. The type checker should only check function signatures in stub files; It is recommended that function bodies in stub files just be a single ellipsis (``...``). The type checker should have a configurable search path for stub files. If a stub file is found the type checker should not read the corresponding "real" module. While stub files are syntactically valid Python modules, they use the ``.pyi`` extension to make it possible to maintain stub files in the same directory as the corresponding real module. This also reinforces the notion that no runtime behavior should be expected of stub files. Additional notes on stub files: * Modules and variables imported into the stub are not considered exported from the stub unless the import uses the ``import ... as ...`` form. Function overloading -------------------- The ``@overload`` decorator allows describing functions that support multiple different combinations of argument types. This pattern is used frequently in builtin modules and types. For example, the ``__getitem__()`` method of the ``bytes`` type can be described as follows:: from typing import overload class bytes: ... @overload def __getitem__(self, i: int) -> int: ... @overload def __getitem__(self, s: slice) -> bytes: ... This description is more precise than would be possible using unions (which cannot express the relationship between the argument and return types):: from typing import Union class bytes: ... def __getitem__(self, a: Union[int, slice]) -> Union[int, bytes]: ... Another example where ``@overload`` comes in handy is the type of the builtin ``map()`` function, which takes a different number of arguments depending on the type of the callable:: from typing import Callable, Iterable, Iterator, Tuple, TypeVar, overload T1 = TypeVar('T1') T2 = TypeVar('T2) S = TypeVar('S') @overload def map(func: Callable[[T1], S], iter1: Iterable[T1]) -> Iterator[S]: ... @overload def map(func: Callable[[T1, T2], S], iter1: Iterable[T1], iter2: Iterable[T2]) -> Iterator[S]: ... # ... and we could add more items to support more than two iterables Note that we could also easily add items to support ``map(None, ...)``:: @overload def map(func: None, iter1: Iterable[T1]) -> Iterable[T1]: ... @overload def map(func: None, iter1: Iterable[T1], iter2: Iterable[T2]) -> Iterable[Tuple[T1, T2]]: ... The ``@overload`` decorator may only be used in stub files. While it would be possible to provide a multiple dispatch implementation using this syntax, its implementation would require using ``sys._getframe()``, which is frowned upon. Also, designing and implementing an efficient multiple dispatch mechanism is hard, which is why previous attempts were abandoned in favor of ``functools.singledispatch()``. (See PEP 443, especially its section "Alternative approaches".) In the future we may come up with a satisfactory multiple dispatch design, but we don't want such a design to be constrained by the overloading syntax defined for type hints in stub files. In the meantime, using the ``@overload`` decorator or calling ``overload()`` directly raises ``RuntimeError``. Storing and distributing stub files ----------------------------------- The easiest form of stub file storage and distribution is to put them alongside Python modules in the same directory. This makes them easy to find by both programmers and the tools. However, since package maintainers are free not to add type hinting to their packages, third-party stubs installable by ``pip`` from PyPI are also supported. In this case we have to consider three issues: naming, versioning, installation path. This PEP does not provide a recommendation on a naming scheme that should be used for third-party stub file packages. Discoverability will hopefully be based on package popularity, like with Django packages for example. Third-party stubs have to be versioned using the lowest version of the source package that is compatible. Example: FooPackage has versions 1.0, 1.1, 1.2, 1.3, 2.0, 2.1, 2.2. There are API changes in versions 1.1, 2.0 and 2.2. The stub file package maintainer is free to release stubs for all versions but at least 1.0, 1.1, 2.0 and 2.2 are needed to enable the end user type check all versions. This is because the user knows that the closest *lower or equal* version of stubs is compatible. In the provided example, for FooPackage 1.3 the user would choose stubs version 1.1. Note that if the user decides to use the "latest" available source package, using the "latest" stub files should generally also work if they're updated often. Third-party stub packages can use any location for stub storage. Type checkers should search for them using PYTHONPATH. A default fallback directory that is always checked is ``shared/typehints/python3.5/`` (or 3.6, etc.). Since there can only be one package installed for a given Python version per environment, no additional versioning is performed under that directory (just like bare directory installs by ``pip`` in site-packages). Stub file package authors might use the following snippet in ``setup.py``:: ... data_files=[ ( 'shared/typehints/python{}.{}'.format(*sys.version_info[:2]), pathlib.Path(SRC_PATH).glob('**/*.pyi'), ), ], ... The Typeshed Repo ----------------- There is a shared repository where useful stubs are being collected [typeshed]_. Note that stubs for a given package will not be included here without the explicit consent of the package owner. Further policies regarding the stubs collected here will be decided at a later time, after discussion on python-dev, and reported in the typeshed repo's README. Exceptions ========== No syntax for listing explicitly raised exceptions is proposed. Currently the only known use case for this feature is documentational, in which case the recommendation is to put this information in a docstring. The ``typing`` Module ===================== To open the usage of static type checking to Python 3.5 as well as older versions, a uniform namespace is required. For this purpose, a new module in the standard library is introduced called ``typing``. It defines the fundamental building blocks for constructing types (e.g. ``Any``), types representing generic variants of builtin collections (e.g. ``List``), types representing generic collection ABCs (e.g. ``Sequence``), and a small collection of convenience definitions. Fundamental building blocks: * Any, used as ``def get(key: str) -> Any: ...`` * Union, used as ``Union[Type1, Type2, Type3]`` * Callable, used as ``Callable[[Arg1Type, Arg2Type], ReturnType]`` * Tuple, used by listing the element types, for example ``Tuple[int, int, str]``. Arbitrary-length homogeneous tuples can be expressed using one type and ellipsis, for example ``Tuple[int, ...]``. (The ``...`` here are part of the syntax, a literal ellipsis.) * TypeVar, used as ``X = TypeVar('X', Type1, Type2, Type3)`` or simply ``Y = TypeVar('Y')`` (see above for more details) Generic variants of builtin collections: * Dict, used as ``Dict[key_type, value_type]`` * List, used as ``List[element_type]`` * Set, used as ``Set[element_type]``. See remark for ``AbstractSet`` below. * FrozenSet, used as ``FrozenSet[element_type]`` Note: ``Dict``, ``List``, ``Set`` and ``FrozenSet`` are mainly useful for annotating return values. For arguments, prefer the abstract collection types defined below, e.g. ``Mapping``, ``Sequence`` or ``AbstractSet``. Generic variants of container ABCs (and a few non-containers): * ByteString * Callable (see above, listed here for completeness) * Container * Generator, used as ``Generator[yield_type, send_type, return_type]``. This represents the return value of generator functions. It is a subtype of ``Iterable`` and it has additional type variables for the type accepted by the ``send()`` method (which is contravariant -- a generator that accepts sending it ``Employee`` instance is valid in a context where a generator is required that accepts sending it ``Manager`` instances) and the return type of the generator. * Hashable (not generic, but present for completeness) * ItemsView * Iterable * Iterator * KeysView * Mapping * MappingView * MutableMapping * MutableSequence * MutableSet * Sequence * Set, renamed to ``AbstractSet``. This name change was required because ``Set`` in the ``typing`` module means ``set()`` with generics. * Sized (not generic, but present for completeness) * ValuesView A few one-off types are defined that test for single special methods (similar to ``Hashable`` or ``Sized``): * Reversible, to test for ``__reversed__`` * SupportsAbs, to test for ``__abs__`` * SupportsComplex, to test for ``__complex__`` * SupportsFloat, to test for ``__float__`` * SupportsInt, to test for ``__int__`` * SupportsRound, to test for ``__round__`` * SupportsBytes, to test for ``__bytes__`` Constants for platform-specific type hinting: * PY2 * PY3, equivalent to ``not PY2`` * WINDOWS * POSIX, equivalent to ``not WINDOWS`` Convenience definitions: * AnyStr, defined as ``TypeVar('AnyStr', str, bytes)`` * NamedTuple, used as ``NamedTuple(type_name, [(field_name, field_type), ...])`` and equivalent to ``collections.namedtuple(type_name, [field_name, ...])``. This is useful to declare the types of the fields of a a named tuple type. * cast(), described earlier * @no_type_check, a decorator to disable type checking per class or function (see below) * @no_type_check_decorator, a decorator to create your own decorators with the same meaning as ``@no_type_check`` (see below) * @overload, described earlier * get_type_hints(), a utility function to retrieve the type hints from a function or method. Given a function or method object, it returns a dict with the same format as ``__annotations__``, but evaluating forward references (which are given as string literals) as expressions in the context of the original function or method definition. Types available in the ``typing.io`` submodule: * IO (generic over ``AnyStr``) * BinaryIO (a simple subtype of ``IO[bytes]``) * TextIO (a simple subtype of ``IO[str]``) Types available in the ``typing.re`` submodule: * Match and Pattern, types of ``re.match()`` and ``re.compile()`` results (generic over ``AnyStr``) Rejected Alternatives ===================== During discussion of earlier drafts of this PEP, various objections were raised and alternatives were proposed. We discuss some of these here and explain why we reject them. Several main objections were raised. Which brackets for generic type parameters? ------------------------------------------- Most people are familiar with the use of angular brackets (e.g. ``List<int>``) in languages like C++, Java, C# and Swift to express the parametrization of generic types. The problem with these is that they are really hard to parse, especially for a simple-minded parser like Python. In most languages the ambiguities are usually dealt with by only allowing angular brackets in specific syntactic positions, where general expressions aren't allowed. (And also by using very powerful parsing techniques that can backtrack over an arbitrary section of code.) But in Python, we'd like type expressions to be (syntactically) the same as other expressions, so that we can use e.g. variable assignment to create type aliases. Consider this simple type expression:: List<int> >From the Python parser's perspective, the expression begins with the same four tokens (NAME, LESS, NAME, GREATER) as a chained comparison:: a c # I.e., (a < b) and (b > c) We can even make up an example that could be parsed both ways:: a [ c ] Assuming we had angular brackets in the language, this could be interpreted as either of the following two:: (a)[c] # I.e., (a).__getitem__(c) a ([c]) # I.e., (a < b) and (b > [c]) It would surely be possible to come up with a rule to disambiguate such cases, but to most users the rules would feel arbitrary and complex. It would also require us to dramatically change the CPython parser (and every other parser for Python). It should be noted that Python's current parser is intentionally "dumb" -- a simple grammar is easier for users to reason about. For all these reasons, square brackets (e.g. ``List[int]``) are (and have long been) the preferred syntax for generic type parameters. They can be implemented by defining the ``__getitem__()`` method on the metaclass, and no new syntax is required at all. This option works in all recent versions of Python (starting with Python 2.2). Python is not alone in this syntactic choice -- generic classes in Scala also use square brackets. What about existing uses of annotations? ---------------------------------------- One line of argument points out that PEP 3107 explicitly supports the use of arbitrary expressions in function annotations. The new proposal is then considered incompatible with the specification of PEP 3107. Our response to this is that, first of all, the current proposal does not introduce any direct incompatibilities, so programs using annotations in Python 3.4 will still work correctly and without prejudice in Python 3.5. We do hope that type hints will eventually become the sole use for annotations, but this will require additional discussion and a deprecation period after the initial roll-out of the typing module with Python 3.5. The current PEP will have provisional status (see PEP 411) until Python 3.6 is released. The fastest conceivable scheme would introduce silent deprecation of non-type-hint annotations in 3.6, full deprecation in 3.7, and declare type hints as the only allowed use of annotations in Python 3.8. This should give authors of packages that use annotations plenty of time to devise another approach, even if type hints become an overnight success. Another possible outcome would be that type hints will eventually become the default meaning for annotations, but that there will always remain an option to disable them. For this purpose the current proposal defines a decorator ``@no_type_check`` which disables the default interpretation of annotations as type hints in a given class or function. It also defines a meta-decorator ``@no_type_check_decorator`` which can be used to decorate a decorator (!), causing annotations in any function or class decorated with the latter to be ignored by the type checker. There are also ``# type: ignore`` comments, and static checkers should support configuration options to disable type checking in selected packages. Despite all these options, proposals have been circulated to allow type hints and other forms of annotations to coexist for individual arguments. One proposal suggests that if an annotation for a given argument is a dictionary literal, each key represents a different form of annotation, and the key ``'type'`` would be use for type hints. The problem with this idea and its variants is that the notation becomes very "noisy" and hard to read. Also, in most cases where existing libraries use annotations, there would be little need to combine them with type hints. So the simpler approach of selectively disabling type hints appears sufficient. The problem of forward declarations ----------------------------------- The current proposal is admittedly sub-optimal when type hints must contain forward references. Python requires all names to be defined by the time they are used. Apart from circular imports this is rarely a problem: "use" here means "look up at runtime", and with most "forward" references there is no problem in ensuring that a name is defined before the function using it is called. The problem with type hints is that annotations (per PEP 3107, and similar to default values) are evaluated at the time a function is defined, and thus any names used in an annotation must be already defined when the function is being defined. A common scenario is a class definition whose methods need to reference the class itself in their annotations. (More general, it can also occur with mutually recursive classes.) This is natural for container types, for example:: class Node: """Binary tree node.""" def __init__(self, left: Node, right: None): self.left = left self.right = right As written this will not work, because of the peculiarity in Python that class names become defined once the entire body of the class has been executed. Our solution, which isn't particularly elegant, but gets the job done, is to allow using string literals in annotations. Most of the time you won't have to use this though -- most *uses* of type hints are expected to reference builtin types or types defined in other modules. A counterproposal would change the semantics of type hints so they aren't evaluated at runtime at all (after all, type checking happens off-line, so why would type hints need to be evaluated at runtime at all). This of course would run afoul of backwards compatibility, since the Python interpreter doesn't actually know whether a particular annotation is meant to be a type hint or something else. A compromise is possible where a ``__future__`` import could enable turning *all* annotations in a given module into string literals, as follows:: from __future__ import annotations class ImSet: def add(self, a: ImSet) -> List[ImSet]: ... assert ImSet.add.__annotations__ == {'a': 'ImSet', 'return': 'List[ImSet]'} Such a ``__future__`` import statement may be proposed in a separate PEP. The double colon ---------------- A few creative souls have tried to invent solutions for this problem. For example, it was proposed to use a double colon (``::``) for type hints, solving two problems at once: disambiguating between type hints and other annotations, and changing the semantics to preclude runtime evaluation. There are several things wrong with this idea, however. * It's ugly. The single colon in Python has many uses, and all of them look familiar because they resemble the use of the colon in English text. This is a general rule of thumb by which Python abides for most forms of punctuation; the exceptions are typically well known from other programming languages. But this use of ``::`` is unheard of in English, and in other languages (e.g. C++) it is used as a scoping operator, which is a very different beast. In contrast, the single colon for type hints reads naturally -- and no wonder, since it was carefully designed for this purpose (the idea long predates PEP 3107 [gvr-artima]_). It is also used in the same fashion in other languages from Pascal to Swift. * What would you do for return type annotations? * It's actually a feature that type hints are evaluated at runtime. * Making type hints available at runtime allows runtime type checkers to be built on top of type hints. * It catches mistakes even when the type checker is not run. Since it is a separate program, users may choose not to run it (or even install it), but might still want to use type hints as a concise form of documentation. Broken type hints are no use even for documentation. * Because it's new syntax, using the double colon for type hints would limit them to code that works with Python 3.5 only. By using existing syntax, the current proposal can easily work for older versions of Python 3. (And in fact mypy supports Python 3.2 and newer.) * If type hints become successful we may well decide to add new syntax in the future to declare the type for variables, for example ``var age: int = 42``. If we were to use a double colon for argument type hints, for consistency we'd have to use the same convention for future syntax, perpetuating the ugliness. Other forms of new syntax ------------------------- A few other forms of alternative syntax have been proposed, e.g. the introduction of a ``where`` keyword [roberge]_, and Cobra-inspired ``requires`` clauses. But these all share a problem with the double colon: they won't work for earlier versions of Python 3. The same would apply to a new ``__future__`` import. Other backwards compatible conventions -------------------------------------- The ideas put forward include: * A decorator, e.g. ``@typehints(name=str, returns=str)``. This could work, but it's pretty verbose (an extra line, and the argument names must be repeated), and a far cry in elegance from the PEP 3107 notation. * Stub files. We do want stub files, but they are primarily useful for adding type hints to existing code that doesn't lend itself to adding type hints, e.g. 3rd party packages, code that needs to support both Python 2 and Python 3, and especially extension modules. For most situations, having the annotations in line with the function definitions makes them much more useful. * Docstrings. There is an existing convention for docstrings, based on the Sphinx notation (``:type arg1: description``). This is pretty verbose (an extra line per parameter), and not very elegant. We could also make up something new, but the annotation syntax is hard to beat (because it was designed for this very purpose). It's also been proposed to simply wait another release. But what problem would that solve? It would just be procrastination. PEP Development Process ======================= A live draft for this PEP lives on GitHub [github]_. There is also an issue tracker [issues]_, where much of the technical discussion takes place. The draft on GitHub is updated regularly in small increments. The official PEPS repo [peps_] is (usually) only updated when a new draft is posted to python-dev. Acknowledgements ================ This document could not be completed without valuable input, encouragement and advice from Jim Baker, Jeremy Siek, Michael Matson Vitousek, Andrey Vlasovskikh, Radomir Dopieralski, Peter Ludemann, and the BDFL-Delegate, Mark Shannon. Influences include existing languages, libraries and frameworks mentioned in PEP 482. Many thanks to their creators, in alphabetical order: Stefan Behnel, William Edwards, Greg Ewing, Larry Hastings, Anders Hejlsberg, Alok Menghrajani, Travis E. Oliphant, Joe Pamer, Raoul-Gabriel Urma, and Julien Verlaguet. References ========== .. [mypy] http://mypy-lang.org .. [gvr-artima] http://www.artima.com/weblogs/viewpost.jsp?thread=85551 .. [wiki-variance] http://en.wikipedia.org/wiki/Covariance_and_contravariance_%28computer_scie… .. [typeshed] https://github.com/JukkaL/typeshed/ .. [pyflakes] https://github.com/pyflakes/pyflakes/ .. [pylint] http://www.pylint.org .. [roberge] http://aroberge.blogspot.com/2015/01/type-hinting-in-python-focus-on.html .. [github] https://github.com/ambv/typehinting .. [issues] https://github.com/ambv/typehinting/issues .. [peps] https://hg.python.org/peps/file/tip/pep-0484.txt Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: -- --Guido van Rossum (python.org/~guido)

5 years, 10 months

segfaults due to hash randomization in C OrderedDict

by Eric Snow

(see http://bugs.python.org/issue16991) I an working on resolving an intermittent segfault that my C OrderedDict patch introduces. The failure happens in test_configparser (RawConfigParser uses OrderedDict internally), but only sporadically. However, Ned pointed out to me that it appears to be related to hash randomization, which I have verified. I'm looking into it. In the meantime, here's a specific question. What would lead to the pattern of failures I'm seeing? I've verified that the segfault happens consistently for certain hash randomization seeds and never for the rest. I don't immediately recognize the pattern but expect that it would shed some light on where the problem lies. I ran the following command with the OrderedDict patch applied: for i in `seq 1 100`; do echo $i; PYTHONHASHSEED=$i ./python -m test.regrtest -m test_basic test_configparser ; done Through 100 I get segfaults with seeds of 7, 15, 35, 37, 39, 40, 42, 47, 50, 66, 67, 85, 87, 88, and 92. I expect the distribution across all seeds is uniform, but I haven't verified that. Thoughts? -eric

5 years, 10 months

← Newer
1
2
3
4
5
6
7
8
9
Older →

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

Python-Dev May 2015