Python-Dev April 2017

python-dev@python.org

39 participants
31 discussions

by Mariatta Wijaya

Hi everyone, >From now on, when proposing a new PEP, please use the number 9999. This allows Travis build to pass, while waiting for a PEP editor to assign you a number. This means, in the header of your draft PEP, instead of ``PEP: XXX``, you should now use ``PEP: 9999``. You should name the file ``pep-9999.rst``, instead of ``pep-xxxx.rst``. Both PEP 1 and PEP 12 have been updated to address this. Thanks :) Mariatta Wijaya

3 years, 4 months

windows installer and python list mention

by Ethan Furman

Apparently the Windows installer mentions asking for help on Python List. If accessing Python List directly via email it is necessary to subscribe first -- perhaps that could be mentioned? Bonus Python Points for mentioning the other methods that do not require subscribing. :) Rejection text sent to non-subscribers follows: ---- "Hello! I see you want to post a message to the Python List. We would be happy to help, but you must subscribe first: https://mail.python.org/mailman/listinfo/python-list After you have subscribed, please send your message again. Alternatively, this list is mirrored both ways with the comp.lang.python newsgroup (news:comp.lang.python). Some people find it easier to follow this and other lists via gmane (http://news.gmane.org/gmane.comp.python.general), a service which offers a newsgroup interface to many online mailing lists. *NB all posts to the mailing list are publicly archived at:* https://mail.python.org/pipermail/python-list " Any questions or comments should be directed to the list administrator at: python-list-owner(a)python.org --- -- ~Ethan~

3 years, 4 months

I will be deleting the cpython-mirror repo on April 10

by Brett Cannon

On the two-month anniversary of the GitHub migration I'm going to delete the old git mirror: https://github.com/python/cpython-mirror. If you have a old PR that got closed with comments or something, now is the time to get those comments off.

3 years, 4 months

CPython benchmark status, April 2017

by Victor Stinner

Hi, I'm still working on analyzing past optimizations to guide future optimizations. I succeeded to identify multiple significant optimizations over the last 3 years. At least for me, some were unexpected like "Use the test suite for profile data" which made pickle 1.28x faster and pidigts 1.16x faster. Here is a report of my benchmark work of last weeks. I succeeded to compute benchmarks on CPython master on the period April, 2014-April,2017: we now have have a timeline over 3 years of CPython performance! https://speed.python.org/timeline/ I started to take notes on significant performance changes (speedup and slowdown) of this timeline: http://pyperformance.readthedocs.io/cpython_results_2017.html To identify the change which introduced a significant performance change, I wrote a Python script running a Git bisection: compile CPython, run benchmark, repeat. https://github.com/haypo/misc/blob/master/misc/bisect_cpython_perf.py It uses a configuration file which looks like: --- [config] work_dir = ~/prog/bench_python/bisect-pickle src_dir = ~/prog/bench_python/master old_commit = 133138a284be1985ebd9ec9014f1306b9a42 new_commit = 10427f44852b6e872034061421a8890902b8f benchmark = ~/prog/bench_python/performance/performance/benchmarks/bm_pickle.py pickle benchmark_opts = --inherit-environ=PYTHONPATH -p5 -v configure_args = --- I succeeded to identify many significant optimizations, examples: * PyMem_Malloc() now uses the fast pymalloc allocator * Add a C implementation of collections.OrderedDict * Use the test suite for profile data * Speedup method calls 1.2x * Added C implementation of functools.lru_cache() * Optimized ElementTree.iterparse(); it is now 2x faster perf, performance, server configuration, etc. evolve quicker than expected, so I created a Git project to keep a copy of JSON files: https://github.com/haypo/performance_results I lost data of my first milestone (november-december 2016), but you have data from the second (december 2016-february 2017) and third (march 2016-today) milestones. Victor

3 years, 4 months

Summary of Python tracker Issues

by Python tracker

ACTIVITY SUMMARY (2017-03-31 - 2017-04-07) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 5882 (+27) closed 35891 (+37) total 41773 (+64) Open issues with patches: 2413 Issues opened (48) ================== #29939: Compiler warning in _ctypes_test.c http://bugs.python.org/issue29939 reopened by vinay.sajip #29956: math.exp documentation is misleading http://bugs.python.org/issue29956 opened by belopolsky #29957: unnecessary LBYL for key contained in defaultdict, lib2to3/btm http://bugs.python.org/issue29957 opened by selik #29960: _random.Random state corrupted on exception http://bugs.python.org/issue29960 opened by bryangeneolson #29963: Remove obsolete declaration PyTokenizer_RestoreEncoding in tok http://bugs.python.org/issue29963 opened by Jim Fasarakis-Hilliard #29964: %z directive has no effect on the output of time.strptime http://bugs.python.org/issue29964 opened by Paul Pinterits #29966: typing.get_type_hints doesn't really work for classes with For http://bugs.python.org/issue29966 opened by simon.percivall #29967: "AMD64 FreeBSD 9.x 3.x" tries to rebuild Include/opcode.h, tim http://bugs.python.org/issue29967 opened by haypo #29970: Severe open file leakage running asyncio SSL server http://bugs.python.org/issue29970 opened by kyuupichan #29971: Lock.acquire() not interruptible on Windows http://bugs.python.org/issue29971 opened by pitrou #29972: Skip tests known to fail on AIX http://bugs.python.org/issue29972 opened by haypo #29974: Change typing.TYPE_CHECKING doc example http://bugs.python.org/issue29974 opened by rav #29975: Issue in extending documentation http://bugs.python.org/issue29975 opened by Namjun Kim #29976: urllib.parse clarify what ' ' in schemes mean http://bugs.python.org/issue29976 opened by orsenthil #29979: cgi.parse_multipart is not consistent with FieldStorage http://bugs.python.org/issue29979 opened by quentel #29980: OSError: multiple exceptions should preserve the exception typ http://bugs.python.org/issue29980 opened by r.david.murray #29981: Update Index for set, dict, and generator 'comprehensions' http://bugs.python.org/issue29981 opened by terry.reedy #29982: tempfile.TemporaryDirectory fails to delete itself http://bugs.python.org/issue29982 opened by max #29983: Reference TOC: expand 'Atoms' and 'Primaries' http://bugs.python.org/issue29983 opened by terry.reedy #29984: Improve test coverage for 'heapq' module http://bugs.python.org/issue29984 opened by Robert Day #29985: make install doesn't seem to support --quiet http://bugs.python.org/issue29985 opened by chris.jerdonek #29986: Documentation recommends raising TypeError from tp_richcompare http://bugs.python.org/issue29986 opened by Devin Jeanpierre #29987: inspect.isgeneratorfunction not working with partial functions http://bugs.python.org/issue29987 opened by Thomas Antony #29988: (async) with blocks and try/finally are not as KeyboardInterru http://bugs.python.org/issue29988 opened by njs #29989: subprocess.Popen does not handle file-like objects without fil http://bugs.python.org/issue29989 opened by rtpg #29990: Range checking in GB18030 decoder http://bugs.python.org/issue29990 opened by Ma Lin #29991: http client marks valid multipart headers with defects. http://bugs.python.org/issue29991 opened by pareshverma91 #29992: Expose parse_string in JSONDecoder http://bugs.python.org/issue29992 opened by oberstet #29994: site.USER_SITE is None for Windows embeddable Python 3.6 http://bugs.python.org/issue29994 opened by brechtm #29995: re.escape() escapes too much http://bugs.python.org/issue29995 opened by serhiy.storchaka #29996: Use terminal width by default in pprint http://bugs.python.org/issue29996 opened by serhiy.storchaka #29997: Suggested changes for https://docs.python.org/3.6/extending/ex http://bugs.python.org/issue29997 opened by ArthurGoldberg #29998: Pickling and copying ImportError doesn't preserve name and pat http://bugs.python.org/issue29998 opened by serhiy.storchaka #29999: repr() of ImportError misses keyword arguments name and path http://bugs.python.org/issue29999 opened by serhiy.storchaka #30000: Inconsistency in the zlib module http://bugs.python.org/issue30000 opened by Ellison Marks #30002: Minor change to https://docs.python.org/3.6/extending/building http://bugs.python.org/issue30002 opened by ArthurGoldberg #30003: Remove hz codec http://bugs.python.org/issue30003 opened by Ma Lin #30004: in regex-howto, improve example on grouping http://bugs.python.org/issue30004 opened by Cristian Barbarosie #30005: Pickling and copying exceptions doesn't preserve non-__dict__ http://bugs.python.org/issue30005 opened by serhiy.storchaka #30006: Deadlocks in `concurrent.futures.ProcessPoolExecutor` http://bugs.python.org/issue30006 opened by tomMoral #30008: OpenSSL 1.1.0 deprecated functions http://bugs.python.org/issue30008 opened by floppymaster #30011: HTMLParser class is not thread safe http://bugs.python.org/issue30011 opened by ale2017 #30012: gzip.open(filename, "rt") fails on Python 2.7.11 on win32, inv http://bugs.python.org/issue30012 opened by maubp #30013: Compiler warning in Modules/posixmodule.c http://bugs.python.org/issue30013 opened by louielu #30014: Speedup DefaultSelectors.modify() by 1.5x http://bugs.python.org/issue30014 opened by giampaolo.rodola #30015: Windows also treats full-width spaces as a delimiter when pars http://bugs.python.org/issue30015 opened by LCY #30016: No sideways scrolling in IDLE http://bugs.python.org/issue30016 opened by Jensen Taylor #30017: zlib.error: Error -2 while flushing: inconsistent stream state http://bugs.python.org/issue30017 opened by Jeremy Heiner Most recent 15 issues with no replies (15) ========================================== #30017: zlib.error: Error -2 while flushing: inconsistent stream state http://bugs.python.org/issue30017 #30016: No sideways scrolling in IDLE http://bugs.python.org/issue30016 #30015: Windows also treats full-width spaces as a delimiter when pars http://bugs.python.org/issue30015 #30013: Compiler warning in Modules/posixmodule.c http://bugs.python.org/issue30013 #30012: gzip.open(filename, "rt") fails on Python 2.7.11 on win32, inv http://bugs.python.org/issue30012 #30011: HTMLParser class is not thread safe http://bugs.python.org/issue30011 #30008: OpenSSL 1.1.0 deprecated functions http://bugs.python.org/issue30008 #30006: Deadlocks in `concurrent.futures.ProcessPoolExecutor` http://bugs.python.org/issue30006 #30005: Pickling and copying exceptions doesn't preserve non-__dict__ http://bugs.python.org/issue30005 #29998: Pickling and copying ImportError doesn't preserve name and pat http://bugs.python.org/issue29998 #29995: re.escape() escapes too much http://bugs.python.org/issue29995 #29994: site.USER_SITE is None for Windows embeddable Python 3.6 http://bugs.python.org/issue29994 #29985: make install doesn't seem to support --quiet http://bugs.python.org/issue29985 #29984: Improve test coverage for 'heapq' module http://bugs.python.org/issue29984 #29982: tempfile.TemporaryDirectory fails to delete itself http://bugs.python.org/issue29982 Most recent 15 issues waiting for review (15) ============================================= #30014: Speedup DefaultSelectors.modify() by 1.5x http://bugs.python.org/issue30014 #29999: repr() of ImportError misses keyword arguments name and path http://bugs.python.org/issue29999 #29998: Pickling and copying ImportError doesn't preserve name and pat http://bugs.python.org/issue29998 #29996: Use terminal width by default in pprint http://bugs.python.org/issue29996 #29995: re.escape() escapes too much http://bugs.python.org/issue29995 #29990: Range checking in GB18030 decoder http://bugs.python.org/issue29990 #29974: Change typing.TYPE_CHECKING doc example http://bugs.python.org/issue29974 #29963: Remove obsolete declaration PyTokenizer_RestoreEncoding in tok http://bugs.python.org/issue29963 #29957: unnecessary LBYL for key contained in defaultdict, lib2to3/btm http://bugs.python.org/issue29957 #29951: PyArg_ParseTupleAndKeywords exception messages containing "fun http://bugs.python.org/issue29951 #29947: In SocketServer, why not passing a factory instance for the Re http://bugs.python.org/issue29947 #29930: Waiting for asyncio.StreamWriter.drain() twice in parallel rai http://bugs.python.org/issue29930 #29926: time.sleep ignores _thread.interrupt_main() http://bugs.python.org/issue29926 #29914: Incorrect signatures of object.__reduce__() and object.__reduc http://bugs.python.org/issue29914 #29897: itertools.chain behaves strangly when copied with copy.copy http://bugs.python.org/issue29897 Top 10 most discussed issues (10) ================================= #29947: In SocketServer, why not passing a factory instance for the Re http://bugs.python.org/issue29947 15 msgs #30014: Speedup DefaultSelectors.modify() by 1.5x http://bugs.python.org/issue30014 11 msgs #29996: Use terminal width by default in pprint http://bugs.python.org/issue29996 10 msgs #28157: Document time module constants (timezone, tzname, etc.) as dep http://bugs.python.org/issue28157 9 msgs #29941: Confusion between asserts and Py_DEBUG http://bugs.python.org/issue29941 8 msgs #29957: unnecessary LBYL for key contained in defaultdict, lib2to3/btm http://bugs.python.org/issue29957 7 msgs #29964: %z directive has no effect on the output of time.strptime http://bugs.python.org/issue29964 7 msgs #29956: math.exp documentation is misleading http://bugs.python.org/issue29956 6 msgs #5906: Risk of confusion in multiprocessing module - daemonic process http://bugs.python.org/issue5906 5 msgs #19225: lack of PyExc_BufferError doc http://bugs.python.org/issue19225 5 msgs Issues closed (35) ================== #11066: cgi.py proposals : sys.stdout encoding + rewriting of parsing http://bugs.python.org/issue11066 closed by quentel #13290: get vars for object with __slots__ http://bugs.python.org/issue13290 closed by rhettinger #15083: Rewrite ElementTree tests in a cleaner and safer way http://bugs.python.org/issue15083 closed by serhiy.storchaka #16510: Using appropriate checks in tests http://bugs.python.org/issue16510 closed by serhiy.storchaka #20545: Use specific asserts in unicode tests http://bugs.python.org/issue20545 closed by serhiy.storchaka #20547: Use specific asserts in bigmem tests http://bugs.python.org/issue20547 closed by serhiy.storchaka #24117: Wrong range checking in GB18030 decoder. http://bugs.python.org/issue24117 closed by Ma Lin #26947: Hashable documentation improvement needed http://bugs.python.org/issue26947 closed by orsenthil #28115: Use argparse for the zipfile module http://bugs.python.org/issue28115 closed by serhiy.storchaka #29339: Interactive: Move to same indentation level as previousline http://bugs.python.org/issue29339 closed by terry.reedy #29549: Improve docstring for str.index http://bugs.python.org/issue29549 closed by rhettinger #29569: threading.Timer class: Continue periodical execution till acti http://bugs.python.org/issue29569 closed by rhettinger #29649: struct.pack_into check boundary error message ignores offset http://bugs.python.org/issue29649 closed by serhiy.storchaka #29654: SimpleHTTPRequestHandler should support browser cache http://bugs.python.org/issue29654 closed by serhiy.storchaka #29725: sqlite3.Cursor doesn't properly document "arraysize" http://bugs.python.org/issue29725 closed by orsenthil #29762: Use "raise from None" http://bugs.python.org/issue29762 closed by serhiy.storchaka #29949: sizeof set after set_merge() is doubled from 3.5 http://bugs.python.org/issue29949 closed by inada.naoki #29953: Memory leak in the replace() method of datetime and time objec http://bugs.python.org/issue29953 closed by serhiy.storchaka #29954: multiprocessing.Pool.__exit__() calls terminate() instead of c http://bugs.python.org/issue29954 closed by pitrou #29955: logging decimal point should come from locale http://bugs.python.org/issue29955 closed by vinay.sajip #29958: Use add_mutually_exclusive_group(required=True) in zipfile and http://bugs.python.org/issue29958 closed by serhiy.storchaka #29959: re.match failed to match left square brackets as the first cha http://bugs.python.org/issue29959 closed by serhiy.storchaka #29961: More compact sets and frozensets created from sets http://bugs.python.org/issue29961 closed by rhettinger #29962: Add math.remainder operation http://bugs.python.org/issue29962 closed by mark.dickinson #29965: MatchObject __getitem__() should support slicing and len http://bugs.python.org/issue29965 closed by serhiy.storchaka #29968: Document that no characters are allowed to proceed \ in explic http://bugs.python.org/issue29968 closed by Jim Fasarakis-Hilliard #29969: Typo in decimal error message http://bugs.python.org/issue29969 closed by skrah #29973: Travis CI docs broken: UnboundLocalError: local variable 'pref http://bugs.python.org/issue29973 closed by haypo #29977: re.sub stalls forever on an unmatched non-greedy case http://bugs.python.org/issue29977 closed by serhiy.storchaka #29978: Remove remove merge=union attribute for Misc/NEWS in 3.6 and 2 http://bugs.python.org/issue29978 closed by Mariatta #29993: error of parsing encoded words in email of standard library http://bugs.python.org/issue29993 closed by r.david.murray #30001: CPython contribution docs reference missing /issuetracker page http://bugs.python.org/issue30001 closed by Mariatta #30007: report bug http://bugs.python.org/issue30007 closed by xiang.zhang #30009: Integer conversion failure http://bugs.python.org/issue30009 closed by eryksun #30010: Initial bytes to BytesIO cannot be seeked to http://bugs.python.org/issue30010 closed by serhiy.storchaka

3 years, 4 months

Wider Call For Participation for the 2017 Python Language Summit

by Larry Hastings

The 2017 Python Language Summit is coming! The Python Language Summit is an annual day-long meeting of CPython core developers. It’s a highly technical meeting, designed to explore and resolve existing discussions regarding the Python programming language, the development of its reference implementation CPython, and the impact of the language’s evolution on the existing alternative implementations. It’s a once-a-year opportunity for Python’s core development team to get together in a room and work things out. The meeting is kept small on purpose, as we think that maximizes its productivity and effectiveness. Nearly all attendees are CPython core developers, but we do accept presentations from anyone in the greater Python community who has something interesting to say to the core developers. And that could be you! In order to be eligible, you must be able to attend the Summit in person. The Summit will be held May 17 2017, all day, in the same convention center where PyCon itself is held. You have to get there yourself; we literally have no discretionary budget to help people attend the Summit. However, you don’t have to buy a ticket to PyCon in order to attend the summit--it’s a completely separate event, and it’s free! But mere eligibility is not enough. The presentations are carefully hand-picked by the Language Summit organizers, and must be exceedingly relevant and of high quality in order to be considered. The Summit only comes once a year, and we the organizers want to keep it interesting and maximally productive for the developers who are kind enough to attend. To be brutally honest, we expect most proposals from non-core-developers will be turned down--again, sorry. PyCon is large, diverse, welcoming, and vibrant, and there are lots of great avenues (e.g. lightning talks, BoFs, open spaces, etc.) for discussing Python-related topics. If your proposed talk isn't accepted for the Language Summit, we highly encourage you to explore these other options. Here are the criteria you should keep in mind if you submit a presentation: * Is this a question about the *design* of Python? Or, to a lesser extent, the implementation of CPython or one of the alternative implementations? The Summit is about Python itself--we don’t discuss other projects. * Is this a *technical* debate? The Python universe is large and diverse, but the Summit is explicitly a highly technical meeting, and a deeper technical context on the part of participants is essential. * Is this topic going to spark conversation? A good rule of thumb: your presentation should raise questions, not provide answers. If your topic could be adequately covered in a blog post, it’s probably not interactive enough. * Is this already an ongoing discussion in the Python development community? As a rule the Language Summit is not a venue for new business--it’s for working through existing technical debates. * Is this topic going to be interesting to a lot of people? While it doesn’t have to be universally interesting, it can’t be too narrow either. As a rule, most people in the room should be interested. * Is this a topic that’s still considered "open" by the core developers? There’s no point in proposing a talk saying "you should abandon Python 3 and go back to 2", or even "you should make a Python 2.8". From the perspective of the core developers, these are resolved, closed issues. Examples of interesting topics: * Python’s new async syntax is flawed, here’s why * The design of CPython’s extension API makes it difficult to do what we want it to do * My patch for CPython makes it ten times faster for common workloads * I’m engaged in a lively and long-lived discussion on python-dev and I want to bring it up with the core devs in person, so that a resolution can be found Examples of irrelevant / uninteresting / ineligible topics: * A useful Python library you want people to know about * A new language feature you want to propose * There’s a bug in CPython you want to raise awareness about * The Python community needs to... The process for submitting a talk proposal is exactly the same as for core developers: fill out the Google form with your contact information, your affiliation, and a summary of your proposal. The form is reachable via the Language Summit page under the PyCon events menu: https://us.pycon.org/2017/events/language-summit/ Since this wider call for proposals comes so late in the process, we’re extending the deadline for submissions. The deadline is now Thursday, April 20th, 2017, two weeks from today. If your submission is accepted, we will notify you by May 1st. Finally, even if you don’t get to attend, stay tuned to Linux Weekly News (LWN) in the days and weeks following the Language Summit. Jake Edge from LWN has done a fantastic job of reporting on the Language Summit the previous two years, and is planning on attending and reporting on it again this year. We all look forward to his thorough reporting of the event! [BL]arry (Barry Warsaw and Larry Hastings)

3 years, 4 months

Sorting

by Ethan Furman

Just want to drop a note and say THANK YOU to everyone for the work in improving Python to the 3 branch. And if you're curious as to why: I just spent three hours trying to figure out why my comparisons were succeeding when they should be raising exceptions -- and then remembered that in 2.x there's a fallback to comparing type names. Grrrr. :/ So, THANK YOU!! -- ~Ethan~

3 years, 4 months

why _PyGen_Finalize(gen) propagates close() to _PyGen_yf() ?

by Oleg Nesterov

Hello, Let me first clarify, I do not claim this is a bug, I am trying to learn python and now I trying to understand yield-from. This simple test-case g = (x for x in range(10)) def fg(): for x in g: yield x print(next(fg())) print(next(g)) works as expected and prints: 0 1 However, if I change fg() to use yield-from g = (x for x in range(10)) def fg(): yield from g print(next(fg())) print(next(g)) then next(g) raises StopIteration: 0 Traceback (most recent call last): File "/tmp/T.py", line 10, in <module> print(next(g)) StopIteration because g.close() is called by destructor of the object returned by fg(). To me this looks strange and confusing. I tried to google, and found https://docs.python.org/3/whatsnew/3.3.html#pep-380 but it doesn't document this behaviour. I understand that yield-from should propagate .close(), but why _PyGen_Finalize() should send close() to the gi_yieldfrom object? I applied the patch below just to verify that I actually understand what's going on, and with this patch the 2nd test-case works as I'd expect. But since I am very new to python I'd suspect that the code is fine and I simply do not understand why it works this way. So. could someone please explain the rationale behind this behaviour? And probably update the docs should be updated? Thanks in advance. Oleg. --- diff --git a/Objects/genobject.c b/Objects/genobject.c index 24a1da6..d5152eb 100644 --- a/Objects/genobject.c +++ b/Objects/genobject.c @@ -6,6 +6,7 @@ #include "opcode.h" static PyObject *gen_close(PyGenObject *, PyObject *); +static PyObject *do_gen_close(PyGenObject *, PyObject *); static PyObject *async_gen_asend_new(PyAsyncGenObject *, PyObject *); static PyObject *async_gen_athrow_new(PyAsyncGenObject *, PyObject *); @@ -71,7 +72,7 @@ _PyGen_Finalize(PyObject *self) } } else { - res = gen_close(gen, NULL); + res = do_gen_close(gen, NULL); } if (res == NULL) { @@ -373,10 +374,9 @@ _PyGen_yf(PyGenObject *gen) } static PyObject * -gen_close(PyGenObject *gen, PyObject *args) +do_gen_close(PyGenObject *gen, PyObject *yf) { PyObject *retval; - PyObject *yf = _PyGen_yf(gen); int err = 0; if (yf) { @@ -407,6 +407,11 @@ gen_close(PyGenObject *gen, PyObject *args) return NULL; } +static PyObject * +gen_close(PyGenObject *gen, PyObject *args) +{ + return do_gen_close(gen, _PyGen_yf(gen)); +} PyDoc_STRVAR(throw_doc, "throw(typ[,val[,tb]]) -> raise exception in generator,\n\

3 years, 4 months

Fix or drop AIX buildbot?

by Victor Stinner

Hi, We have a "PPC64 AIX 3.x" buildbot slave which fails on cloning the GitHub repository: "SSL certificate problem: unable to get local issuer certificate". It started to fail around Feb 11, 2017 (Build #294), probably when buildbots moved to GitHub, after CPython moved to GitHub. First build which failed: http://buildbot.python.org/all/builders/PPC64%20AIX%203.x/builds/294 Moreover, some tests are failing since at least 2 years on AIX. Some examples: * test_locale.test_strcoll_with_diacritic() * test_socket.testIPv4toString() * test_strptime Last build which ran unit tests: http://buildbot.python.org/all/builders/PPC64%20AIX%203.x/builds/293/steps/… For me, the principle of a CI is to detect *regressions*. But the AIX buildbot is always failing because of known bugs. There are 3 options: * Find a maintainer who quickly fix all known bugs. Unlike * Skip tests known to fail on AIX. I know that many core developers dislike this option, "hiding" bugs. * Drop the buildbot My favorite option is to drop the buildbot, since I'm tired of these red buildbot slaves. I added David Edelsohn to this email, he owns the buildbot, and I know that he wants to get the best AIX support in Python. The question is also which kind of support level do we provide per platform? Full support, "best effort" or no support? * Full support requires active maintainers, a CI with tests which pass, etc. * "Best effort": fix bugs when someone complains and someone (else?) provides a fix * No support: reject proposed patches to add a partial support for a platform. Victor

3 years, 4 months

__del__ is not called after creating a new reference

by Oleg Nesterov

Hello, I already tried to ask on python-list, see https://mail.python.org/pipermail/python-list/2017-March/720037.html but it seems that this list is not for technical questions. Let me resend my question to python-dev. Please tell me if I should not spam this list with newbiesh questions, and thanks in advance. --------------------------------------------------------------------------- I started to learn python a few days ago and I am trying to understand what __del__() actually does. https://docs.python.org/3/reference/datamodel.html says: object.__del__(self) ... Note that it is possible (though not recommended!) for the __del__() method to postpone destruction of the instance by creating a new reference to it. It may then be called at a later time when this new reference is deleted. However, this trivial test-case class C: def __del__(self): print("DEL") global X X = self C() print(X) X = 0 print(X) shows that __del__ is called only once, it is not called again after "X = 0": DEL <__main__.C object at 0x7f067695f4a8> 0 (Just in case, I verified later that this object actually goes away and its memory is freed, so the problem is not that it still has a reference). I've cloned https://github.com/python/cpython.git and everything looks clear at first glance (but let me repeat that I am very new to python): PyObject_CallFinalizerFromDealloc() calls PyObject_CallFinalizer() which finally calls "__del__" method in slot_tp_finalize(), then it notices that "X = self" creates the new reference and does: /* tp_finalize resurrected it! Make it look like the original Py_DECREF * never happened. */ refcnt = self->ob_refcnt; _Py_NewReference(self); self->ob_refcnt = refcnt; However, PyObject_CallFinalizer() also does _PyGC_SET_FINALIZED(self, 1) and that is why __del__ is not called again after "X = 0": /* tp_finalize should only be called once. */ if (PyType_IS_GC(tp) && _PyGC_FINALIZED(self)) return; The comment and the code are very explicit, so this does nt look like a bug in cpython. Probably the docs should be fixed? Or this code is actually wrong? The test-case works as documented if I remove _PyGC_SET_FINALIZED() in PyObject_CallFinalizer() or add another _PyGC_SET_FINALIZED(self, 0) into PyObject_CallFinalizerFromDealloc() after _Py_NewReference(self), but yes, yes, I understand that this is not correct and won't really help. Oleg.

3 years, 4 months

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

Python-Dev April 2017