Mailman 3 January 2020 - Python-Dev

PEP 587
by Zak Mabbott Jan. 21, 2020

Jan. 21, 2020

I am really confused on writing python and what app I need to write it in any tips or any ideas for apps??

2 1

About "python-porting" mail list
by Facundo Batista Jan. 20, 2020

Jan. 20, 2020

Hello! This list (which I co-admin, with Georg) is getting less and less traffic as months pass by. See: https://mail.python.org/pipermail/python-porting/ The interwebs has been collecting ton of resources about porting py2 to 3 during these years. Any not-yet-answered question surely can be done in a list with more participants. Can we kill this list? Thanks! Regards, -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org.ar/ Twitter: @facundobatista

4 7

Re: Questions about CPython's behavior on addition operator -- binary_op1 (abstract.c) and nb_add (typeobject.c)
by Raphaël Monat Jan. 20, 2020

Jan. 20, 2020

> I'm also keen to understand this and think I can elucidate a little, > from my study of the code. The perspective of someone who *didn't* > write it might help, but I defer to the authors' version when it appears. > Thank you! I'd be happy to know the authors' opinion as well. > A crucial observation is that there is only one nb_add slot in a type > definition. Think about adding a int(1) + float(2). Where it lands in > long_add (v->ob_type->tp_as_number->nb_add) will return > NotImplemented, because it does not understand the float right-hand > argument, but float_add (w->ob_type->tp_as_number->nb_add) is able to > give an answer, since it can float the int, behaving as > float.__radd__(f, i). So the slots have to implement both __add__ and > __radd__. > I agree. And by default, this is done by expanding the macro "SLOT1BIN(slot_nb_add, nb_add, "__add__", "__radd__")". > The logic is the same as binary_op1, but the function has to deal with > the possibility that one or other type may already provide a special > function wrapper function in its nb_add slot, which is what the test > tp_as_number->SLOTNAME == TESTFUNC is about. TESTFUNC is nearly always > the same as FUNCNAME. Then it also deals with quite a complex decision > in method_is_overloaded(). > I have overlooked this test, and now I'm confused by it. In the definition of slot_nb_add, it seems to check that tp_as_number->nb_add == slot_nb_add. Is it (a convoluted way) to check that tp_as_number->nb_add != NULL? Otherwise, I guess this always hold for standard library modules, but someone may change tp_as_number->nb_add for another function? > The partial repetition of the logic, which I think is now nested > (because binary_op1() may have called slot_nb_add) is necessary to > insert the more complex version into the decision tree. But this is > roughly where my ability to visualise the paths runs out. > I have an hypothesis: when binary_op1(v, w, ...) is called: 1) either slotv (v->ob_type->tp_as_number->nb_add) is defined. This calls v's slot_nb_add. This slot_nb_add may then call either v's "__add__" or w's "__radd__" using the calls to vectorcall_maybe. 2) or slotv is not defined. In that case we start the whole process using w's slot_nb_add, which should be stored in w->ob_type->tp_as_number->nb_add. In particular, does this mean that if slotv is defined, the if(slotw) <https://github.com/python/cpython/blob/3.8/Objects/abstract.c#L813> will never be reached (meaning this if could be changed by an else if)? Letting the case where w is a subtype of v aside for now, this would mean that some tests may be performed twice, but only one slot_nb_add will be called? (of course, this slot_nb_add will be able to call either its __add__ or the __radd__ of the other) > I would not say they are "defined" by the slot_nb_add function. > Rather, if one or other has been defined (in Python), v.__add__ or > w.__radd__ is called by the single function slot_nb_add. They cannot > be called directly from C, but call_maybe() supplies the mechanism. > > A confusing factor is that for types defined in C and filling the > nb_add slot, the slot function (float_add, or whatever) has to be > wrapped by two descriptors that can then sit in the type's dictionary > as "__add__" and "__radd__". In that case these *are* defined by a > wrapped C function, but that function is the function in the > implementation of the type (float_add, say), not slot_nb_add. There > are two kinds of wrapper: one used "Python-side out", so > Python-calling "__add__" leads to nb_add's behaviour, and one "C-side > out" so C-calling via nb_add leads to "__add__". > This makes sense. Thank you! > I *think* it is changed to contain slot_nb_add as defined by the macro. > And then slot_nb_add is able to dispatch to __add__ or __radd__ with vectorcall_maybe. Thanks! When this is all clearer (and hopefully accurate), I'll wrap it up in a blog post or something. -- Raphaël

1 0

Remove formatter module from Python 3.9, deprecated for 7 years (Python 3.4)
by Victor Stinner Jan. 17, 2020

Jan. 17, 2020

Hi, I proposed https://bugs.python.org/issue39352 to remove the formatter module. It's deprecated since Python 3.4. The main reason why it's still around is the PEP 4 rule: "In order to facilitate writing code that works in both Python 2 & 3 simultaneously, any module that exists in both Python 3.5 and Python 2.7 will not be removed from the standard library until Python 2.7 is no longer supported as specified by PEP 373." https://www.python.org/dev/peps/pep-0004/#for-modules-existing-in-both-pyth… Python 2.7 is not longer supported. So can we now remove the formatter module? The module has no test, I failed to find any user in a dummy GitHub code search and I didn't know that this module even existed. We can still revert the module before 3.9.0 final release if someone shows up and asks to keep it for one more cycle. My intent here is to reduce the size of the Python standard library to reduce the maintenance burden. Python became quite large and it's more and more expensive to maintain it. Wait... I'm not asking to remove all modules of the standard library :-) The decision must be taken on a case by case basis, for each module. Here I'm only taking about the formatter module. Victor -- Night gathers, and now my watch begins. It shall not end until my death.

3 3

Summary of Python tracker Issues
by Python tracker Jan. 17, 2020

Jan. 17, 2020

ACTIVITY SUMMARY (2020-01-10 - 2020-01-17) Python tracker at https://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 7268 (+17) closed 43855 (+63) total 51123 (+80) Open issues with patches: 2847 Issues opened (54) ================== #16576: ctypes: structure with bitfields as argument https://bugs.python.org/issue16576 reopened by vinay.sajip #37751: In codecs, function 'normalizestring' should convert both spa https://bugs.python.org/issue37751 reopened by vstinner #38076: Make struct module PEP-384 compatible https://bugs.python.org/issue38076 reopened by pablogsal #39293: Windows 10 64-bit needs reboot https://bugs.python.org/issue39293 opened by ToKa #39294: zipfile.ZipInfo objects contain invalid 'extra' fields. https://bugs.python.org/issue39294 opened by Bram Stolk #39295: usage of bitfields in ctypes structures changed between 3.7.5 https://bugs.python.org/issue39295 opened by Matthew Newville #39296: Windows register keys https://bugs.python.org/issue39296 opened by ToKa #39298: add BLAKE3 to hashlib https://bugs.python.org/issue39298 opened by larry #39299: Improve test coverage for mimetypes module https://bugs.python.org/issue39299 opened by xtreak #39301: Specification of bitshift on integers should clearly state flo https://bugs.python.org/issue39301 opened by ncoghlan #39302: Language reference does not clearly describe modern operand co https://bugs.python.org/issue39302 opened by ncoghlan #39305: Merge nntplib._NNTPBase and nntplib.NNTP https://bugs.python.org/issue39305 opened by corona10 #39306: Lib/configparser.py - RawConfigParser.set does not pass non-tr https://bugs.python.org/issue39306 opened by Strijker #39309: Please delete my account https://bugs.python.org/issue39309 opened by sfjwlejfawnf #39314: Autofill the closing paraenthesis during auto-completion for f https://bugs.python.org/issue39314 opened by opensource-assist #39316: settrace skips lines when chaining methods without arguments https://bugs.python.org/issue39316 opened by alexmojaki #39318: NamedTemporaryFile could cause double-close on an fd if _Tempo https://bugs.python.org/issue39318 opened by nneonneo #39320: Handle unpacking of */** arguments and rvalues in the compiler https://bugs.python.org/issue39320 opened by Mark.Shannon #39323: Add test for imghdr cli https://bugs.python.org/issue39323 opened by xtreak #39324: Add mimetype for extension .md (markdown) https://bugs.python.org/issue39324 opened by Ryan Batchelder #39325: Original window focus when opening IDLE by double clicking Pyt https://bugs.python.org/issue39325 opened by IrvKalb #39326: Python-3.8.1 "test_importlib" failed https://bugs.python.org/issue39326 opened by Divyansh_tiwari #39327: shutil.rmtree using vagrant synched folder fails https://bugs.python.org/issue39327 opened by PeterFS #39328: Allow filename mismatch in local and central directories in zi https://bugs.python.org/issue39328 opened by cheryl.sabella #39331: 2to3 mishandles indented imports https://bugs.python.org/issue39331 opened by galun.guy #39334: python specific index directives in our doc has been deprecate https://bugs.python.org/issue39334 opened by mdk #39336: Immutable module type can't be used as package in custom loade https://bugs.python.org/issue39336 opened by dino.viehland #39337: codecs.lookup() ignores non-ASCII characters, whereas encoding https://bugs.python.org/issue39337 opened by vstinner #39339: ProcessPoolExecutor() Exception in thread QueueManagerThread https://bugs.python.org/issue39339 opened by pythonpython #39340: shutil.rmtree and write protected files https://bugs.python.org/issue39340 opened by PeterFS #39341: [security] zipfile: ZIP Bomb vulnerability, don't check announ https://bugs.python.org/issue39341 opened by vstinner #39342: Expose X509_V_FLAG_ALLOW_PROXY_CERTS in ssl https://bugs.python.org/issue39342 opened by chrisburr #39343: Travis CI: documentation job fails in library/nntplib.rst with https://bugs.python.org/issue39343 opened by vstinner #39344: Getting error while importing ssl " import _ssl # if we can't https://bugs.python.org/issue39344 opened by Ajaya #39345: Py_Initialize Hangs on Windows 10 https://bugs.python.org/issue39345 opened by dhamilton #39346: gzip module only supports half of possible read/write scenario https://bugs.python.org/issue39346 opened by ddevault #39347: Use of argument clinic like parsing and `METH_FASTCALL` suppor https://bugs.python.org/issue39347 opened by seberg #39349: Add "cancel_futures" parameter to concurrent.futures.Executor. https://bugs.python.org/issue39349 opened by aeros #39352: Remove the formatter module, deprecated since Python 3.4 https://bugs.python.org/issue39352 opened by vstinner #39353: Deprecate the binhex module, binhex4 and hexbin4 standards https://bugs.python.org/issue39353 opened by vstinner #39355: The Python library will not compile with a C++2020 compiler be https://bugs.python.org/issue39355 opened by aCuria #39358: test_code.CoExtra leads to double-free when ce_size >1 https://bugs.python.org/issue39358 opened by slebedev #39359: zipfile: add missing "pwd: expected bytes, got str" exception https://bugs.python.org/issue39359 opened by dhillier #39360: python3.8 regression - ThreadPool join via __del__ hangs forev https://bugs.python.org/issue39360 opened by gaborbernat #39361: [C API] Document PyTypeObject.tp_print removal in What's New I https://bugs.python.org/issue39361 opened by vstinner #39362: add option to make chunksize adaptive for multiprocessing.pool https://bugs.python.org/issue39362 opened by fgregg #39363: zipfile with multiprocessing: zipfile.BadZipFile https://bugs.python.org/issue39363 opened by maxime-lemonnier #39364: Automatically tabulate module contents in the docs https://bugs.python.org/issue39364 opened by alexchandel #39365: Support (SEEK_END/SEEK_CUR) relative seeking in StringIO https://bugs.python.org/issue39365 opened by random832 #39366: Remove deprecated nntplib method https://bugs.python.org/issue39366 opened by corona10 #39369: Doc: Update mmap readline method documentation https://bugs.python.org/issue39369 opened by wellpardim #39370: askopenfilename is missing from the Tkinter filedialog library https://bugs.python.org/issue39370 opened by rdnielsen #39371: http.client.HTTPResponse raises IncompleteRead on chunked enco https://bugs.python.org/issue39371 opened by Arden #39372: The header files in Include/ have many declarations with no de https://bugs.python.org/issue39372 opened by pablogsal Most recent 15 issues with no replies (15) ========================================== #39372: The header files in Include/ have many declarations with no de https://bugs.python.org/issue39372 #39370: askopenfilename is missing from the Tkinter filedialog library https://bugs.python.org/issue39370 #39369: Doc: Update mmap readline method documentation https://bugs.python.org/issue39369 #39365: Support (SEEK_END/SEEK_CUR) relative seeking in StringIO https://bugs.python.org/issue39365 #39364: Automatically tabulate module contents in the docs https://bugs.python.org/issue39364 #39362: add option to make chunksize adaptive for multiprocessing.pool https://bugs.python.org/issue39362 #39361: [C API] Document PyTypeObject.tp_print removal in What's New I https://bugs.python.org/issue39361 #39359: zipfile: add missing "pwd: expected bytes, got str" exception https://bugs.python.org/issue39359 #39358: test_code.CoExtra leads to double-free when ce_size >1 https://bugs.python.org/issue39358 #39347: Use of argument clinic like parsing and `METH_FASTCALL` suppor https://bugs.python.org/issue39347 #39346: gzip module only supports half of possible read/write scenario https://bugs.python.org/issue39346 #39345: Py_Initialize Hangs on Windows 10 https://bugs.python.org/issue39345 #39342: Expose X509_V_FLAG_ALLOW_PROXY_CERTS in ssl https://bugs.python.org/issue39342 #39339: ProcessPoolExecutor() Exception in thread QueueManagerThread https://bugs.python.org/issue39339 #39337: codecs.lookup() ignores non-ASCII characters, whereas encoding https://bugs.python.org/issue39337 Most recent 15 issues waiting for review (15) ============================================= #39372: The header files in Include/ have many declarations with no de https://bugs.python.org/issue39372 #39366: Remove deprecated nntplib method https://bugs.python.org/issue39366 #39359: zipfile: add missing "pwd: expected bytes, got str" exception https://bugs.python.org/issue39359 #39353: Deprecate the binhex module, binhex4 and hexbin4 standards https://bugs.python.org/issue39353 #39352: Remove the formatter module, deprecated since Python 3.4 https://bugs.python.org/issue39352 #39343: Travis CI: documentation job fails in library/nntplib.rst with https://bugs.python.org/issue39343 #39342: Expose X509_V_FLAG_ALLOW_PROXY_CERTS in ssl https://bugs.python.org/issue39342 #39336: Immutable module type can't be used as package in custom loade https://bugs.python.org/issue39336 #39334: python specific index directives in our doc has been deprecate https://bugs.python.org/issue39334 #39324: Add mimetype for extension .md (markdown) https://bugs.python.org/issue39324 #39323: Add test for imghdr cli https://bugs.python.org/issue39323 #39320: Handle unpacking of */** arguments and rvalues in the compiler https://bugs.python.org/issue39320 #39318: NamedTemporaryFile could cause double-close on an fd if _Tempo https://bugs.python.org/issue39318 #39299: Improve test coverage for mimetypes module https://bugs.python.org/issue39299 #39298: add BLAKE3 to hashlib https://bugs.python.org/issue39298 Top 10 most discussed issues (10) ================================= #39318: NamedTemporaryFile could cause double-close on an fd if _Tempo https://bugs.python.org/issue39318 10 msgs #39295: usage of bitfields in ctypes structures changed between 3.7.5 https://bugs.python.org/issue39295 8 msgs #17005: Add a topological sort algorithm https://bugs.python.org/issue17005 7 msgs #37751: In codecs, function 'normalizestring' should convert both spa https://bugs.python.org/issue37751 6 msgs #39343: Travis CI: documentation job fails in library/nntplib.rst with https://bugs.python.org/issue39343 6 msgs #39353: Deprecate the binhex module, binhex4 and hexbin4 standards https://bugs.python.org/issue39353 6 msgs #29282: Fused multiply-add: proposal to add math.fma() https://bugs.python.org/issue29282 5 msgs #38076: Make struct module PEP-384 compatible https://bugs.python.org/issue38076 5 msgs #39207: concurrent.futures.ProcessPoolExecutor does not properly reap https://bugs.python.org/issue39207 5 msgs #39296: Windows register keys https://bugs.python.org/issue39296 5 msgs Issues closed (65) ================== #17718: boolop constant checking for if/while https://bugs.python.org/issue17718 closed by benjamin.peterson #19557: ast - docs for every node type are missing https://bugs.python.org/issue19557 closed by cheryl.sabella #27195: Crash when RawIOBase.write(b) evaluates b.format https://bugs.python.org/issue27195 closed by cheryl.sabella #31031: Unify duplicate bits_in_digit and bit_length https://bugs.python.org/issue31031 closed by vstinner #32021: Brotli encoding is not recognized by mimetypes https://bugs.python.org/issue32021 closed by maxking #32524: Python 2.7 leaks a packages __init__.py module object on Synta https://bugs.python.org/issue32524 closed by cheryl.sabella #32615: Inconsistent behavior if globals is a dict subclass https://bugs.python.org/issue32615 closed by vstinner #33471: string format with 'n' failling with french locales https://bugs.python.org/issue33471 closed by cheryl.sabella #34922: hashlib segmentation fault https://bugs.python.org/issue34922 closed by ned.deily #36556: Trashcan causing duplicated __del__ calls https://bugs.python.org/issue36556 closed by cheryl.sabella #36710: Pass _PyRuntimeState as an argument rather than using the _PyR https://bugs.python.org/issue36710 closed by vstinner #36746: Create test for fcntl.lockf() https://bugs.python.org/issue36746 closed by vstinner #37958: Adding get_profile_dict to pstats https://bugs.python.org/issue37958 closed by gregory.p.smith #38293: Deepcopying property objects results in unexpected TypeError https://bugs.python.org/issue38293 closed by cheryl.sabella #38330: httplib specifies content-length when transfer-encoding presen https://bugs.python.org/issue38330 closed by cheryl.sabella #38356: test_asyncio: SubprocessThreadedWatcherTests leaks threads https://bugs.python.org/issue38356 closed by asvetlov #38361: syslog: Default "ident" in syslog.openlog() shouldn't contain https://bugs.python.org/issue38361 closed by brandtbucher #38630: subprocess.Popen.send_signal() should poll the process https://bugs.python.org/issue38630 closed by vstinner #38901: [venv] Add a CLI flag to venv to use the pwd basename as the p https://bugs.python.org/issue38901 closed by vinay.sajip #38903: #if 0 block on parsetok.c https://bugs.python.org/issue38903 closed by pablogsal #39033: zipimport raises NameError: name '_boostrap_external' is not d https://bugs.python.org/issue39033 closed by petr.viktorin #39048: Look up __aenter__ before __aexit__ in the async with statemen https://bugs.python.org/issue39048 closed by ncoghlan #39156: Break up COMPARE_OP into logically distinct operations. https://bugs.python.org/issue39156 closed by Mark.Shannon #39160: ./configure --help has inconsistencies in style https://bugs.python.org/issue39160 closed by ned.deily #39164: Add private _PyErr_GetExcInfo() function which accepts a tstat https://bugs.python.org/issue39164 closed by vstinner #39179: pandas tz_convert() seems to report incorrect date conversion https://bugs.python.org/issue39179 closed by cheryl.sabella #39200: Fix inaccurate TypeError messages when calling with insufficie https://bugs.python.org/issue39200 closed by vstinner #39233: glossary entry for parameter out-of-date for positional-only p https://bugs.python.org/issue39233 closed by mark.dickinson #39249: difflib SequenceMatcher 200 char length limitation for ratio c https://bugs.python.org/issue39249 closed by terry.reedy #39259: poplib.POP3/POP3_SSL should reject timeout = 0 (non-blocking m https://bugs.python.org/issue39259 closed by vstinner #39275: Traceback off by one line when https://bugs.python.org/issue39275 closed by facundobatista #39288: Add math.nextafter(a, b) https://bugs.python.org/issue39288 closed by vstinner #39289: crypt.crypt fail with OSError "[Errno 22] Invalid argument" on https://bugs.python.org/issue39289 closed by vinay.sajip #39291: "pathlib.Path.link_to()" and "pathlib.Path.symlink_to()" have https://bugs.python.org/issue39291 closed by pitrou #39292: syslog constants behind rfc https://bugs.python.org/issue39292 closed by vinay.sajip #39297: Synchronize importlib.metadata with importlib_metadata 1.4 https://bugs.python.org/issue39297 closed by jaraco #39300: dataclasses non-default argument follows default argument https://bugs.python.org/issue39300 closed by eric.smith #39303: Refactor cmd module https://bugs.python.org/issue39303 closed by cheryl.sabella #39304: Don't accept a negative number for the count argument in str.r https://bugs.python.org/issue39304 closed by rhettinger #39307: Memory leak in parsetok https://bugs.python.org/issue39307 closed by pablogsal #39308: Literal[True] prints as Literal[1] in some cases https://bugs.python.org/issue39308 closed by gvanrossum #39310: Add math.ulp(x): unit in the last place https://bugs.python.org/issue39310 closed by vstinner #39311: difflib pathlike support for {unified,context}_diff() {from,to https://bugs.python.org/issue39311 closed by rhettinger #39312: Expose placeholder reparse points in Windows https://bugs.python.org/issue39312 closed by steve.dower #39313: lib2to3 provide a way to use exec as a function in Refactoring https://bugs.python.org/issue39313 closed by pablogsal #39315: Lists of objects containing lists https://bugs.python.org/issue39315 closed by hmathers #39317: This new feature or bug about operator "- -"? https://bugs.python.org/issue39317 closed by rhettinger #39319: ntpath module must not be available on POSIX platforms https://bugs.python.org/issue39319 closed by eryksun #39321: AMD64 FreeBSD Non-Debug 3.x: main regrtest process killed by S https://bugs.python.org/issue39321 closed by vstinner #39322: Add gc.is_finalized to check if an object has been finalised b https://bugs.python.org/issue39322 closed by pablogsal #39329: smtplib.LMTP needs timeout parameter https://bugs.python.org/issue39329 closed by vstinner #39330: Way to build without IDLE https://bugs.python.org/issue39330 closed by zach.ware #39332: Python 3.6 compiler protections from Ubuntu distros https://bugs.python.org/issue39332 closed by benjamin.peterson #39333: argparse should offer an alternative to SystemExit in case a p https://bugs.python.org/issue39333 closed by geophile #39335: round Decimal edge case https://bugs.python.org/issue39335 closed by benjamin.peterson #39338: Data lost randomly from dictionary after creating the dictiona https://bugs.python.org/issue39338 closed by zach.ware #39348: wrong rst syntax in socket.rst https://bugs.python.org/issue39348 closed by Mariatta #39350: Remove deprecated fractions.gcd() https://bugs.python.org/issue39350 closed by vstinner #39351: Remove base64.encodestring() and base64.decodestring() aliases https://bugs.python.org/issue39351 closed by vstinner #39354: collections.UserString format and format_map return a string https://bugs.python.org/issue39354 closed by rhettinger #39356: zipfile suprising "except DeprecationWarning:" block https://bugs.python.org/issue39356 closed by vstinner #39357: bz2: Remove deprecated buffering parameter of bz2.BZ2File https://bugs.python.org/issue39357 closed by vstinner #39367: readline module core dumps Python 3.8.1 when calling exit() https://bugs.python.org/issue39367 closed by vstinner #39368: A matrix (list of lists) behaves differently, depending how it https://bugs.python.org/issue39368 closed by ezio.melotti #1531415: parsetok.c emits warnings by writing to stderr https://bugs.python.org/issue1531415 closed by pablogsal

1 0

Questions about CPython's behavior on addition operator -- binary_op1 (abstract.c) and nb_add (typeobject.c)
by Raphaël Monat Jan. 17, 2020

Jan. 17, 2020

Hi all, I'm looking at CPython's behavior when an addition is called. From what I understand, binary_op1 <https://github.com/python/cpython/blob/3.8/Objects/abstract.c#L786> is eventually called, and it calls either slotv or slotw, which seems to be the binaryfunc defined as nb_add in the field tp_as_number of respectively v / w. I have a few questions: 1) In the default case, tp_as_number->nb_add is defined by the function slot_nb_add <https://github.com/python/cpython/blob/3.8/Objects/typeobject.c#L6312> itself stemming from the macro expansion SLOT1BINFULL <https://github.com/python/cpython/blob/3.8/Objects/typeobject.c#L6140> defined in typeobject.c. Both binary_op1(v, w) and slot_nb_add(v, w) appear to perform similar checks (if their second argument is a subtype of the first, etc), to decide if v's add or w's reverse add must be called and in which order. I find this repetition weird, and I guess I'm missing something... Any ideas? 2) From the SLOT1BINFULL macro, both __add__ and __radd__ are defined by the slot_nb_add function (with some argument swapping done by wrap_binaryfunc_l / wrap_binaryfunc_r). If I want to define a different behavior for the reverse operator during a definition with a PyTypeObject, I guess I should add an "__radd__" method? 3) If I create a user-defined class A, having different methods __add__ and __radd__, these methods are added in A's dictionary. From what I understand, the function update_one_slot <https://github.com/python/cpython/blob/3.8/Objects/typeobject.c#L7209> is then called to change A's tp_as_number->nb_add to point to the methods defined by __add__ and __radd__? From the code documentation, I think that "a wrapper for the special methods is installed". Where exactly is this wrapper applied, and how does it know when to dispatch to __add__ or __radd__? Kind regards, Raphaël

2 1

Windows - rebuilding grammar files
by Abdur-Rahmaan Janhangeer Jan. 14, 2020

Jan. 14, 2020

Greetings list, On windows i want to play with the grammar file, but according to this article <https://realpython.com/cpython-source-code-guide/>: > For Windows, there is no officially supported way of running pgen. However, you can clone my fork <https://github.com/tonybaloney/cpython/tree/pcbuildregen> and run build.bat --regen from within the PCBuild directory. But i don't want to work from his fork. Is there an official way of regenerating the files? I just want to have my own keywords. As far as i've understood, rebuilding won't do it. Yours, Abdur-Rahmaan Janhangeer Mauritius

2 2

Why doesn't venv also install python3*-config?
by Musbur Jan. 13, 2020

Jan. 13, 2020

Hello, I'm experimenting with package development on different versions of Python in different virtualenvs. After running "make" I don't do "make install", but rather I set up virtualenvs by running /path/to/source/python -m venv env_dir. This works for as long as I don't need to compile extensions. Once I do that I'm running into trouble because there is no python3-config binary in the venv, so it uses the "system" python3-config which of course returns results for the /usr(/local)/.... tree. This seems to go against the idea of an encapsulated and self-contained environment. And the venv created straight from the "not-installed" source tree works so well that having a venv/bin/python3-config whose output points into the source tree seems a logical step. Is this an omission or is there a rationale for not doing it? Of course I can "properly" install different Python versions by using different "configure --prefix" directories. But I like the venv so much in general that this rubs me the wrong way. Thanks!

10 15

Should we pass tstate in the VECTORCALL calling convention before making it public?
by Victor Stinner Jan. 11, 2020

Jan. 11, 2020

Hi, I started to modify Python internals to pass explicitly the Python thread state ("tstate") to internal C a functions: https://vstinner.github.io/cpython-pass-tstate.html Until subinterpreters will be fully implemented, it's unclear if getting the current Python thread state using _PyThreadState_GET() macro (which uses an atomic read) will remain efficient. For example, right now, the "GIL state" API doesn't support subinterpreters: fixing this may require to add a lock somewhere which may make _PyThreadState_GET() less efficient. Sorry, I don't have numbers, since I didn't experiment to implement these changes yet: I was blocked by other issues. We can only guess at this point. To me, it sounds like a good practice to pass the current Python thread state to internal C functions. It seems like most core developers agreed with that in my previous python-dev thread "Pass the Python thread state to internal C functions": https://mail.python.org/archives/list/python-dev@python.org/thread/PQBGECVG… The question is now if we should "propagate" tstate to function calls in the latest VECTORCALL calling convention (which is currently private). Petr Viktorin plans to make VECTORCALL APIs public in Python 3.9, as planned in the PEP 590: https://bugs.python.org/issue39245 I added explicitly Stefan Behnel in copy, since Cython should be directly impacted by such change. Cython is the kind of project which may benefit of having tstate accessible directly. I started to move more and more things from "globals" to "per interpreter". For example, the garbage collector is now "per interpreter" (lives in PyThreadState). Small integer singletons are now also "per singleton": int("1") are now different objects in each interpreter, whereas they were shared previously. Later, even "None" singleton (and all other singletons) should be made "per interpreter". Getting a "per interpreter" object requires to state from the Python thread state: call _PyThreadState_GET(). Avoiding _PyThreadState_GET() calls reduces any risk of making Python slower with incoming subinterpreter changes. For the long term, the goal is that each subinterpreter has its own isolated world: no Python object should be shared, no state should be shared. The intent is to avoid any need of locking, to maximize performance when running interpreters in parallel. Obviously, each interpreter will have its own private GIL ;-) Py_INCREF() and Py_DECREF() would require slow atomic operations if Python objects are shared. If objects are not shared between interpreters, Py_INCREF() and Py_DECREF() can remain as fast as they are today. Any additional locking may kill performance. Victor -- Night gathers, and now my watch begins. It shall not end until my death.

6 11

Object deallocation during the finalization of Python program
by Pau Freixes Jan. 11, 2020

Jan. 11, 2020

Hi, Recently I've been facing a really weird bug where a Python program was randomly segfaulting during the finalization, the program was using some C extensions via Cython. Debugging the issue I realized that during the deallocation of one of the Python objects the deallocation function was trying to release a pointer that was surprisingly assigned to NULL. The pointer was at the same time held by another Python object that was an attribute of the Python object that had the deallocation function, something like this: class Foo: my_type * value class Bar def __cinit__: self._foo = Foo() self._foo->value = initialize() def __dealloc__: destroy(self._foo->value) Seems that randomly the instance of the object Foo held by the Bar object was deallocated by the CPython interpreter before the Foo deallocation, so after being deallocated - and zeroing the memory space of the instance of Foo - the execution of the `destroy(self._foo->value)` was in fact given as a parameter a NULL address and raising a segfault. It was a surprise for me, If I'm not missing something the deallocation of the Foo instance happened even though there was still an active reference held by the Bar object. As a kind of double-checking I changed the program for making an explicit `gc.collect()` before the last line of the Python program. As a result, I couldn't reproduce the segfault, which theoretically would mean that objects were deallocated "in order". So my question would be, could CPython deallocate the objects during the finalization step without considering the dependencies between objects? If this is not the right list to make this kind of questions, just let me know what would be the best place for making this kind of questions Thanks in advance, -- --pau

3 3