Mailman 3 February 2019 - Python-Dev

Asking for reversion
by Antoine Pitrou 25 Feb '19

25 Feb '19

Hello, I'd like to ask for the reversion of the changes done in https://github.com/python/cpython/pull/11664 The reason is simple: the PR isn't complete, it lacks docs and tests. It also didn't pass any review (this was pointed by Ronald), even though it adds 1300 lines of code. No programmer is perfect, so it's statistically likely that the PR is defective. With git, forks and branches, we definitely don't need to commit unfinished PRs to the main repo. It's perfectly fine to maintain some non-trivial piece of work in a separate fork. People do it on a regular basis (for example I have currently two such long-lived branches: one for PEP 556 and one for PEP 574). Also, this is not the first time this happened. Another multiprocessing PR was merged some years ago without any docs or tests: https://bugs.python.org/issue28053 Today that work /still/ lacks docs or tests, and there is a suspicion that it doesn't work as intended (see issue comments). It's probably too late to revert it, but it's definitely a slippery slope. Regards Antoine.

13 30

"Good first issues" on the bug tracker
by Cheryl Sabella 25 Feb '19

25 Feb '19

Hello! Due to the upcoming sprints at PyCon, there will be issues marked in the bug tracker as 'Good first issues'. Please do *not* submit pull requests for these as it is difficult to find suitable issues. Instead of working on these issues, I'd like to propose a much more difficult challenge to current contributors looking for something to work on. Please find issues which can be tackled at the sprint! They are out there, but the trick is finding them. :-) Here's the link to the mentored sprint page to understand the audience: https://us.pycon.org/2019/hatchery/mentoredsprints/ I don't think this needs to be limited to documentation changes, but those are the obvious choice (even the devguide suggests it). The goal is really workflow, meaning the new contributor doesn't necessarily have to solve the problem themselves - if an issue says: x function in y module needs to have z applied, then there's a lot of hurdles there for a newcomer which may not involve solving the problem. If you find anything you think is suitable, please add a comment with 'good first issue' and nosy me or Mariatta on it. If you're unsure, then nosy us anyway. It would be awesome to have too many issues to choose from than not enough. If an issue isn't worked on at the sprint, then it would still be available to new contributors afterwards, so the exercise of flagging the issues wouldn't be wasted effort. Thanks! Cheryl

9 14

Add minimal information with a new issue?
by Stephane Wirtel 22 Feb '19

22 Feb '19

Hi, What do you think if we suggest a "template" for the new bugs? For example: * Python version (exact version) * Operating System * on Linux -> Distribution (python can have some patches) * Add a script for the reproduction of the bug * Eventually, try with the docker images (on i386,x86_64, ...) * etc... We can lost a lot of time just trying to find the right information for the reproduction of the issue. Cheers, Stéphane -- Stéphane Wirtel - https://wirtel.be - @matrixise

6 7

Making PyInterpreterState an opaque type
by Eric Snow 21 Feb '19

21 Feb '19

Hi all, I've been working on the runtime lately, particularly focused on my multi-core Python project. One thing that would help simplify changes in this area is if PyInterpreterState were defined in Include/internal. This would mean the type would be opaque unless Py_BUILD_CORE were defined. The docs [1] already say none of the struct's fields are public. Furthermore, Victor already moved it into Include/cpython (i.e. not in the stable ABI) in November. Overall, the benefit of making internal types like this opaque is realized in our ability to change the internal details without further breaking C-API users. Realistically, there may be extension modules out there that use PyInterpreterState fields directly. They would break. I expect there to be few such modules and fixing them would not involve great effort. We'd have to add any missing accessor functions to the public C-API, which I see as a good thing. I have an issue [2] open for the change and a PR open. My PR already adds an entry to the porting section of the 3.8 What's New doc about dealing with PyInterpreterState. Anyway, I just wanted to see if there are any objections to making PyInterpreterState an opaque type outside of core use. -eric p.s. I'd like to do the same with PyThreadState, but that's a bit trickier [3] and not one of my immediate needs. :) [1] https://docs.python.org/3/c-api/init.html#c.PyInterpreterState [2] https://bugs.python.org/issue35886 [3] https://bugs.python.org/issue35949

15 40

Making PyInterpreterState an opaque type
by Stefan Krah 21 Feb '19

21 Feb '19

Victor Stinner wrote: > Premature optimization is the root of all evil. Most C extensions use > premature optimization which are causing us a lot of troubles nowadays > when we want to make the C API evolve and cause issues to PyPy which > has issues to reimplement the C API on top of their different object > model with a different GC. I'm intimately familiar with several C extensions in the math/science space and this is not the impression I get at all. Most people are happy if they get things to work, because the actual problems are much harder than focusing on private vs. public API. As for PyPy, if I understood correctly, Armin Rigo was skeptical of the proposed plan and favored publishing an API as a third party package. Stefan Krah

1 0

Question - Bug Triage for 3.4 & 3.5
by Stephane Wirtel 21 Feb '19

21 Feb '19

Hi, As you know, Python 3.4 and 3.5 are in security mode and the EOL for these versions are respectively 2019-03-16 and 2020-09-13. Number of issues 3.4: 1530 issues 3.5: 1901 issues But some issues are not related to the security. Could we update these issues (non-security) to 3.6/3.7 & 3.8? Cheers, Stéphane -- Stéphane Wirtel - https://wirtel.be - @matrixise

5 7

Re: [Python-Dev] Is distutils.util.get_platform() the "current" or the "target" platform
by Xavier de Gaye 20 Feb '19

20 Feb '19

> [Any reason for dropping python-dev?] Sorry. just clicked the wrong button. > And the answer is a resounding "yes, it returns the target platform"? It seems you're saying this, but the wording of your email sounds just enough of a question that I'm not sure whether you are definitively answering it or not. The answer is yes indeed, it returns the target platform. This is a listing of the non-obvious steps that lead to this conclusion. Xavier On 2/19/19 8:45 PM, Steve Dower wrote: > [Any reason for dropping python-dev?] > > On 19Feb2019 1139, Xavier de Gaye wrote: >> Is distutils.util.get_platform() the "current" or the "target" platform ? > > I *think* you're answering this below, yes? > >> * When cross-compiling on posix platforms using autoconf, configure is >> run with the '--host=host-type' [1] command line option to specify >> the target platform. >> >> * The AC_CANONICAL_HOST macro is used by configure.ac to get the >> canonical variables `host' and `host_cpu' [2]. >> Those variables are used to compute _PYTHON_HOST_PLATFORM. >> >> * The Makefile generated by configure runs setup.py, >> generate-posix-vars, etc... on the build platform using >> PYTHON_FOR_BUILD, a native python interpreter that is set >> to run with the _PYTHON_HOST_PLATFORM environment variable. >> >> * get_platform() in setup.py and in Lib/distutils/util.py returns the >> value of _PYTHON_HOST_PLATFORM when cross-compiling. >> >> So the process of cross-compilation on posix platforms has >> get_platform() return the target ('host' in autoconf terminology) platform. >> >> [1] https://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/autoconf-2.69/ht… >> [2] https://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/autoconf-2.69/ht… > > And the answer is a resounding "yes, it returns the target platform"? It seems you're saying this, but the wording of your email sounds just enough of a question that I'm not sure whether you are definitively answering it or not. > > Cheers, > Steve

2 1

buildbottest on Android emulator with docker
by Xavier de Gaye 20 Feb '19

20 Feb '19

The following command runs the buildbottest on an Android emulator with docker (it will use a little bit more than 11 GB): $ docker run -it --privileged xdegaye/abifa:r14b-24-x86_64-master This command does: * pull an image from the Docker hub (only the first time that the command is run, note that this a 2 GB download !) and start a container * pull the latest changes from the GitHub cpython repository and cross-compile python * start an Android emulator and install python on it * run the buildbottest target of cpython Makefile The image is built from a Dockerfile [2]. This same image can also be used with the 'bash' command line argument to enter bash in the container and run python interactively on the emulator [1]. If the 'docker run' command also sets a bind mount to a local cpython repository, then it is possible to develop/debug/fix python on the emulator running in this container using one's own clone of cpython. [1] documentation at https://xdegaye.gitlab.io/abifa/docker.html [2] Dockerfile at https://gitlab.com/xdegaye/abifa/blob/master/docker/Dockerfile.r14b-24-x86_… Xavier

3 5

new binary wheels PEP idea
by Alexander Revin 19 Feb '19

19 Feb '19

Hi all, I have an idea regarding Python binary wheels on non-glibc platforms, and it seems that initially I've posted it to the wrong list ([1]) Long story short, the proposal is to use platform tuples (like compiler ones) for wheel names, which will allow much broader platform support, for example: package-1.0-cp36-cp36m-amd64_linux_gnu.whl package-1.0-cp36-cp36m-amd64_linux_musl.whl So eventually only {platform tag} part will be modified. Glibc/musl detection is quite trivial and eventually will be based on existing one in PEP 513 [2]. Let me know what you think. Best regards, Alex [1] https://mail.python.org/pipermail/python-list/2019-February/739524.html [2] https://www.python.org/dev/peps/pep-0513/#id49

3 3

Adding test.support.safe_rmpath()
by Giampaolo Rodola' 19 Feb '19

19 Feb '19

Hello, after discovering os.makedirs() has no unit-tests ( https://bugs.python.org/issue35982) I was thinking about working on a PR to increase the test coverage of fs-related os.* functions. In order to do so I think it would be useful to add a convenience function to "just delete something if it exists", regardless if it's a file, directory, directory tree, etc., and include it into test.support module. Basically it would be very similar to "rm -rf". I use something like this into psutil: https://github.com/giampaolo/psutil/blob/3ea94c1b8589891a8d1a5781f0445cb508… I find this paradigm especially useful when testing functions involving two files ("src" and "dst"). E.g. in case of os.renames() unit-tests I would write something like this: class RenamesTest(unittest.TestCase): srcname = support.TESTFN dstname = support.TESTFN + '2' def setUp(self): test.support.rmpath(self.srcname) test.support.rmpath(self.dstname) tearDown = setUp def test_rename_file(self): ... def test_rename_dir(self): ... def test_rename_failure(self): # both src and dst will not exist ... With the current utilities included in test.support the setUp function above would be written as such: def setUp(self): for path in (self.srcname, self.dstname): if os.path.isdir(path): test.support.rmtree(path) elif os.path.exists(path): test.support.unlink(path) Extra: one may argue whether this utility could be included into shutil module instead. The extra advantage of test.support.rmtree and test.support.unlink though, is that on Windows they use a timeout, catching "file is currently in use" exceptions for some time before giving up. That IMO would probably make this utility function not palatable for inclusion into shutil module, so test.support would probably be a better landing place. Thoughts? -- Giampaolo - http://grodola.blogspot.com

11 17