Mailman 3 - Distutils-SIG

PEP 440 on fully autonomous forks versioning
by Sofia Nunes 01 Mar '19

01 Mar '19

Hello, Regarding fork versioning, I found this thread: https://mail.python.org/archives/list/distutils-sig@python.org/thread/XAAN4… Now, this is relating to forks that incorporate small changes, like bug fixes, and that are meant to be integrated upstream. When the fork implies greater changes, and is meant to be independent from the original package and have a different name, how should versioning work? From the following discussion, I get that probably the best method is to start as a fully independent distribution, for instance from version 1.0. https://bitbucket.org/pypa/pypi-metadata-formats/issues/1/add-local-numberi… So, let’s assume I fork a package which is on version 2.1. and change its name. Would my distribution version be 1.0 or 2.2/3.0? Or none of the options? Thanks, Sofia Nunes

2 1

PEP-582 concerns
by Alex Walters 01 Mar '19

01 Mar '19

I have 2 main concerns about PEP 582 that might just be me misunderstanding the pep. My first concern is the use of CWD, and prepending ./_pypackages_ for scripts. For example, if you were in a directory with a _pypackages_ subdirectory, and had installed the module "super.important.module". My understanding is that any scripts you run will have "super.important.module" available to it before the system site-packages directory. Say you also run "/usr/bin/an_apt-ly_named_python_script" that uses "super.important.module" (and there is no _pypackages_ subdirectory in /usr/bin). You would be shadowing "super.important.module". In this case, this adds no more version isolation than "pip install --user", and adds to the confoundment factor for a new user. If this is a misunderstanding of the pep (which it very well might be!), then ignore that concern. If it's not a misunderstanding, I think that should be emphasized in the docs, and perhaps the pep. My second concern is a little more... political. This pep does not attempt to cover all the use-cases of virtualenvs - which is understandable. However, this also means that we have to teach new users *both* right away in order to get them up and running, and teach them the complexities of both, and when to use one over the other. Instead of making it easier for the new user, this pep makes it harder. This also couldn't have come at a worse time with the growing use of pipenv which provides a fully third way of thinking about application dependencies (yes, pipenv uses virtualenvs under the hood, but it is a functionally different theory of operation from a user standpoint compared to traditional pip/virtualenv or this pep). Is it really a good idea to do this pep at this time? In a vacuum, I like this pep. Aside from the (possible) issue of unexpected shadowing, it's clean and straight forward. It's easy to teach. But it doesn't exist in a vacuum, and we have to teach the methods it is intended to simplify anyways, and it exists in competition with other solutions. I am not a professional teacher; I don't run python training courses. I do, however, volunteer quite a bit of time on the freenode channel. I get that the audience there is self-selecting to those who want to donate their time, and those who are having a problem (sometimes, those are the same people). This is the kind of thing that generates a lot of confusion and frustration to the new users I interact with there.

11 21

PEP 440 on fully autonomous forks versioning
by Sofia Nunes 28 Feb '19

28 Feb '19

Hello, Regarding fork versioning, I found this thread: https://mail.python.org/archives/list/distutils-sig@python.org/thread/XAAN4… Now, this is relating to forks that incorporate small changes, like bug fixes, and that are meant to be integrated upstream. When the fork implies greater changes, and is meant to be independent from the original package and have a different name, how should versioning work? From the following discussion, I get that probably the best method is to start as a fully independent distribution, for instance from version 1.0. https://bitbucket.org/pypa/pypi-metadata-formats/issues/1/add-local-numberi… So, let's assume I fork a package which is on version 2.1. and change its name. Would my distribution version be 1.0 or 2.2/3.0? Or none of the options? Thanks, Sofia Nunes

1 0

PEP idea regarding non-glibc systems
by Alexander Revin 27 Feb '19

27 Feb '19

Hi all, I have an idea regarding Python binary wheels on non-glibc platforms, and it seems that initially I've posted it to the wrong list ([1]) Long story short, the proposal is to use platform tuples (like compiler ones) for wheel names, which will allow much broader platform support, for example: package-1.0-cp36-cp36m-amd64_linux_gnu.whl package-1.0-cp36-cp36m-amd64_linux_musl.whl So eventually only {platform tag} part will be modified. Glibc/musl detection is quite trivial and eventually will be based on existing one in PEP 513 [2]. Let me know what you think. Best regards, Alex [1] https://mail.python.org/pipermail/python-list/2019-February/739524.html [2] https://www.python.org/dev/peps/pep-0513/#id49

2 8

content clashes
by Robin Becker 19 Feb '19

19 Feb '19

Is there a way for a package to recognize that its content clashes with that of another package? This can happen when a package becomes unmaintained and another differently named package takes over with perhaps clashing modules/__package__ paths. -- Robin Becker

2 1

Depending on external library
by Jeroen Demeyer 15 Feb '19

15 Feb '19

Hello, I have a Python project that depends on an external C library (which is unrelated to Python and NOT part of the OS). For an sdist, this is easy: my setup.py assumes that the library is pre-installed somewhere on the system where setuptools can find it. However, is there a standard solution for packaging such a project as wheel? Ideally, the project should "just work" when doing pip install on it, which means that the external library should somehow be bundled in the wheel. I know that for example numpy does that, but it also has a very complicated build system (containing a fork of distutils!). Does anybody know any pointers for this? Or if you think that this is a minefield which is not really supported, feel free to say so. Thanks, Jeroen.

3 3

Generalize sys.executable replacement from scripts to data_files (or more)
by Min RK 13 Feb '19

13 Feb '19

This was opened as an issue (https://github.com/pypa/pip/issues/4032) some years ago, but it was more recently recommended to open a thread here and that issue was closed due to inactivity, so here goes. pip currently rewrites sys.executable in the shebang line of scripts to be installed from wheels. This is necessary for portability, envs, etc. On the far opposite end of the spectrum, conda finds and rewrites the install prefix in any file anywhere in the package (including binaries). In IPython/Jupyter (specifically ipykernel), we have a file that we want to install that is a data_file in `share/jupyter/kernels/python3/kernel.json`. Like scripts, this file contains a reference to sys.executable, and like scripts, this should be rewritten at install time, since the wheel doesn't know the right value at build time. However, the sys.executable rewrite from pip is not configurable or accessible, so only scripts are allowed to have this modification. It would be useful to us to be able to opt-in to this modification for data_files as well. Our current choices are: 1. disable wheels for our package because we need to know sys.executable to create these files, and thus must rely on arbitrary code execution to get it right 2. use `python` and add special runtime-handling (this is what we do, and it can be wrong, e.g. when sys.executable at runtime is not actually the sys.executable used for installation, exactly the problem script shebang rewrite solves for scripts) 3. don't install the kernelspec with the package and tell users that proper installation is a two step process: `pip install ipykernel; ipython kernel install --sys-prefix`. We used to do this, and it caused lots of problems due to uninstall/upgrade causing the package and kernelspec to fall out of sync since part of the package is not managed by the package manager. A "works for us" version of this could be to allow specifying that a given data_file is a script (or entrypoint) and should be treated as one. This would allow us to place a script next to our json file in a way that it is found by our kernelspec mechanism. This isn't as general a solution, but it works for us without needing to generalize sys.executable (or sys.prefix) handling as much to locations in files other than the shebang, it only needs generalizing the installation location of scripts.

2 1

Building local versions with pip wheel
by Bert JW Regeer 13 Feb '19

13 Feb '19

Hey all, Patching 3rd party repositories comes up every so often at $WORK and one of the things we do is build a local version that is generally some released version + a couple of local patches. We follow the usual version scheme of using the public version identifier + local version label. The steps are as follows: python setup.py egg_info -b "+local<build number>" sdist bdist_wheel twine upload dist/*.whl We only really care about the wheel though, and for all of our other dependencies we basically just run: pip wheel <some pip valid URL> and upload it to our local pypi instance. This also allows us to directly build from Github repositories. With for example: pip wheel git+https://github.com/org/somerepo.git@ourpatchset#egg=somerepo <git+https://github.com/org/somerepo.git@ourpatchset#egg=somerepo> However we would like to add the build number so that when a new release is provided upstream we can easily update our local repository (or preferably we have upstreamed our patch in the meantime) and build a new release with our patches. Is there some way to influence the egg info for the build when using pip wheel? Some thing like: pip wheel git+https://github.com/org/somerepo.git@ourpatchset#egg=somerepo&egg_info= <git+https://github.com/org/somerepo.git@ourpatchset#egg=somerepo&egg_info=>"-b +local<build number>" Or is there a better method for dealing with this scenario? Thanks, Bert JW Regeer

2 4

API for SHA-256 fingerprints
by Eric Peterson 13 Feb '19

13 Feb '19

Hi all, When in the "Download files" section of a project on PyPI, next to each download there is a convenient "SHA256" link that will copy the SHA-256 fingerprint for that file to the clipboard. I am wondering if there is a programmatic way to access the SHA-256 for a file (besides just scraping the web page)? Ideally there would be some way to construct a URL based on the name of the file that, when called, would return the fingerprint. Thanks, Eric

10 22

pip + safety
by Julian Berman 12 Feb '19

12 Feb '19

Hi. I recently found myself installing a node.js package, and in the process noticed that (sometime recently?) it started automatically warning about known vulnerabilities during installation of package.jsons (see https://docs.npmjs.com/cli/audit). At work, we run safety (https://pypi.org/project/safety/) on all our projects (which has both free and paid versions). It's great. I know there's a ton of wonderful work happening at the minute to improve underlying scaffolding + specification to enable tools other than setuptools + pip to thrive, so maybe this is the wrong moment, but I figured I'd ask anyways :) -- what are opinions on running a similar thing during pip install? -J

4 7