Mailman 3 September 2013 - Distutils-SIG

Two fixes (at last)
by Greg Ward 29 Apr '18

29 Apr '18

Hi all -- at long last, I have fixed two problems that a couple people noticed a while ago: * I folded in Amos Latteier's NT patches almost verbatim -- just changed an `os.path.sep == "/"' to `os.name == "posix"' and added some comments bitching about the inadequacy of the current library installation model (I think this is Python's fault, but for now Distutils is slavishly aping the situation in Python 1.5.x) * I fixed the problem whereby running "setup.py install" without doing anything else caused a crash (because 'build' hadn't yet been run). Now, the 'install' command automatically runs 'build' before doing anything; to make this bearable, I added a 'have_run' dictionary to the Distribution class to keep track of which commands have been run. So now not only are command classes singletons, but their 'run' method can only be invoked once -- both restrictions enforced by Distribution. The code is checked into CVS, or you can download a snapshot at http://www.python.org/sigs/distutils-sig/distutils-19990607.tar.gz Hope someone (Amos?) can try the new version under NT. Any takers for Mac OS? BTW, all parties involved in the Great "Where Do We Install Stuff?" Debate should take a good, hard look at the 'set_final_options()' method of the Install class in distutils/install.py; this is where all the policy decisions about where to install files are made. Currently it apes the Python 1.5 situation as closely as I could figure it out. Obviously, this is subject to change -- I just don't know to *what* it will change! Greg -- Greg Ward - software developer gward(a)cnri.reston.va.us Corporation for National Research Initiatives 1895 Preston White Drive voice: +1-703-620-8990 Reston, Virginia, USA 20191-5434 fax: +1-703-620-0913

2 1

Package versioning
by Timothy Docker 29 Apr '18

29 Apr '18

Hi all, I've been aware that the distutils sig has been simmerring away, but until recently it has not been directly relevant to what I do. I like the look of the proposed api, but have one question. Will this support an installed system that has multiple versions of the same package installed simultaneously? If not, then this would seem to be a significant limitation, especially when dependencies between packages are considered. Assuming it does, then how will this be achieved? I am presently managing this with a messy arrangement of symlinks. A package is installed with its version number in it's name, and a separate directory is created for an application with links from the unversioned package name to the versioned one. Then I just set the pythonpath to this directory. A sample of what the directory looks like is shown below. I'm sure there is a better solution that this, and I'm not sure that this would work under windows anyway (does windows have symlinks?). So, has this SIG considered such versioning issues yet? Cheers, Tim -------------------------------------------------------------- Tim Docker timd(a)macquarie.com.au Quantative Applications Division Macquarie Bank -------------------------------------------------------------- qad16:qad $ ls -l lib/python/ total 110 drwxr-xr-x 2 mts mts 512 Nov 11 11:23 1.1 -r--r----- 1 root mts 45172 Sep 1 1998 cdrmodule_0_7_1.so drwxr-xr-x 2 mts mts 512 Sep 1 1998 chart_1_1 drwxr-xr-x 3 mts mts 512 Sep 1 1998 Fnorb_0_7_1 dr-xr-x--- 3 mts mts 512 Nov 11 11:21 Fnorb_0_8 drwxr-xr-x 3 mts mts 1536 Mar 3 12:45 mts_1_1 dr-xr-x--- 7 mts mts 512 Nov 11 11:22 OpenGL_1_5_1 dr-xr-x--- 2 mts mts 1024 Nov 11 11:23 PIL_0_3 drwxr-xr-x 3 mts mts 512 Sep 1 1998 Pmw_0_7 dr-xr-x--- 2 mts mts 512 Nov 11 11:21 v3d_1_1 qad16:qad $ ls -l lib/python/1.1 total 30 lrwxrwxrwx 1 root other 29 Apr 10 10:43 _glumodule.so -> ../OpenGL_1_5_1/_glumodule.so lrwxrwxrwx 1 root other 30 Apr 10 10:43 _glutmodule.so -> ../OpenGL_1_5_1/_glutmodule.so lrwxrwxrwx 1 root other 22 Apr 10 10:43 _imaging.so -> ../PIL_0_3/_imaging.so lrwxrwxrwx 1 root other 36 Apr 10 10:43 _opengl_nummodule.so -> ../OpenGL_1_5_1/_opengl_nummodule.so lrwxrwxrwx 1 root other 27 Apr 10 10:43 _tkinter.so -> ../OpenGL_1_5_1/_tkinter.so lrwxrwxrwx 1 mts mts 21 Apr 10 10:43 cdrmodule.so -> ../cdrmodule_0_7_1.so lrwxrwxrwx 1 mts mts 12 Apr 10 10:43 chart -> ../chart_1_1 lrwxrwxrwx 1 root other 12 Apr 10 10:43 Fnorb -> ../Fnorb_0_8 lrwxrwxrwx 1 mts mts 12 Apr 10 10:43 mts -> ../mts_1_1 lrwxrwxrwx 1 root other 15 Apr 10 10:43 OpenGL -> ../OpenGL_1_5_1 lrwxrwxrwx 1 root other 33 Apr 10 10:43 opengltrmodule.so -> ../OpenGL_1_5_1/opengltrmodule.so lrwxrwxrwx 1 root other 33 Apr 10 10:43 openglutil_num.so -> ../OpenGL_1_5_1/openglutil_num.so lrwxrwxrwx 1 root other 10 Apr 10 10:43 PIL -> ../PIL_0_3 lrwxrwxrwx 1 mts mts 10 Apr 10 10:43 Pmw -> ../Pmw_0_7 lrwxrwxrwx 1 root other 10 Apr 10 10:43 v3d -> ../v3d_1_1

3 2

[Distutils-sig] distutils charter and interscript
by John Skaller 29 Apr '18

29 Apr '18

3 2

error in windows 7 installation
by Manuel López 13 May '14

13 May '14

Hello Everybody: I am trying to install distribute-0.6.25 in a windows 7 machine. I have python 2.7.3 in 32 bits, although the machine is 64 bits. I installed the 32 bit python version because I want to install ipython and there is no 64 bit builds of the Windows installer for ipython. In any case, upon installing disuitils with python setup.py install I get the following error error: C:\manuel\prog_red\python\distribute-0.6.25\distribute-0.6.25\setuptools\cli-32.exe: No such file or directory Any help will be greatly appeciated. Thanks, Manuel -- Manuel López Mariscal Depto. de Oceanografía Física/CICESE malope(a)cicese.mx

2 1

503 errors from PyPI
by Paul Moore 23 Feb '14

23 Feb '14

Whenever I do a PyPI search at the moment, I get Error 503 backend read error backend read error Guru Meditation: XID: 477806858 ________________________________ Varnish cache server Paul

3 6

unparseable sdist filenames
by Marcus Smith 01 Oct '13

01 Oct '13

so, take a case like so "pytest-xdist-dev.tar.gz" (or any sdist with "-" in the project name, and a version starting with a string) I think it's like so: - pkg_resources.Distribution.from_location will treat "xdist-dev" as the version. - distlib.util.split_filename won't parse it because versions have to start with [0-9]. - pip will accept this as a "pytest" archive and install it potentially if no other version matches greater. what's the right answer? The historical-compatible answer is to be confused when projects have "-". so stay confused? or get rigid like distlib? Marcus

5 9

Problem: couldn't find a setup script in C:\Users\[user]\[package]
by Martin Fiers 30 Sep '13

30 Sep '13

Hello! When a folder inside a user's home directory has the same name as a package, then installation fails. *What steps will reproduce the problem?* 1. On a Windows system, start with a vanilla python, with pip not installed 2. Create the folder pip in C:\Users\[username] (so now there's a folder C:\Users\[username]\pip) 3. run easy_install pip (the error is reproduced with any other package as well, as long as there's a home folder). *What is the expected output? What do you see instead?* pip is installed Instead, this gives the error: Couldn't find a setup script in /home/[username]/pip *What version of the product are you using? On what operating system?* Windows 7, Python 2.7.5, with ez_setup.py called so setuptools is installed. *Please provide any additional information below.* This is rather annoying, since our software creates a folder in the user's home directory. But also the pip packages creates a log folder. So when I accidentially remove pip and want to re-install, easy_install does not work. Thanks very much, Martin

2 1

Remove the "Mirror Authenticity" API
by Donald Stufft 29 Sep '13

29 Sep '13

I believe we should remove the /serverkey and /serversig/* API's from PyPI. * I am not aware of *any* implementation that actually verifies packages against this API * In the light of PEP449 users now make a very conscious choice of which mirror they are using, which means they are no longer downloading random things from indiscriminate mirrors. * It uses DSA, which is a cryptographic primitive where if you reuse the random number or *any* bias in your random number you completely leak the private key. Given the nature of PyPI it's completely possible for a malicious user to essentially create an unbounded number of signatures making it more likely that a random nonce will be reused. * Moving forward something like TUF is a much better answer to the problems this attempts to solve as well as other problems. So it's basically unused with questionable primitives and better solutions exist. Does anyone have any objections to this being removed? ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

4 9

Twine 1.10 Releases - Upload things to PyPI
by Donald Stufft 26 Sep '13

26 Sep '13

I just released twine, it's a library for uploading things to PyPI - https://pypi.python.org/pypi/twine * Uses all the same flags as ``python setup.py upload`` * Doesn't force you to create the distribution file in the same command as the upload * Enables you to test the exact files you're about to upload prior to uploading * Enables you to manually sign the files, possibly on a different computer then created them * Enables you to automatically generate distributions in CI but have a trusted computer do releases * Uses TLS and verifies it (python setup.py upload does not) ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

1 0

devpi-1.1: improved pypi cache, data migration, wheels, ...
by holger krekel 25 Sep '13

25 Sep '13

The devpi-{server,client}-1.1 releases bring a lot of refinements and improvements for serving standalone pypi mirrors or company/private indexes which seemlessly merge in PyPI releases, mainly: - improved speed and semantics of pypi cache. devpi-server has been tested to correctly mirror all ~34000 projects on pypi.python.org now, and minimizes upstream interactions by fully exploiting the existing mirroring protocols. - data migration: new --upgrade/import/export options. versioning of serverstate and API calls so that future changes can be detected by clients. - better support for ".whl" files and many other little improvements and bug fixes, see the changelog below for more info. Note that devpi-1.1 will require to ``--upgrade`` your 1.0 server state before you can serve with devpi-server-1.1 release. Docs also were updated. Here is a Quickstart tutorial for efficient pypi-mirroring on your laptop:: http://doc.devpi.net/1.1/quickstart-pypimirror.html And if you want to manage your releases or implement staging as an individual or within an organisation:: http://doc.devpi.net/1.1/quickstart-releaseprocess.html If you want to permanently install devpi-server and potentially access it from many clients:: http://doc.devpi.net/1.1/quickstart-server.html More documentation and the beginning of an exhaustive user manual:: http://doc.devpi.net/latest/ have fun, holger CHANGELOG 1.1 ---------------------------- devpi-server: - systematically test pypi/mirror code against all 34K pypi projects so that we know that all http/https installable archive links that pypi offers are correctly recognized by devpi-server's root/pypi index. - if no pypi mirror state is known, devpi-server now calls pypi to obtain names/serials. It will fail to start if no such initial connection is possible. Once a first mirror state is known, subsequent devpi-server starts will not perform this initial query. - speed up and make more reliable all operations on private packages which have no pypi.python.org release: we can now determine if a project exists on pypi and under which name exactly without remote queries or redirects to pypi.python.org. - fix issue45: register/upload package names are now properly validated and redirects take place if e.g. a project was registered as "name-sub" and "+simple/name_sub" is queried. - new --upgrade-state command to allow for easy and safe in-place upgrading of server state. This is not guranteed to be possible for all future releases which might require using --export with an older version and --import with a newer version. - new --export/--import options to dump and import server contents: users, indexes, docs, release files and (test) attachments. Note that root/pypi (PyPI-caching information) will not be exported/imported. (maybe in the future if there is demand). - fix issue49: both push and import/export now support docfiles. Note, however, that docfiles relate to a project as a whole and are not tied to a particular version. This property is inherited from the PyPI standard upload_docs action and cannot be changed without interfering or replacing the upload_docs protocol of setuptools/sphinx. - fix issue51: return 200 code if release file is successfully uploaded but jenkins could not be triggered (previously returned 500) - reject simple/NAME if NAME contains non-ascii characters (PEP426 naming rules) - devpi-server now returns a X-DEVPI-API-VERSION and X-DEVPI-SERVER-VERSION header. For future incompatible changes these versions allow clients to reject interactions. - also add ".serverversion" file and write it if it does not exist, and make devpi-server use it to verify if operating on a compatible server data layout, otherwise bail out. - address issue43: --gendeploy now uses pip without --pre and explicitely instructs pip to install the exact same version of devpi-server with which --gendeploy is issued. - fix issue46 -- for GET /root/pypi/ only show a link to the simple page instead of computing "latest in-stage packages" which is only useful for devpi's user indices. - fix issue37: upload with expired login causes proper 401 devpi-client: - detect "X-DEVPI-API-VERSION" header and check for compatibility. devpi-client currently supports version "1" and warns if no version is known (defaulting to "1"). - getjson now prints http reply headers if "-v|--verbose" was supplied. - fix issue52: add "--no-vcs" option to force "devpi upload" to not vcs-export files before executing build/upload - fix issue50: introduce "--toxargs" to "devpi test" invocations in order to add arbitrary arguments to tox. - fix issue43: set PIP_PRE environment variable when executing "devpi install ..." so that the behaviour between pip < 1.4 and >= 1.4 is normalized to always install development versions. - fix issue47: set PIP_USE_WHEEL with "devpi install ..." so that it will automatically pick up wheel files if pip>1.4 is used. - fix issue42: allow to set empty bases for an index, so that it doesn't inherit anything. - fix issue44: "use -l" doesn't break when a user has no index - devpi test now invokes tox in-process (by importing tox) rather than a subprocess.

1 0