Mailman 3 August 2012 - Distutils-SIG

Two fixes (at last)
by Greg Ward 29 Apr '18

29 Apr '18

Hi all -- at long last, I have fixed two problems that a couple people noticed a while ago: * I folded in Amos Latteier's NT patches almost verbatim -- just changed an `os.path.sep == "/"' to `os.name == "posix"' and added some comments bitching about the inadequacy of the current library installation model (I think this is Python's fault, but for now Distutils is slavishly aping the situation in Python 1.5.x) * I fixed the problem whereby running "setup.py install" without doing anything else caused a crash (because 'build' hadn't yet been run). Now, the 'install' command automatically runs 'build' before doing anything; to make this bearable, I added a 'have_run' dictionary to the Distribution class to keep track of which commands have been run. So now not only are command classes singletons, but their 'run' method can only be invoked once -- both restrictions enforced by Distribution. The code is checked into CVS, or you can download a snapshot at http://www.python.org/sigs/distutils-sig/distutils-19990607.tar.gz Hope someone (Amos?) can try the new version under NT. Any takers for Mac OS? BTW, all parties involved in the Great "Where Do We Install Stuff?" Debate should take a good, hard look at the 'set_final_options()' method of the Install class in distutils/install.py; this is where all the policy decisions about where to install files are made. Currently it apes the Python 1.5 situation as closely as I could figure it out. Obviously, this is subject to change -- I just don't know to *what* it will change! Greg -- Greg Ward - software developer gward(a)cnri.reston.va.us Corporation for National Research Initiatives 1895 Preston White Drive voice: +1-703-620-8990 Reston, Virginia, USA 20191-5434 fax: +1-703-620-0913

2 1

Package versioning
by Timothy Docker 29 Apr '18

29 Apr '18

Hi all, I've been aware that the distutils sig has been simmerring away, but until recently it has not been directly relevant to what I do. I like the look of the proposed api, but have one question. Will this support an installed system that has multiple versions of the same package installed simultaneously? If not, then this would seem to be a significant limitation, especially when dependencies between packages are considered. Assuming it does, then how will this be achieved? I am presently managing this with a messy arrangement of symlinks. A package is installed with its version number in it's name, and a separate directory is created for an application with links from the unversioned package name to the versioned one. Then I just set the pythonpath to this directory. A sample of what the directory looks like is shown below. I'm sure there is a better solution that this, and I'm not sure that this would work under windows anyway (does windows have symlinks?). So, has this SIG considered such versioning issues yet? Cheers, Tim -------------------------------------------------------------- Tim Docker timd(a)macquarie.com.au Quantative Applications Division Macquarie Bank -------------------------------------------------------------- qad16:qad $ ls -l lib/python/ total 110 drwxr-xr-x 2 mts mts 512 Nov 11 11:23 1.1 -r--r----- 1 root mts 45172 Sep 1 1998 cdrmodule_0_7_1.so drwxr-xr-x 2 mts mts 512 Sep 1 1998 chart_1_1 drwxr-xr-x 3 mts mts 512 Sep 1 1998 Fnorb_0_7_1 dr-xr-x--- 3 mts mts 512 Nov 11 11:21 Fnorb_0_8 drwxr-xr-x 3 mts mts 1536 Mar 3 12:45 mts_1_1 dr-xr-x--- 7 mts mts 512 Nov 11 11:22 OpenGL_1_5_1 dr-xr-x--- 2 mts mts 1024 Nov 11 11:23 PIL_0_3 drwxr-xr-x 3 mts mts 512 Sep 1 1998 Pmw_0_7 dr-xr-x--- 2 mts mts 512 Nov 11 11:21 v3d_1_1 qad16:qad $ ls -l lib/python/1.1 total 30 lrwxrwxrwx 1 root other 29 Apr 10 10:43 _glumodule.so -> ../OpenGL_1_5_1/_glumodule.so lrwxrwxrwx 1 root other 30 Apr 10 10:43 _glutmodule.so -> ../OpenGL_1_5_1/_glutmodule.so lrwxrwxrwx 1 root other 22 Apr 10 10:43 _imaging.so -> ../PIL_0_3/_imaging.so lrwxrwxrwx 1 root other 36 Apr 10 10:43 _opengl_nummodule.so -> ../OpenGL_1_5_1/_opengl_nummodule.so lrwxrwxrwx 1 root other 27 Apr 10 10:43 _tkinter.so -> ../OpenGL_1_5_1/_tkinter.so lrwxrwxrwx 1 mts mts 21 Apr 10 10:43 cdrmodule.so -> ../cdrmodule_0_7_1.so lrwxrwxrwx 1 mts mts 12 Apr 10 10:43 chart -> ../chart_1_1 lrwxrwxrwx 1 root other 12 Apr 10 10:43 Fnorb -> ../Fnorb_0_8 lrwxrwxrwx 1 mts mts 12 Apr 10 10:43 mts -> ../mts_1_1 lrwxrwxrwx 1 root other 15 Apr 10 10:43 OpenGL -> ../OpenGL_1_5_1 lrwxrwxrwx 1 root other 33 Apr 10 10:43 opengltrmodule.so -> ../OpenGL_1_5_1/opengltrmodule.so lrwxrwxrwx 1 root other 33 Apr 10 10:43 openglutil_num.so -> ../OpenGL_1_5_1/openglutil_num.so lrwxrwxrwx 1 root other 10 Apr 10 10:43 PIL -> ../PIL_0_3 lrwxrwxrwx 1 mts mts 10 Apr 10 10:43 Pmw -> ../Pmw_0_7 lrwxrwxrwx 1 root other 10 Apr 10 10:43 v3d -> ../v3d_1_1

3 2

[Distutils-sig] distutils charter and interscript
by John Skaller 29 Apr '18

29 Apr '18

3 2

error in windows 7 installation
by Manuel López 13 May '14

13 May '14

Hello Everybody: I am trying to install distribute-0.6.25 in a windows 7 machine. I have python 2.7.3 in 32 bits, although the machine is 64 bits. I installed the 32 bit python version because I want to install ipython and there is no 64 bit builds of the Windows installer for ipython. In any case, upon installing disuitils with python setup.py install I get the following error error: C:\manuel\prog_red\python\distribute-0.6.25\distribute-0.6.25\setuptools\cli-32.exe: No such file or directory Any help will be greatly appeciated. Thanks, Manuel -- Manuel López Mariscal Depto. de Oceanografía Física/CICESE malope(a)cicese.mx

2 1

distribute buildbot
by Adam GROSZER 22 Oct '12

22 Oct '12

Hello, Are there any buildbots for distribute? -- Best regards, Adam GROSZER -- Quote of the day: Men do not reject the Bible because it contradicts itself but because it contradicts them. - The Defender

5 4

Struggling with a modular buildout configuration
by Lele Gaifax 02 Oct '12

02 Oct '12

Hi all, I'm trying to setup a modular buildout configuration, but I'm hitting my head on an invisible wall... Basically, I'd like to have a top-level buildout.cfg that "collects" settings from various "subprojects"; in particular, each subproject may "augment" a set of eggs, some of which may be defined as "develop" (in buildout parlance). Finally, with a single "zc.recipe.egg" I wanna install a custom interpreter with all those eggs inside. So, I have the top-level configuration that uses "extends" to include a "projects.cfg": [buildout] extends = base.cfg projects.cfg that in turn brings in the various subprojects snippets thru "extends": [buildout] extends = src/this-project.cfg src/that-project.cfg these snippets typically contain a section like [projects] eggs += this-project-egg some-other-dependency-egg develop += src/this-project parts += this-project-specific-part [this-project-specific-part] ... The problem I'm having is that for incomprensible (to me, of course :) reason some of those settings do not "emerge" at the top level (others do!), as if a later "extends" is clobbering a previous one. I built a minimalistic example [1] that shows the unexpected behaviour; doing a "buildout annotate" I get something like the following: Annotated sections ================== [buildout] ... develop= ${projects:develop} /tmp/tb/buildout.cfg parts= ${projects:parts} /tmp/tb/buildout.cfg ... [extra-parts-from-proj2] [projects] develop= src/proj2 += /tmp/tb/src/proj2/devel.cfg eggs= extra-needed-by-proj1 proj1 proj2 += /tmp/tb/src/proj1/devel.cfg += /tmp/tb/src/proj1/base.cfg += /tmp/tb/src/proj2/base.cfg parts= extra-parts-from-proj2 += /tmp/tb/src/proj2/devel.cfg [python] eggs= ${projects:eggs} /tmp/tb/base.cfg interpreter= python /tmp/tb/base.cfg recipe= zc.recipe.egg /tmp/tb/base.cfg If you look at the [projects] section, you can see that while the "eggs" setting carries the three expected values (one for each subproject plus one extra), the "develop" setting contains just a single value, the one injected by proj2, and the one that should be coming from proj1 is missing. If I omit "src/proj2" from the extends option, then the "develop" setting correctly contains just "src/proj1"... Of course I tried hard to understand what's going, even walking into buildout 1.5.2 code with pdb. I tried also with the latest development version (the upcoming 1.6) as well as the github branch (IIUC it derivates from 1.4), but all exhibit the same behaviour. The 1.6 code is slightly different, and at first it seemed doing the right thing, but at a closer look it really moved the problem from the "develop" setting "eggs". As far as I can tell, there is something strange happening in the _update_section() function when the _open() function handles the "extends" setting: at some point it "loses" the cognition that a certain assignment is an expansion (tecnically, that the key is actually "develop +", instead of "develop"), so the final "s1.update(s2)" overrides it, instead of augmenting it... Of course I may be doing the wrong thing or missing the obvious, and if so I'll be happy if you can shed some light on me. Thanks in advance for any help and sorry for the long post, ciao, lele. [1] http://artiemestieri.tn.it/~lele/test-buildout.tar.gz -- nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia. lele(a)metapensiero.it | -- Fortunato Depero, 1929.

3 6

the 'wheel' binary package format
by Daniel Holth 30 Sep '12

30 Sep '12

I got tired of waiting for lxml to compile over and over again, so I invented a binary packaging format called 'wheel' (of cheese) that uses Metadata 1.2 and .dist-info directories instead of .egg-info, patched pkg_resources.py to be able to load .dist-info directories, implemented "python setup.py bdist_wheel", and patched pip to be able to install .whl files. The gist of the spec is that it is a zip file with the .whl extension containing the contents of 'purelib' for a distribution, plus a Name-1.0.dist-info/ directory with the metadata files, plus Name-1.0.data/subdir directories for scripts, platlib, packaging's "categories", ... My specification so far is at https://docs.google.com/document/d/1mWPyvoeiqCrAy4UPNnvaz7Cgrqm4s_cfaTauAeJ… and an lxml compiled for linux-x86-64 is at https://docs.google.com/open?id=0BxHz5bC4iN5TN0VWTFNrZGtCbWs http://bitbucket.org/dholth/distribute http://github.com/dholth/pip http://pypi.python.org/pypi/wheel Perhaps it will be useful. The implementation is still pretty rough, for example it does not check the architecture while installing, but it could be a handy way to speed up repeated virtualenv builds. Daniel Holth

5 21

wheel file signatures scheme
by Daniel Holth 08 Sep '12

08 Sep '12

I've made what I think is exciting progress on the digital signatures design for wheel (updated built/binary packages for Python; intended to replace egg). The insight is that we can overload the "extras" syntax as a convenient way to mention the public key we expect: package[extra, ed25519=ouBJlTJJ4SJXoy8Bi1KRlewWLU6JW7HUXTgvU1YRuiA] This line in a pip-style requires.txt specifies that we want to install package, the normal optional dependency "extra" and that we expect it to have a valid signature made with the mentioned ed25519 public key. Distribute your application, assemble its requirements in this format, sign that file, and Bob's your uncle -- your users can install from that file and know that the requirements they download have the same publishers as the packages you developed with. This is far more powerful than a file hash because it is valid for more than one version of the package. For backwards compatibility, packages can say they provide the extra ed25519=ouBJlTJJ4SJXoy8Bi1KRlewWLU6JW7HUXTgvU1YRuiA (adding no dependencies) and easy_install should parse and ignore it, installing from an unsigned egg or source distribution because it doesn't know about wheels. bdist_wheel will be updated to always sign the wheels it generates if possible, making up a new signing key if one is not stored on the building machine. That way signed wheels will be commonplace, and the remaining problem is simply to decide which signing keys you would like to trust. wheel now includes a command line tool `wheel sign wheelfile.whl` that adds a digital signature to a wheel file. http://wheel.readthedocs.org/en/latest/index.html#signed-wheel-files

4 5

Question on distutils.cfg and problem with installation of yappi
by Virgil Stokes 28 Aug '12

28 Aug '12

I have only recently joined this list --- hopefully my first mail to this list is acceptable. *First*, is it possible to place comments in the distutils.cfg file? *Second*, I have been unable to install yappi (included in the PyPI list) on my Windows 7 (64-bit) platform with Python 2.7 (32-bit) and would greatly appreciate help on this installation. Note, I have installed many other PyPI packages on this same system with no problems where I have cygwin, minGW (32-bit) and minGW (64-bit) gcc and g++ compilers installed and working properly. I have installed yappi on my Windows Vista (32-bit) platform with Python 2.7. I have also studied the information posted by Greg Ward at http://docs.python.org/install/index.html. Here are some of the results of my attempts to install yappi on the Windows 7 system. I looked at what compilers are available for building from my setup.py file: c:\Software\Python\yappi\yappi-0.62>python setup.py build --help-compiler List of available compilers: --compiler=bcpp Borland C++ Compiler --compiler=cygwin Cygwin port of GNU C Compiler for Win32 --compiler=emx EMX port of GNU C Compiler for OS/2 --compiler=mingw32 Mingw32 port of GNU C Compiler for Win32 --compiler=msvc Microsoft Visual C++ --compiler=unix standard UNIX-style compiler I then tried the following: c:\Software\Python\yappi\yappi-0.62>python setup.py build --compiler=cygwin running build running build_py running build_ext building '_yappi' extension gcc -mcygwin -mdll -O -Wall -IC:\Python27\include -IC:\Python27\PC -c _yappi.c -o build\temp.win32-2.7\Release\_yappi.o error: command 'gcc' failed: No such file or directory I then tried to be more specific, c:\Software\Python\yappi\yappi-0.62>python setup.py build --compiler=c:\cygwin\bin\gcc.exe running build running build_py running build_ext building '_yappi' extension error: don't don't know how to compile C/C++ code on platform 'nt' with 'c:\cygwin\bin\gcc.exe' compiler I also tried many other possible (at least IMHO) solutions --- these also failed. For example, c:\Software\Python\yappi\yappi-0.62>python setup.py build --compiler=msvc running build running build_py running build_ext building '_yappi' extension error: Unable to find vcvarsall.bat I then modified the c:\python27\Lib\distutils\distutils.cfg file and set different compilers in it --- these also failed. The bottom line --- all my trials (attempts) failed for one reason or another. Any suggestions as to how to proceed? I will be glad to provide any additional information on my trials. --V

1 0

Distribute tests.
by Lennart Regebro 22 Aug '12

22 Aug '12

Most of the tests for distribute are run with "python setup.py test", but there is a whole bunch of tests in ./tests and they are not run. Anyone know how the *should* be run? Just running them creates various errors, clearly there is some sort of setup needed first. //Lennart

1 0