Mailman 3 March 2013 - Distutils-SIG

Two fixes (at last)
by Greg Ward 29 Apr '18

29 Apr '18

Hi all -- at long last, I have fixed two problems that a couple people noticed a while ago: * I folded in Amos Latteier's NT patches almost verbatim -- just changed an `os.path.sep == "/"' to `os.name == "posix"' and added some comments bitching about the inadequacy of the current library installation model (I think this is Python's fault, but for now Distutils is slavishly aping the situation in Python 1.5.x) * I fixed the problem whereby running "setup.py install" without doing anything else caused a crash (because 'build' hadn't yet been run). Now, the 'install' command automatically runs 'build' before doing anything; to make this bearable, I added a 'have_run' dictionary to the Distribution class to keep track of which commands have been run. So now not only are command classes singletons, but their 'run' method can only be invoked once -- both restrictions enforced by Distribution. The code is checked into CVS, or you can download a snapshot at http://www.python.org/sigs/distutils-sig/distutils-19990607.tar.gz Hope someone (Amos?) can try the new version under NT. Any takers for Mac OS? BTW, all parties involved in the Great "Where Do We Install Stuff?" Debate should take a good, hard look at the 'set_final_options()' method of the Install class in distutils/install.py; this is where all the policy decisions about where to install files are made. Currently it apes the Python 1.5 situation as closely as I could figure it out. Obviously, this is subject to change -- I just don't know to *what* it will change! Greg -- Greg Ward - software developer gward(a)cnri.reston.va.us Corporation for National Research Initiatives 1895 Preston White Drive voice: +1-703-620-8990 Reston, Virginia, USA 20191-5434 fax: +1-703-620-0913

2 1

Package versioning
by Timothy Docker 29 Apr '18

29 Apr '18

Hi all, I've been aware that the distutils sig has been simmerring away, but until recently it has not been directly relevant to what I do. I like the look of the proposed api, but have one question. Will this support an installed system that has multiple versions of the same package installed simultaneously? If not, then this would seem to be a significant limitation, especially when dependencies between packages are considered. Assuming it does, then how will this be achieved? I am presently managing this with a messy arrangement of symlinks. A package is installed with its version number in it's name, and a separate directory is created for an application with links from the unversioned package name to the versioned one. Then I just set the pythonpath to this directory. A sample of what the directory looks like is shown below. I'm sure there is a better solution that this, and I'm not sure that this would work under windows anyway (does windows have symlinks?). So, has this SIG considered such versioning issues yet? Cheers, Tim -------------------------------------------------------------- Tim Docker timd(a)macquarie.com.au Quantative Applications Division Macquarie Bank -------------------------------------------------------------- qad16:qad $ ls -l lib/python/ total 110 drwxr-xr-x 2 mts mts 512 Nov 11 11:23 1.1 -r--r----- 1 root mts 45172 Sep 1 1998 cdrmodule_0_7_1.so drwxr-xr-x 2 mts mts 512 Sep 1 1998 chart_1_1 drwxr-xr-x 3 mts mts 512 Sep 1 1998 Fnorb_0_7_1 dr-xr-x--- 3 mts mts 512 Nov 11 11:21 Fnorb_0_8 drwxr-xr-x 3 mts mts 1536 Mar 3 12:45 mts_1_1 dr-xr-x--- 7 mts mts 512 Nov 11 11:22 OpenGL_1_5_1 dr-xr-x--- 2 mts mts 1024 Nov 11 11:23 PIL_0_3 drwxr-xr-x 3 mts mts 512 Sep 1 1998 Pmw_0_7 dr-xr-x--- 2 mts mts 512 Nov 11 11:21 v3d_1_1 qad16:qad $ ls -l lib/python/1.1 total 30 lrwxrwxrwx 1 root other 29 Apr 10 10:43 _glumodule.so -> ../OpenGL_1_5_1/_glumodule.so lrwxrwxrwx 1 root other 30 Apr 10 10:43 _glutmodule.so -> ../OpenGL_1_5_1/_glutmodule.so lrwxrwxrwx 1 root other 22 Apr 10 10:43 _imaging.so -> ../PIL_0_3/_imaging.so lrwxrwxrwx 1 root other 36 Apr 10 10:43 _opengl_nummodule.so -> ../OpenGL_1_5_1/_opengl_nummodule.so lrwxrwxrwx 1 root other 27 Apr 10 10:43 _tkinter.so -> ../OpenGL_1_5_1/_tkinter.so lrwxrwxrwx 1 mts mts 21 Apr 10 10:43 cdrmodule.so -> ../cdrmodule_0_7_1.so lrwxrwxrwx 1 mts mts 12 Apr 10 10:43 chart -> ../chart_1_1 lrwxrwxrwx 1 root other 12 Apr 10 10:43 Fnorb -> ../Fnorb_0_8 lrwxrwxrwx 1 mts mts 12 Apr 10 10:43 mts -> ../mts_1_1 lrwxrwxrwx 1 root other 15 Apr 10 10:43 OpenGL -> ../OpenGL_1_5_1 lrwxrwxrwx 1 root other 33 Apr 10 10:43 opengltrmodule.so -> ../OpenGL_1_5_1/opengltrmodule.so lrwxrwxrwx 1 root other 33 Apr 10 10:43 openglutil_num.so -> ../OpenGL_1_5_1/openglutil_num.so lrwxrwxrwx 1 root other 10 Apr 10 10:43 PIL -> ../PIL_0_3 lrwxrwxrwx 1 mts mts 10 Apr 10 10:43 Pmw -> ../Pmw_0_7 lrwxrwxrwx 1 root other 10 Apr 10 10:43 v3d -> ../v3d_1_1

3 2

[Distutils-sig] distutils charter and interscript
by John Skaller 29 Apr '18

29 Apr '18

3 2

error in windows 7 installation
by Manuel López 13 May '14

13 May '14

Hello Everybody: I am trying to install distribute-0.6.25 in a windows 7 machine. I have python 2.7.3 in 32 bits, although the machine is 64 bits. I installed the 32 bit python version because I want to install ipython and there is no 64 bit builds of the Windows installer for ipython. In any case, upon installing disuitils with python setup.py install I get the following error error: C:\manuel\prog_red\python\distribute-0.6.25\distribute-0.6.25\setuptools\cli-32.exe: No such file or directory Any help will be greatly appeciated. Thanks, Manuel -- Manuel López Mariscal Depto. de Oceanografía Física/CICESE malope(a)cicese.mx

2 1

special compiler options for only one file
by Andrew Dalke 23 Apr '13

23 Apr '13

Hi all, I have a Python extension which uses CPU-specific features, if available. This is done through a run-time check. If the hardware supports the POPCNT instruction then it selects one implementation of my inner loop, if SSSE3 is available then it selects another, otherwise it falls back to generic versions of my performance critical kernel. (Some 95%+ of the time is spent in this kernel.) Unfortunately, there's a failure mode I didn't expect. I use -mssse3 and -O3 to compile all of the C code, even though only one file needs that -mssse3 option. As a result, the other files are compiled with the expectation that SSSE3 will exist. This causes a segfault for the line start_target_popcount = (int)(query_popcount * threshold); because the compiler used fisttpl, which is an SSSE-3 instruction. After all, I told it to assume that ssse3 exists. The Debian packager for my package recently ran into this problem, because the test machine has a gcc which understands -mssse3 but the machine itself has an older CPU without those instructions. I'm trying to come up with a solution that can be automated for the Debian distribution. I want a solution where the same binary can work on older machines and on newer ones Ideally I would like to say that only one file is compiled with the -mssse3 option. Since my selector code isn't part of this file, SSSE-3 code will never be executed unless the CPU supports is. However, I can't figure out any way to tell distutils that a set of compiler options are specific to a single file. Is that even possible? Cheers, Andrew dalke(a)dalkescientific.com

3 3

PEP: Improving Python ZIP Application Support
by Daniel Holth 16 Apr '13

16 Apr '13

Python ZIP Application Support - https://docs.google.com/document/d/1MKXgPzhWD5wIUpoSQX7dxmqgTZVO6l9iZZis8dn… PEP: 4XX Title: Improving Python ZIP Application Support Author: Daniel Holth <dholth(a)gmail.com> Status: Draft Type: Standards Track Python-Version: 3.4 Created: 30 March 2013 Post-History: 30 March 2013 Improving Python ZIP Application Support Python has had the ability to execute directories or ZIP-format archives as scripts since version 2.6. When invoked with a zip file or directory as its first argument the interpreter adds that directory to sys.path and executes the __main__ module. These archives provide a great way to publish software that needs to be distributed as a single file script but is complex enough to need to be written as a collection of modules. This feature is not as popular as it should be for two reasons: a) users haven’t heard of it because it wasn’t mentioned in earlier versions of Python 2.6 “What’s New” document, and b) Windows users don’t have a file extension (other than .py) to associate with the launcher. This PEP proposes to fix these problems by re-publicising the feature, defining the .pyz and .pyzw extensions as “Python ZIP applications” and “Windowed Python Zip Applications”, and providing some simple tooling to manage the format. A New Python ZIP Application Extension The Python 3.4 installer will associate .pyz and .pyzw “Python ZIP Applications” with itself so they can be executed by the Windows launcher. A .pyz archive is a console application and a .pyzw archive is a windowed application. This indicates whether the console should appear when running the app. Why not use .zip or .py? Users expect a .zip file would be opened with an archive tool, and users expect .py to be opened with a text editor. Both would be confusing for this use case. For UNIX users, .pyz applications should be prefixed with a #! line pointing to the correct Python interpreter and an optional explanation. #!/usr/bin/env python # This is a Python application stored in a ZIP archive. (binary contents of archive) As background, ZIP archives are defined with a footer containing relative offsets from the end of the file. They remain valid when concatenated to the end of any other file. This feature is completely standard and is how self-extracting ZIP archives and the bdist_wininst installer format work. Minimal Tooling: The pyzaa Module This PEP also proposes including a simple application for working with Python ZIP Archives: The Python Zip Application Archiver “pyzaa” (rhymes with “huzzah” or “pizza”). “pyzaa” can archive or extract these files, compile bytecode, and can write the __main__ module if it is not present. Usage python -m pyzaa (pack | unpack | compile) python -m pyzaa pack [-o path/name] [-m module.submodule:callable] [-c] [-w] [-p interpreter] directory: ZIP the contents of directory as directory.pyz or [-w] directory.pyzw. Adds the executable flag to the archive. -c compile .pyc files and add them to the archive -p interpreter include #!interpreter as the first line of the archive -o path/name archive is written to path/name.pyz[w] instead of dirname. The extension is added if not specified. -m module.submodule:callable __main__.py is written as “import module.submodule; module.submodule.callable()” pyzaa pack will warn if the directory contains C extensions or if it doesn’t contain __main__.py. python -m pyzaa unpack arcname.pyz[w] The archive is unpacked into a directory [arcname] python -m pyzaa compile arcname.pyz[w] The Python files in arcname.pyz[w] are compiled and appended to the ZIP file. References [1] http://bugs.python.org/issue1739468 “Allow interpreter to execute a zip file” Copyright This document has been placed into the public domain.

8 12

"packaging-user" mailing list?
by Marcus Smith 04 Apr '13

04 Apr '13

Some of the pypa people have been discussing beginning a "packaging-user" mailing list. - It would be open to *any* packaging or install user issues. - It would be on python.org - pip/virtualenv would use it instead of our "virtualenv" list - Other projects could(would) use it too: Setuptools (old and new), Distribute, wheel, buildout, bento? etc... I think python users would appreciate the simplicity of this, and the support would be better too I think. distutils-sig would maintain it's current focus as a place for development discussions. thoughts? Marcus

5 6

Help with buildout
by John Anderson 04 Apr '13

04 Apr '13

I'm trying to use buildout to pull in 4 github repositories and combine their wsgi applications together using paste's urlmap. A few things aren't working the way I expect and so I'm looking for some more information: This is my buildout.cfg: https://github.com/sontek/notaliens.com/blob/master/buildout.cfg 1. I had to declare a [pyramid] section that pulls in Pyramid with the zc.recipe.egg recipe so that it would drop the entry_point scripts in the generated bin/ folder. Is there a way for it to do this from dependencies declared in my install_requires in setup.py rather than being declared in the buildout? 2. The 2nd problem is after I run buildout, I'm not able to import paste.deploy: I see the egg in my sys.path: bin/py >>> import sys >>> sys.path <cut> '/home/sontek/code/test_notaliens/ notaliens.com/eggs/PasteDeploy-1.5.0-py2.7.egg', but if I try to import from it: >>> from paste.deploy import loadserver Traceback (most recent call last): File "<console>", line 1, in <module> ImportError: No module named deploy PasteDeploy is defined as a dependency and gets pulled in properly but for some reason the interpreter isn't seeing it. any ideas?

5 4

Self-contained boostrap scripts [was: Re: A new, experimental packaging tool: distil]
by Philippe Ombredanne 31 Mar '13

31 Mar '13

On Tue, Mar 26, 2013 at 11:08 AM, Paul Moore <p.f.moore(a)gmail.com> wrote: > On 26 March 2013 09:49, Philippe Ombredanne <pombredanne(a)nexb.com> wrote: >> Would anyone know of a better way to package things in a single >> python-executable bootstrapping script file without obfuscating the >> source contents in compressed/encoded/obfuscated byte arrays? > > Packaging as a zip file is a good way - but on Windows the file needs > to be named xxx.py (which is surprising, to say the least :-)) for the > relevant file association to be triggered (and on Unix, a #! line > needs to be prepended). Paul: I was not talking about this type of zips, but rather the same used in virtualenv, i.e. a string in a .py file that contains an encoded zip. That string is then decoded and unzipped at runtime as in here: https://github.com/pypa/virtualenv/blob/develop/virtualenv.py#L1933 This is not a zip, not an egg, not a wheel but some egg-in-py, zip-in-py or wheel-in-py and is similar to a shar shell archive. My point was that on the one hand, I like the fact that everything is self contained in one single .py file that you can execute right away. On the other hand, I find it somewhat discomforting as an emerging best way to package and distribute self-contained bootstrap scripts. Yet I cannot think of a better way atm: for instance splitting things in non-encoded non-binary plain strings would be quite weird too. Virtualenv does it, distil is doing it now, pip tried some of it here https://github.com/pypa/pip/blob/develop/contrib/get-pip.py In contrast, buildout, distribute and setuptools bootstrap scripts do not embed their dependencies and either try to get them satisfied locally or attempt to download the requirements. Having some support to do self-contained bootstrap scripts (as in requiring no network access and embedding all their dependencies) using this shar style could be something to consider normalizing? -- Philippe Ombredanne +1 650 799 0949 | pombredanne(a)nexB.com DejaCode Enterprise at http://www.dejacode.com nexB Inc. at http://www.nexb.com

9 26

Re: [Distutils] [Catalog-sig] Merge catalog-sig and distutils-sig
by Richard Jones 30 Mar '13

30 Mar '13

On 29 March 2013 14:45, Tres Seaver <tseaver(a)palladion.com> wrote: > If we leave the main list the 'distutils-sig', and just announce that > 'catalog-sig' is retired, folks who want to follow the new list just > switch over. All the archives (mailman / gmane / etc.) stay valid, but > the list goes into moderated mode. Whoever has the power to do this, do it please. Richard

3 3