Hi all --
at long last, I have fixed two problems that a couple people noticed a
* I folded in Amos Latteier's NT patches almost verbatim -- just
changed an `os.path.sep == "/"' to `os.name == "posix"' and added
some comments bitching about the inadequacy of the current library
installation model (I think this is Python's fault, but for now
Distutils is slavishly aping the situation in Python 1.5.x)
* I fixed the problem whereby running "setup.py install" without
doing anything else caused a crash (because 'build' hadn't yet
been run). Now, the 'install' command automatically runs 'build'
before doing anything; to make this bearable, I added a 'have_run'
dictionary to the Distribution class to keep track of which commands
have been run. So now not only are command classes singletons,
but their 'run' method can only be invoked once -- both restrictions
enforced by Distribution.
The code is checked into CVS, or you can download a snapshot at
Hope someone (Amos?) can try the new version under NT. Any takers for
BTW, all parties involved in the Great "Where Do We Install Stuff?"
Debate should take a good, hard look at the 'set_final_options()' method
of the Install class in distutils/install.py; this is where all the
policy decisions about where to install files are made. Currently it
apes the Python 1.5 situation as closely as I could figure it out.
Obviously, this is subject to change -- I just don't know to *what* it
Greg Ward - software developer gward(a)cnri.reston.va.us
Corporation for National Research Initiatives
1895 Preston White Drive voice: +1-703-620-8990
Reston, Virginia, USA 20191-5434 fax: +1-703-620-0913
I've been aware that the distutils sig has been simmerring away, but
until recently it has not been directly relevant to what I do.
I like the look of the proposed api, but have one question. Will this
support an installed system that has multiple versions of the same
package installed simultaneously? If not, then this would seem to be a
significant limitation, especially when dependencies between packages
Assuming it does, then how will this be achieved? I am presently
managing this with a messy arrangement of symlinks. A package is
installed with its version number in it's name, and a separate
directory is created for an application with links from the
unversioned package name to the versioned one. Then I just set the
pythonpath to this directory.
A sample of what the directory looks like is shown below.
I'm sure there is a better solution that this, and I'm not sure that
this would work under windows anyway (does windows have symlinks?).
So, has this SIG considered such versioning issues yet?
Tim Docker timd(a)macquarie.com.au
Quantative Applications Division
qad16:qad $ ls -l lib/python/
drwxr-xr-x 2 mts mts 512 Nov 11 11:23 1.1
-r--r----- 1 root mts 45172 Sep 1 1998 cdrmodule_0_7_1.so
drwxr-xr-x 2 mts mts 512 Sep 1 1998 chart_1_1
drwxr-xr-x 3 mts mts 512 Sep 1 1998 Fnorb_0_7_1
dr-xr-x--- 3 mts mts 512 Nov 11 11:21 Fnorb_0_8
drwxr-xr-x 3 mts mts 1536 Mar 3 12:45 mts_1_1
dr-xr-x--- 7 mts mts 512 Nov 11 11:22 OpenGL_1_5_1
dr-xr-x--- 2 mts mts 1024 Nov 11 11:23 PIL_0_3
drwxr-xr-x 3 mts mts 512 Sep 1 1998 Pmw_0_7
dr-xr-x--- 2 mts mts 512 Nov 11 11:21 v3d_1_1
qad16:qad $ ls -l lib/python/1.1
lrwxrwxrwx 1 root other 29 Apr 10 10:43 _glumodule.so -> ../OpenGL_1_5_1/_glumodule.so
lrwxrwxrwx 1 root other 30 Apr 10 10:43 _glutmodule.so -> ../OpenGL_1_5_1/_glutmodule.so
lrwxrwxrwx 1 root other 22 Apr 10 10:43 _imaging.so -> ../PIL_0_3/_imaging.so
lrwxrwxrwx 1 root other 36 Apr 10 10:43 _opengl_nummodule.so -> ../OpenGL_1_5_1/_opengl_nummodule.so
lrwxrwxrwx 1 root other 27 Apr 10 10:43 _tkinter.so -> ../OpenGL_1_5_1/_tkinter.so
lrwxrwxrwx 1 mts mts 21 Apr 10 10:43 cdrmodule.so -> ../cdrmodule_0_7_1.so
lrwxrwxrwx 1 mts mts 12 Apr 10 10:43 chart -> ../chart_1_1
lrwxrwxrwx 1 root other 12 Apr 10 10:43 Fnorb -> ../Fnorb_0_8
lrwxrwxrwx 1 mts mts 12 Apr 10 10:43 mts -> ../mts_1_1
lrwxrwxrwx 1 root other 15 Apr 10 10:43 OpenGL -> ../OpenGL_1_5_1
lrwxrwxrwx 1 root other 33 Apr 10 10:43 opengltrmodule.so -> ../OpenGL_1_5_1/opengltrmodule.so
lrwxrwxrwx 1 root other 33 Apr 10 10:43 openglutil_num.so -> ../OpenGL_1_5_1/openglutil_num.so
lrwxrwxrwx 1 root other 10 Apr 10 10:43 PIL -> ../PIL_0_3
lrwxrwxrwx 1 mts mts 10 Apr 10 10:43 Pmw -> ../Pmw_0_7
lrwxrwxrwx 1 root other 10 Apr 10 10:43 v3d -> ../v3d_1_1
I am trying to install distribute-0.6.25 in a windows 7 machine. I have
python 2.7.3 in 32 bits, although the machine is 64 bits. I installed
the 32 bit python version because I want to install ipython and there
is no 64 bit builds of the Windows installer for ipython.
In any case, upon installing disuitils with
python setup.py install
I get the following error
No such file or directory
Any help will be greatly appeciated. Thanks,
Manuel López Mariscal
Depto. de Oceanografía Física/CICESE
I have a Python extension which uses CPU-specific features,
if available. This is done through a run-time check. If the
hardware supports the POPCNT instruction then it selects one
implementation of my inner loop, if SSSE3 is available then
it selects another, otherwise it falls back to generic versions
of my performance critical kernel. (Some 95%+ of the time is
spent in this kernel.)
Unfortunately, there's a failure mode I didn't expect. I
use -mssse3 and -O3 to compile all of the C code, even though
only one file needs that -mssse3 option.
As a result, the other files are compiled with the expectation
that SSSE3 will exist. This causes a segfault for the line
start_target_popcount = (int)(query_popcount * threshold);
because the compiler used fisttpl, which is an SSSE-3 instruction.
After all, I told it to assume that ssse3 exists.
The Debian packager for my package recently ran into this problem,
because the test machine has a gcc which understands -mssse3 but
the machine itself has an older CPU without those instructions.
I'm trying to come up with a solution that can be automated for
the Debian distribution. I want a solution where the same binary
can work on older machines and on newer ones
Ideally I would like to say that only one file is compiled
with the -mssse3 option. Since my selector code isn't part of this
file, SSSE-3 code will never be executed unless the CPU supports is.
However, I can't figure out any way to tell distutils that
a set of compiler options are specific to a single file.
Is that even possible?
Python ZIP Application Support -
Title: Improving Python ZIP Application Support
Author: Daniel Holth <dholth(a)gmail.com>
Type: Standards Track
Created: 30 March 2013
Post-History: 30 March 2013
Improving Python ZIP Application Support
Python has had the ability to execute directories or ZIP-format
archives as scripts since version 2.6. When invoked with a zip file or
directory as its first argument the interpreter adds that directory to
sys.path and executes the __main__ module. These archives provide a
great way to publish software that needs to be distributed as a single
file script but is complex enough to need to be written as a
collection of modules.
This feature is not as popular as it should be for two reasons: a)
users haven’t heard of it because it wasn’t mentioned in earlier
versions of Python 2.6 “What’s New” document, and b) Windows users
don’t have a file extension (other than .py) to associate with the
This PEP proposes to fix these problems by re-publicising the feature,
defining the .pyz and .pyzw extensions as “Python ZIP applications”
and “Windowed Python Zip Applications”, and providing some simple
tooling to manage the format.
A New Python ZIP Application Extension
The Python 3.4 installer will associate .pyz and .pyzw “Python ZIP
Applications” with itself so they can be executed by the Windows
launcher. A .pyz archive is a console application and a .pyzw archive
is a windowed application. This indicates whether the console should
appear when running the app.
Why not use .zip or .py? Users expect a .zip file would be opened with
an archive tool, and users expect .py to be opened with a text editor.
Both would be confusing for this use case.
For UNIX users, .pyz applications should be prefixed with a #! line
pointing to the correct Python interpreter and an optional
# This is a Python application stored in a ZIP archive.
(binary contents of archive)
As background, ZIP archives are defined with a footer containing
relative offsets from the end of the file. They remain valid when
concatenated to the end of any other file. This feature is completely
standard and is how self-extracting ZIP archives and the bdist_wininst
installer format work.
Minimal Tooling: The pyzaa Module
This PEP also proposes including a simple application for working with
Python ZIP Archives: The Python Zip Application Archiver “pyzaa”
(rhymes with “huzzah” or “pizza”). “pyzaa” can archive or extract
these files, compile bytecode, and can write the __main__ module if it
is not present.
python -m pyzaa (pack | unpack | compile)
python -m pyzaa pack [-o path/name] [-m module.submodule:callable]
[-c] [-w] [-p interpreter] directory:
ZIP the contents of directory as directory.pyz or [-w]
directory.pyzw. Adds the executable flag to the archive.
-c compile .pyc files and add them to the archive
-p interpreter include #!interpreter as the first line of the archive
-o path/name archive is written to path/name.pyz[w] instead of
dirname. The extension is added if not specified.
-m module.submodule:callable __main__.py is written as “import
pyzaa pack will warn if the directory contains C extensions or if
it doesn’t contain __main__.py.
python -m pyzaa unpack arcname.pyz[w]
The archive is unpacked into a directory [arcname]
python -m pyzaa compile arcname.pyz[w]
The Python files in arcname.pyz[w] are compiled and appended to
the ZIP file.
 http://bugs.python.org/issue1739468 “Allow interpreter to execute
a zip file”
This document has been placed into the public domain.
Some of the pypa people have been discussing beginning a "packaging-user"
- It would be open to *any* packaging or install user issues.
- It would be on python.org
- pip/virtualenv would use it instead of our "virtualenv" list
- Other projects could(would) use it too: Setuptools (old and new),
Distribute, wheel, buildout, bento? etc...
I think python users would appreciate the simplicity of this, and the
support would be better too I think.
distutils-sig would maintain it's current focus as a place for development
I'm trying to use buildout to pull in 4 github repositories and combine
their wsgi applications together using paste's urlmap. A few things aren't
working the way I expect and so I'm looking for some more information:
This is my buildout.cfg:
1. I had to declare a [pyramid] section that pulls in Pyramid with the
zc.recipe.egg recipe so that it would drop the entry_point scripts in the
generated bin/ folder. Is there a way for it to do this from dependencies
declared in my install_requires in setup.py rather than being declared in
2. The 2nd problem is after I run buildout, I'm not able to import
I see the egg in my sys.path:
>>> import sys
but if I try to import from it:
>>> from paste.deploy import loadserver
Traceback (most recent call last):
File "<console>", line 1, in <module>
ImportError: No module named deploy
PasteDeploy is defined as a dependency and gets pulled in properly but for
some reason the interpreter isn't seeing it.
On Tue, Mar 26, 2013 at 11:08 AM, Paul Moore <p.f.moore(a)gmail.com> wrote:
> On 26 March 2013 09:49, Philippe Ombredanne <pombredanne(a)nexb.com> wrote:
>> Would anyone know of a better way to package things in a single
>> python-executable bootstrapping script file without obfuscating the
>> source contents in compressed/encoded/obfuscated byte arrays?
> Packaging as a zip file is a good way - but on Windows the file needs
> to be named xxx.py (which is surprising, to say the least :-)) for the
> relevant file association to be triggered (and on Unix, a #! line
> needs to be prepended).
I was not talking about this type of zips, but rather the same used in
virtualenv, i.e. a string in a .py file that contains an encoded zip.
That string is then decoded and unzipped at runtime as in here:
This is not a zip, not an egg, not a wheel but some egg-in-py,
zip-in-py or wheel-in-py and is similar to a shar shell archive.
My point was that on the one hand, I like the fact that everything is
self contained in one single .py file that you can execute right away.
On the other hand, I find it somewhat discomforting as an emerging
best way to package and distribute self-contained bootstrap scripts.
Yet I cannot think of a better way atm: for instance splitting things
in non-encoded non-binary plain strings would be quite weird too.
Virtualenv does it, distil is doing it now, pip tried some of it here
In contrast, buildout, distribute and setuptools bootstrap scripts do
not embed their dependencies and either try to get them satisfied
locally or attempt to download the requirements.
Having some support to do self-contained bootstrap scripts (as in
requiring no network access and embedding all their dependencies)
using this shar style could be something to consider normalizing?
+1 650 799 0949 | pombredanne(a)nexB.com
DejaCode Enterprise at http://www.dejacode.com
nexB Inc. at http://www.nexb.com
On 29 March 2013 14:45, Tres Seaver <tseaver(a)palladion.com> wrote:
> If we leave the main list the 'distutils-sig', and just announce that
> 'catalog-sig' is retired, folks who want to follow the new list just
> switch over. All the archives (mailman / gmane / etc.) stay valid, but
> the list goes into moderated mode.
Whoever has the power to do this, do it please.