This is the only obvious up-to-date address or mailing list I could find, so
feel free to direct me where I should go.
I'm a Linux user, and normally I have a 0027 umask. This means that when I
build a Python package using distutils, the files in the build directory are
not world-readable (Python copies them there without preserving permissions,
according to the comment in build_py.py:
# XXX copy_file by default preserves mode, which appears to be the
# wrong thing to do: if a file is read-only in the working
# directory, we want it to be installed read/write so that the next
# installation of the same module distribution can overwrite it
# without problems. (This might be a Unix-specific issue.) Thus
# we turn off 'preserve_mode' when copying to the build directory,
# since the build directory is supposed to be exactly what the
# installation will look like (ie. we preserve mode when
This is all very well, but turning off "preserve_mode" is not enough to
ensure this in the presence of a restrictive umask like mine; the
permissions need rather to be explicitly set (as done for example by the GNU
install tool, though that assumes that permissions are set at install time
rather than build time).
The obvious workaround is for me to remember to change my umask when I'm
building python packages. That's OK, but is this problem recognised as such
and might it be fixed in future?
http://rrt.sc3d.org/ | God is the name of our ignorance (Grayling)
At 10:30 AM 4/9/2008 -0700, zooko wrote:
>PEP 262 sounds like a non-starter to me because
>1. It appears to be backwards-incompatible with setuptools/
>easy_install/eggs, thus losing all the recently gained cooperation
>that I mentioned in the previous paragraph, and
No. It provides a forward-compatibility path for the distutils, so
that easy_install doesn't have to install things in .egg format in
the future. There's no compatibility breakage at all.
>2. It defines a new database file,
No, it defines *files*. One file per installed distribution,
containing (among other things) an installation manifest.
>where I would prefer either:
> a. Doing away with database files entirely and relying on the
>filesystem alone to hold that information, or
...which is what PEP 262 *does*.
Unfortunately, PEP 262's title is bad for marketing, as you've
effectively pointed out. It would be better titled something
"package installation manifests" or "package contents files", or
something of that sort.
> b. Continuing to use the current ".pth" database file format,
>possibly improved by having native support for .pth files in the
>Python import machinery.
These mechanisms are orthogonal to this issue.
>3. Because of #2, it triggers programmers to exclaim "They are
>planning to reinvent apt!", thus making it unlikely that the new
>proposal will recapture the cooperation that setuptools has already
Yeah, we need a new name. Everybody is going off of "database of
installed packages" and thinking "apt", because they aren't paying
any closer attention. However, given that we are discussing this on
Python-Dev and distutils-sig, I do think it's reasonable to expect
(if perhaps not reasonable to receive) that people discussing the PEP
have *read* the freaking PEP first, prior to trashing it or offering
And it's not like I'm personally offended or anything -- I didn't
even write the PEP in question. But what's the point of having PEPs
if people read nothing but their titles? We could just delete
everything but PEP 0. :)
>Perhaps PEP 262 and my proposal are not actually alternatives, but
As I've already pointed out, your proposal does not address multiple
installed versions of a package, and I see no sane way to modify it to do so.
>What I want is for the already implemented, tested, and deployed
>code- re-use features of setuptools/easy_install to be more widely
>accepted. This is best and most easily achieved by fixing the two
>most frequent objections to setuptools/easy_install: 1. That you
>can't conveniently install into an arbitrary directory, and 2. that
>it subverts the meaning of your PYTHONPATH.
As I've already stated, the only way for these problems to be fixed
is for easy_install to not install files in .egg form -- which also
solves the general objection to using .eggs in the first place.
And the only way to do that, is to have a way to keep track of what
files are installed. Rather than have easy_install come up with its
own way of doing that, I would prefer to share a standard with the
distutils. Hence, the PEP discussion.
For earlier versions of Python, it will still be possible to install
and uninstall with setuptools using this approach. You just won't be
able to uninstall pure distutils-based packages, unless you installed
them using easy_install.
Meanwhile, it has occurred to me that the easiest way of handling
compatibility is not to require that other packaging tools mark their
files for non-removability, but simply not allow easy_install to
remove or overwrite anything that *isn't* claimed by a manifest. In
that way, easy_install would be immediately usable in the new mode,
without any updates to Python or to system packaging tools.
On Wed, April 9, 2008 4:27 pm, distutils-sig-request(a)python.org wrote:
> Message: 5
> Date: Wed, 9 Apr 2008 21:21:09 +0100
> From: Floris Bruynooghe <floris.bruynooghe(a)gmail.com>
> Subject: Re: [Distutils] how to easily consume just the parts of eggs
> that are good for you
> To: distutils-sig(a)python.org
> On Wed, Apr 09, 2008 at 02:26:31PM -0400, Stanley A. Klein wrote:
>> I don't know what Windows add/remove
>> programs function does, but all it might do is to run the executable to
>> install packages and record the installation (as was previously done by
>> third party programs) to facilitate clean removal. Unless you can
>> more of the other functions I listed above, I doubt I would call
>> add/remove a package manager.
> Ugh, you have yet to discover the horrors/wonders of MSI (I wish I was
> as naive as you here!). A properly installed windows program will
> install using an MSI database, registering each file, registry setting
> etc. Often a setup.exe will still interface with the MSI database in
> the background (they should, there's a C API for it too). MSI will
> even do stuff like reference counting how many programs need a certain
> file (in case you have something installed in a shared directory),
> figure out what to do on conflict etc. They have many anc complicated
> rules, options and features.
> As far as I am concerned MSI (and thus Add/Remove Programs) can be
> treated as a package manager in windows.
I have 3 desktops and a laptop. They are all at least dual boot. One of
the systems on each machine is Windows. All the others are Linux,
including Fedora 7, Fedora Core 5, Ubuntu 7, or CENTOS 5 in some
combination on each machine. My greatest use of Windows is at tax time,
because the good tax programs aren't released for Linux. I also have a
mid-1990's version of QuickBooks that I still use. Aside from those
applications, my use of Windows is sporadic at best, maybe a few times
every few months.
I do everything else in Linux. My exposure to Windows is minimal, so my
exposure to MSI is even less. I wouldn't call it naivete. I just don't
do Windows. :-)
I'll continue my fool hearty effort  to build a concrete proposal
for "a database of installed packages" by offering up a sketch of a
possible straw-man "solution". I realize that this is likely
oversimplified to a fault, but I hope it will help us move forward.
Apologies if the equivalent of this has been proposed and rejected
before. My proposal is basically to make PKG-INFO functional and
* Fixing the technical issues with requirements (i.e. dependencies)
and naming as specified by PEP 314/345.
* Modifying distutils to install PKG-INFO alongside each module file
or package directory as a side-car file of the same name but with a
special extension (.pyi or whatever). These files would be the place
to include the optional list of installed files as well as the
optional md5sums, if desired by the installer. Files in the package
will be listed using relative paths, while far flung files (bin,
shared, etc) will get full paths so that there is full allowance for
relocating simple (nothing in bin or shared) modules and packages.
Although optional, "python setup.py install" will include the
installed file list by default.
That's it. The intent is to provide just enough information to allow
the development of tools to use it, for those that are interested,
while being minimally invasive to developers that are not interested
in such tools. To determine the current state of your python
environment, walk sys.path looking for modules and packages,
collecting PKG-INFO when available. No standard centralized database.
Some of us will choose to opt-in to a particular installation
management tool that might maintain a cache (centralized or
per-directory) for efficiency, but that would be considered a
performance optimization for that particular tool.
We can also bootstrap older python installations by creating an online
database (that can of course be downloaded by security conscious
individuals for offline querying) that maps
(module-file/package-directory name, md5sum) pairs to their respective
PKG-INFO contents (no list of installed files) which can be queried by
an automated sys.path walker to fill-in missing side-car files. Thus,
I can opt-in to this scheme for python 2.5 by installing a distutils
patch and meta-data side-car bootstrapper that does its best to
identify what's on my sys.path. It would be quite tractable to
maintain this for the python standard library and perhaps the official
installations of a few major OS versions. Such a database could even
be used for the community to provide metadata for packages that the
developer didn't (again, furthering an opt-in mentality). Of course,
even though it worked for CDDB, it would likely be too much to expect
this level of coverage through user submitted entries.
On Tue, April 8, 2008 9:37 pm, Ben Finney
> Date: Wed, 09 Apr 2008 11:37:07 +1000
> From: Ben Finney <bignose+hates-spam(a)benfinney.id.au>
> Subject: Re: [Distutils] how to easily consume just the parts of eggs
> that are good for you
> To: Distutils-Sig(a)Python.Org
> zooko <zooko(a)zooko.com> writes:
>> I am skeptical that prorgammers are going to be willing to use a new
>> database format. They already have a database -- their filesystem --
>> and they already have the tools to control it -- mv, rm, and
>> PYTHONPATH. Many of them already hate the existence the
>> "easy_instlal.pth" database file, and I don't see why a new database
>> file would be any different.
> Moreover, many of us already have a database of *all* packages on the
> system, not just Python-language ones: the package database of our
> operating system. Adding another, parallel, database which needs
> separate maintenance, and only applies to Python packages, is not a
> step forward in such a situation.
>> They both agreed that it made perfect sense. I told one of them
>> about the alternate proposal to define a new database file to
>> contain a list of installed packages, and he sighed and rolled his
>> eyes and said "So they are planning to reinvent apt!".
> That's pretty much my reaction, too.
I have the same reaction. I don't install eggs (unless they are installed
through my operating system package manager) or use easy_install. My
systems have rpms (for Fedora and CENTOS) and debs (for Ubuntu). There
are also a Linux Standards Base and a Unix Filesystem Hierarchy Standard
(cited by the LSB) that rpms and debs generally enforce, but eggs often do
I have tried in the past to use easy_install, but have run into problems
because there is no communication between easy_install and the rpm
database, resulting in failure of easy_install to recognize that
dependencies have already been installed using rpms. Also, there are
tools provided with rpm and apt to perform functions such as querying
installed packages for their contents. I use "rpm -qil" frequently to
find what files a package has installed on my system and where they are
A database focused only on Python packages is highly inappropriate for
Linux systems, violates the Linux standards, and creates problems because
eggs are not coordinated with the operating system package manager. The
way to achieve a database for Python would be to provide tools for
conversion of eggs to rpms and debs, to have eggs support conformance to
the LSB and FHS, and to use rpm or apt for package management.
I've just added this patch which adds basic UAC support to bdist_wininst in a fairly unobtrusive manner. I've added Thomas to the nosy list, but I'd welcome any feedback or concerns from everyone.
From: Mark Hammond [mailto:email@example.com]
Sent: Tuesday, 8 April 2008 12:43 PM
Subject: [issue2581] Vista UAC/elevation support for bdist_wininst
New submission from Mark Hammond <mhammond(a)users.sourceforge.net>:
The attached patch adds basic UAC support to bdist_wininst created
installers. A new option '--user-access-control' has been added to
bdist_wininst, which is written to the INI file read by the installer.
The installer reads this value: if it is 'force', elevation is always
requested, if it is 'auto', elevation is requested when Python is
installed in HKLM. 'none' (the default) means nothing UAC related
happens at all.
The elevation happens by having the installer re-execute itself using
I've also ensured the code builds for all versions of VS we support. As
a result, it was necessary to change the old bdist_wininst project files
to point to the later zlib version Python itself uses. All these
changes are in the patch.
keywords: patch, patch
nosy: mhammond, theller
title: Vista UAC/elevation support for bdist_wininst
versions: Python 2.6
Added file: http://bugs.python.org/file9978/bdist_wininst_uac.patch
I'm using the orginal Enthought suite 2.4.3
and added Pygame to it.
Now everything works fine, until ..
... I try to build a windows executable with Py2Exe.
All kinds of numarray modules are not detected by Py2Exe.
Adding the missing numarray (which one, Enthought contains more of them),
later on manual to the dirstibution didn't help.
Does anyone recognize this problem,
and maybe even have a solution ?
As I was preparing to check-in my cross-compilation patch, which includes a
new x64 executable in the Lib/distutils/command directory, it occurs to me
that we don't actually need it in subversion - and nor do we need
I believe these executables are in svn for historical reasons. The build
process for these executables are now integrated into the core Python build
process (although they are disabled by default). There is a good use-case
for keeping executables built with older MSVC versions, but I don't see one
for executables built with the current version.
I'd like to propose we delete Lib/Distutils/command/wininst-9.0.exe, and
enable the building of that project by default in the standard build process
(and I'll setup the x64 build of the executable similarly). They will then
need to be re-added when a future version moves away from Visual Studio
2008, but that seems like a reasonable way to approach things. Are there
any objections to this, or should we stick with the status-quo and I'll add
the new x64 executable? Or do things that way *just* for x64?
I installed setuptools-0.6c8 in a Python 2.5 AMD64 machine by dropping
its .egg in site-packages, a 'temp.pth' file alongside pointing to it,
and doing "python -m easy_install setuptools".
Was there an easier way?
Sorry I must be naive, but I don't understand how setuptools community works
at this time.
In the README.txt of the package, it is written
"Questions, comments, and bug reports should be directed to the
distutils-sig mailing list..."
But what happens when some bug reports or patches are just ignored in the
mailing list ? (even after months)
I would expect patch or bug reports to be added in a bug tracker, then
rejected or accepted, even if it takes
months for them to be treated. At least, registered somewhere.
I have read that a bug tracker was going to be set for setuptools, but the
current status is rather fuzzy,
so I was just wondering if this is stilled planned...
Tarek Ziadé | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/