Mailman 3 April 2008 - Distutils-SIG

Distutils setting permissions during build
by Reuben Thomas April 10, 2008

April 10, 2008

Hi, This is the only obvious up-to-date address or mailing list I could find, so feel free to direct me where I should go. I'm a Linux user, and normally I have a 0027 umask. This means that when I build a Python package using distutils, the files in the build directory are not world-readable (Python copies them there without preserving permissions, according to the comment in build_py.py: # XXX copy_file by default preserves mode, which appears to be the # wrong thing to do: if a file is read-only in the working # directory, we want it to be installed read/write so that the next # installation of the same module distribution can overwrite it # without problems. (This might be a Unix-specific issue.) Thus # we turn off 'preserve_mode' when copying to the build directory, # since the build directory is supposed to be exactly what the # installation will look like (ie. we preserve mode when # installing). This is all very well, but turning off "preserve_mode" is not enough to ensure this in the presence of a restrictive umask like mine; the permissions need rather to be explicitly set (as done for example by the GNU install tool, though that assumes that permissions are set at install time rather than build time). The obvious workaround is for me to remember to change my umask when I'm building python packages. That's OK, but is this problem recognised as such and might it be fixed in future? -- http://rrt.sc3d.org/ | God is the name of our ignorance (Grayling)

1 0

Re: [Distutils] [Python-Dev] how to easily consume just the parts of eggs that are good for you
by Phillip J. Eby April 10, 2008

April 10, 2008

At 10:30 AM 4/9/2008 -0700, zooko wrote: >PEP 262 sounds like a non-starter to me because > >1. It appears to be backwards-incompatible with setuptools/ >easy_install/eggs, thus losing all the recently gained cooperation >that I mentioned in the previous paragraph, and No. It provides a forward-compatibility path for the distutils, so that easy_install doesn't have to install things in .egg format in the future. There's no compatibility breakage at all. >2. It defines a new database file, No, it defines *files*. One file per installed distribution, containing (among other things) an installation manifest. >where I would prefer either: > a. Doing away with database files entirely and relying on the >filesystem alone to hold that information, or ...which is what PEP 262 *does*. Unfortunately, PEP 262's title is bad for marketing, as you've effectively pointed out. It would be better titled something "package installation manifests" or "package contents files", or something of that sort. > b. Continuing to use the current ".pth" database file format, >possibly improved by having native support for .pth files in the >Python import machinery. These mechanisms are orthogonal to this issue. >3. Because of #2, it triggers programmers to exclaim "They are >planning to reinvent apt!", thus making it unlikely that the new >proposal will recapture the cooperation that setuptools has already >(slowly) gained. Yeah, we need a new name. Everybody is going off of "database of installed packages" and thinking "apt", because they aren't paying any closer attention. However, given that we are discussing this on Python-Dev and distutils-sig, I do think it's reasonable to expect (if perhaps not reasonable to receive) that people discussing the PEP have *read* the freaking PEP first, prior to trashing it or offering alternatives. And it's not like I'm personally offended or anything -- I didn't even write the PEP in question. But what's the point of having PEPs if people read nothing but their titles? We could just delete everything but PEP 0. :) >Perhaps PEP 262 and my proposal are not actually alternatives, but >are complementary. As I've already pointed out, your proposal does not address multiple installed versions of a package, and I see no sane way to modify it to do so. >What I want is for the already implemented, tested, and deployed >code- re-use features of setuptools/easy_install to be more widely >accepted. This is best and most easily achieved by fixing the two >most frequent objections to setuptools/easy_install: 1. That you >can't conveniently install into an arbitrary directory, and 2. that >it subverts the meaning of your PYTHONPATH. As I've already stated, the only way for these problems to be fixed is for easy_install to not install files in .egg form -- which also solves the general objection to using .eggs in the first place. And the only way to do that, is to have a way to keep track of what files are installed. Rather than have easy_install come up with its own way of doing that, I would prefer to share a standard with the distutils. Hence, the PEP discussion. For earlier versions of Python, it will still be possible to install and uninstall with setuptools using this approach. You just won't be able to uninstall pure distutils-based packages, unless you installed them using easy_install. Meanwhile, it has occurred to me that the easiest way of handling compatibility is not to require that other packaging tools mark their files for non-removability, but simply not allow easy_install to remove or overwrite anything that *isn't* claimed by a manifest. In that way, easy_install would be immediately usable in the new mode, without any updates to Python or to system packaging tools.

2 2

Re: [Distutils] how to easily consume just the parts of eggs that are good for you
by Stanley A. Klein April 10, 2008

April 10, 2008

On Wed, April 9, 2008 4:27 pm, distutils-sig-request(a)python.org wrote: > Message: 5 > Date: Wed, 9 Apr 2008 21:21:09 +0100 > From: Floris Bruynooghe <floris.bruynooghe(a)gmail.com> > Subject: Re: [Distutils] how to easily consume just the parts of eggs > that are good for you > To: distutils-sig(a)python.org > > On Wed, Apr 09, 2008 at 02:26:31PM -0400, Stanley A. Klein wrote: >> I don't know what Windows add/remove >> programs function does, but all it might do is to run the executable to >> install packages and record the installation (as was previously done by >> third party programs) to facilitate clean removal. Unless you can >> perform >> more of the other functions I listed above, I doubt I would call >> add/remove a package manager. > > Ugh, you have yet to discover the horrors/wonders of MSI (I wish I was > as naive as you here!). A properly installed windows program will > install using an MSI database, registering each file, registry setting > etc. Often a setup.exe will still interface with the MSI database in > the background (they should, there's a C API for it too). MSI will > even do stuff like reference counting how many programs need a certain > file (in case you have something installed in a shared directory), > figure out what to do on conflict etc. They have many anc complicated > rules, options and features. > > As far as I am concerned MSI (and thus Add/Remove Programs) can be > treated as a package manager in windows. > I have 3 desktops and a laptop. They are all at least dual boot. One of the systems on each machine is Windows. All the others are Linux, including Fedora 7, Fedora Core 5, Ubuntu 7, or CENTOS 5 in some combination on each machine. My greatest use of Windows is at tax time, because the good tax programs aren't released for Linux. I also have a mid-1990's version of QuickBooks that I still use. Aside from those applications, my use of Windows is sporadic at best, maybe a few times every few months. I do everything else in Linux. My exposure to Windows is minimal, so my exposure to MSI is even less. I wouldn't call it naivete. I just don't do Windows. :-) Stan Klein

1 0

A Modest Proposal for "A Database of Installed Packages"
by Alexander Michael April 10, 2008

April 10, 2008

I'll continue my fool hearty effort [1] to build a concrete proposal for "a database of installed packages" by offering up a sketch of a possible straw-man "solution". I realize that this is likely oversimplified to a fault, but I hope it will help us move forward. Apologies if the equivalent of this has been proposed and rejected before. My proposal is basically to make PKG-INFO functional and usable by: * Fixing the technical issues with requirements (i.e. dependencies) and naming as specified by PEP 314/345. * Modifying distutils to install PKG-INFO alongside each module file or package directory as a side-car file of the same name but with a special extension (.pyi or whatever). These files would be the place to include the optional list of installed files as well as the optional md5sums, if desired by the installer. Files in the package will be listed using relative paths, while far flung files (bin, shared, etc) will get full paths so that there is full allowance for relocating simple (nothing in bin or shared) modules and packages. Although optional, "python setup.py install" will include the installed file list by default. That's it. The intent is to provide just enough information to allow the development of tools to use it, for those that are interested, while being minimally invasive to developers that are not interested in such tools. To determine the current state of your python environment, walk sys.path looking for modules and packages, collecting PKG-INFO when available. No standard centralized database. Some of us will choose to opt-in to a particular installation management tool that might maintain a cache (centralized or per-directory) for efficiency, but that would be considered a performance optimization for that particular tool. We can also bootstrap older python installations by creating an online database (that can of course be downloaded by security conscious individuals for offline querying) that maps (module-file/package-directory name, md5sum) pairs to their respective PKG-INFO contents (no list of installed files) which can be queried by an automated sys.path walker to fill-in missing side-car files. Thus, I can opt-in to this scheme for python 2.5 by installing a distutils patch and meta-data side-car bootstrapper that does its best to identify what's on my sys.path. It would be quite tractable to maintain this for the python standard library and perhaps the official installations of a few major OS versions. Such a database could even be used for the community to provide metadata for packages that the developer didn't (again, furthering an opt-in mentality). Of course, even though it worked for CDDB, it would likely be too much to expect this level of coverage through user submitted entries. [1] http://mail.python.org/pipermail/distutils-sig/2008-March/009108.html

3 7

Re: [Distutils] how to easily consume just the parts of eggs that are good for you
by Stanley A. Klein April 9, 2008

April 9, 2008

On Tue, April 8, 2008 9:37 pm, Ben Finney <bignose+hates-spam(a)benfinney.id.au> wrote: > Date: Wed, 09 Apr 2008 11:37:07 +1000 > From: Ben Finney <bignose+hates-spam(a)benfinney.id.au> > Subject: Re: [Distutils] how to easily consume just the parts of eggs > that are good for you > To: Distutils-Sig(a)Python.Org > > > zooko <zooko(a)zooko.com> writes: > >> I am skeptical that prorgammers are going to be willing to use a new >> database format. They already have a database -- their filesystem -- >> and they already have the tools to control it -- mv, rm, and >> PYTHONPATH. Many of them already hate the existence the >> "easy_instlal.pth" database file, and I don't see why a new database >> file would be any different. > > Moreover, many of us already have a database of *all* packages on the > system, not just Python-language ones: the package database of our > operating system. Adding another, parallel, database which needs > separate maintenance, and only applies to Python packages, is not a > step forward in such a situation. > >> They both agreed that it made perfect sense. I told one of them >> about the alternate proposal to define a new database file to >> contain a list of installed packages, and he sighed and rolled his >> eyes and said "So they are planning to reinvent apt!". > > That's pretty much my reaction, too. I have the same reaction. I don't install eggs (unless they are installed through my operating system package manager) or use easy_install. My systems have rpms (for Fedora and CENTOS) and debs (for Ubuntu). There are also a Linux Standards Base and a Unix Filesystem Hierarchy Standard (cited by the LSB) that rpms and debs generally enforce, but eggs often do not. I have tried in the past to use easy_install, but have run into problems because there is no communication between easy_install and the rpm database, resulting in failure of easy_install to recognize that dependencies have already been installed using rpms. Also, there are tools provided with rpm and apt to perform functions such as querying installed packages for their contents. I use "rpm -qil" frequently to find what files a package has installed on my system and where they are installed. A database focused only on Python packages is highly inappropriate for Linux systems, violates the Linux standards, and creates problems because eggs are not coordinated with the operating system package manager. The way to achieve a database for Python would be to provide tools for conversion of eggs to rpms and debs, to have eggs support conformance to the LSB and FHS, and to use rpm or apt for package management. Stan Klein

1 0

FW: [issue2581] Vista UAC/elevation support for bdist_wininst
by Mark Hammond April 8, 2008

April 8, 2008

Hi all, I've just added this patch which adds basic UAC support to bdist_wininst in a fairly unobtrusive manner. I've added Thomas to the nosy list, but I'd welcome any feedback or concerns from everyone. Thanks, Mark -----Original Message----- From: Mark Hammond [mailto:report@bugs.python.org] Sent: Tuesday, 8 April 2008 12:43 PM To: mhammond(a)users.sourceforge.net Subject: [issue2581] Vista UAC/elevation support for bdist_wininst New submission from Mark Hammond <mhammond(a)users.sourceforge.net>: The attached patch adds basic UAC support to bdist_wininst created installers. A new option '--user-access-control' has been added to bdist_wininst, which is written to the INI file read by the installer. The installer reads this value: if it is 'force', elevation is always requested, if it is 'auto', elevation is requested when Python is installed in HKLM. 'none' (the default) means nothing UAC related happens at all. The elevation happens by having the installer re-execute itself using ShellExecute. I've also ensured the code builds for all versions of VS we support. As a result, it was necessary to change the old bdist_wininst project files to point to the later zlib version Python itself uses. All these changes are in the patch. ---------- assignee: mhammond components: Distutils files: bdist_wininst_uac.patch keywords: patch, patch messages: 65139 nosy: mhammond, theller severity: normal status: open title: Vista UAC/elevation support for bdist_wininst type: behavior versions: Python 2.6 Added file: http://bugs.python.org/file9978/bdist_wininst_uac.patch __________________________________ Tracker <report(a)bugs.python.org> <http://bugs.python.org/issue2581> __________________________________

1 0

Py2exe, on PyGame gives problems ...
by Stef Mientki April 7, 2008

April 7, 2008

hello, I'm using the orginal Enthought suite 2.4.3 and added Pygame to it. Now everything works fine, until .. ... I try to build a windows executable with Py2Exe. All kinds of numarray modules are not detected by Py2Exe. Adding the missing numarray (which one, Enthought contains more of them), later on manual to the dirstibution didn't help. Does anyone recognize this problem, and maybe even have a solution ? thanks, Stef Mientki

1 0

Remove "current" Windows executables from Lib/distutils/command in svn?
by Mark Hammond April 7, 2008

April 7, 2008

As I was preparing to check-in my cross-compilation patch, which includes a new x64 executable in the Lib/distutils/command directory, it occurs to me that we don't actually need it in subversion - and nor do we need wininst-9.0.exe. I believe these executables are in svn for historical reasons. The build process for these executables are now integrated into the core Python build process (although they are disabled by default). There is a good use-case for keeping executables built with older MSVC versions, but I don't see one for executables built with the current version. I'd like to propose we delete Lib/Distutils/command/wininst-9.0.exe, and enable the building of that project by default in the standard build process (and I'll setup the x64 build of the executable similarly). They will then need to be re-added when a future version moves away from Visual Studio 2008, but that seems like a reasonable way to approach things. Are there any objections to this, or should we stick with the status-quo and I'll add the new x64 executable? Or do things that way *just* for x64? Thanks, Mark.

5 6

[setuptools] Windows AMD64 installation
by Luis Bruno April 7, 2008

April 7, 2008

Hi all, I installed setuptools-0.6c8 in a Python 2.5 AMD64 machine by dropping its .egg in site-packages, a 'temp.pth' file alongside pointing to it, and doing "python -m easy_install setuptools". Was there an easier way?

1 0

[setuptools] how to report bugs and patchs ?
by Tarek Ziadé April 6, 2008

April 6, 2008

Hello, Sorry I must be naive, but I don't understand how setuptools community works at this time. In the README.txt of the package, it is written "Questions, comments, and bug reports should be directed to the distutils-sig mailing list..." But what happens when some bug reports or patches are just ignored in the mailing list ? (even after months) I would expect patch or bug reports to be added in a bug tracker, then rejected or accepted, even if it takes months for them to be treated. At least, registered somewhere. I have read that a bug tracker was going to be set for setuptools, but the current status is rather fuzzy, so I was just wondering if this is stilled planned... Regards, Tarek -- Tarek Ziadé | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/

2 1