Mailman 3 December 2005 - Distutils-SIG

Setuptools' goal in rpm/deb-based systems
by Jeff Pitman 29 Dec '05

29 Dec '05

Hi, This may have been discussed ad nauseum already, but as I tried to package turbogears for Fedora/CentOS, I thought I'd throw in 2 cents. From a .deb and .rpm perspective, it is highly discouraged to install non-system managed files into non-$HOME areas (esp. /usr). Even for many, /usr/local is highly frowned upon (eg. ./configure; make; make install mantra). I see that --single-version-externally-managed has been added as an option so that .deb/.rpm packaging can take advantage of that relying upon apt/yum/smart to take care of business. Hopefully, this will be pushed out soon. (Because turbogears script generation is not trivial to package. I think I'll just include a generated script as one of the Sources, for now.) I think the goal for setuptools on rpm/deb-based systems should focus on getting $HOME setup and managed correctly for packages not provided by the system. For example, if I had an ISP which provided python2.4, but not TurboGears, then I could use .egg to manage a setup in $HOME to use the packages correctly. Currently, -d/-s and other options can be used. But, if the user isn't root, maybe it should be the default behavior: install and manage in $HOME. Also, are there efforts to detect if a package is already installed? For example, if "python-sqlobject" is installed, can setuptools detect the lib if the install_requires has "SQLObject"? I think peering between .egg and .deb/.rpm to be a difficult problem. So, having it all or nothing would be a cleaner policy. .deb/.rpm to manage /usr installed packages/modules and .egg to manage $HOME installed packages. Thoughts? -- -jeff

2 5

Re: [Distutils] [build_scripts] executable
by Phillip J. Eby 26 Dec '05

26 Dec '05

At 10:52 AM 12/26/2005 -0800, Brian Zhou wrote: >Never mind. I think it's problem of the way turbogears uses setuptools, in >the case of sqlobject for example, it works ok. The issue is that setuptools-generated scripts were not respecting build_scripts --executable. I've fixed that now. Projects that don't use entry-point scripts wouldn't have been affected, and projects with both old and new-style scripts would have had some scripts work and some not.

1 0

Re: [Distutils] [build_scripts] executable
by Phillip J. Eby 26 Dec '05

26 Dec '05

At 10:41 AM 12/26/2005 -0800, Brian Zhou wrote: >Hi all, > >I'm working on some system packages for ><http://nslu2-linux.org>http://nslu2-linux.org optware with setuptools >(svn dev so I can use the --single-version-externally-managed). Thanks to >your hard work, so far it looks very promising. > >However, since the platform I'm working on requires cross build, the >python on debian host /usr/bin/python2.4 should not be used for scripts in >the package. In particular we need all scripts in the package to have > > #!/opt/bin/python > >As the first line. With traditional distutils, we can append the following >to setup.cfg > > [build_scripts] > executable=/opt/bin/python > >Can we add that back please? I've checked a fix in now. I didn't know that option existed. Note that it needs to be specified in the config file or on the command line at the time install_scripts is run, since setuptools-generated scripts don't have a separate "build" step. Also, easy_install does not respect this option; I haven't evaluated yet whether it should, and the change is more complex. So at the moment, I've only changed it for "install --single-version-externally-managed".

1 0

[build_scripts] executable
by Brian Zhou 26 Dec '05

26 Dec '05

Hi all, I'm working on some system packages for http://nslu2-linux.org optware with setuptools (svn dev so I can use the --single-version-externally-managed). Thanks to your hard work, so far it looks very promising. However, since the platform I'm working on requires cross build, the python on debian host /usr/bin/python2.4 should not be used for scripts in the package. In particular we need all scripts in the package to have #!/opt/bin/python As the first line. With traditional distutils, we can append the following to setup.cfg [build_scripts] executable=/opt/bin/python Can we add that back please? Thanks, -Brian Zhou

1 1

Re: [Distutils] Multiple version support (default installs and plugins)
by Phillip J. Eby 23 Dec '05

23 Dec '05

At 02:13 PM 12/22/2005 +0000, Paul Moore wrote: >On 12/22/05, M.-A. Lemburg <mal(a)egenix.com> wrote: > > Ian Bicking wrote: > > > I'm still finding it impossible to use multiple versions of a package > > > unless none of them show up in a .pth file (i.e., none are available > > > without requiring), I get a VersionConflict. > > > > This comes up every now and then when discussing the benefits > > of easy_install. > > > > I'd really like to understand what the use case is for having > > multiple versions of a package around. > > > > Note that sys.modules (the registry of loaded modules) does > > not support having multiple versions of a module loaded. > >I think the point is to allow multiple versions to be installed on the >system, but allow an individual program, at run time, to specify which >one is to be loaded. There will never be more than one version loaded >in any specific instance, but if program A requires version 1 of >package X, and program B requires version 2 of package X, both can run >without needing a reinstall of package X in between. > >Of course, this is also possible via PYTHONPATH manipulations, or >sys.path modifications in the program before doing the import. All >setuptools is doing is to provide a common infrastructure for handling >this. Yes, Paul is correct; supporting multiple programs that use different versions of a module is indeed the primary use case. Also, during development and debugging it can be quite convenient to switch versions of a dependency back and forth to identify the source of a problem.

2 2

Egg support for system packages (including bdist_wininst)
by Phillip J. Eby 23 Dec '05

23 Dec '05

Setuptools in SVN now provides preliminary support for installing eggs in .egg-info directory style, so that setuptools-based projects can be wrapped by system packagers who wish to avoid using .egg files or directories. In addition, you can now use setuptools to install non-setuptools packages in such a way that an .egg-info directory is included, so that setuptools-based packages can detect the presence of the non-setuptools-based package. If you are building packages using a distutils command such as 'bdist_wininst' that internally invokes the distutils "install" command, you do not need to do anything special. Setuptools detects when the "install" command is invoked programmatically (via run_command()) and automatically enters its backward-compatibility mode. However, if you are using a command such as 'bdist_rpm' that generates external scripts which then run "setup.py install", you will need to modify the generated scripts to run this instead: setup.py install --single-version-externally-managed You will also need to include the --root or --record options. For the 'bdist_rpm' command, setuptools automatically adjusts the 'spec' files before they're written, so bdist_rpm *should* work correctly, although I haven't tested it yet. Please let me know if you find any problems, as I have not yet tested this installation mode with anything but bdist_wininst or direct manual installation. For persons like Paul Moore who want to wrap egg-based packages in Windows installers, note that you can now take a setuptools-based package's source and run "setup.py bdist_wininst" and it will build a usable .exe. In addition, you can build Windows installer-wrapped eggs for non-setuptools packages by changing your command line from: python setup.py bdist_wininst to: python -c "import setuptools;execfile('setup.py')" bdist_wininst This will build an .exe that includes .egg-info, so that other packages will be able to detect it at runtime. Similarly, if you are building any other kind of system package using "python setup.py install", you can force .egg-info to be included by using this instead: python -c "import setuptools;execfile('setup.py')" install \ --single-version-externally-managed ... This support for system packages of eggs is still early and experimental; please let me know about any problems you may encounter or any questions you may have. Thanks.

3 9

Generation of script during installation
by Giovanni Bajo 22 Dec '05

22 Dec '05

Hello, I have a problem with distutils. During installation, I need to generate a Python module containing information about the target installation paths. For instance, if an user does: setup.py install --install-headers=/foo my module will have to contain something along the lines of: headers= "/foo" I managed to do this by subclassing install_lib (overriding method run) and generating the module there. I access the target paths with things like self.get_finalized_command("install_headers").install_dir. The module is created into the build_dir, so that install_lib.run() then copy it together with the other modules at the correct location. This works like a charm *but* breaks when doing a bdist_wininst (and probably a bdist_rpm, I haven't checked). I'm not sure how to deal with that, as the final paths will be decided at installation time (depending on the user's system), and that's outside of the scope of my setup.py script. Suggestions? For instance, is there a way so that the Windows installer generate an installation log, which my Python script can parse to extract the directory information? -- Giovanni Bajo

4 6

bdist_nsis, bdist_deb, bdist_msi, bdist_inno, bdist_pkgtool, bdist_sdux, etc.
by M.-A. Lemburg 22 Dec '05

22 Dec '05

Hi, I'd like to focus attention on these distribution formats that distutils is missing: bdist_nsis: Nullsoft installer for Windows (NSIS); by Jean-Christophe G. http://mail.python.org/pipermail/distutils-sig/2005-July/004724.html http://bdist-nsi.sourceforge.net/ bdist_deb: Debian package; by Geoffrey T. Dairiki http://lists.debian.org/debian-python/2004/10/msg00017.html bdist_msi: MS Installer (MSI); by Martin von Löwis http://mail.python.org/pipermail/distutils-sig/2005-August/005008.html bdist_inno: InnoSetup installer for Windows Included in py2exe by Thomas Heller: http://starship.python.net/crew/theller/py2exe/ bdist_pkgtool: Package format used on Solaris There was a code contribution by Mark Alexander for this, but it had to be removed due to licensing issues: http://cvs.sourceforge.net/viewcvs.py/python/python/dist/src/Lib/distutils/… bdist_sdux: Package format used on HP-UX Same issues as with bdist_pkgtool. Ideally, I'd like to see all of these added to distutils. The standard procedure for this is to: 1. Find a maintainer for the code; ideally the author him/herself 2. Get the author to sign the PSF contribution form: http://www.python.org/psf/contrib.html 3. Add the code to distutils Since these are plain additions to distutils, I don't think we need a PEP for the integration. Would any of the authors (I've tried to CC them) be willing to contribute their distutils command to the PSF for integration ? Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Dec 13 2005) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

8 24

Re: [Distutils] Generation of script during installation
by Phillip J. Eby 22 Dec '05

22 Dec '05

At 04:06 PM 12/21/2005 -0800, Bob Ippolito wrote: > > OK, so you're suggesting to install two copies of the same files: one in a > > fixed location (within site-packages) so that it's always accessible, and > > a > > copy for the user wherever he wants. This might work for my case, but how > > do I > > implement it? Do I need to override most of the install_* cmd to implement > > this > > logic, or there is some trick I can exploit? > >Why not just include the header files in both install_headers and >install_data (or equivalent)? Well, if you're using Python 2.4 (or Python 2.3 with setuptools), just put the header files inside your package directories and use: package_data = {'': ['*.h']}, headers = ['srcdir/mypkg/foo.h', 'srcdir/mypkg/bar.h', ...], to include all .h files that are found in your package directories, and to also install the same files as headers.

1 0

Re: [Distutils] bdist_nsis, bdist_deb, bdist_msi, bdist_inno, bdist_pkgtool, bdist_sdux, bdist_mpkg, etc.
by Phillip J. Eby 21 Dec '05

21 Dec '05

At 01:09 PM 12/20/2005 -0800, Bob Ippolito wrote: >The ez_setup bootstrap >definitely checks the md5 for known versions of setuptools, and I'm >assuming that a similar feature is in easy_install for normal packages. No need to assume; see the 'check_md5' method of the PackageIndex class in setuptools.package_index. ;) *Any* URL processed by PackageIndex that ends with a "#md5=" anchor will have its md5 verified upon download. If the match fails, the downloaded file is immediately deleted and an error is raised. (In addition, PyPI pages' MD5 sums for downloads are preprocessed into the "#md5=" format using a regular expression search.)

1 0