Mailman 3 October 2005 - Distutils-SIG

Re: [Distutils] setuptools in a cross-compilation packaging environment
by Phillip J. Eby 11 Oct '05

11 Oct '05

At 04:56 PM 10/11/2005 +0100, Paul Moore wrote: >But for general packages (PIL, pywin32, cx_Oracle, whatever), I >actively want a managed installation. I have made noises about >building management tools for eggs, but I think I've been hitting the >"absolutely requiring install/uninstall steps, which is anathema" >philosophy, which makes such tools awkward to define. I don't think that's really the case, it's just that implementing management tools is lower on my to-do list; I don't see them coming in until 0.7 or 0.8, and at the moment I don't have a strong vision of what those commands will look like, except that they'll be things like 'nest rm', 'nest ls', etc. (abbreviated forms) or 'nest uninstall', 'nest list', etc. (unabbreviated). Since I don't have a particularly rigid vision of what those tools will do, I'm wide open to suggestions and especially use cases for what you want to be able to do with them. In the meantime, some of the things that are on my list ahead of the "nest" command subsystem are: * Making it possible for wrapper scripts on Windows to not conflict with a same-named module they import * --no-deps option for easy_install * changing the activation mechanism so eggs get added with higher precedence on sys.path than their containing directory, and that package name conflicts are checked at runtime * Automatic support for creating new "site" directories that allow .pth files, so that Unix users can install eggs anywhere without needing to have root access or create a virtual Python installation. Also, the --allow-hosts option, and maybe even signature checking, may end up landing before the nest command. Meanwhile, there's plenty of room for someone else to create management tools using the existing API, as most everything I've got on my list will not be changing the API or file formats or anything like that.

2 1

setuptools in a cross-compilation packaging environment
by Brian Zhou 11 Oct '05

11 Oct '05

Hi all, I am the package maintainer of python and a couple of python packages for the nslu2 optware platform, see http://www.nslu2-linux.org and http://ipkg.nslu2-linux.org/feeds/unslung/cross/ There were some distutils features I really appreciate: 1) setup.py install --prefix This allows us to install into a separate staging area. 2) setup.cfg [build_scripts] executable option 3) setup.cfg [build_ext] include-dirs, library-dirs and rpath options The new setuptools is all nice and easy for end user, but as a package maintainer, I'd like to have the option of building a binary package without all the dependencies. Is it possible? How about the above distutils features, are there any equivalent? Thanks in advance for any advice, -Brian Zhou

5 9

linking modules to a shared library
by Stephen Langer 11 Oct '05

11 Oct '05

Hi -- I have a problem that is similar to one discussed in a thread from a year ago, http://mail.python.org/pipermail/distutils-sig/2004-September/ 004160.html, but that thread doesn't quite have a resolution in the archives. I'd like to use distutils to build a bunch of python modules that all link to the same shared library. The shared library is all C++ code. On Linux, without distutils, I can build the libraries like this: g++ -o libBase.so -shared base.C # builds the common shared library g++ -o _mymodule.so -shared mymodule.C -lBase # builds a python module # linking to the shared lib. g++ -o _othermodule.so -shared othermodule.C -lBase # and so on _mymodule.so and _othermodule.so can be imported by Python, and share a single copy of the code in libBase.so. On the Mac I can get the same effect with this: g++ -dynamiclib base.C -o libBase.dylib g++ -o _mymodule.so mymodule.C -lBase -bundle -undefined dynamic_lookup -flat_namespace g++ -o _othermodule.so othermodule.C -lBase -bundle -undefined dynamic_lookup -flat_namespace Is it possible to create libBase portably with distutils? It's possible to do it on Linux by subclassing build_ext.build_ext and explicitly using self.compiler.compile() and self.compiler.link_shared_lib() to build the shared library before calling build_ext.build_ext.build_extensions(). But the same thing on Mac OS X only creates libBase.so, whereas I need it to create libBase.dylib. If it matters, I'm using OS X 10.4.2 and gcc 3.3, with Python 2.3.5. Many thanks, Steve -- -- EMail: stephen.langer(a)nist.gov Phone: (301) 975-5423 -- -- WWW: http://math.nist.gov/mcsd/Staff/SLanger/ Fax: (301) 990-4127 -- -- Mail: NIST; 100 Bureau Drive -- Stop 8910; Gaithersburg, Md 20899-8910 -- -- "I don't think this will work. That's why it's science." -- -- Naomi Langer, 17 Feb 2003 --

3 7

install_lib lib versus lib64
by Matthew Rodriguez DSD staff 07 Oct '05

07 Oct '05

Greetings, I've been working a few different 64 bit machines with a few different python installations. I'm trying to determine a foolproof way to see if when install_lib is run libraries are installed in a lib or lib64 directory. Here is what is the best that I've come up with so far, and it doesn't always work. import distutils.sysconfig libdir = distutils.sysconfig.get_python_lib() libdir.find('lib64') Has anyone else dealt with this, and if so, how did they handle it? Thanks, Matt Rodriguez

1 0

find_package_data function
by Ian Bicking 06 Oct '05

06 Oct '05

I wrote a simple find_package_data function: http://svn.pythonpaste.org/Paste/trunk/paste/util/finddata.py (if you want to use it, you should copy it into your own source) At first I made it just generate a list of wildcards with all the (non-package) subdirectories enumerated, e.g., 'mypackage': ['*.pt', 'templates/*.pt', 'templates/admin/*.pt']. But I decided it's too easy to miss files that way. So now it enumerates everything except files and directories you exclude with patterns (or with explicit paths). There's still no way to include empty directories. You use it like: ... package_data=find_package_data() Which should give a reasonable set. Pass show_ignored=True to see more of what it is doing. -- Ian Bicking / ianb(a)colorstudy.com / http://blog.ianbicking.org

1 0

py2exe has a new maintainer
by Jimmy Retzlaff 05 Oct '05

05 Oct '05

I am taking over the maintenance and support of py2exe from Thomas Heller. As he announced a few weeks ago he is looking to focus on other things. py2exe has been very useful to me over the years and I look forward to keeping it every bit as useful in the future. I plan to make the transition as smooth as possible for users of py2exe. I don't plan to make changes to the license other than adding my name to the list of people not to sue. I will try to be as helpful as Thomas has been in supporting py2exe on the py2exe mailing list and comp.lang.python. The mailing list, the SourceForge project, and the Wiki will continue in their current locations. The web site is moving to http://www.py2exe.org and the old site will forward to the new one so any bookmarks should still work. I will be releasing version 0.6.3 very soon with a few changes Thomas and others have made over the last few weeks. After that my priorities for py2exe will be: - Support - Documentation (which should help familiarize me with the code) - Automated tests (to point out when I haven't familiarized myself enough) - Bug fixes Any help on any of these fronts will be greatly appreciated. After I feel comfortable with things, I hope to work with other projects in the Python packaging community (e.g., cx_Freeze, PyInstaller/McMillan, py2app, setuptools, etc.) to see if we can't find synergies that will make all of them better. I recognize that different packagers are better for different audiences because of licensing, platform, Python versions, and module support among other things. Working together on the common parts (identifying dependencies, customized handling of modules with unique needs, etc.) should make all of the packagers serve their niches better. I'd like to thank Thomas for the great work he's done with py2exe over the years. He's set a very high standard for me to try and maintain. Jimmy

4 3

Re: [Distutils] py2exe has a new maintainer
by Jimmy Retzlaff 05 Oct '05

05 Oct '05

Phillip J. Eby wrote: > At 05:16 PM 10/4/2005 -0700, Bob Ippolito wrote: > >On the other hand, setuptools/eggs solves a lot of these problems > >(given appropriate metadata), so I'd target integration with that > >first and then bring in the "legacy" support that py2app does pretty > >well. Requiring appropriate metadata from the packages themselves > >has a much brighter future than maintaining an external "quirks" > >infrastructure in (each) application packager. I agree with Bob here. I know that in the past I've wanted to add metadata to packages I distribute to make them work better with py2exe (e.g. automatically including DLLs accessed through ctypes), so I think this is the right approach. > Also, setuptools supports adding setup() keywords and commands in a more > extensible way than subclassing Distribution, so there's no need to > maintain a bunch of code to hook into the distutils if you just plug in > with setuptools. See: > > http://peak.telecommunity.com/DevCenter/setuptools#creating-distutils- > extensions > > At this point, the easy_install command is capable of dumping all the > needed eggs and any generated wrapper scripts into a target > directory. What it doesn't have is: > > * frontend wrappers/packaging to actually build the .exe or app and > resources > > * module-finding or stdlib packaging to locate and bundle the needed > portion of the standard library > > It would be cool if these things could be integrated in such a way that a > single setup() could be used to create both an OS X application and a > Windows .exe. As it happens, setuptools supports specifying additional > eggs needed to run a particular setup command, so we can actually delay > the > requirement of the egg containing py2exe or py2app until somebody actually > requests the command. That would be awesome. There might be a few platform specific bits that need to be specified, but it would be nice to let the executable packagers focus on the job of creating a platform specific executable. > On the other hand, it's also possible that the metadata could be bundled > into the egg metadata directory, which would then make it possible to have > external bundling tools that just use the eggs to build the application, > without needing to run the setup script at all. One of my thoughts for py2exe has been a small GUI tool that helps people new to distutils/py2exe to create (and maybe maintain) a setup script graphically. A lot of people who just want to bundle one small utility seem to get confused about options on the command line vs. in parameters and that type of thing. With what you are describing, this could degenerate into just a builder tool, and any GUI could be applied further upstream where more of the metadata lives. If enough of the metadata lives in the individual packages, then the options needed for the executable builder part might be simple enough that the confusion goes away. Alas I won't have time for much of this for a while. I expect I'll have my hands full getting up to speed on py2exe's internals and providing support until I get a break from my current deadlines at work (which is looking like December). When I get to that point, I'll try to reopen the discussion. Regards, Jimmy

1 0

Include (.h) files not installed?
by Roberto De Almeida 05 Oct '05

05 Oct '05

Hi, I'm using setuptools to distribute my DAP (Data Access Protocol) module. The module depends on Numeric Python and Scientific Python, so I added the following to the setup.py script: install_requires = [ 'Numeric>=22.0', 'scientificpython>=2.4.9' ], There's no download URL for Numeric python in PyPI, so I created a package index at http://pydap.org/package_index.html. I then try to install my module with: easy_install -f http://pydap.org/package_index.html dap Easy_install downloads and install Numeric, and then proceeds to install Scientific. Scientific also depends on Numeric, and requires the include file "Numeric/arrayobject.h" to be compiled. The problem is that this file is not being installed when easy_install installs Numeric. Any hints? TIA, Roberto

2 1

easy_install, find-links, etc
by Ian Bicking 04 Oct '05

04 Oct '05

The whole find-links thing is getting confusing -- right now each system has to give its own set of find-links, and they aren't well inherited. So, for instance, if Paste has some custom packages on an index page, you can't just require Paste -- you have to require Paste plus put whatever find-links settings are needed. What would be mightily useful is if you could at least advise -- from within a package -- on places to find some of the requirements for that package. E.g., .egg-info/find_links.txt. I think it is inevitable that packages will require custom builds, and we can't always push those changes into PyPI or track those changes in aggregate package indexes. -- Ian Bicking / ianb(a)colorstudy.com / http://blog.ianbicking.org

2 1

Re: [Distutils] specifying --with-featurename with ez_setup.py / easy_install
by Phillip J. Eby 03 Oct '05

03 Oct '05

At 08:19 PM 10/2/2005 -0500, Matthew Scott wrote: >It does beg the question though, what is the intended use case of >'Feature' if features cannot be selected when installing using >easy_install? "Features" predated even the whisper of a concept of Python eggs by at least 8-10 months, and the idea of having easy_install by about a year and a half. So, they weren't designed together. Features were originally conceived as a way of trimming down mega-distributions like PEAK and especially Zope 3. You'll notice that setuptools' documentation doesn't mention them at all, though, because eggs are more useful. Currently, I'm only using "features" for a few of my packages, mostly to control building of the associated optional C code. However, that feature could be handled using an "optional" extensions ala mxSetup. So, I'd probably say that the "features" facility is kind of deprecated, especially since it has a few quirks that don't work so well. For example, if you build without the feature, and then run "install" or something, the package doesn't "stay configured" for the options you gave it previously. So, it doesn't really work all that well and will probably die out of setuptools as soon as I have a chance to rework my packages that use it. Certainly, I'd like to know if anyone else has found productive uses for it. If there are any, I'd consider keeping it in, but it seems to me like what's really needed is a "configure" command and a way to write the configuration to a file that then is used by subsequent runs to ensure consistency.

1 0