Mailman 3 October 2002 - Distutils-SIG

Sharing code between C extensions
by Anders J. Munch Oct. 22, 2002

Oct. 22, 2002

Hi all, I have a set of C extensions that I developed while using my own Borland compiler Win32 Python build, statically linking the whole thing. Recently I gave up on using that special build, too much hassle to get third-party stuff (like wxWindows) to work. So now I'm using distutils, and I must say it's been surprisingly smooth sailing so far. Good work guys! I've hit a snag, though: My C extensions (are supposed to) share code and data. But distutils insists that each extension be self-contained and include everything that it depends on. So that means that some of my C extensions and library code now exists in multiple copies, one copy per extension. I don't mind the memory used; but the duplicated static data and duplicated type objects are causing problems. I guess placing the shared stuff in a DLL is an option but not one I'm too fond of, for various reasons. (Or perhaps I should switch to Linux where shared objects behave differently? Well, some other time, perhaps ;-)) What I'd like to do is to have multiple C extensions in the same .pyd. Like, instead of writing (simplified): ext_modules = [Extension('foo', source=['foo.cc']), Extension('bar', source=['bar.cc'])] if I could do: ext_modules = [Extension(['foo','bar'], source=['foo.cc','bar.cc'])] and have both extensions in the same .pyd. Hmm... come to think of it I guess I could fake it with ext_modules = [Extension('foo', source=['foo.cc', 'bar.cc'])] and then calling initbar() from initfoo(). I'd like to find a cleaner solution though. regards, Anders

3 4

Re: python installation question...
by Greg Ward Oct. 21, 2002

Oct. 21, 2002

[In the future, please ask distutils questions on distutils-sig(a)python.org.] On 21 October 2002, Tom Rauch said: > I'm trying to install MySQL-python-0.9.0 on a RH 7.3 box. I get this > message about "sklpping byte compilation" which I suspect is > preventing the Zope MySQL database adapter from being installed. I > saw on the web that you've had some experience with this error > message...can you offer any suggestions on how to work around this? The "skipping byte compilation" message simply means that you already have a version of MySQL-python installed on that box. Since the distutils skipped byte compilation for *all* files, you probably already have the exact version that you're trying to install. You might try blowing away the existing MySQL-python installation and starting from scratch. Greg -- Greg Ward <gward(a)python.net> http://www.gerg.ca/ Rules for Urban Cycling, #1: Green means go; yellow means go like hell; red means proceed with caution.

1 0

source distributions for compiler-less machines
by Thomas Heller Oct. 18, 2002

Oct. 18, 2002

I'm currently hacking together a python package management system - I have some free time over the weekend ;-). I just had an idea how to prepare source-distributions for people (on windows) without a compiler: The developer could just call 'setup.py build' before calling 'setup.py sdist', add the '--no-prune' flag to the sdist command, add 'recursive-include build *.*' to the MANIFEST.in file, and now the build directory, with the object files and binary extension files is included in the zip file. This source distribution can now be installed by 'setup.py install' just as usual - since all targets are up to date, the (nonexistant) C compiler will not be called. Any comments on this idea? Thomas

2 1

distutils & AFS
by Christian Pfaffel Oct. 17, 2002

Oct. 17, 2002

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello! I have a problem with using distutils on a filesystem residing on AFS. AFS does not allow hard-linking across directory boundaries. So the copy method from file_util.py using hard links files fails, e.g.: # ./setup.py sdist running sdist reading manifest file 'MANIFEST' making hard links in foo-1.0... hard linking MANIFEST.in -> foo-1.0 error: Invalid cross-device link The system I run is linux, thus it tries to hard link, I did not dig too deep into file_util.py but I figured out that it is the link mode which has to be set to None to make it working on AFS so that it copies files. I made a rather non elegant change to the code, which I attached. I am sure that you can come up with a better solution. I do, unfortunately not have the time, to fix it elsewhere. Just wanted to let you know. Best regards, Christian Pfaffel - -- PGP-Key: http://fubphpc.tu-graz.ac.at/~flash/pubkey.gpg -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.6 (GNU/Linux) Comment: Processed by Mailcrypt 3.5.7 <http://mailcrypt.sourceforge.net/> iD8DBQE9rqAzzNp7/ndBhMQRAoO3AJ90Q77x3TUU6S/OaP/ehzKbFYrIWACcDBBZ UK4XthK4zZjpSYRxxRCBoTY= =bfqa -----END PGP SIGNATURE-----

1 0

Re: [Distutils] lineendings
by Michael Hudson Oct. 16, 2002

Oct. 16, 2002

"Andy Robinson" <andy(a)reportlab.com> writes: > > Of course, the right solution would be getting a decent > > text editor ;-) > > No, it wouldn't. We have decent editors. But we ship > code to people maintaining Unix systems, who should > neither know nor care if it is Java or Python or Algol. > If they have to edit some config file in VI when logged > into a remote server, it's just wrong to ship it full of > ^M thingies; someone actually phoned me and asked if it was > safe to promote! At the same time I would quite like the > freedom to make a package on my Windows machine, or even > on a Mac. In Python 2.3, this will be better. PEP 278, I think. Maybe this could be ported to the 2.2 line after 2.3 is out and all the bugs have been shaken out of it. > (BTW, does OS/X have Unix line endings?) I think that depends :-/ > I think the right solution is "a few subclasses" and one > day some options in distutils to set this if you choose > to. Distutils knows which files are source, tar and zip > cannot be expected to. I think so far distutils has made the right choice on this one: be dumb. This is much less aggravating than trying to do something smart and stuffing it up would be, IMHO. > Putting it in perspective: we had a quiet day and all > decided to learn distutils, so we may ask some pretty > dumb things before we grasp its inner beauty, elegance > and fundamental rightness ;-) If you can fix all the horrible bits while you're at it, that would be even better :) Cheers, M. -- "Well, the old ones go Mmmmmbbbbzzzzttteeeeeep as they start up and the new ones go whupwhupwhupwhooopwhooooopwhooooooommmmmmmmmm." -- Graham Reed explains subway engines on asr

1 0

UCS-2/4 conflicts in distutils-produced RPM
by Henry S. Thompson Oct. 10, 2002

Oct. 10, 2002

I've just hit a major pblm with a package I distribute which includes extension code which in turn uses Unicode functions -- it fails if the UCS2/UCS4 build state of the source/target are not the same. I've checked the archives and can't see anything about this -- is this a known problem? What's the recommended solution? Thanks ht -- Henry S. Thompson, HCRC Language Technology Group, University of Edinburgh W3C Fellow 1999--2002, part-time member of W3C Team 2 Buccleuch Place, Edinburgh EH8 9LW, SCOTLAND -- (44) 131 650-4440 Fax: (44) 131 650-4587, e-mail: ht(a)cogsci.ed.ac.uk URL: http://www.ltg.ed.ac.uk/~ht/ [mail really from me _always_ has this .sig -- mail without it is forged spam]

2 3

RE: [Distutils] suggestions for distutil
by Philippe FREMY Oct. 9, 2002

Oct. 9, 2002

> > I like distutils, it is a very good tools but I have some suggestions: > > > > - data_files are copied directly to the python lib directory while python > > modules are copied in site-packages. > > I would find more logical to copy data file also in site-packages, so that > > all your product data is available in one place in the python directory. > > Funny, I have a subclass in mxSetup.py (see e.g. egenix-mx-base) which > does just that. Would it be possible to contribute it to distutils ? > > I imagine it is not possible to break the existing behaviour. But it is > > probably possible to specify in the doc that is is best to copy to > > 'Lib/site-packages/...' > > I found the data_files handling pretty useless, which is why I > wrote my own version. However, people are probably relying on the > existing behaviour, so the only way to change defaults is by > providing options which let you change them in e.g. setup.cfg. Indeed. > > - the documentation says how to install data file but not how to use them > > from inside a python module. How do I fetch my data files ? > > If you install them inside a Python package it's really easy > to get at them, because the path to the package's root dir > is listed in the module as attribute. If I have a code sample in the documentation that explains how to do it, it will be even easier. :-) It is not that I am lazy (I am), but I would like a documented supported way. I am afraid any attempt I make to code this would not be portable. > > - I would like my module to be compiled/installed with optimisation by > > default. Is it possible ? > > The install command let's you pass in a parameter: > > --optimize (-O) also compile with optimization: -O1 > for "python -O", -O2 > for "python -OO", and -O0 to disable > [default: -O0] Yes, but I want my installed module to be optimised by default. Is this possible without tweaking too much with setup.py ? I would have imagine that standard python install would optimise everything. > > - how do I uninstall ? 'python setup.py uninstall' does not work. > > That's true. See mxSetup.py for a way to implement that command. Is mxSetup a complete rewrite, or just some improvement over setup.py ? Is it possible to contribute to distutils ? regards, Philippe ########################################### This message has been scanned by F-Secure Anti-Virus for Microsoft Exchange. For more information, connect to http://www.F-Secure.com/

1 0

How do I prepend compile/link switches?
by Todd Miller Oct. 9, 2002

Oct. 9, 2002

Is there an easy way to insert additional compile/link parameters for a Python extension, where the new parameters appear in the command line *before* the source/object modules? It looks like the distutils is carrying that "feature" in about half the code, but in the end, extra_compile_args and extra_link_args wind up appended to the command line. Is there a work-around, or does more work need to be done to distutils to support this? Todd -- Todd Miller jmiller(a)stsci.edu STSCI / SSB

1 0

suggestions for distutil
by Philippe FREMY Oct. 9, 2002

Oct. 9, 2002

Hi, I am using Python 2.2 from active state, windows 2k but this is a generic problem. I like distutils, it is a very good tools but I have some suggestions: - data_files are copied directly to the python lib directory while python modules are copied in site-packages. I would find more logical to copy data file also in site-packages, so that all your product data is available in one place in the python directory. I imagine it is not possible to break the existing behaviour. But it is probably possible to specify in the doc that is is best to copy to 'Lib/site-packages/...' - the documentation says how to install data file but not how to use them from inside a python module. How do I fetch my data files ? - the syntax to use with scripts files is not given in the documentation. I am speaking about the chapter "3.4 Listing scripts" - I would like my module to be compiled/installed with optimisation by default. Is it possible ? - python setup.py --help lists many options (--license, --long-description, ...) which are not described in the documentation. I suggest to add a list of all supported keywords arguments with their meaning. - how do I uninstall ? 'python setup.py uninstall' does not work. - the user that just downloads a python program and want to install it will probably run 'python setup.py'. The help doesn't give any hint on how to install the stuff. The second command 'python setup.py --help' still doesn't give any hint about how to install. This is only after a third step that the help mention the work install for the first time. After a fourth command, the guy is able to install the packages. For such a common and easy thing as simply running the installer, it should be simpler than that. I suggest to add a "To install this package, type 'python setup.py install'" at the end of the standard help of 'python setup.py'. This would certainly help many users. regards, Philippe ########################################### This message has been scanned by F-Secure Anti-Virus for Microsoft Exchange. For more information, connect to http://www.F-Secure.com/

2 1

MANIFEST not included
by John J Lee Oct. 8, 2002

Oct. 8, 2002

I'm not getting MANIFEST included when running sdist, though the file does get created. If I explicitly add it to MANIFEST.in, I get a message from distutils telling me that MANIFEST doesn't exist (presumably because distutils just deleted it, so it can remake it). What might be wrong? Thanks for any help John

2 2