Greg Ward wrote:
On 04 September 2000, M.-A. Lemburg said:
Correct. MANIFEST is a simple list of files, one per line. If you want to include whole directories, you should write a MANIFEST.in file.
I didn't mean that distutils should scan the goven directory -- it should simply ignore it instead of raising an exception.
Hmm, are you really getting an exception? When I put a directory name into a MANIFEST file, I get:
error: can't copy 'pkg': doesn't exist or not a regular file
and immediate termination of the setup script (hey, it's an error, not a warning). I'll admit that's not as useful as it could be, but as long as there's no traceback it's not a bug! ;-)
You're right; it's not an exception, but still something which causes distutils to stop ;-)
Would it hurt much adding a "continue" somewhere to remedy this minor "caveat" :-)
Actually, I spent some time documenting the "sdist" command today, and tracked down some bugs in the handling of the manifest files. I also now understand the rules a bit better -- amazing what reading the code will tell you. ;-) Here are the rules as implemented in the current code (not even checked in yet, because of the 2.0b1 code freeze):
cases: 1) no manifest, template exists: generate manifest (covered by 2a: no manifest == template newer) 2) manifest & template exist: 2a) template or setup script newer than manifest: regenerate manifest 2b) manifest newer than both: do nothing (unless --force or --manifest-only) 3) manifest exists, no template: do nothing (unless --force or --manifest-only) 4) no manifest, no template: generate w/ warning ("defaults only")
In the 0.9.2 code (and all Distutils releases since at least 0.2), case (4) wasn't handled properly. A workaround is to use the -f (--force-manifest) option, which causes the sdist command to unconditionally regenerate the MANIFEST file.
Also, case (3) implies that the "default file set" -- setup.py, README.txt, and any source code specified in the setup script -- will *not* be used if you supply your own MANIFEST file. Additionally, the default "prune list" -- which strips RCS and CVS directories, as well as Distutils-generated temporary directories -- does not apply. IOW, if you generate your own MANIFEST, you have to get it exactly right. I think this is the right thing to do, because only control freaks will want to supply their own MANIFEST file, and taking away any control from control freaks makes them, well, freak out. ;-)
See the 'get_file_list()' method in distutils/command/sdist.py if you're curious about the implementation. (And see if you can spot the bug that I fixed this morning.) (No fair watching python-checkins!)
I'm currently using my own tools for generating the MANIFEST file. The most important difference is that they allow per directory MANIFEST.in style files which are appended to the general MANIFEST.in logic while scanning the directory.
Hmmm, another good reason to roll your own manifest -- maybe it's not just for control freaks?
My version if rpm 3.0.3.
I also noted another bug: When building an RPM which contains more than one Extension(), bdist_rpm fails on the second Extension(): it can't find the C file. """ running build_ext building 'mx.DateTime.mxDateTime.mxDateTime' extension creating build/temp.linux2 creating build/temp.linux2/mx creating build/temp.linux2/mx/DateTime creating build/temp.linux2/mx/DateTime/mxDateTime gcc -g -O2 -fpic -Imx/DateTime/mxDateTime -I/usr/local/include/python2.0 -c mx/DateTime/mxDateTime/mxDateTime.c -o build/temp.linux2/mx/DateTime/mxDateTime/mxDateTime.o -O2 -m486 -fno-strength-reduce creating build/lib.linux2/mx/DateTime creating build/lib.linux2/mx/DateTime/mxDateTime gcc -shared build/temp.linux2/mx/DateTime/mxDateTime/mxDateTime.o -o build/lib.linux2/mx/DateTime/mxDateTime/mxDateTime.so building 'mx.Proxy.mxProxy.mxProxy' extension error: file 'mx/Proxy/mxProxy/mxProxy.c' does not exist Bad exit status from /var/tmp/rpm-tmp.85854 (%build) error: command 'rpm' failed with exit status 1 """
??? Very weird. Does "python setup.py build" work on its own, ie. when run by you rather than by rpm in its build directory?
Yes. Works just fine.
One simple possibility is that you forgot to include mx/Proxy/mxProxy/mxProxy.c in your tarball -- make sure that you can unpack your tarball in a fresh directory and run "python setup.py build", which is just what rpm is doing.
That could have been the cause. While testing this I stumbled over another bugglet:
Executing: %build + umask 022 + cd /data/home/lemburg/projects/tmp/build/bdist.linux2/rpm/BUILD + cd mx-Extensions-BASE-1.0.0 + env 'CFLAGS=-O2 -m486 -fno-strength-reduce' python setup.py build Traceback (innermost last): File "setup.py", line 7, in ? from distutils.core import setup, Extension File "/home/lemburg/lib/distutils/core.py", line 146 raise ^ SyntaxError: invalid syntax
This is due to RPM finding a Python 1.5 version under the simple name "python".... distutils doesn't seem to be compatible with multiple installed Python versions :-( [I did run the setup.py file using Python 2.0].
I then tried:
projects/tmp> python2.0 setup.py bdist_rpm --prep-script ./prep-script invalid command name './prep-script'
Looks like somethings wrong there: I can't seem to pass a script filename to the command.
I had to hack the bdist_rpm.py file to make it work (perhaps time for yet another option ?-)
OK, after that hack the RPM compile runs through, but it now finishes with the following lines (and no apparent reason):
creating /var/tmp/mx-Extensions-BASE-buildroot/usr/local/lib/python2.0/site-packages/mx/Tools creating /var/tmp/mx-Extensions-BASE-buildroot/usr/local/lib/python2.0/site-packages/mx/Tools/mxTools copying build/lib.linux2/mx/Tools/mxTools/mxTools.so -> /var/tmp/mx-Extensions-BASE-buildroot/usr/local/lib/python2.0/site-packages/mx/Tools/mxTools byte-compiling /var/tmp/mx-Extensions-BASE-buildroot/usr/local/lib/python2.0/site-packages/mx/__init__.py to __init__.pyc writing list of installed files to 'INSTALLED_FILES' warning: install: modules installed to '/var/tmp/mx-Extensions-BASE-buildroot/usr/local/lib/python2.0/site-packages/', which is not in Python's module search path (sys.path) -- you'll have to change the search path yourself + exit 0 Processing files: mx-Extensions-BASE File listed twice: /usr/local/lib/python2.0/site-packages/mx/Tools/mxTools/mxTools.so Finding provides... Finding requires... Provides: mxDateTime.so mxProxy.so mxQueue.so mxStack.so mxTextTools.so mxTools.so Requires: ld-linux.so.2 libc.so.6 libc.so.6(GLIBC_2.0) libc.so.6(GLIBC_2.1) error: command 'rpm' failed with exit status 1
Does this mean that my system doesn't provide the "required" packages ? (SuSE has a somewhat different naming scheme than RedHat so this could be a possible cause)
For a straight Distutil build (ie. no RPM involved), setting CFLAGS should work, but it only applies to extensions (not C libraries). When RPM is involved, I'm not sure what the Right Way to do it is. Maybe CFLAGS will work? Maybe editing /etc/rpmrc or /usr/lib/rpm/rpmrc? I dunno.
Wouldn't an option to distutils be the right way for these kind of things ?
Yes. Definitely. It's on the TODO list.
Now, on to your other "bdist_rpm" failure...
setup.py bdist_rpm output:
rpm -ba --define _topdir /data/home/lemburg/projects/tmp/build/bdist.linux2/rpm --clean build/bdist.linux2/rpm/SPECS/mx-Extensions-BASE.spec Executing: %prep
Wrote: /data/home/lemburg/projects/tmp/build/bdist.linux2/rpm/SRPMS/mx-Extensions-BASE-1.0.0-1.src.rpm Could not open /data/home/lemburg/projects/tmp/build/bdist.linux2/rpm/RPMS/i386/mx-Extensions-BASE-1.0.0-1.i386.rpm
error: command 'rpm' failed with exit status 1
Hmmm, this smells like the archetypal Distutils bug: someone's trying to write a file into a directory that doesn't exist yet. In this case, it would be rpm's fault, since the "bdist_rpm" command doesn't know the name of the architecture directory that rpm will try to write to. Things to look for:
- does /data/home/lemburg/projects/tmp/build/bdist.linux2/rpm/RPMS/i386 exist?
- if you create it, does the bdist_rpm command then run successfully?
I haven't had problems with rpm failing to create directories -- I have RPM 3.0.4 on my Red Hat 6.2 box, and I just cranked out an RPM of mxDateTime 1.3.0 without a hitch. Perhaps RPM 3.0.3 joins RPM 2.x on the index of forbidden versions... sigh...
Dunno. It did work after I manually created the i386 directory and applied the python2.0 hack to bdist_rpm.py.