Mailman 3 August 2007 - Distutils-SIG

Re: [Distutils] bug in svn detection
by Phillip J. Eby 06 Aug '07

06 Aug '07

At 06:46 PM 8/5/2007 -0700, Keith Dart â wrote: >Phillip J. Eby wrote the following on 2007-08-05 at 18:31 PDT: >=== > > I don't see why; the fixes are already in setuptools SVN repository, > > as you can see here in r56277. AFAIK, this code is in setuptools > > 0.6c6, i.e., the current release. > >=== > >Oh, ok. It's fixed already. I didn't know you knew about it. Yes, I am >using 0.6c6, since that is the recommended release (and what my Linux >distribution packages). So, the only fix then is to install setuptools >from SVN HEAD? It would appear so; I guess I'm wrong about this being in 0.6c6; I just looked at my checkout, which is actually an 0.6c7dev version. Use "easy_install setuptools==dev06" to get the same version as I'm using.

1 0

CVS/Entries.Log ignored by setuptools default CVS file-finder logic
by Jeff Shell 06 Aug '07

06 Aug '07

Hi. I'm trying to update our internal code to use distutils/setuptools at long last. We work heavily in Zope 3, so our packages contain many "data" files (ZCML, Page Templates, Javascript, CSS, etc). All of our internal code is maintained in CVS. I noticed that using ``include_package_data = True`` was having little or no effect: at most, only data files in the root directory might be included. This seems to be related to our directories being listed in `CVS/Entries.Log` instead of `CVS/Entries`, and setuptools doesn't read this file. I don't know why our directories are in this file: these are fresh checkouts (retrieved over the network using :ext: and SSH) with no local changes having yet been applied. Apparently programs that are reading 'Entries' should also read 'Entries.Log', which is slightly different from Entries. I have more details below. Is there any particular reason why this file isn't being read? I'd be surprised if I'm the only one encountering this issue. A basic fix appears to be easy: add another `re_finder` for `CVS/Entries.Log` with a regex like ``^A\s\w?/([^/]+)/``. I'm sure I could add something like this with an entry-point and our own egg, but it seems ridiculously small for us to maintain internally. Details ======= I traced this issue down to setuptools/command/sdist.py and the default set of finders:: finders = [ (convert_path('CVS/Entries'), re_finder(re.compile(r"^\w?/([^/]+)/", re.M))), (convert_path('.svn/entries'), entries_finder), (convert_path('.svn/dir-props'), externals_finder), (convert_path('.svn/dir-prop-base'), externals_finder), # svn 1.4 ] That looked right to me. Then I looked in a CVS directory in a couple of different packages and looked at Entries:: CVS> cat Entries /__init__.py/1.1/Thu Aug 18 23:29:21 2005//Tfixup-branch /addhelp.py/1.1.2.2/Wed Dec 20 22:55:09 2006//Tfixup-branch /configure.zcml/1.5.2.10/Wed Jul 18 19:32:51 2007//Tfixup-branch # .... (more lines - no directories) /tests.py/1.1.2.2/Thu Jan 18 22:10:50 2007//Tfixup-branch /traversing.py/1.1.2.2/Tue Mar 20 23:29:36 2007//Tfixup-branch /valuelib.py/1.1.2.3/Fri Jul 27 20:26:38 2007//Tfixup-branch D Hmmm, no directories... Then I noticed an "Entries.Log" file. I looked at that:: CVS> cat Entries.Log A D/authentication//// A D/batching//// # ... (more lines, all directories, all lines beginning with ``A D``) A D/undo//// Whoa! Here are all of the directories! I'm not sure why these are in there - this is showing up in a fresh code checkout, and it's showing up regardless of whether the checked out code is a branch or not. This is showing up on Mac OS X with the CVS that comes with the development tools package. CVS> cvs --version Concurrent Versions System (CVS) 1.11.18 (client/server) Our server has ``1.11.1p1``. I don't know if these version numbers mean anything in relation to this issue. Does anybody else? Anyways, the CVS manual says this about Entries.Log:: > Programs which are reading the 'Entries' file should also check > for 'Entries.Log'...The format of a line in 'Entries.Log' is > a single character command followed by a space followed by a line in > the format specified for a line in 'Entries'. The single character > command is 'A' to indicate that the entry is being added, 'R' > to indicate that the entry is being removed, or any other character to > indicate that the entire line in 'Entries.Log' should be silently > ignored (for future expansion). http://www.network-theory.co.uk/docs/cvsmanual/Workingdirectorystorage.html So, it looks like a decent solution would be to add another `re_finder` in setuptools' `sdist.py` module as mentioned above that took the Entries.Log file and format into account.... Yes? Thanks, Jeff Shell

1 0

bug in svn detection
by Keith Dart 06 Aug '07

06 Aug '07

Greetings. I would like to report a bug in setuptools here, since I can't find a bug tracking system for it anywhere. 598 $ sudo python setup.py install running install running bdist_egg exceptions.ValueError : max() arg is an empty sequence > /usr/lib/python2.4/site-packages/setuptools/command/egg_info.py(224) in get_svn_revision() localrev = max([int(d[9]) for d in data if len(d)>9 and d[9]]) Debug:15/15> l 214 continue # no sense walking uncontrolled subdirs 215 dirs.remove('.svn') 216 f = open(os.path.join(base,'.svn','entries')) 217 data = f.read() 218 f.close() 219 220 if data.startswith('8'): 221 data = map(str.splitlines,data.split('\n\x0c\n')) 222 del data[0][0] # get rid of the '8' 223 dirurl = data[0][3] 224 -> localrev = max([int(d[9]) for d in data if len(d)>9 and d[9]]) 225 elif data.startswith('<?xml'): 226 dirurl = urlre.search(data).group(1) # get repository URL 227 localrev = max([int(m.group(1)) for m in revre.finditer(data)]) 228 else: 229 log.warn("unrecognized .svn/entries format; skipping %s", base) 230 dirs[:] = [] 231 continue 232 if base==os.curdir: 233 base_url = dirurl+'/' # save the root url 234 elif not dirurl.startswith(base_url): Debug:15/15> show get_svn_revision ( self = <setuptools.command.egg_info.egg_info instance at 0xb7b43f2c>, ) Compiled locals: files = ['mkpydocindex'] revre = <_sre.SRE_Pattern object at 0xb7cf4458> base_url = 'https://pycopia.googlecode.com/svn/trunk/vim/' localrev = 11 revision = 53 dirs = [] base = './bin' data = [['', 'dir', '0', 'https://pycopia.googlecode.com/svn/trunk/vim/bin', 'https://pycopia.googlecode.com/svn', 'add', ...], ['mkpydocindex', 'file', '', '', '', 'add', ...], []] d = [] f = <closed file './bin/.sv... mode 'r' at 0xb7c81530> m = '*** undefined ***' _[1] = '*** undefined ***' dirurl = 'https://pycopia.googlecode.com/svn/trunk/vim/bin' urlre = <_sre.SRE_Pattern object at 0xb7ba49a0> 601 !$ svn --version svn, version 1.4.4 (r25188) compiled Jun 29 2007, 12:11:09 (However, the repository may have been created with an older version of subversion). I guess Phillip would be most interested in this: 51900 phillip.eby 51900 phillip.eby if data.startswith('8'): 51900 phillip.eby data = map(str.splitlines,data.split('\n\x0c\n')) 51900 phillip.eby del data[0][0] # get rid of the '8' 51900 phillip.eby dirurl = data[0][3] 56277 phillip.eby localrev = max([int(d[9]) for d in data if len(d)>9 and d[9]]+[0]) 51900 phillip.eby elif data.startswith('<?xml'): 51900 phillip.eby dirurl = urlre.search(data).group(1) # get repository URL 56277 phillip.eby localrev = max([int(m.group(1)) for m in revre.finditer(data)]+[0]) 51900 phillip.eby else: 52009 phillip.eby log.warn("unrecognized .svn/entries format; skipping %s", base) 51900 phillip.eby dirs[:] = [] 51900 phillip.eby continue -- -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Keith Dart <keith(a)dartworks.biz> public key: ID: 19017044 <http://www.dartworks.biz/> =====================================================================

2 1

Best practices on distributing documentation and examples with eggs?
by Stanley A. Klein 02 Aug '07

02 Aug '07

On Thu, July 26, 2007 9:50 pm, Phillip J. Eby wrote: >At 02:13 PM 7/26/2007 -0500, Dave Peterson wrote: >>In further discussions at Enthought, we're now thinking that creating a 'docs' dir as a top-level under python itself makes sense, and then the installation of eggs could copy docs and/or examples there in a manner similar to how it handles scripts into the platforms scripts dir. It would make sense that they should be put in dirs named with a format like <project name>-<version> so that the user could differentiate the docs for each project they installed. If this was done, it seems like it wouldn't be too hard for Stan, and other rpm builders, to either create symbolic links or copy these to /usr/share/doc in order to maintain compatibility with the LSB. >This isn't a bad idea, although I think that there are some hurdles that need to be worked out. My general thought is that, were >easy_install to support documentation, it would need to take an >option to determine the documentation base directory, and have a --no-docs install option. >One issue is that under most RPM systems, package names are more like 'python-foobar-1.2', and 'foobar' may not even be the exact distutils name of the package. So, installing the docs to the correct location in the general case may be more complex. Note, however, that >bdist_rpm offers a --doc-files option, which of course can be >configured via setup.cfg. I have no idea how it works, though, so don't ask me. :) The practice of changing the name of the whatever package to python-whatever may be distro dependent. Also (thankfully), python-whatever is installed in site-packages/whatever and not in site-packages/python-whatever. It was not always that way. Back at something like RedHat 7, they did change the names and the installation locations. If you had another program that imported the package, you sometimes had to go in and change the code to match their changed name. I can understand the reasons of the distros and repositories for doing this (keeping all the python or other language packages together in the list) but that does not make it less annoying. In looking to see if a package is in the repository, you have to look in various places beyond what the original developer/maintainer of the package calls it. Also, I'm not sure of the split between /usr/share/whatever and usr/share/doc/whatever for things like docs, tests, examples, and other miscellaneous files. It tends to be at the whim of the packager. In terms of what bdist_rpm does, it seems to be in the preparation of the rpm spec file. The spec file contains scripts (some of which point to macros in /usr/lib/rpm/macros on a Fedora system) and bdist_rpm seems to build the scripts and execute them using rpm-build. The build and install scripts do "python setup.py build" or "python setup.py install" with whatever options are specified. I think the actual location of the files may be governed by the %files section of the spec, but I will have to spend some time looking at other spec files to be sure that is where it happens. Stan Klein

1 0

Best practices on distributing documentation and examples with eggs?
by Dave Peterson 02 Aug '07

02 Aug '07

Over on the enthought-dev mailing list we're having a bit of a discussion on what the best way to distribute documenation and examples is for projects that we distribute binary eggs for. The general consensus is that it would be very nice indeed if there was a way to generate a tarball, or platform install, of just documentation and examples so that people wouldn't need to download a full, presumably significantly larger, source tarball. Another option would be that eggs included the documentation and examples and that, during the installation of the egg, those docs and examples were relocated to a common location (outside of the zip) to make access by users more convenient. This latter idea is similar to how Ruby Gems deal with docs. I don't claim to be a distutils or setuptools guru, so it shouldn't be too surprising that I can't seem to find anything about a setuptools or distutils command to do either of these. Am I missing something? If not, does it seem like something that might be worthy of putting into setuptools? -- Dave

5 9