Mailman 3 January 2007 - Distutils-SIG

sdist command doesn't include data files
by Arve Knudsen 15 Jan '07

15 Jan '07

Hi I'm experiencing a problem when using setuptools (0.6c5) to generate a source tarball, using the 'sdist' command. What happens is that data files (PNGs) are omitted even though I include them in package_data (keyword to setup), and they are correctly included when installing. Is it the intended behaviour of 'sdist' to ignore package_data, or is this a bug? Thanks, Arve Knudsen

1 0

Re: [Distutils] custom egg metadata
by "Lutz Pälike" 10 Jan '07

10 Jan '07

> > there is no real need to have the package_dir directly alongside the setup.py ? > > I have only a single setup.py, but lots of subprojects that i want to > > keep in subfolders. And there are no setup.py in the project subfolders > > but everything is controlled through the single setup.py and a config > > file within each project. > That's not really a supported arrangement; note that you won't be able to > make proper source distributions, for example. (Source distributions must > contain a setup.py to be installable.) I am aware that this is not a standard setup, but nevertheless that's the way i want to go. All the distribution will happen in-house without source and never hit the real world so source distributions won't be a issue here. Thanks for pointing this out though. > To be blunt, the distutils doesn't really support what you're trying to > do. And setuptools inherits certain of its limitations, like the fact > that it is built to assume that there is a setup.py in the current > directory and the current directory is the root of a project. I really would like to have more flexibility for the distribution of packages and setuptools are a major improvement over standard distutils. Thanks for all your effort to improve the situation. This is highly appreciated. As i already mentioned i would like to have better api support to use some functionality isolated from the rest of the package. I can see that distutils dictates a lot of decisions here but some design decisions are not clear to me. What comes to my mind for example is get_svn_revision(). This is a method of the egg_info command but it would also be useful to have this as a standalone function. Personally i replicated it as a function to take a filesystem path as parameter to get the svn revision of the given path. Very useful functionality but not very accessible within setuptools ... wouldn't it nice and easy to convert the method call to a function call ? Probably there are more low hanging fruits but i still need to take a closer look to the code. christmas is over but if the wishlist is still open, i would like to see the core setuptools functionality split off and to have a distutils compatibility layer on top. It is great to have pkg_resources.py on the distribution side, but it would be nice to have the same flexibility on the building side ... Regards, Lutz Paelike -- Der GMX SmartSurfer hilft bis zu 70% Ihrer Onlinekosten zu sparen! Ideal für Modem und ISDN: http://www.gmx.net/de/go/smartsurfer

1 0

setuptools 0.6c4 released
by Phillip J. Eby 10 Jan '07

10 Jan '07

As of this release, the installation instructions have been changed to support installation via RPM, .exe (Windows) or "sh setuptools-whatever.egg". In other words, ez_setup.py is no longer needed or recommended for user installation of setuptools. Prior to the final release of setuptools 0.6, I intend to update ez_setup to do one or both of the following things: 1. It will detect and download the current stable setuptools release from the Cheeseshop, AND/OR 2. It will instruct the user to visit the cheeseshop page and follow the installation instructions there. I'm not sure yet which is the best approach, so I'll probably just do both. :) I'd like to hear some input, but no "voting" please; just explain your rationale one way or the other, with detailed information about your situation/use cases. Thanks!

2 2

Re: [Distutils] Adding a setuptool command that runs when setup.py build is invoked
by Phillip J. Eby 09 Jan '07

09 Jan '07

At 02:47 PM 1/9/2007 -0500, Phillip J. Eby wrote: >At 09:51 AM 1/9/2007 +0100, Ronald Oussoren wrote: Sorry for the duplicate message, I was trying to see if there was a problem with the list, which has held my two setuptools release announcements

1 0

Re: [Distutils] Adding a setuptool command that runs when setup.py build is invoked
by Phillip J. Eby 09 Jan '07

09 Jan '07

At 09:51 AM 1/9/2007 +0100, Ronald Oussoren wrote: >Another alternative is to tweak >distutils.commands.build.build.sub_commands, which is more convenient >than subclassing. But will crash other setup scripts if they're being run by easy_install, unless you use a try/finally block to remove your "tweak" afterwards. Subclassing is therefore the recommended (and documented) procedure.

1 0

Re: [Distutils] Adding a setuptool command that runs when setup.py build is invoked
by Phillip J. Eby 09 Jan '07

09 Jan '07

At 04:35 PM 1/5/2007 -0800, Scott Robertson wrote: >I'm trying to create a package that provides a setuptool command that will >compile idl files when you run python setup.py build or python setup.py >install. > >I've figured out how to add an additional command (build_omniidl) which I >can run with: > python setup.py build_omniidl > >But I'm not sure how to wire it up so that > python setup.py build > >automatically invokes build_omniidl for me. Basically I want build_omniidl >to act just like build_python. > > >Is this even possible with setuptools? It's possible even with the distutils. What you need is to subclass the distutils 'build' command class, and then pass your subclass in via the 'cmdclass' dictionary argument to setup(). In setuptools 0.7, I plan to allow adding support for these kinds of extensions using entry points, so that subclassing isn't needed. >If so anyone know where I can see an example? Not right off, no. Check the distutils documentation; I seem to recall it at least has some information about subclassing and using cmdclass.

2 2

Re: [Distutils] custom egg metadata
by "Lutz Pälike" 09 Jan '07

09 Jan '07

> Your custom writer has to be on sys.path in order to work, but your > package_dir isn't on sys.path. Well this is the case, since i want to use a general setuptools writer and need no fancy custom writer (yet). > You'll have to either install the writer as a separate egg and use > setup_requires to ensure it gets on the path, or else move your > package_dir to the setup.py directory. Ok. But the writer just needs to be importable right? there is no real need to have the package_dir directly alongside the setup.py ? I have only a single setup.py, but lots of subprojects that i want to keep in subfolders. And there are no setup.py in the project subfolders but everything is controlled through the single setup.py and a config file within each project. > The general intention of the design, however, was for egg_info writers to > live in a different egg than the one being built, and to be activated > using setup_requires. Ok that's fine. > Still, you *can* make it work, it's just that you'll have to run the > egg_info command (or a command that runs egg_info) more than once in > order to get your custom metadata written, because the first time the > egg_info is written, your entry points won't be on disk yet. The second > time, it will. Ok i tried now what you suggested and it worked as you described if i split the build process in 2 steps. However it does not if i want to put the two steps in a single script, which indicates that some data is persistent and the freshly generated meta information is not yet respected. I tried to set pkg_resources.working_set , but it does not seem to have the necessary impact for the bdist_egg command. I guess the working_set instance is imported at an earlier stage into setuptools and thus does'nt get updated. I think the main problems i got is because i want to use setuptools from an api perspective and not from a tool perspective. Finally i want to integrate the setup script into a gui and make the building accessible to non-developers (this is a necessity). And because i want to catch exceptions and all that i don't like to use something like os.system() to launch different steps of the build. Maybe you can give me some more hints why this fails. At present i invoke the egg_info and the bdist_egg build step by setting the 'script_args' parameter for each step and call setup(). As already mentioned this fails. Regards, Lutz Paelike -- Der GMX SmartSurfer hilft bis zu 70% Ihrer Onlinekosten zu sparen! Ideal für Modem und ISDN: http://www.gmx.net/de/go/smartsurfer

2 1

Re: [Distutils] custom egg metadata
by "Lutz Pälike" 09 Jan '07

09 Jan '07

Ok i solved the problem now. I expanded sys.path to reflect my environment but the changes did not have any effect on setuptools. Now i noticed that i have to use pkg_resources.working_set.add_entry() to let setuptools take notice of the change. That did the trick. Thanks for your time. Cheers, Lutz -- Der GMX SmartSurfer hilft bis zu 70% Ihrer Onlinekosten zu sparen! Ideal für Modem und ISDN: http://www.gmx.net/de/go/smartsurfer

1 0

Re: [Distutils] custom egg metadata
by Lutz Paelike 09 Jan '07

09 Jan '07

Ok i solved the problem now. I expanded sys.path to reflect my environment but the changes did not have any effect on setuptools. Now i noticed that i have to use pkg_resources.working_set.add_entry() to let setuptools take notice of the change. That did the trick. Thanks for your time. Cheers, Lutz

1 0

custom egg metadata
by "Lutz Pälike" 08 Jan '07

08 Jan '07

I have some problems customizing the egg building process. I want to add a custom file to the egg metadata and the documentation says a egg_info.writers entry should work, but it looks like my definition is not evaluated. I tried to trace this down and found some more information. This is how my args to setup look like: {'entry_points': { 'distutils.setup_keywords': ['xyz_settings=setuptools.dist:assert_string_list'], 'egg_info.writers': ['xyz_settings.txt = setuptools.command.egg_info:write_arg'] }, 'name': u'XYZ_Project', 'description': 'My XYZ Project', 'package_dir': {'': 'data/project'}, 'packages': ['XYZ_Project'], 'version': '0.1' 'xyz_settings' : [ "Line 1", "Line 2" ] } Now if i step through the building process in the debugger i get the following: - distutils/dist.py:236: UserWarning: Unknown distribution option: 'xyz_settings' - if i step until 'setuptools/command/egg_info.py(169)run()' i see the following line: for ep in iter_entry_points('egg_info.writers') so this looks like the place where my 'egg_info.writers' definition is evaluated, but if i inspect iter_entry_points('egg_info.writers') i get the following: [ EntryPoint.parse('requires.txt = setuptools.command.egg_info:write_requirements'), EntryPoint.parse('PKG-INFO = setuptools.command.egg_info:write_pkg_info'), EntryPoint.parse('eager_resources.txt = setuptools.command.egg_info:overwrite_arg'), EntryPoint.parse('namespace_packages.txt = setuptools.command.egg_info:overwrite_arg'), EntryPoint.parse('top_level.txt = setuptools.command.egg_info:write_toplevel_names'), EntryPoint.parse('dependency_links.txt = setuptools.command.egg_info:overwrite_arg'), EntryPoint.parse('entry_points.txt = setuptools.command.egg_info:write_entries'), EntryPoint.parse('depends.txt = setuptools.command.egg_info:warn_depends_obsolete') ] There appears no definition of xyz_settings.txt ? Am i missing something here ? If i look closer inside iter_entry_points() i see the following code: for dist in self: entries = dist.get_entry_map(group) if i inspect the list of Distributions in self i get all the Distributions i installed via easy_install to my system but not the package i am currently building. So how is this supposed to work ? Should'nt the Distribution i am building be in this list? How are my entry points found then if not? Sorry if this is obvious, but i would be glad if somebody could give some more information here. Thanks, Lutz Paelike -- Der GMX SmartSurfer hilft bis zu 70% Ihrer Onlinekosten zu sparen! Ideal für Modem und ISDN: http://www.gmx.net/de/go/smartsurfer

2 1