Mailman 3 February 2010 - Distutils-SIG

stdeb vs dependencies?
by K. Richard Pixley 08 Feb '10

08 Feb '10

What is the expected work flow around package dependencies between python packages and debs in stdeb? I was expecting it to declare any deb dependencies found, then translate any python dependencies into the names they would have if those pypi packages were translated using stdeb. What I'm seeing with stdeb-0.5.1 is that no dependencies are declared at all, but instead, I'm seeing unresolved macro references to ${python:Depends}. --rich

2 6

Re: [Distutils] Distutils roadmap of refactoring for 2.7
by P.J. Eby 08 Feb '10

08 Feb '10

At 06:29 PM 1/23/2010 +0100, Tarek Ziadé wrote: >2010/1/23 P.J. Eby <pje(a)telecommunity.com>: > > At 01:00 PM 1/23/2010 +0100, Tarek Ziadé wrote: > >> > >> 3 - dir_util, archive_util and file_util are going to be removed in > >> favor of calls to shutil. > > > > By removed, do you simply mean that distutils will stop using them, but the > > modules will still be there? (i.e., deprecated and phased out, rather than > > simply dropped from existence altogether.) > >I'll use the same strategy than sysconfig : > >- if the API is just moved to another place and works exactly the same way > (e.g. like what's planned for make_archive), it will be dropped, and the > documentation will refer to the new place. Is this the standard procedure for relocation of stdlib APIs across 2.x releases? I was under the impression that the standard is to do such things across two release cycles with a deprecation.

7 8

Re: [Distutils] PEP 376 -- new round
by P.J. Eby 05 Feb '10

05 Feb '10

At 01:55 AM 2/5/2010 +0100, Tarek Ziadé wrote: >I think it's still useful, because it points the root packages that >can be removed >safely without breaking the system -- even if leaving orphaned >packages behind. > >Any opinion ? I think it's a good idea to have a way to tell what packages were *not* installed to satisfy dependencies. Merely plotting the dependency graph doesn't tell you this, because you could have a non-root orphan - i.e., something that was installed to fill a dependency, but the depender(s) have now vanished. I guess what I'm saying is, a thing that is not needed by anything else could either be an orphan (due to other uninstalls) *or* a root (manually chosen for install), and there is no way to tell them apart just by following the graph.

5 7

Re: [Distutils] PEP 376 -- new round
by Laura Creighton 05 Feb '10

05 Feb '10

You need to think about dependencies more, I think. Having worked on building such a thing, the hellish problem comes when there are more than one way to support a package .. i.e. this is my new testing plugin, it works with either py.test or nose. Laura

2 1

PEP 386 - suggest_normalized version
by Tarek Ziadé 05 Feb '10

05 Feb '10

Hi, Find attached the latest state of PEP 386 suggest_normalized() over PyPI. Summary: Total Packages : 9058 Already Match : 8055.0 (88.93%) Have Suggestion : 792.0 (8.74%) No Suggestion : 211.0 (2.33%) -> the attached file contains the details on the suggestion made, and the versions that couldn't be treated I am sure it still can be improved but I think it is already quite good. Regards Tarek -- Tarek Ziadé | http://ziade.org

1 0

PEP 376 -- new round
by Tarek Ziadé 05 Feb '10

05 Feb '10

Hello, As mentioned on python-dev, I would like to do a new round of work on PEP 376 before Pycon, We had a short off-list conversation with Guido on PEP 376, I'd like to share here so we can discuss the point, it's about the REQUESTED file: Guido's question: I have a question about REQUESTED. It seems that the absence of this file means that the package is installed as a dependency of another package, so that it can be removed if that other package is removed . But since there is no record of the requesting package, it would seem problematic to find out when a dependent package can be removed, since there could possibly be multiple packages depending on it. Maybe it would be better to keep track of a list of packages that depend on a given package, adding a special "root" or "explicit" line for packages that were explicitly requested by the user? My answer: Distutils doesn't provide a feature where dependencies gets installed or removed. It works at the package level only, and provides tools to install or uninstall one package at a time. So we didn't want to provide a record of a full dependency graph through this file because we have considered that this work is up to a full-featured install/uninstall tool that wants to install and remove dependencies. This tool can calculate this graph when working, as it does when installing dependencies, since everything is available in the metadata once we have PEP 345 -- with the "requires-dist" field. IOW: 1/ an orphaned package is a package that doesn't have a REQUESTED file, *and* that is not used by another package anymore. So this can only be known by a uninstaller tool that deals with dependencies. 2/ a requested package is a package that has the REQUESTED file Now for Distutils, the REQUESTED file is useful to mark the packages that were explicitely wanted. For example to develop against them. vs things that where installed as a dependency. Of course the caveat is that a full-featured uninstall tool might be slow to compute the full dependency graph when trying to remove dependencies, because it will have to take a look at all other REQUESTED packages. Installers/uninstallers that are built on the top of Distutils could probably create their own dependency records inside each .egg-info placeholders, to speed up the work. But I am not sure it's such a big deal to look at all metadata files when performing an uninstallation. It's ok for this operation to be kind of slow I guess. And I am also not sure how this could work if the user uses different kind of installers. This REQUESTED file is at the boundary of what we want to provide in Distutils. Going further seemed dangerous at this point because as we said last year at the summit, we want to make distutils a toolkit that provides some standards and a common ground for installers out there, rather than a full-featured package manager. IOW: we don't want to tell the tools out there how they should deal with the dependencies installation / uninstallation process, but give them tools to deal efficiently with one package at a time. I realize this is not clear enough in the PEP. Let me know if what I've explained makes sense to you. If so I'll work on adding details on these points. If not, it means we need more work on this. So Guido wonders if adding this file is really necessary since higher-level installers/uninstallers will have to build the graph anyways. I think it's still useful, because it points the root packages that can be removed safely without breaking the system -- even if leaving orphaned packages behind. Any opinion ? Regards, Tarek -- Tarek Ziadé | http://ziade.org

2 1

modules in different folders but under same namespace
by Riccardo-Maria BIANCHI 04 Feb '10

04 Feb '10

Hi, I have a package structured like this: package/__init__.py src/ __init__.py mod1.py share/__init__.py mod2.py Now I can import them as: package.src.mod1 package.share.mod2 How can I use Distutils to be able to import both of them under the same "package" namespace as: import package.mod1 import package.mod2 ? Thanks a lot in advance for your kind help! Best regards, Ric.

5 10

nailed-down versions by having custom index
by Reinout van Rees 03 Feb '10

03 Feb '10

Hi all, I'd like to do a bit of experimenting with a custom index with only the right packages in just the right versions. Some possible advantages I'm hoping to see for my usecase: - Isolation from one of various servers being down (pypi, sourceforge). - Guaranteed versions even when not using buildout. (Stuff needs "python setup.py install" in the system python, apparently, for arcgis/arcview on windows to be able to use it). Personally, I'd just let everyone use buildout, but I think I won't be able to get that working with that "install it in site-packages" requirement. (Ideas welcome, of course). Question: how to maintain such a custom index? iirc chrism has a script to build an apache-served folder structure from a bunch of eggs/tarballs, so just having the tarballs is enough. Probably just putting those tarballs in one location is enough anyway. But how to get the right collection of tarballs in there? Is that a by-hand approach? Can you automate it? Question 2: I thought you could only have one index. So installing something extra from pypi seems out. How do people handle that? I'm totally used to buildout and its [versions] list and buildout.dumppickedversions and KGS lists, so I need to reset my thinking on this a little. I need some input :-) Reinout -- Reinout van Rees - reinout(a)vanrees.org - http://reinout.vanrees.org Programmer/advisor at http://www.nelen-schuurmans.nl "Military engineers build missiles. Civil engineers build targets"

6 8