[Distutils] Re: Dependencies as feature for PEP Python software catalog entries

[Pete Shinners]

note, this was a letter sent to me and a few others "off-list"
i thought i'd send my reply to this mail list also...

From: <cognite@zianet.com>
> First, Thanks for your work on the python packages collection!
> 
> SUGGEST:  adding a Dependencies section to the tags of a
> Python software catalog
> 
> The dependencies of a package are needed a priori.  They are
> not mentioned in the Catalog-SIG PEP. Dependencies are tied
> to versions, of course. Size of the total and its pieces would
> help, too. A partial example, sort of like the ones in the
> posted PEP at python.org , exemplifies.
> 
> EXAMPLE:
> The Facts found in this case:
> Dependencies of pyUI include not only pygame but also SDL.
> The licenses are not the same.
> 
> Thus notes on installing it included the DOWNLOAD LOCATIONS
> (not shown; elsewhere in the catalog entry, presumably) and
> "ORDERED RULES" of the package's footing in code and licenses.
> Putting that together took considerable study time for info
> that the package's author could have readily provided in a
> catalog entry, as it was part of the package's basic design.
> Not that these notes are ideally complete, but they were
> necessary to get things straight enough to get the package up.
> So they provide an example of some of the background info that
> would make a catalog entry useful.
> 
> ==============
> ..\GUIs\pyUI\pygame\pygame-1.0>less pygame-testing.nb
> ------------- pyUI NOTES
>         SC 5June2001
> 
> pyUI may be easy as claimed, and look slick,
> but it has dependencies
> on GUI manager pygame and GUI runtime environment  SDL (Simple
> DirectMedia Layer).  SDL is GNU; there are
> windows binaries.  pygame is LGPL.
> 
> 
> ------------- pygame NOTES
> 
> sophisticated setup.py and preparatory/fallback config.
> Sorts type of files by suffix, puts into resp. dirs.
> Gathers dlls on Windows, compiles C components,
> links to Numeric python if available.  Python >=2.0.
> 
> (1) put python >=2.0
>         add NumPy/Numeric if desired; ...
> 
> (2) put SDL via
> put SDL.dll 217k from binary SDL-1.2.0-win32.zip 91k into path
> (tgz 1343k)
> 
> (3) put pygame via
>         pygameDir/> python setup.py install
> 
> =========


great idea, but tricky to implement...

* in your example, pyui does not directly make use of SDL.
  so why should it be listed in the pyui dependencies?
  shouldn't the only dependency be "pygame". and then pygame
  has its own list of dependencies? what if the dependencies
  for pygame change? this would mean you'd have to go and
  change the dependencies for pyui also, that doesn't seem
  quite right.

* does distutils check for the dependencies itself and report
  any unfound ones? this isn't too bad for other python projects
  that have been installed with distutils (since we can control
  the environment), but what about non-python dependencies. should
  the distutils package provide its own code for detecting the
  dependencies? that might be best, it's the most flexible. i've
  seen distutils already has a bunch of "autoconfig" type functions
  which allow it to compile small test programs and return the
  results.

* of course, it would be nice if this was tied into the python
  catalog PEP, so dependencies could be easily found? perhaps we
  also allow non-python dependencies to live in the online catalog
  too? as long as their license allows redistribution and whatnot?
  unfortunately they all have potentially varied compile steps.
  for example, i just looked at freetype2 (an optional pygame
  dependency) and it is not the standard "configure;make" routine.

* one other problem is supporting multiple versions of the same
  python package. in fact versioning alltogether might be tricky.
  what if one package says it needs Numeric v20 and another needs
  Numeric v18. just about all python packages i've seen install to
  the same directory. there is no support for both versions.
  this could potentially cause the most trouble. the best solution
  might be to change python and distutils a bit. distutils installs
  into versioned directory names. it then installs some sort of
  special .PTH file which python uses to determine the "default"
  version for that library. then we change the import command
  (or maybe sys too?) to specify needed version numbers for a
  python package. something like "import Numeric[20.1]" or setting
  something in sys, like "sys.versioninfo['Numeric'] = 20.1".
  then when doing "import Numeric" python would check the
  sys.versioninfo and import Numeric from the correct directory.
  as for the install directories, perhaps distutils would create
  the versioned package names using some sort of invalid python
  namevalue character to separate the package name and installed
  version, "Numeric-20.1".

owell, there's the initial problems and potential solutions that
come to my head. its not all an easy thing to fix, but the more
solutions python provides for these problems, the better off we
may be?