[Pythonmac-SIG] PackMan - in defence of Python code in the database

[Jack Jansen]

On 10-okt-03, at 22:45, Bob Ippolito wrote:
>>> 1.  This is because we're using bdist_dumb, which is just as smart 
>>> as it sounds.  We'll inevitably be using something else in the 
>>> future that's a lot smarter.
>>
>> Definitely (and actually fairly simple[*]), but even then you cannot 
>> be 100% sure that any binary distribution can be re-targeted to a 
>> different location. There's nothing stopping the author of setup.py 
>> to store sys.prefix in a datafile somewhere.
>
> There's nothing stopping the author of setup.py from writing an 
> application that doesn't work either.. or stopping the author from 
> using Makefile instead of setup.py.  I'm not really concerned with 
> these people.

That's not fair: having sys.prefix be correct has been the case in 
distutils since day one.
So if you're creating an extension module that needs a pathname at 
runtime or something like
that it's perfectly legitimate to add dependencies on sys.prefix. But, 
all that said: there's
little we can do about it...

>> [*] Now that Python has its own tar module we could have a 
>> bdist_slightlylessdumb that creates a tarfile that has magic tokens 
>> for places like sys.prefix, etc. While a normal tar couldn't extract 
>> these to the right place a Python script to do so is dead simple.
>
> That's what I was thinking.  I was actually really surprised that you 
> were using bdist_dumb and rewriting the sys.prefix out of the paths to 
> do user installs.  Whatever works, right? :)

Partly, yes. And it's also a result of my attempts to use source 
distributions unadultered
(which lowers the maintainance load on the scapegoat). There's one 
alternative to bdist_dumb,
that looks much better from the description, but it's not generally 
available: ActiveState
have a way of building binary distributions where they basically do 
"setup.py build", then
freeze the tree, remove all the sources and pack that into an archive. 
On the destination
system they then do something quite like "setup.py install". This 
doesn't work with standard
distutils, because the "install" always does the "build" too.

I would like to see something like this standardised in distutils, 
because it keeps the
setup.py script, so it allows the package writer to do things like 
sys.prefix dependencies,
as long as they're done at install time, not build time.

>>> 2.   We shouldn't really be putting things in /System, ever.  
>>> Anything that does system-wide installation should use the 
>>> authorization API in order to acquire root access, even if the user 
>>> has admin gid.
>>
>> In theory I agree with this. In practice there are a number of 
>> problems:
>> - Not putting anything into /System conflicts with the accepted 
>> Python usage of packages adding stuff to include/python2.3. Until 
>> Python/distutils learn about a secondary include directory I think we 
>> have no options but putting things into /System (or not allowing it, 
>> at the expensive of disabling Numeric support in many packages).
>
> Let's put that into the distutils discussion.  Python packages should 
> have a canonical way of having headers, scripts, dynamic libraries, 
> and other arbitrary data (like fonts, icons, etc.) associated with 
> them *in the same or folder (or a closely related folder with 
> identical permissions)* as the package itself.  We could propose 
> something like Apple's bundles for this purpose (sans versioning, most 
> likely).

For datafiles: yes, definitely. But you can already do that by stuffing 
the datafile into your package. For headers and scripts the situation 
is different, as these are really for use outside of the package. I 
think I see more in a scheme similar to what we discussed here a couple 
of months ago for site-packages: having a number of these for different 
"authorization groups". So there's site-packages for vendor, 
system-wide, per-user, and similar for include files.

Dynamic libraries are a whole different can of worms, I have no ready 
solution for that.

>> - We don't have the authorization API right now
>
> That's not a really good excuse, I can fix that this weekend.

Then please please please do it with bgen, or with another technology 
that will allow me to import this into the Python core distribution and 
keep it maintainable.....

>> Actually, that wouldn't even be needed due to pimp's ability to 
>> update itself: just add a module pimp_update to site-packages and 
>> pimp will override itself with the contents of that module (if the 
>> version number is higher).
>
> That works too, forgot about that.  It's a shame that site-packages 
> isn't allowed to just replace standard library stuff (by way of 
> sys.path ordering), since there are so many batteries these days that 
> don't necessarily update at the same speed as Python and it's quite 
> possible for someone to botch packaging of a battery, such as you 
> forgetting to have bsddb installed when you packaged Python

That's by design: installing something should never break the standard 
stuff. There is a way around it, but I'll only tell you if you promise 
never to use it:-)

Seriously: I discussed this with Guido and he sort-of agreed that while 
standard third-parties should never override parts of the standard lib 
there is one exception, and that's to allow Python core developers 
responsible for a specific platform to override parts of the standard 
library in case the platform ships with Python pre-installed. You're 
free to read that long snippet as "that's me":-).
--
Jack Jansen, <Jack.Jansen at cwi.nl>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma 
Goldman