Tarek Ziadé wrote:

On Wed, Jul 8, 2009 at 7:09 AM, P.J. Eby wrote:

...

So I punned on that in order to imply that nobody who takes on the job of packaging expects to be asked the kind of detailed, unanswerable questions that come with the territory of packaging, or to be subjected to torture by third parties merely for trying to do something useful.

And, by further implication, I was thus expressing camaraderie with you on your initiation into the wonderful world of trying to satisfy everyone in Pythonland, and our shared experience in the area of "if I'd really known what I was getting into, I probably wouldn't have". ;-)

Hehe very nice, thanks for the explanation :)

(thanks to the others too)

I think is exchange actually a good reminder that the bar for acceptance of PEP 376 shouldn't be "Addresses every possible packaging issue we can come up with". That would be setting expectations far too high because packaging cross-platform is so messy. A more appropriate bar is probably: a) Better that what we have now; and b) Provides possible avenues for even more improvement in the future For example, I quite like the concept behind the various ideas for "location" or "prefix" definitions either in the RECORD file itself or in a separate PREFIXES file, since such approaches feeds directly in to part b) above. Once some kind prefix definition mechanism is in place in the metadata, it becomes possible to leverage it to categorise files for the benefit of system packagers while still allowing simple internal-use-only packages to just bundle everything together. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------

Antoine Pitrou wrote:

Nick Coghlan writes:

...

For example, I quite like the concept behind the various ideas for "location" or "prefix" definitions either in the RECORD file itself or in a separate PREFIXES file, since such approaches feeds directly in to part b) above.

I don't understand the point. Since RECORD is specific to a particular machine and installation (it is generated at install time), what would you gain by adding a fake genericity to its entries?

My understanding is that RECORD isn't generated at install time for the bdist_* commands -it's generated at the time the binary distribution is created. Hence the problems raised with the idea of having absolute paths in RECORD - if they were written directly into RECORD they would reflect where the files were when the distribution was created, not where they end up when it is installed. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------

Paul Moore writes:

I have no answer here. But I think we need feedback from the people who will ultimately be building bdist_xxx formats - Debian packagers, people wrting alternate RPM builders, etc.

I think any bdist_xxx command which targets externally handled package formats (rpm, deb, msi, etc.) shouldn't generate RECORD. If you install with rpm or dpkg, you will uninstall with the same tool, won't you?

Bottom line - Is RECORD to be created on the target machine at install time, or must it be relocatable?

This is quite an uninformed opinion, but a relocatable RECORD looks like a can of worms to me. As for creating RECORD at install time, I don't understand the argument about duplicate code. Distutils can provide useful primitives for generating RECORD, so that interested subcommands don't have a lot of additional work to do.

Antoine Pitrou wrote:

Paul Moore writes:

...

I have no answer here. But I think we need feedback from the people who will ultimately be building bdist_xxx formats - Debian packagers, people wrting alternate RPM builders, etc.

I think any bdist_xxx command which targets externally handled package formats (rpm, deb, msi, etc.) shouldn't generate RECORD. If you install with rpm or dpkg, you will uninstall with the same tool, won't you?

I agree with this. For RPM's, there's a whole other database of what files were installed. Is it really the intent that a file will be managed by multiple different installers? That I can install with RPM but remove with some python-installer (or other) tool? That way lies madness. In fact, I see RECORD as an installer-specific detail that doesn't need to be standardized at all. For bdist_rpm (which I'm contemplating taking over, but no promises), the only thing that needs to be done with the metadata is be able to build a .spec file. Once the .spec file is generated, rpmbuild (or equivalent) will produce the .rpm files. I image that .deb's are similar, but I'm no expert. Which brings up the "static metadata" argument I was making at PyCon. I want there to be enough static metadata that the new bdist_rpm can read so that it produces a well-behaved .spec file. I need to know the intent of some of the files (which are documentation, which are config files, which are scripts, which are libraries, etc.) and not much else. I envision the new bdist_rpm as a standalone tool, not a distutils extension.

...

Bottom line - Is RECORD to be created on the target machine at install time, or must it be relocatable?

This is quite an uninformed opinion, but a relocatable RECORD looks like a can of worms to me.

Agreed.

As for creating RECORD at install time, I don't understand the argument about duplicate code. Distutils can provide useful primitives for generating RECORD, so that interested subcommands don't have a lot of additional work to do.

I envision a "distlib" that contains new support code that's not part of the current distutils morass.

On Wed, Jul 8, 2009 at 5:11 PM, P.J. Eby wrote:

At 01:13 PM 7/8/2009 +0100, Paul Moore wrote:

...

An alternative solution would be for the bdist_xxx commands to ignore the RECORD file generated in the temp area, and build its own on the target machine when the installer is run. This is conceptuially far cleaner, it's in line with the implicit assumptions in the PEP, and it avoids a lot of the problems with path formats (the filenames in RECORD could be absolute local pathnames, and that's the end of the story). But - and it's a big but - this dumps RECORD file handling on the bdist_xxx command, and they all need to implement the same code, resulting in a lot of duplication and opportunity for inconsistency.

The PREFIXES approach I suggested doesn't have these problems; the bdist installer only needs to write a PREFIXES file, and doesn't need to touch RECORD (which would be generated by the install --root).  In fact, since PREFIXES is strictly advisory, it can be generated by the install --root and left alone at installation time.  And, if it's generated without the --root prefix included, RPMs would have a correct PREFIXES file without further manipulation.  Plus, the bdist_wininst .exe installer can easily write a PREFIXES file, since it has to know the prefixes in order to do an installation in the first place.

(We still need to hash out an initial list of prefixes, of course.)

Instead of building this list of prefixes from scratch, let's use the ones that are *actually* used when the install command is called, have a look at the list I've suggested : http://mail.python.org/pipermail/python-dev/2009-July/090514.html And let's drop other absolute paths,

At 01:58 PM 7/8/2009 -0400, Eric Smith wrote:

I really don't see this use case. Supporting multiple installers for the same file (or even just trying to prevent two installers from writing the same file)? Wouldn't you be better off just saying an installer can't overwrite any existing file?

Likewise, I don't see a use case for installing with one installer and uninstalling with another.

Put those two together, and the mechanism that an installer uses to record what files it installed is a private implementation detail.

You'll have to ask Tarek about that. As I've said in another email, if I were doing this for setuptools I'd just go and do it; it's already on my backlog of things to do. The point of the PEP is to provide an uninstall facility for the distutils, and to expose those records via an API so that other tools can be built on top of it. Whether anything other than distutils-based tools will actually use the format directly is entirely irrelevant to the PEP's merits, since bdist_* tools and platform-specific distutils installers may need to read or write the format. Hence, a PEP is appropriate to describe a format and API for the distutils family of tools to operate on. The specific proposed format is based on an existing de facto standard for installed package metadata, so that the many existing packaging tools based on that other format and existing APIs can easily read or write this data, and so that the API can also be used as a way to detect the presence of already-installed distributions of software -- even ones installed other ways.