[Python-Dev] Packaging and binary distributions for Python 3.3

[Paul Moore]

On 10 October 2011 21:38, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
>> I'm not sure what you mean by a "setup.cfg-based directory". Could
>
>> you
>> clarify, and maybe explain how you'd expect to create such an archive?
>> We may be talking at cross-purposes here.
>
> Here's how I see it: at present, you can install a project by specifying
>
> pysetup3 install path-to-directory
>
> where the named directory contains a setup.cfg (replacing setup.py) and a bunch of things to install. Exactly what to install where is specified in the setup.cfg:
> it covers not only python packages and modules but also arbitrary binary files. The setup.cfg format is extensible enough to allow specifying where files are
> installed not only declaratively (as static paths in setup.cfg) but also according to criteria computed at installation time (e.g., write some PowerShell scripts to
> the installing user's PowerShell profile location).
>
> Of course, since you can install arbitrary data (and record what was installed where, to allow uninstallation to work), you can certainly install DLLs too (modulo
> the user having write permissions for the installation location, but that's true for data files, too).
>
> In theory, therefore, a binary distribution could be installed from a directory containing a setup.cfg, some DLLs, Python source files, and other text and binary
> data files. Moreover, it would be just as easy to zip that whole directory up (using any zipping tools), and pass it around as a .zip file; at installation time, the
> packaging code would unpack the directory in a temporary location and install from there.
>
> The zip archive can, of course, be appended to an executable which does the relevant unpacking and calls to packaging code to actually do the installation.
> The scheme is conceptually the same as the wininst-x.y.exe does - only the details differ. This gives a one (double-)click installer.
>

Ah, I see what you are saying now. I hadn't realised that the
setup.cfg format was that flexible. I'll look into it a bit more -
you're right that it would be better to reuse the existing technology
than to extend it if that's not needed.

>> Agreed - but I'm looking at a pysetup install approach to work for
>> source and binary packages, essentially replacing the use of
>> bdist_wininst and bdist_msi with sdist/bdist_simple archives. That's a
>> change of heart for me, as I used to argue for wininst/msi over
>> setuptools and similar - but pysetup includes all the listing and
>> uninstalling features I wanted, so the "one unified approach" has won
>> me over in preference to the platform integration.
>
> Right, but AFAICT pysetup3 will work now with a binary distribution, other than it does not contain mechanisms for checking Python version and platform compatibilities.
> Being a command line script, it will even support virtual environments without too much trouble - I've been working on this in the pythonv branch with some success.
> What's missing from things is a .exe installer; even though you might be happy without one, not having it may be seen by some as a retrograde step.
>

Now I understand what you mean. I agree that an exe installer should
be available. And given that it can be used like a zipfile as well if
it follows the exe stub plus zipfile approach of bdist_wininst) then
that sounds ideal.

>> Ideally bdist_wininst and bdist_msi would also integrate with pysetup
>> and with virtual environments, but I imagine that could be pretty hard
>> to make work cleanly, as Windows doesn't really support multiple
>> installations of a software package...
>
> I don't think Windows itself cares in general, it's more about the specifics of what's being installed. Obviously some things like COM components would need to be
> managed centrally, but I would imagine that if you had two projects with separate versions of e.g. C-based extensions, you could install the relevant DLLs in separate
> virtual environments and not necessarily have problems with them coexisting.

Agreed, it's more common Windows conventions than windows itself. Plus
having to invent distinct names for each entry to go into add/remove
programs, which could get to be a pain with multiple venvs.

>> (Plus, I've no real idea about how bdist_wininst works, so while you
>> may be right, I wouldn't know how to do anything with your suggestion
>> :-))
>
> Though I can't claim to have looked at the codebase in detail, the overall scheme would appear to be this: bdist_wininst creates an executable from wininst-x.y.exe
> (part of Python, in a distutils directory), appends some metadata (used in the UI of wininst-x.y.exe - things like the package name, icon etc.) and appends to that
> an archive containing all the stuff to install. When the executable is run, the UI is presented incorporating relevant metadata, user input solicited and the archive
> contents installed according to that input. However, the installation locations are determined from the registry information on installed Pythons only, with no nod to
> the possibility of users having installed multiple virtual environments from those installed Pythons.

Sorry, to be clear, yes I'm aware that's the general scheme. But I
don't know much about writing installers in general, or the code of
wininst-x.y.exe in particular, which is why I added the proviso. And I
don't have the free time to work on a C-based installer to replace
wininst-x.y.exe, which is why my focus is on pysetup.

To summarise, then:

1. By using setup.cfg technology, it would be easy enough to zip up a
binary build in a way that pysetup could unpack and install.
   1a. A packaging command to build such an archive would be worth providing.
2. A GUI installer would still be valuable for many people
   2a. Having the GUI work by doing a pysetup install passing the
installer exe (which would have a zipfile as noted in 1 above
appended) could make sense to avoid duplicating work.
   2b. The GUI could do the extra needed to integrate with the OS,
which pysetup wouldn't do
   2c. There's a question over a GUI install followed by a pysetup
uninstall, which wouldn't remove the add/remove entry...
3. Ideally, the GUI should co-operate with venvs, by offering some
form of browse facility. The command line does this automatically.

I'll do some research into setup.cfg capabilities and do some proof of
concept work to see how all this would work.

Does the above make sense?

Paul.