Le 09/10/2011 13:54, Paul Moore a écrit :
> On 9 October 2011 08:15, Éric Araujo  wrote:
>> Are there that many distributions with extension modules?  sdists should
>> work well even on Windows for pure Python projects.
> [...]
> Looking at my installations, I see:
> - database drivers (cx_Oracle, in my case)
> - lxml
> - pywin32
> - pyQT
> - pyzmq (that's just for playing a bit with IPython, so doesn't really count...)
> - I've also used in the past PIL mod_python (mod_wsgi more recently)
> and wxPython,

These are good examples.  Even if the number is not high, they are
widely used, so support for binary distributions really seems needed.

(When pip switches to distutils2 as underlying lib, they’ll get bdist
support for free!  IOW, proper bdist support would be another argument
to make the world switch to the new standards.)

> The pysetup features for uninstalling packages aren't going to work with
> bdist_wininst/bdist_msi (that's an assumption, I haven't tried them
> but I can't see how they would, and it'd certainly be a lot of
> marginally-useful effort to do even if it were possible).
> 
> The virtual environment stuff also wouldn't work that well with the
> installers, because they wouldn't have any way of finding which
> environments existed to ask where to install to. The same problem
> exists with virtualenv. (Again this is speculation backed by a small
> amount of playing with virtualenv, so I may be wrong here).

wininst and msi bdists can continue to be used as previously, for people
who want clicky installation to their system Python.  With built-in
package management and virtual environments in 3.3+, we can just
recommend that people publish bdist_simple instead of wininst or eggs.

> It may be that the bdist_dumb format would be OK. I haven't checked it
> out (to be honest, I don't think it's ever been used much).
It may or may not be.  bdist_dumb just makes a tarball or zipfile of
Python modules and built extension modules and is supposed to be
unpacked under sys.prefix or sys.exec_prefix.  However, that won’t play
nice with install options (--home, --user, --install-lib or redefined
--prefix) or sysconfig categories (i.e. I may want config files under
/usr/local/etc, scripts in /usr/local/bin, etc.).  I think we could
revamp bdist_dumb so that it’s really “sdist with compiled files” and
then we let pysetup install things to the right places.

>> Yes!  We need feedback to provide a much better tool than distutils,
>> before the API is locked by backward compatibility rules.
> Always the chicken and egg problem :-)
I’d rather say it’s “code in the stdlib has one foot in the grave” +
“stdlib code without active maintainer is effectively frozen” (hi
asyncore changes in 2.6!).

>> I actually wanted to talk about that, so let me take the opportunity.
>> What if we released packaging in Python 3.3 (and distutils2 1.0 on PyPI)
>> as a not-quite-final release?  [...]
> My immediate thought is that it would actually put people off using
> packaging for 3.3, they'd wait until "it is stable".
OK.  Too bad.

I’ll probably post that question again in its own message to get more
feedback.

> What is the status of distutils2? Is that (still?) intended to be
> effectively a backport of packaging to earlier Python versions?
Yes.  It works with 2.4-3.3.  I maintain it synchronized with packaging
in 3.3.  There are a small number of test failures which needs fixing
before I release distutils2 1.0a4 on PyPI.

> I'd suggest getting a distutils2 release available, and promoted,
> as the "early adopter" version of packaging.
We may do that, but I fear we’re going to lack time for that.  As part
of the stdlib, the packaging module API will be frozen in June, for the
first 3.3 beta.  We still have a lot to do: defining __all__ in all
public modules, changing some important internals (Tarek wants to kill
the subcommands system for example), fixing a number of bugs which may
imply incompatible API changes, etc.  The pace of development has slowed
much these last months, so I’m not sure we’ll reach 1.0 status months
before June.

> Maybe even with an option to install it as "packaging" so that people
> can use it in 3.2 and earlier and expect to need no changes when 3.3
> is released.
Not gonna happen!  d2 changed name on purpose when entering the stdlib,
so that in the future code can choose to use packaging (in the stdlib,
for example 1.0) or distutils2 (external backport, possibly of 2.0).
Code wanting to use “packaging if available otherwise distutils2” will
use the same import try/except as what’s done with unittest/unittest2,
json/simplejson and similar.

> A python-announce article "Python 3.3 new packaging features - early
> adopter release" publicising it, would be what I'm thinking of...
Here’s a plan:
1) make the docs usable (I’m on it)
2) fix the three test failures we currently have in d2
3) test on Windows and Mac
4) release 1.0a4 (I’ll do it)
5) announce and request feedback
6) work frantically before Python 3.3b1 to improve stuff and limit the
public API so as not to lock ourselves

Regards

On 13 October 2011 18:30, "Martin v. Löwis" wrote:

...

wininst and msi bdists can continue to be used as previously, for people who want clicky installation to their system Python. With built-in package management and virtual environments in 3.3+, we can just recommend that people publish bdist_simple instead of wininst or eggs.

Pardon me for jumping in - but I fail to see why those missing features can't be provided by bdist_wininst and bdist_msi in a straight-forward manner. Giving people even more choice is bad, IMO, as it will confuse users. There should be one obvious way.

I don't particularly disagree - although I would point out that the two formats bdist_wininst and bdist_msi already offer more than one obvious way... My contention is that there *are* two distinct use cases - platform integrated installation (on Windows that implies a GUI to most people), and Python's native installation process (pysetup). This isn't new, before packaging the "python-native" form was setuptools/eggs, for better or worse. Ideally, both forms should have full capabilities, making the decision a style/preference choice rather than a functionality choice. But this particular choice is always with us, and people are familiar with it. (Native vs cross-platform GUIs, cygwin vs mingw, etc, etc). So we need two obvious ways, one for each case. (It would be nice if one way could cover both cases, of course - having pysetup consume bdist_wininst files is my attempt to achieve that). I don't really understand the benefits of bdist_msi over bdist_wininst, and I certainly don't understand the MSI technology well enough to comment on what it's capable of, so I'm going to stick to bdist_wininst in the following. My apologies for anything I miss as a result. But it does strike me that the existence of both MSI and wininst is where the confusing duplication exists, rather than having GUI and command line alternatives. The GUI and platform integration aspects of the bdist_wininst format are all part of the executable "bit". I haven't looked at that code at all, but I am certain it can be modified to provide whatever user experience is desired. The only real problem here is how many people have the knowledge and/or inclination to work on that code. When it comes to installing the actual package, I don't know how the bdist_wininst code does it - the data is there in zip format, and I suspect that the code simply unzips the data in the expected directories. But the zipped up data in bdist_wininst could be consumed by the packaging module, just by writing a new install method. This would reuse all of the various packaging support routines and infrastructure. The bdist_wininst executable code *could* be modified to invoke that packaging method - whether that's worthwhile isn't clear to me (I don't know how extensive the changes would be to get the benefit reusing the same implementation). As MSI format is a specialised format, I don't believe this option is open for bdist_msi.

In particular wrt. virtual environments: I see no need to actually *install* files multiple times. It's rather sufficient that the distributions to be installed are *available* in the virtual env after installation, and unavailable after being removed. Actually copying them into the virtual environment might not be necessary or useful.

So I envision a setup where the MSI file puts the binaries into a place on disk where pysetup (or whatever tool) finds them, and links them whereever they need to go (using whatever linking mechanism might work). For MSI in particular, there could be some interaction with pysetup, e.g. to register all virtualenvs that have linked the installation, and warn the user that the file is still in use in certain locations. Likewise, automated download might pick an MSI file, and tell it not to place itself into the actual Python installation, but instead into a location where pysetup will find it.

I can't really comment on this. I agree in principle with what you're saying, but I know little about the MSI format so I can't say much more. It feels to me like you're suggesting that the MSI file encapsulate the file layout logic that already has to exist in pysetup, though, which sounds like duplication of effort. Can MSI call out to pysetup to actually install the files and save this duplication? Paul.

On 13 October 2011 20:28, Tim Golden wrote:

On 13/10/2011 19:36, Paul Moore wrote:

...

I don't really understand the benefits of bdist_msi over bdist_wininst

Just commenting on this particular issue: in essence, the .MSI format is the Microsoft standard, something which is especially important for corporate rollouts. We're not particularly bureaucratic, but I recently had to bundle a small number of common extensions as .msi packages so they could be deployed easily onto our baseline machines.

I'm not saying that Python *must* have .msi support for this reason: if it didn't already, you could argue that it could be provided by corporates who needed this, or by 3rd party service providers, if only by providing light .msi wrappers round standard installers.

Thanks for the clarification. I can see why this would be important. But maintaining 3 different interfaces to do essentially the same thing (collect some data from the user, then based on that data put the same set of files in the same places) seems a waste of effort, and a recipe for discrepancies in capabilities. Maybe the wininst and MSI installers should ultimately become simple UIs around a zipfile and an invocation of the packaging APIs? Not that I'm offering to do that work, I'm afraid... Paul.

On Thursday, October 13, 2011 04:02:27 PM Jeremy Kloth wrote:

That said, I have been working on a drop-in replacement for the current bdist_wininst executable stub with the following features: - install to 32- or 64-bit Python installations from a single installer; currently one installer for each architechure is required - install to any Python from version 2.4 to the latest; currently one installer is needed for each major version - updated look and feel (Wizard97) with the new (as of Python 2.5!) logo; for some screen shots see: http://www.flickr.com/photos/67460826@N04/sets/72157627653603530/ - unicode metadata support (name, summary, description) - runs on Win95 through Win7 (that is, all support platforms for the supported Python versions for packaging) - per-user installs (as in, setup.py install --user); currently only system-wide or per-user based on permissions and how Python itself was installed

I missed a few additional features: - UAC (Vista, Win7) is handled at the install phase depending on the selected Python target (system or user installed Python or user site-packages); currently just at the running of the installer - pre-/post- install and remove script support with the scripts no longer needing to be installed with the distribution - MSVCRT agnostic; built completely with the Windows API meaning only one stub EXE required; currently there is one stub per MSVCRT version -- Jeremy Kloth

On Sunday, October 16, 2011 02:24:58 PM Vinay Sajip wrote:

Jeremy Kloth writes:

...

That said, I have been working on a drop-in replacement for the current

...

bdist_wininst executable stub with the following features: [snip]

...

http://www.flickr.com/photos/67460826 <at> N04/sets/72157627653603530/

[snip]

Sounds interesting, but your flickr link did not work for me, even after I tried replacing "<at>" with "@". Care to post a shortened link?

Hmm, clicking the link in the email works here. but just to be safe: http://goo.gl/pC48e Thanks -- Jeremy Kloth

Thanks for the clarification. I can see why this would be important. But maintaining 3 different interfaces to do essentially the same thing (collect some data from the user, then based on that data put the same set of files in the same places) seems a waste of effort, and a recipe for discrepancies in capabilities.

Maybe the wininst and MSI installers should ultimately become simple UIs around a zipfile and an invocation of the packaging APIs? Not that I'm offering to do that work, I'm afraid...

I think you are mixing issues: even if they were simple wrappers, they would still be 3 different interfaces (presented to the user). I.e. the user doesn't really care whether there is a zip file inside (as in bdist_wininst) or a cab file (as in bdist_msi), and whether or not the packaging APIs are invoked during installation. So if you want to get rid of interfaces, you really have to drop one of the formats. Making maintenance of the interfaces simpler and more homogenous by having them call into installed Python code is certainly worthwhile. The challenge here is that the installers also work on older Python versions (unless there are extension modules in there), so they could only use the packaging API when installing into 3.3 or newer. Regards, Martin

- On formats, I strongly believe that having multiple formats is a problem. But I need to be clear here - an installer (MSI, wininst) is a bundle containing executable code (which drives the interface), plus a chunk of data that is the objects to be installed. (I am oversimplifying here, but bear with me).

Beyond oversimplifying, I think this is actually wrong: MSI deliberately is *not* an executable format, but just a "dumb" database, to be interpreted by the installation routines that are already on the system. In that sense, it is very similar to pysetup and bdist_simple. Regards, Martin

On Sat, Oct 15, 2011 at 4:42 AM, Paul Moore wrote:

I wish I felt more comfortable with MSI as a format (as opposed to an opaque clickable installer). I'd be interested to know what others think.

Compilation can be a problem on Linux systems as well, so a platform neutral format is a better idea. Just have a mechanism that allows pysetup to create a bdist_msi from a bdist_simple. Similar, bdist_rpm and bdist_deb plugins could be taught to interpret bdist_simple. However, you do get into architecture problems (x86 vs x86_64 vs ARM) if you go that route. Cheers, Nick. -- Nick Coghlan   |   ncoghlan@gmail.com   |   Brisbane, Australia

Nick Coghlan writes:

Compilation can be a problem on Linux systems as well, so a platform neutral format is a better idea. Just have a mechanism that allows pysetup to create a bdist_msi from a bdist_simple. Similar, bdist_rpm and bdist_deb plugins could be taught to interpret bdist_simple.

I agree that a platform-neutral format is a good idea, but there might be other complications with binary formats, which I'm not sure we've considered. For example, if we're just bundling self-contained C extensions which just link to libc/msvcrt, that's one thing. But what if those extensions link to particular versions of other libraries? Are those referenced binaries supposed to be bundled in the archive, too? I don't know that the dependency language supported by packaging extends to these kinds of dependencies on external, non-Python components. If we leave it to the packager to include all relevant binary dependencies, I'm not sure how satisfactory that'll be - possibly, not very. Regards, Vinay Sajip

On 14 October 2011 15:07, "Martin v. Löwis" wrote:

...

I can't really comment on this. I agree in principle with what you're saying, but I know little about the MSI format so I can't say much more. It feels to me like you're suggesting that the MSI file encapsulate the file layout logic that already has to exist in pysetup, though, which sounds like duplication of effort. Can MSI call out to pysetup to actually install the files and save this duplication?

I'm not sure what exactly it is that pysetup does, so I can't say whether there is any duplication. It's possibly to have post-install actions in MSI, so if the files get put into (some) place at first, it would be possible to copy/link them to whatever layout is needed.

OK, that might be a useful approach.

What I'd like to avoid is that people need to create too many different packages on Windows, for different use cases. It would be better if the author/packager could create one Windows distribution, and have that work in all use cases.

Absolutely. That is crucial if we're to avoid the sort of fragmentation we've seen in the past (with bdist_wininst, bdist_msi and binary eggs). I don't know if we'll ever get down to one format, but that would be the ideal.

As for MSI: it's primary advantages over bdist_wininst is the higher flexibility of integration into systems maintenance infrastructures. UI-less installation, installation through Active Directory, nested installation (as part of some bundle of installers) are all supported by MSI out of the box. IMO, the primary reason to keep bdist_wininst (besides popularity) is that you need to run the packaging on Windows to create an MSI file, whereas bdist_wininst can be created cross-platform (as long as there are no binary extension modules). In addition, bdist_wininst is better wrt. to repeated installations. I'd prefer a setup though where the same package can work in multiple installations without requiring physical copies.

One other aspect is that MSI format is essentially opaque (correct me if I'm wrong here). With bdist_msi, if I want to get the compiled binaries out for some reason (maybe to install them in a virtual environment or some type of other custom build) I just unzip the file - the exe header gets ignored. With bdist_msi, I have no idea if there's any way of doing that. Also, there are fewer people with expertise in MSI format. I suspect that even a Unix developer could have a go at modifying the C code in bdist_msi, it's not too MS-specific. I don't know if that's possible for bdist_msi. Speaking personally, the msilib documentation is pretty unreadable, as I don't know anything about the MSI format. Whenever I've tried reading the MS documentation in the past, I've found it pretty impenetrable (a link to a simple tutorial, and some examples of use, in the msilib documentation might help). Paul.

Martin v. Löwis writes:

In particular wrt. virtual environments: I see no need to actually *install* files multiple times. It's rather sufficient that the distributions to be installed are *available* in the virtual env after installation, and unavailable after being removed. Actually copying them into the virtual environment might not be necessary or useful.

So I envision a setup where the MSI file puts the binaries into a place on disk where pysetup (or whatever tool) finds them, and links them whereever they need to go (using whatever linking mechanism might work). For MSI in particular, there could be some interaction with pysetup, e.g. to register all virtualenvs that have linked the installation, and warn the user that the file is still in use in certain locations. Likewise, automated download might pick an MSI file, and tell it not to place itself into the actual Python installation, but instead into a location where pysetup will find it.

While it seems a little inelegant, copying might actually be simpler and less error-prone. Firstly, AFAIK you can't do true symlinks in relatively recent, widely-used versions of Windows like XP. Also, while some people use virtualenvs in a central location (such as virtualenvwrapper users), others have their envs under a project folder. I don't know that the complication of a centralised registry of virtual envs is necessarily a good thing. Regards, Vinay Sajip

On Sun, Oct 9, 2011 at 9:31 PM, PJ Eby wrote: ...

...

What we can do however is to see what bdist_egg does and define a new bdist command inspired by it, but without zipping, pkg_resource calls, etc.

Why?  If you just want a dumb bdist format, there's already bdist_dumb.  Conversely, if you want a smarter format, why reinvent wheels?

Just to make sure we're on the same page here. PEP 376 provide the installation format for the 'future' -- http://www.python.org/dev/peps/pep-0376/ Introducing back another *installation* format would be against the goal we've initially had with PEP 376 : have a single installation format all tools out there would support, for the sake of standardization of interoperability. (and for consumers in other communities) So, while 'eggs' are interesting as plugins for a given application (that was the initial use case right ?), please do not consider them as an installation format for Python. Now for a binary archive, that would get installed ala PEP 376, why not ? I'd just be curious to have someone list the advantage of having a project released that way besides the "importable as-is" feature. Cheers Tarek -- Tarek Ziadé | http://ziade.org

On 9 Oct 2011, at 21:14, Paul Moore wrote:

On 9 October 2011 20:47, Tarek Ziadé wrote:

...

On Sun, Oct 9, 2011 at 9:31 PM, PJ Eby wrote: ...

...

...

What we can do however is to see what bdist_egg does and define a new bdist command inspired by it, but without zipping, pkg_resource calls, etc.

Why? If you just want a dumb bdist format, there's already bdist_dumb. Conversely, if you want a smarter format, why reinvent wheels?

Just to make sure we're on the same page here.

PEP 376 provide the installation format for the 'future' -- http://www.python.org/dev/peps/pep-0376/ [...] Now for a binary archive, that would get installed ala PEP 376, why not ? I'd just be curious to have someone list the advantage of having a project released that way besides the "importable as-is" feature.

Agreed. I'm not looking at a new binary installed format - PEP 376 covers this fine. What I am looking at is how/if users without a compiler can get a file that contains all the bits they need to install a distribution.

Just to agree with Paul, a typical Windows Python user will not be able to install a non-binary version of a distribution that includes C code. Even on the Mac it is common to distribute binaries. Michael

My expectation would be that the user would type pysetup install some_binary_format_file.zip and have that file unpacked and all the "bits" put in the appropriate place. Basically just like installing from a source archive - pysetup install project-1.0.tar.gz - but skipping the compile steps because the compiler output files are present. That may need some extra intelligence in pysetup if it doesn't have this feature already (I sort of assumed it would, but that's likely because of my interest in binary formats) but if not it shouldn't be hard to add - just unzip the bits into the right place, or something similar.

As regards the format, bdist_dumb is about the right level - but having just checked it has some problems (which if I recall, have been known for some time, and are why bdist_dumb doesn't get used). Specifically, bdist_dumb puts the location of site-packages ON THE BUILD SYSTEM into the archive, making it useless for direct unzipping on a target system which has Python installed somewhere else.

See the following for an example:

PS D:\Data\python-sample> unzip -l .\dist\PackageName-1.0.win32.zip Archive: ./dist/PackageName-1.0.win32.zip Length EAs ACLs Date Time Name -------- --- ---- ---- ---- ---- 6656 0 0 09/10/11 20:56 Apps/Python32/Lib/site-packages/hello.pyd 208 0 0 09/10/11 20:56 Apps/Python32/Lib/site-packages/PackageName-1.0-py3.2.egg-info -------- ----- ----- ------- 6864 0 0 2 files

It should be simple enough to fix this in bdist_dumb, although a new name might be needed if backward compatibility of the old broken format matters...

If pysetup doesn't have support for binary archives at all, then I'm happy to take a look at what might be involved in adding this. But I don't know the code at all, and I have little time, so I'm rather hoping I won't need to...

Paul.

PS The problem for me is that if pysetup only handles source builds, it's STILL annoyingly incomplete for my requirements (and possibly many Windows users') So I feel this is a hole that needs to be filled before 3.3 is released, or pysetup won't be suitable as "the way to do all packaging in Python". _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.u...

-- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html

2011/10/10 PJ Eby :

On Sun, Oct 9, 2011 at 4:14 PM, Paul Moore wrote:

...

As regards the format, bdist_dumb is about the right level - but having just checked it has some problems (which if I recall, have been known for some time, and are why bdist_dumb doesn't get used). Specifically, bdist_dumb puts the location of site-packages ON THE BUILD SYSTEM into the archive, making it useless for direct unzipping on a target system which has Python installed somewhere else.

I don't know about the case for packaging/distutils2, but I know that in original distutils, you can work around this by making bdist_dumb call the install commands with different arguments.  That is, it's a relatively shallow flaw in bdist_dumb.

Agreed.

bdist_wininst, for example, is basically a zipped bdist_dumb with altered install arguments and an .exe header tacked on the front.  (Along with a little extra data crammed in between the two.)

I'd propose that the install arguments used in bdist_wininst be transferred to bdist_dumb (or a new command bdist_binary created based on the same), because the bdist_wininst zip format has the following advantages: 1. Proven format, so it should deal with any edge cases like header files reasonably. And the code already exists. 2. Easily recognisable directory names (PLATLIB, PURELIB, etc) making detection easy without needing extra metadata. 3. At a pinch, a bdist_wininst installer could be treated as a dumb distribution without change (assuming the stdlib zip handling correctly ignores prepended data like the exe header). Then pysetup could be enhanced to recognise and install the binary format in pretty much the same way as it does source formats (add another install_method to _run_install_from_dir that copies the files to the target locations along with appropriate checking and/or metadata handling). There might be a small amount of extra work to do, to check binary version compatibility, but that shouldn't be hard. If this is useful, I could look at creating a patch. (Once I get my build environment fixed so I can get 3.3 up and running - it looks like Python 3.3 can't be built with Visual C++ Express these days, the IDE can't convert the solution files because Express Edition doesn't support 64-bit. I'll have to fish out the full version and install that...) Paul.

On 10 October 2011 12:47, Nick Coghlan wrote:

IIRC, even the Express edition should still work once the 64 bit Platform SDK is installed. Regardless, the intent is that it should be possible to build Python with only the free tools from MS. If they broke the Express editions such that extra tools are needed, suggested updates to the devguide [1] would be appreciated.

[1] http://docs.python.org/devguide/setup.html#windows

It's the "once the 64 bit platform SDK is installed" bit that's the pain. To just build a test setup for a 32-bit PC it's a shame that you need to install a load of 64-bit tools. Hmm, looking at the devguide it says you should use VS 2008. If that's still the case, ignore me - I was using VS 2010 as that's what's (easily) downloadable. I've now installed VS Pro 2010. We'll see how that goes. I'd rather avoid downgrading to VS2008 (or having both at once) just for personal builds. But will if I have to. Nothing to see here (other than me not reading the docs), move right along... :-) Paul.

Paul Moore writes:

I'd propose that the install arguments used in bdist_wininst be transferred to bdist_dumb (or a new command bdist_binary created based on the same), because the bdist_wininst zip format has the following advantages:

1. Proven format, so it should deal with any edge cases like header files reasonably. And the code already exists. 2. Easily recognisable directory names (PLATLIB, PURELIB, etc) making detection easy without needing extra metadata. 3. At a pinch, a bdist_wininst installer could be treated as a dumb distribution without change (assuming the stdlib zip handling correctly ignores prepended data like the exe header).

Then pysetup could be enhanced to recognise and install the binary format in pretty much the same way as it does source formats (add another install_method to _run_install_from_dir that copies the files to the target locations along with appropriate checking and/or metadata handling).

A simple change to packaging will allow an archive containing a setup.cfg-based directory to be installed in the same way as a source directory. AFAICT this gives a more useful result than bdist_wininst (as you typically want to install in more places than PURELIB and PLATLIB, and the setup.cfg scheme allows for writing files to locations such as Powershell script directories for a user).

There might be a small amount of extra work to do, to check binary version compatibility, but that shouldn't be hard.

If this is useful, I could look at creating a patch. (Once I get my build environment fixed so I can get 3.3 up and running - it looks like Python 3.3 can't be built with Visual C++ Express these days, the IDE can't convert the solution files because Express Edition doesn't support 64-bit. I'll have to fish out the full version and install that...)

There's one thing that you touched on in an earlier post, which hasn't been further discussed: support for virtual environments. The executable installer format covers two things: packaging of version specific/compiled code, and the simplicity of point-and-click installation. This latter convenience is worth having, but the current installer stub (wininst-x.y.exe) does not know anything about virtual environments. If we care about virtual environment support (and I think we should), wininst.exe could be enhanced to provide a "Browse..." button to allow a user to select a virtual environment to install into, in addition to the detection of installed Pythons from the registry. If this is coupled with the ability to invoke a setup.cfg-based installation when the appended archive is for a setup.cfg-based directory tree, won't this pretty much tick all the boxes? Regards, Vinay Sajip

I'm not sure what you mean by a "setup.cfg-based directory". Could

you clarify, and maybe explain how you'd expect to create such an archive? We may be talking at cross-purposes here.

Here's how I see it: at present, you can install a project by specifying pysetup3 install path-to-directory where the named directory contains a setup.cfg (replacing setup.py) and a bunch of things to install. Exactly what to install where is specified in the setup.cfg: it covers not only python packages and modules but also arbitrary binary files. The setup.cfg format is extensible enough to allow specifying where files are installed not only declaratively (as static paths in setup.cfg) but also according to criteria computed at installation time (e.g., write some PowerShell scripts to the installing user's PowerShell profile location). Of course, since you can install arbitrary data (and record what was installed where, to allow uninstallation to work), you can certainly install DLLs too (modulo the user having write permissions for the installation location, but that's true for data files, too). In theory, therefore, a binary distribution could be installed from a directory containing a setup.cfg, some DLLs, Python source files, and other text and binary data files. Moreover, it would be just as easy to zip that whole directory up (using any zipping tools), and pass it around as a .zip file; at installation time, the packaging code would unpack the directory in a temporary location and install from there. The zip archive can, of course, be appended to an executable which does the relevant unpacking and calls to packaging code to actually do the installation. The scheme is conceptually the same as the wininst-x.y.exe does - only the details differ. This gives a one (double-)click installer.

Agreed - but I'm looking at a pysetup install approach to work for source and binary packages, essentially replacing the use of bdist_wininst and bdist_msi with sdist/bdist_simple archives. That's a change of heart for me, as I used to argue for wininst/msi over setuptools and similar - but pysetup includes all the listing and uninstalling features I wanted, so the "one unified approach" has won me over in preference to the platform integration.

Right, but AFAICT pysetup3 will work now with a binary distribution, other than it does not contain mechanisms for checking Python version and platform compatibilities. Being a command line script, it will even support virtual environments without too much trouble - I've been working on this in the pythonv branch with some success. What's missing from things is a .exe installer; even though you might be happy without one, not having it may be seen by some as a retrograde step.

Ideally bdist_wininst and bdist_msi would also integrate with pysetup and with virtual environments, but I imagine that could be pretty hard to make work cleanly, as Windows doesn't really support multiple installations of a software package...

I don't think Windows itself cares in general, it's more about the specifics of what's being installed. Obviously some things like COM components would need to be managed centrally, but I would imagine that if you had two projects with separate versions of e.g. C-based extensions, you could install the relevant DLLs in separate virtual environments and not necessarily have problems with them coexisting.

(Plus, I've no real idea about how bdist_wininst works, so while you may be right, I wouldn't know how to do anything with your suggestion :-))

Though I can't claim to have looked at the codebase in detail, the overall scheme would appear to be this: bdist_wininst creates an executable from wininst-x.y.exe (part of Python, in a distutils directory), appends some metadata (used in the UI of wininst-x.y.exe - things like the package name, icon etc.) and appends to that an archive containing all the stuff to install. When the executable is run, the UI is presented incorporating relevant metadata, user input solicited and the archive contents installed according to that input. However, the installation locations are determined from the registry information on installed Pythons only, with no nod to the possibility of users having installed multiple virtual environments from those installed Pythons. Regards, Vinay Sajip

Paul Moore writes:

To summarise, then:

1. By using setup.cfg technology, it would be easy enough to zip up a binary build in a way that pysetup could unpack and install. 1a. A packaging command to build such an archive would be worth providing. 2. A GUI installer would still be valuable for many people 2a. Having the GUI work by doing a pysetup install passing the installer exe (which would have a zipfile as noted in 1 above appended) could make sense to avoid duplicating work. 2b. The GUI could do the extra needed to integrate with the OS, which pysetup wouldn't do 2c. There's a question over a GUI install followed by a pysetup uninstall, which wouldn't remove the add/remove entry... 3. Ideally, the GUI should co-operate with venvs, by offering some form of browse facility. The command line does this automatically.

I'll do some research into setup.cfg capabilities and do some proof of concept work to see how all this would work.

Does the above make sense?

To me it does, and it would be useful to have some validation from the packaging folks. I looked at the dialog resources for wininst-x.y.exe and noticed that there is a "Find other ..." button which is hidden, and its handler (in PC\bdist_wininst\install.c) is commented out. However, the code called by the handler - GetOtherPythonVersion - is still there. Does anyone here know why the button has been made unavailable? Regards, Vinay Sajip

On Tuesday, October 11, 2011 01:59:45 AM Vinay Sajip wrote:

I looked at the dialog resources for wininst-x.y.exe and noticed that there is a "Find other ..." button which is hidden, and its handler (in PC\bdist_wininst\install.c) is commented out. However, the code called by the handler - GetOtherPythonVersion - is still there. Does anyone here know why the button has been made unavailable?

This "feature" has never been active. It has been commented out since before Distutils was imported into Python proper. -- Jeremy Kloth

Hi Nick, Le 11/10/2011 03:29, Nick Coghlan a écrit :

On Mon, Oct 10, 2011 at 2:29 PM, Paul Moore wrote:

...

Ideally bdist_wininst and bdist_msi would also integrate with pysetup and with virtual environments, but I imagine that could be pretty hard to make work cleanly, as Windows doesn't really support multiple installations of a software package... That's OK, the package managers get bypassed by pysetup on POSIX systems as well - that's kind of the point of language level virtual environments I’m not sure I follow you. wininst and msi installers are supposed to work with the Windows programs manager, it is not bypassed at all IIUC. That’s the difficulty: How to make the Add/Remove program aware of many Pythons and venvs?

There are hard to build packages on POSIX (e.g. PIL) that would also benefit from a good, cross-platform approach to binary installation. We haven’t talked about cross-platform binary installers. The current gist of the discussion seems to point to a world where people continue to release an sdist for capable OSes and also publish one bdist_simple with .pyd for one Windows version and one bdist_simple with .so for one Mac OS X version (but possible many arches). (And given that it’s possible to have one setup.cfg and one setup.py coexisting, maybe eggs will be offered for a while too.)

Regards

On 9 October 2011 20:47, Tarek Ziadé wrote:

...

PEP 376 provide the installation format for the 'future' -- http://www.python.org/dev/peps/pep-0376/ [...] Now for a binary archive, that would get installed ala PEP 376, why not ? I'd just be curious to have someone list the advantage of having a project released that way besides the "importable as-is" feature. I think I don’t understand you here. IMO, bdists are just intermediary

Le 09/10/2011 22:14, Paul Moore a écrit : formats that are supposed to be consumed by installation tools. Users are not expected to import from bdists.

My expectation would be that the user would type pysetup install some_binary_format_file.zip and have that file unpacked and all the "bits" put in the appropriate place. Basically just like installing from a source archive - pysetup install project-1.0.tar.gz - but skipping the compile steps because the compiler output files are present. Yep.

That may need some extra intelligence in pysetup if it doesn't have this feature already [...] just unzip the bits into the right place, or something similar. Yes. The bdist can be just like an sdist, but it contains compiled files instead of C source files (maybe setuptools bdist_egg is just that), then pysetup uses the setup.cfg file to find files and install them at the right places.

Alternatively, the bdist format could put build files into a few top-level directories, using sysconfig names: config, appdata, doc, purelib, platlib, etc. pysetup would then move files to the right target directory.

As regards the format, bdist_dumb is about the right level - but having just checked it has some problems (which if I recall, have been known for some time, and are why bdist_dumb doesn't get used). Or maybe because it’s just useless: Windows users want to click many times, Mac OS users want to drag-and-drop things, free OS users are fine with sdists.

Specifically, bdist_dumb puts the location of site-packages ON THE BUILD SYSTEM into the archive, making it useless for direct unzipping on a target system which has Python installed somewhere else. pysetup would not just unzip it though, so maybe this limitation is not really one.

a new name might be needed if backward compatibility of the old broken format matters... The point of forking distutils under a new name is that we can break compat.

I don't know the code at all, and I have little time Your time is best used with giving user expectations and feedback. I hope to get a Windows VM soon-ish, so I should not even pester people for testing my patches :)

PS The problem for me is that if pysetup only handles source builds, it's STILL annoyingly incomplete for my requirements (and possibly many Windows users') Agreed. packaging does not want to exclude Windows users.

[Nick]

bdist_zip, bdist_archive, bdist_simple would all work (bdist_binary is redundant, given what the 'b' stands for). Isn’t calling zipfiles “archives” an abuse? I like bdist_simple.

On a related topic, I’m not sure the bdist comand is useful. Its only role is to translate “bdist --formats zip,targz,wininst” to calls to the other bdist_* commands.

The 'bdist_dumb' name has always irritated me, since the connotations more strongly favour 'stupid' than they do 'simple' I’ve also recently learned that people with mental illness can be hurt by derogatory uses of “dumb”, so if we change the name to reflect the change in behavior, it’d have the nice side-effect of being nicer.

A simple change to packaging will allow an archive containing a setup.cfg-based directory to be installed in the same way as a source directory. Isn’t that already supported, as long as the tarball or zipfile contains

[Vinay] source files? In any case, it was intended to be, and there’s still support code around.

the current installer stub (wininst-x.y.exe) does not know anything about virtual environments. If we care about virtual environment support (and I think we should), wininst.exe could be enhanced to provide a "Browse..." button to allow a user to select a virtual environment to install into, Personally, I’ll focus on sdist and bdist_simple support. When pysetup is run from a virtualenv, projects will be installed into the venv. You are free to work on patches for bdist_wininsts, but I’m not sure it will be needed, if we make the pysetup user experience smooth enough. That said, even if Paul was convinced to forsake clicky installers, maybe some Windows users will absolutely refuse to use the command line.

[Paul]

To summarise, then:

1. By using setup.cfg technology, it would be easy enough to zip up a binary build in a way that pysetup could unpack and install. Correct. I’m still pondering whether I find the idea of registering built files in setup.cfg as elegant or hacky :) We also have the other ideas I wrote to choose from.

1a. A packaging command to build such an archive would be worth providing. Definitely. Maybe we could also decide on one of wininst or msi?

2. A GUI installer would still be valuable for many people 2a. Having the GUI work by doing a pysetup install passing the installer exe (which would have a zipfile as noted in 1 above appended) could make sense to avoid duplicating work. Yes.

2b. The GUI could do the extra needed to integrate with the OS, which pysetup wouldn't do Nice property.

2c. There's a question over a GUI install followed by a pysetup uninstall, which wouldn't remove the add/remove entry... I think we could require that a project installed with a clicky wininst_bdist has to be removed via the Add/Remove GUI. (There is support for that in PEP 376: the INSTALLER file.)

3. Ideally, the GUI should co-operate with venvs, by offering some form of browse facility. The command line does this automatically. Will Windows users want a GUI to create venvs too?

[Vinay]

I looked at the dialog resources for wininst-x.y.exe and noticed that there is a "Find other ..." button which is hidden, and its handler (in PC\bdist_wininst\install.c) is commented out. However, the code called by the handler - GetOtherPythonVersion - is still there. Does anyone here know why the button has been made unavailable? hg blame tells that this has been commented out since the initial commit to Subversion. Maybe the previous CVS history will yield more info.

Regards

On 13 October 2011 17:25, Éric Araujo wrote:

...

1. By using setup.cfg technology, it would be easy enough to zip up a binary build in a way that pysetup could unpack and install. Correct.  I’m still pondering whether I find the idea of registering built files in setup.cfg as elegant or hacky :)  We also have the other ideas I wrote to choose from.

To be honest, I think I prefer the idea of taking the bdist_wininst code which creates a zipped distribution archive with "special" root directories like PLATLIB, and use it to create a bdist_simple (basically, by removing some of the code to prepend the EXE stub). Then teach pysetup install to install that file format.(likely by just plugging in a new function into install_methods). The benefit is that bdist_wininst installers can be consumed unaltered by this install method. [1] Vinay's suggestion of registering the built files in setup.cfg sounds attractive, as the code is already there, but it seems like it'd just move the complexity from the install code to the process of building the bdist_simple archive.

...

   1a. A packaging command to build such an archive would be worth providing. Definitely.  Maybe we could also decide on one of wininst or msi?

I was thinking of a new bdist_simple archive format, which is platform-agnostic. bdist_wininst is a compatible superset restricted to the Windows environment, so promoting bdist_wininst over bdist_msi for people who prefer GUIs and platform integration would make sense.

...

2. A GUI installer would still be valuable for many people    2a. Having the GUI work by doing a pysetup install passing the installer exe (which would have a zipfile as noted in 1 above appended) could make sense to avoid duplicating work. Yes.

Or having bdist_wininst usable by pysetup install directly (because it's a bdist_simple compatible format)...

...

   2b. The GUI could do the extra needed to integrate with the OS, which pysetup wouldn't do Nice property.

And already present with bdist_wininst.

...

   2c. There's a question over a GUI install followed by a pysetup uninstall, which wouldn't remove the add/remove entry... I think we could require that a project installed with a clicky wininst_bdist has to be removed via the Add/Remove GUI.  (There is support for that in PEP 376: the INSTALLER file.)

Agreed - and if pysetup install works with bdist_wininst files, the user can choose whether to use a pysetup or a GUI install (and consequently which approach to management/uninstall they prefer).

...

3. Ideally, the GUI should co-operate with venvs, by offering some form of browse facility. The command line does this automatically. Will Windows users want a GUI to create venvs too?

Quite possibly some will. Personally, I don't. And given that virtualenv has managed OK without a GUI interface, I'd say let's assume it's YAGNI for now. [1] Actually, based on the above, I think the pysetup install method that consumes bdist_wininst files as if they were just bdist_simple archives (i.e., assuming the bdist_simple format takes that layout) would be useful even if bdist_simple is never implemented. The bdist_simple version makes the same facilities available for non-Windows users, if they want it. I think I'll look at coding that option, and see where it takes me. Paul.

Paul Moore writes:

On 13 October 2011 17:25, Éric Araujo wrote:

...

...

My expectation would be that the user would type pysetup install some_binary_format_file.zip and have that file unpacked and all the "bits" put in the appropriate place. Basically just like installing from a source archive - pysetup install project-1.0.tar.gz - but skipping the compile steps because the compiler output files are present. Yep.

...

That may need some extra intelligence in pysetup if it doesn't have this feature already [...] just unzip the bits into the right place, or something similar. Yes.  The bdist can be just like an sdist, but it contains compiled files instead of C source files (maybe setuptools bdist_egg is just that), then pysetup uses the setup.cfg file to find files and install them at the right places.

I have uploaded an initial (working, just needs test, docs and some testing and possibly tidying up) version of bdist_simple in the tracker (http://bugs.python.org/issue13189).

There's one area of pysetup3 functionality which I don't think has been discussed in this thread, though it's pertinent to Windows users. Namely, a completely declarative approach to installation locations will not satisfy all requirements. For example, if you want to install PowerShell scripts, the right way of doing that is to make Shell API calls at installation time to determine the correct location for the installing user. I have this working at present for a project, using pysetup's hook functionality; but any move to a completely passive archive format would lose this kind of flexibility. So, I think whatever archive format we end up with should provide exactly the same level of flexibility we currently get with pysetup3 for pure-Python projects, but extended to include binary deliverables. The simplest way of doing this is to register those binaries in setup.cfg, and to have hook code check for the correct dependencies in the run-time environment before actually installing (e.g. x86/x64/ARM/Python version dependencies). While it's not the slickest solution imaginable, it does allow for just about all use cases. This flexibility is more important under Windows than under Posix, because the installation locations under Posix conform much more closely to a standard (FHS) than anything you find in the Windows environment. Distribution build step = get all source, binary code and data defined in setup.cfg and in a single directory tree (containing the setup.cfg in the root of the tree), then zip that tree. Distribution installation step = unzip aforementioned zip, and run pysetup3 install . BTW, I believe that for virtual env installations, there's no need to provide integration with Add/Remove Programs. I find that the Add/Remove Programs dialog takes long enough to populate as it is :-( Regards, Vinay Sajip

Paul Moore writes:

Interesting. That's not a use case that I have encountered, but I can see it could be an issue. I have been working on the basis that a bdist_simple format that matches the functionality of bdist_wininst is at least good enough for those projects that currently use bdist_wininst. The only potential issue right now is with postinstall scripts, which bdist_simple doesn't support. It would be easy enough to add, and indeed it may be possible to use existing packaging functionality already (I haven't looked into this area).

The packaging machinery contains a reasonably good set of hooks which a setup.cfg can plug into, which is IMO more flexible than just a post- installation script (e.g. sometimes you need your code to run before installation, to determine where to install things).

I don't disagree, but I'm struggling to see how that would be done.

See example below.

Can you give an example of a setup.cfg, that would work like this? Suppose I have two files, foo.py and bar.pyd, which are a pure-python module and a compiled C extension. How would I write a setup.cfg and lay out the directory structure to allow pysetup install to do the right thing? I tried to do this myself, but couldn't get it to work the way I expected. It may be I was just hitting bugs, but it felt to me like I was going at the problem the wrong way.

It may not work for you, because in the default branch, packaging is currently missing some functionality or has bugs (I've raised about a dozen issues since trying to get packaging working with virtual environments). In the pythonv branch (which is pretty up to date with default), I've added the missing functionality/fixed some of the issues. Here's an example: I've created an empty virtual environment. Here are the contents of it at the moment: C:\TEMP\VENV │ env.cfg │ ├───Include ├───Lib │ └───site-packages └───Scripts activate.bat deactivate.bat pysetup3-script.py pysetup3.exe [various stock Python binaries omitted] Here's an example setup.cfg: [global] setup_hooks = hooks.setup [install_data] pre-hook.win32 = hooks.pre_install_data [metadata] name = dory version = 0.1 requires_python = >= 3.3 [other metadata omitted] [files] modules = dory packages = apackage apackage.asubpackage scripts = dory extra_files = hooks.py resources = data/data.bin = {data} compiled/ spam.pyd = {compiled} #[extension: spam] #language = c #sources = spammodule.c The extension section is commented out because we're not building the extension at installation time. This setup.cfg works hand-in-hand with some hooks, in file hooks.py below: def setup(config): # Can do various checks here. For example, platform # compatibility checks, or set up different binaries # to install for x86 vs. x64, etc. # Do this by setting up config['files']['resources'] # appropriately based on installation time environment. pass def pre_install_data(cmd): # assumes os.name == 'nt' for simplicity of this example cmd.categories['compiled'] = '%s/Lib/site-packages' % cmd.install_dir Notice how in setup.cfg, file 'spam.pyd' in 'compiled' is expected to be copied to category 'compiled', whose path is set in hooks.pre_install_data. Here's the project directory: C:\USERS\VINAY\PROJECTS\DORY │ dory │ dory.py │ hooks.py │ setup.cfg │ ├───apackage │ │ __init__.py │ │ │ └───asubpackage │ __init__.py │ ├───compiled │ spam.pyd │ └───data data.bin At the moment, the "spam" module is of course not installed: (venv) c:\Users\Vinay\Projects\dory>python -c "import spam" Traceback (most recent call last): File "<string>", line 1, in <module> ImportError: No module named 'spam' Now, we install the project: (venv) c:\Users\Vinay\Projects\dory>pysetup3 install . Installing from source directory: c:\Users\Vinay\Projects\dory running install_dist running build running build_py running build_scripts copying and adjusting dory -> build\scripts-3.3 running install_lib creating C:\temp\venv\Lib\site-packages\apackage byte-compiling C:\temp\venv\Lib\site-packages\apackage\__init__.py to __init__.c python-33.pyc byte-compiling C:\temp\venv\Lib\site-packages\dory.py to dory.cpython-33.pyc running install_scripts running pre_hook hooks.pre_install_data for command install_data running install_data running install_distinfo creating C:\temp\venv\Lib\site-packages\dory-0.1.dist-info creating C:\temp\venv\Lib\site-packages\dory-0.1.dist-info\METADATA creating C:\temp\venv\Lib\site-packages\dory-0.1.dist-info\INSTALLER creating C:\temp\venv\Lib\site-packages\dory-0.1.dist-info\REQUESTED creating C:\temp\venv\Lib\site-packages\dory-0.1.dist-info\RESOURCES creating C:\temp\venv\Lib\site-packages\dory-0.1.dist-info\RECORD Now, the virtualenv looks like this: C:\TEMP\VENV │ env.cfg │ ├───data │ data.bin │ ├───Include ├───Lib │ └───site-packages │ │ dory.py │ │ spam.pyd │ │ │ ├───apackage │ │ │ __init__.py │ │ │ │ │ ├───asubpackage │ │ │ │ __init__.py │ │ │ │ │ │ │ └───__pycache__ │ │ │ __init__.cpython-33.pyc │ │ │ │ │ └───__pycache__ │ │ __init__.cpython-33.pyc │ │ │ ├───dory-0.1.dist-info │ │ INSTALLER │ │ METADATA │ │ RECORD │ │ REQUESTED │ │ RESOURCES │ │ │ └───__pycache__ │ dory.cpython-33.pyc │ └───Scripts activate.bat deactivate.bat dory-script.py dory.exe pysetup3-script.py pysetup3.exe [various stock Python binaries omitted] Now we can import "spam" and run something from it: (venv) c:\Users\Vinay\Projects\dory>python -c "import spam; spam.system('cd')" c:\Users\Vinay\Projects\dory

I'd like to see a bdist_xxx command to do the build step as you describe, if only to make it trivially simple for developers to produce binary distributions. Having to package stuff up manually is bound to put at least some developers off. If you can give me the example I mentioned above, I could work on modifying the bdist_simple code I posted to the tracker today to produce that format rather than my custom format based on bdist_wininst.

Example as above, though you may need to use the pythonv branch to actually get it working. I can zip up the directory and send it to you, but at the moment there's missing functionality in pythonv in terms of the link step when building the extension. (I overcame this by linking manually .) If you want, I can zip all the project files up and send them to you. In the more general case, one might want an arrangement with a directory structure like compiled/x86/..., compiled/x64/... in the built zip, with the hooks.py code setting up the resources appropriately based on the target environment as determined at installation time.

For the installation step, you shouldn't even need to unzip, as pysetup3 can do the unpacking for you.

Indeed, with pythonv I could just zip the whole "dory" project directory and install with e.g. "pysetup3 install dory-1.0-win32-py3.3.zip".

Agreed. Personally, as I've said, I'm happy not to use Add/Remove even for system installations - pysetup list and pysetup remove do what I need without slowing down the Add/Remove list. But I accept that's not likely to be the view of many Windows users. Anyone using virtual envs, though, is probably by definition comfortable enough with command line tools to be willing to use pysetup3.

A fair subset of those who must have ARP integration will probably also want to install using MSI, so that would be taken care of by having a good bdist_simple -> bdist_msi conversion. Regards, Vinay Sajip

Paul Moore writes:

Correct me if I'm wrong, but if we standardised on a particular structure, the hooks.py contents could actually be integrated into the core, if we wanted? People could still write hooks for more complex cases, but the basic binary build case could work out of the box that way.

Well, the hooks.py is there to allow user-defined setups which are outside the scope of what should be provided in the stdlib - for instance, my earlier example about PowerShell scripts is IMO out-of-scope for the stdlib itself, but perfectly fine for the documentation, say in a set of example recipes in a packaging HOWTO. The hooks aren't needed at all for conventional deployments - only when you need something out of the ordinary. We could certainly extend the setup.cfg scheme to have specific support for pre-compiled binaries, which are currently "out of the ordinary" (which of course is why this thread is here :-)). Life could be made easier for distribution authors by initially having well documented examples or recipes, and later, if the ubiquity of certain patterns is established, better support might be provided in the stdlib for those patterns. But there are other changes we could make now - for example, the list of categories does not include a library location (necessitating my use of a "compiled" category), but perhaps a "lib" category could be built in now. Regards, Vinay Sajip

Hi Philip,

> [...] In any case, it definitely wasn't the case that eggs or setuptools were
> rejected for 2.5; they were withdrawn for reasons that didn't have anything
> to do with the format itself.
Thanks for clarifying.  I nonetheless remember strong opposition to
pulling the code unmodified, from MvL IIRC.

> (And, ironically enough, AFAIK the new packaging module uses code that's
> actually based on the bits of setuptools Fredrik was worried about
> supporting...
setuptools presence in packaging is
- packaging.database can read egg/PKG-INFO (zipped and unzipped) and
egg-info files
- packaging.install can detect that a project uses a setup.py with
setuptools, run that setup.py, and convert egg-info to dist-info

>  but at least there now are more people providing that support.)
Truth be told, I’m not sure it is so.  The student who worked on
packaging.database has not remained a member of our group; his mentor is
also less active.  But that’s not the hardest code in packaging.
Regarding installation, we do have people with distribute knowledge and
experience, so that’s good.

>> What we can do however is to see what bdist_egg does and define a new bdist
>> command inspired by it, but without zipping, pkg_resource calls, etc.
> Why?  If you just want a dumb bdist format, there's already bdist_dumb.
We’re not sure bdist_dumb is what we’re after—see my other messages.

> Conversely, if you want a smarter format, why reinvent wheels?
Recent packaging PEPs and distutils2 are all about reinventing wheels!
Or rather standardizing best practices for wheels.  Some ideas are taken
near-identical from setuptools, other see great changes.  In this case,
we have to define our requirements, and if bdist_egg can work (as a
distribution format, not an installation format!), then we may just take
it.  If it does not, we’ll have to make a new wheel.

Regards