That's a document describing how to use âdistutilsâ, which is what every recipient of Python will already have installed.

...

Can I ask that you also provide a link for windows users to my project:

http://sourceforge.net/projects/pythonpkgmgr/

That doesn't seem at all appropriate; promoting third-party packages isn't at all what the above document should be doing.

Distutils was once seperate and was then included in the standard python. So i guess that I am working with the same goal in mind.

If people want an alternative to the standard tools provided in Python, The Python Package Index <URL:http://pypi.python.org/> is the right place to be promoting (free-software) third-party tools.

Well I can sure try that and thank you for the tip. Btw, at the moment there exists no inbuilt mechanism within python for retrieving packages from pypi. Imho this is an omission for a so called 'batteries included' language. Distutils is a builtin module for 'pushing' a developer package 'to' pypi. But there is no corresponging mechanise for a user to 'pull' packages back. Surely this is a gap in the standard distro? So it is not inappropriate for me to ask about this on this list. Take care David

On Thu, 23 Jul 2009 18:30:58 +1000, Ben Finney <ben+python@benfinney.id.au> wrote:

In which case you should work to get it accepted into standard Python *before* asking for it to be promoted in the standard Python documentation.

I'm very interested in how I would go about doing that. Die-hard users probably know all the python issues relating to package installation and appear to have thick skins. An "easy-way" might not mean to much to them because they've settled down into their own comfort zone. But new users, who's first operating system comes complete with a polished GUI, have an expectation for having a polished GUI tool to download their python packages in. Under Windows, we get IDLE. And ActiveState as I understand are working on their own package manager for their own python distribution. So maybe it is appropriate to consider having a GUI Package Manager within at least the Windows distribution of Python. Yes, please tell me the process... I'm very interested. David

Raising it without at least glancing at the list archives which hold copious amounts of virtual text on that topic is somewhat inappropriate though :)

Well I have consulted every available expert on the distutils list to the point where I feel 'up' with the issues at hand. They're great people.. And as helpful as they possibly can be.. But surely Python isn't only about archives and age old problems. Sure they might be there.. So lets get in and fix them.. Which is totally what i'm attempting to do at the moment even if packaging isnt perceived as being the most interesting part of python develepment going on today. Inapropriate or not, i want to donate my time to it.. Because i think we need 'fresh' thinking - not archive regurgitation. David

On Thu, Jul 23, 2009 at 5:43 AM, <david.lyon@preisshare.net> wrote:

...

Raising it without at least glancing at the list archives which hold copious amounts of virtual text on that topic is somewhat inappropriate though :)

Well I have consulted every available expert on the distutils list to the point where I feel 'up' with the issues at hand.

They're great people.. And as helpful as they possibly can be..

But surely Python isn't only about archives and age old problems.

Sure they might be there.. So lets get in and fix them..

Which is totally what i'm attempting to do at the moment even if packaging isnt perceived as being the most interesting part of python develepment going on today.

Inapropriate or not, i want to donate my time to it.. Because i think we need 'fresh' thinking - not archive regurgitation.

David

Then why not include pip, easy_install, and this bash script I use to install packages into core? The more the merrier, right? Answer: None of these are standards, and as nick points out, there's issues with sysadmins, security, and other things. Not to mention they're fundamentally not part of the language. At the language summit, this was hashed out quite a bit. I think most everyone agreed that tools like easy_install, pip, virtualenv, zc.buildout, etc simply do not belong in core python. The "installation" landscape varies from platform to platform, it can run full in the face of people who just want to use apt or yum, etc. What *does* belong in core (distutils) however, and is something Tarek has been working on, are APIs/Hooks/standards for tools like the ones I've mentioned, and yours, to be able to do their job better, cleaner and easier. Standards on metadata, uninstall hooks, etc are what the stdlib should provide, not applications and tools built on top of those things. IMHO, the only "binary" python-core itself should "ship" is the python binary itself (and pydoc, maybe :-)) - everything else should be built with the idea of making integration with internals easier and standard. That way OS package maintainers can hook into these APIs (because they don't want to use something "non standard" to them), people such as yourself can build GUI apps for downloading and managing things, etc. I know you want to help, and I don't think anyone is discouraging that - but I think instead of solely focusing on your application is a mistake. The bulk of the effort should be spent helping Tarek and others "fix" distutils and spraying down bikeshedders so progress can be made. jesse

On Fri, Jul 24, 2009 at 11:47, Sridhar Ratnakumar <sridharr@activestate.com>wrote:

On Thu, 23 Jul 2009 06:11:38 -0700, Jesse Noller <jnoller@gmail.com> wrote:

Then why not include pip, easy_install, and this bash script I use to

...

install packages into core? The more the merrier, right? Answer: None of these are standards, and as nick points out, there's issues with sysadmins, security, and other things. Not to mention they're fundamentally not part of the language. At the language summit, this was hashed out quite a bit. I think most everyone agreed that tools like easy_install, pip, virtualenv, zc.buildout, etc simply do not belong in core python.

Once PEP 376 is implemented, my suggestion is to at least link to these tools (if not write a paragraph) as way of pointing users to alternative package managers that uses the 'uninstall' API. A mention of PyPI would also be helpful.

You can put that in the wiki, but it does not belong in the core documentation (except the PyPI mention as we do control that). That simply becomes a maintenance nightmare for use as we then have to keep up with the releases of external tools that we have no direct connection with. Plus it also give the illusion that Python has blessed these tools when in fact we have not. -Brett

On Fri, 24 Jul 2009 03:23:57 +0200, Christian Heimes <lists@cheimes.de> wrote:

I'm sorry to inform you that a wxWindows based solution has zero change to get into the Python standard library ever. We are not going to add another GUI toolkit to the core distribution.

In executable form, the Package Manager does not require wxWidgets to be installed. There is no dependency for this to be installed. I'm not requesting that wxPython be added to the standard python distribution.

We might even remove TK from the core and ship it as a separate package like bsddb3.

That doesn't affect the Python Package Manager in any way. David

2009/7/24 Georg Brandl <g.brandl@gmx.net>:

David Lyon schrieb:

...

On Fri, 24 Jul 2009 03:23:57 +0200, Christian Heimes <lists@cheimes.de> wrote:

...

I'm sorry to inform you that a wxWindows based solution has zero change to get into the Python standard library ever. We are not going to add another GUI toolkit to the core distribution.

In executable form, the Package Manager does not require wxWidgets to be installed.

There is no dependency for this to be installed.

What does "in exectuable form" mean?

I read this as meaning that David was proposing to ship a built application (on Windows, bundled up with something like py2exe, I guess) and any supporting DLLs such as the wxWindows ones would be bundled in, but the wxPython package would *not* be shipped as part of the standard library. If my assumption is right, that makes the package manager inappropriate for shipping with Python, whether or not there are other issues. In my view, for an "application" to be shipped with Python, whether it be Idle, or a package manager, or pydoc, or whatever, it must: - be written as a Python script - depend only on packages shipped with the standard library That is before any considerations of the value of the application itself. Given that David's package manager uses setuptools to do the installs (and now it appears it uses wxPython for its GUI) that immediately disqualifies it for inclusion in my view - regardless of its value as a package. Paul.

At 08:09 PM 7/24/2009 -0400, David Lyon wrote:

Presently it used pkg_resources to read the list of packages installed which is part of setuptools. I was told it was the "right" and only way to read a list of packages.

"Right" is relative, but right now it is certainly the *only* way to read a list of installed packages, that's compatible with both distutils and setuptools-based ways of installing packages (including of course pip, enstaller, and buildout). PEP 376 proposes a stdlib replacement for a portion of this functionality, but the status of whether it's actually going to support anything other than distutils, pip, and setuptools' backward-compatibility mode is apparently still up for grabs.

On Fri, 24 Jul 2009 17:54:09 +0200, Georg Brandl <g.brandl@gmx.net> wrote:

David Lyon schrieb:

...

In executable form, the Package Manager does not require wxWidgets to be installed.

There is no dependency for this to be installed.

What does "in exectuable form" mean?

Compiled with py2exe. An executable ".exe" file is created under windows. David

david.lyon@preisshare.net writes:

Distutils was once seperate and was then included in the standard python.

So i guess that I am working with the same goal in mind.

I interpret this as expressing your intent to (eventually) have your application included in standard Python. David Lyon <david.lyon@preisshare.net> writes:

On Fri, 24 Jul 2009 03:23:57 +0200, Christian Heimes <lists@cheimes.de> wrote:

...

I'm sorry to inform you that a wxWindows based solution has zero change to get into the Python standard library ever.

I wasn't suggesting including it in the standard library as I like others know how troublesome it can be.

This appears to contradict what you said above.

...

We are not going to add another GUI toolkit to the core distribution. We might even remove TK from the core and ship it as a separate package like bsddb3.

So is this a bias against native window applications?

No, it's a bias against adding things to Python that depend on things not already in Python. Are you, or are you not, intending for your application to be in standard Python? If that is your intent, I think Christian's answer is valid: applications, especially ones that depend on third-party widget libraries, shouldn't get added to Python. If that is not your intent, then your application shouldn't be mentioned in standard Python documentation. -- \ “Stop — Drive sideways.” —detour sign, Kyushu, Japan | `\ | _o__) | Ben Finney

David Lyon <david.lyon@preisshare.net> writes:

Not at all. In source form the pythonpkgmgr requires wx package. In executable form it does not.

The only way it could be added is in source form; that's essential for free software like Python. So, if it's not suitable for adding to Python in source form, it's not suitable for adding to Python.

...

No, it's a bias against adding things to Python that depend on things not already in Python.

That implies that nothing new can be added then.

You omit the important part: adding a new thing to Python *so long as it doesn't depend on anything outside Python*.

The Python Package Manager can be written to work in console mode.

I think this would be best. The functionality you often discuss around this tool would be best implemented independently of any UI. It would, in fact, be best to work with the team performing ongoing active standardisation of distutils functionality. -- \ “Consider the daffodil. And while you're doing that, I'll be | `\ over here, looking through your stuff.” —Jack Handey | _o__) | Ben Finney

On Sat, 25 Jul 2009 11:25:27 +1000, Ben Finney <ben+python@benfinney.id.au>

You omit the important part: adding a new thing to Python *so long as it doesn't depend on anything outside Python*.

I'm signing out on this silly discussion for now.... Any python program is dependant on things "outside" python. For example an operating system.. a computer... and a user perhaps for without such it wouldn't be possible to generate the all important 1Di0t errors that we so often have to deal with.

...

The Python Package Manager can be written to work in console mode.

I think this would be best.

Haha - I'm glad somebody took this seriously... It was a sort of a joke comment but it's a serious possibility.

The functionality you often discuss around this tool would be best implemented independently of any UI.

It is anyway. That's why doing an entirely different UI isn't any major issue. I have a class and a possible command line interface behind the scenes.

It would, in fact, be best to work with the team performing ongoing active standardisation of distutils functionality.

I am already doing that. But there is a bias against windows development and a bias against native applications. That's fine because I know they are using python on different platforms. I'm working on a proposal to make setup.py object oriented and "modern". http://wiki.python.org/moin/Distutils/Proposals So I'm doing as much as I can - really. David

On Sat, 25 Jul 2009 16:00:13 +1000, Ben Finney <ben+python@benfinney.id.au> wrote:

I think you know quite well what “depend on” means in this instance, so this is taking it to silly extremes.

haha - yes - no offence. It was just bad humour. Have a nice weekend David

On Sat, 25 Jul 2009 10:28:51 +0100, Paul Moore <p.f.moore@gmail.com> wrote:

??? I see no bias as you describe in the distutils enhancement work.

ok

Native applications are by definition not platform neutral. How does your proposal help Linux users? Mac OS? Solaris?

I'm doing a Linux/Solaris version. But I find issues every day that must be addressed.

If your concern is to make things easier for Windows users, then your application is worthwhile, but it should probably remain an external project. If it gets overwhelming support, maybe providing a standardised version with a simplified UI in the core would be an option.

That sounds ok.

How many users do you have currently?

Approx 250 downloads this (first) month for the windows version. I need to extend coverage to Linux and the Mac. Given that I'm relatively new to all this there's a learning curve. But so far so good. David

On Sat, 25 Jul 2009 12:47:21 -0400, Terry Reedy <tjreedy@udel.edu> wrote:

...

...

...

The Python Package Manager can be written to work in console mode. I think this would be best.

Haha - I'm glad somebody took this seriously... It was a sort of a joke comment but it's a serious possibility.

I took it seriously too ;-). It seems to me that you project can have at least 3 components.

It's logical and plausible.

1) the core library module, which might or might not be targeted at eventual stdlib inclusion.

I'm sure they'll fix distutils someday, and if they do, then I'll use that.

2) a small console driver script. 3) one or more GUI scripts.

It's possible. I'll give it serious consideration.

A TK version would be good from my viewpoint as it does not require download and installation of any other binaries.

Sure. Who cares if it's not as slick as something else. I'll try when I have time. David

On Sat, 25 Jul 2009 11:42:13 +0200, "Martin v. Löwis" <martin@v.loewis.de> wrote:

...

It's my intention to get a Package Manager included in standard python - yes.

In addition to the other constraints you'll have to meet for this to happen, you also have to wait a rather long time (several years) before inclusion becomes possible. This time is necessary for the community to accept your tool, and evaluate it. Ideally, the request for inclusion should not come from you, but from your users.

Yes. But users have been asking for a tool and complaining loudly about the lack of such a tool. I know you're an extremely competent and those emails you perphaps floss over. But the pleas are many and when we compare python to other languages, python rates towards at the low end of the spectrum its third party-package management facilities. You can't seriously expect users to wait for years for an integrated package management tool. Especially on Windows - that's cruel :-) To date, there just hasn't been any movement on the building of such a tool due to whatever reasons. A Package Manager isn't a frivolous "nice-to-have" tool. It's essential for a new user. (if you want I can sift the last twelve months worth of ML archives and report on how many times easier package management has been asked for. And how difficult people are finding it)

No. It implies that any addition could only depend on Python (and OS API) - so things *can* be added. For example, setuptools could be added by this principle. OTOH, if your tool depends on setuptools, you'll have to wait for setuptools to get included before inclusion of your tool can be considered.

Let's get precise. It doesn't "depend" on setuptools. But it will install setuptools if the user requests to use setuptools/easy_install. So we should only be back to the lack of a TK interface and the fact that the Package Manager Project is a new project, and needs to be working properly on more platforms. Thanks for your email. I appreciate the feedback, from everyone. Take care. David

On Sun, 26 Jul 2009 08:43:07 +0200, "Martin v. Löwis" <martin@v.loewis.de> wrote:

Hmm. I'm -0 on providing a tool whose only purpose is to download files from a web server. I always use a web browser for that...

It does a lot more than that. Firstly it shows what packages you already have installed and lets you manage them. Namely, deinstall, (manually) upgrade, view documentation and examples. In any case, even if it were only a tool to download packages it's still consistant with modern appliance design (mobile phones, programming languages, dedicated hardware) to have an internal application where a user can pretty easily download "checked" apps to their device/system. Nobody gives people just a browser to do that... and tell them "go browse". It isn't being done like that - except in python.

Hmm. I would expect that a new user is faced with the challenge of finding out what packages to install more so than with actually installing them.

Yes. That's exactly the point. You're 100% right. That's why the pythonpkgmgr provides a useful search capability for pypi packages. It's faster and more natural to use a native app than do it all in the browser. That's true in Perl (cpan) as in Python and I'm honestly saying that I can't find any noteable faults in pypi the way I find it.

If they read examples, they will see import statements, and then they have to find out how to make those work. Does your tool help with that?

Yes. It will open the website or homepage to the project/package in question. And find any documentation that might be stored in the package directory. If you only have "import interbasedb" it's a complicated process for a new user to located the home page or project documentation without pythonpkgmgr. Using pythonpkgmgr, they look on their list of installed packages, find the "interbasedb" or whatever module, and can get quick access to the pypi page or the package homepage. It's much easier. Much faster.. especially if the new user doesn't know the internals of package storage, pypi and the like.

When the user is not so new anymore, I seriously doubt that they still ask for a package management tool - except perhaps for dependencies, and here they use easy_install.

Well, that might be one use-case. It's true that many python programmers will just want to stick to their own "known" packages. However, this tool aids people with more curiosity. Because users can so easily (and safely) install and deinstall anything. It's only a click to install a package, and another to deinstall. At the moment, pythonpkgmgr doesn't handle dependencies except that which is provided internally by easy_install/pip. I'm hoping to change this as my experience and understanding grows. David

On Sun, 26 Jul 2009 19:31:40 +0200, "Martin v. Löwis" <martin@v.loewis.de> wrote:

...

...

If they read examples, they will see import statements, and then they have to find out how to make those work. Does your tool help with that?

Yes. It will open the website or homepage to the project/package in question.

How does it know the project in question?

Because it is either in the pypi search results or list of installed packages.

How do they find interbasedb in the list of installed packages if they haven't installed interbasedb yet?

They'd already installed it. But perphaps through a .exe installer. David

Steven D'Aprano wrote:

I'm a user, and personally I don't want Yet Another Integrated Package Management Tool. What I really want is the ability to install Python packages using the PM tool I already use, namely yum. (And I'd like a pony.)

Picking up on this point... out of curiousity, I threw a couple of package names at "apt-get -s" on my main (Ubuntu 8.04) PC to see what worked. The first two names I tried (numpy, scipy) didn't work, but the next four (python-numpy, python-scipy, python-django, python-turbogears) all had hits. Because Turbo Gears incorporates so many other projects, the simulated run at installing it provided a nice overview of a variety of other Python packages in the Ubuntu repositories as well. No doubt a search in Adept would reveal a whole lot more "python-*" packages that are just a few clicks away for me. </tangent> Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------

David Lyon wrote:

On Sun, 26 Jul 2009 18:05:07 +1000, Steven D'Aprano <steve@pearwood.info> wrote:

...

But you shouldn't expect the Python dev team to accept an unproven tool into the official library before demonstrating both the need and the solution.

Of course... that's why I started off by asking what the process is.

I am accepting that the current status of pythonpkgmgr is "unproven" so it will need some time for things to run their course.

It would be great to have a decent visual package manager for Python. It needs to be built on top of the work that Tarek is doing with distutils (and be compatible with his Distribute fork of setuptools) and it also needs to be established. The good news is that there is time for both... Aren't there other tools that have similar goals? PyPI browser, Yolk etc It is a shame that it needs to be built on Tk if it is to stand a chance of ever being included in the standard library - but with the right expertise it is *possible* to create decent looking UIs with Tk. Michael

...

(Just because users say they want something, doesn't mean they'll actually use the tool you provide -- perhaps they don't know what they really need, and perhaps your tool doesn't match their needs.)

Conversely, the process of satisfying needs is to discuss them. And people have been ever so helpful in that regard.

...

I'm a user, and personally I don't want Yet Another Integrated Package Management Tool. What I really want is the ability to install Python packages using the PM tool I already use, namely yum.

ok - but no alternative to that is available on windows.

pythonpkgmgr is not so different to that. And the idea behind it is to bring consistancy in package management across the different platforms.

It does have configuration that are specific to python, and will continue to do so. Like being able to specify where one wants the package installed.

...

Putting your software on the Cheeseshop, and making regular announcements to the Python community (e.g. on comp.lang.python) will be a good way to publicise the tool, and if does meet a need, people will use it, and then, if it's good enough and popular enough and supported, it may be blessed by inclusion in the standard library.

Thanks very much. Take care.

David _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.u...

-- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog

On Mon, Jul 27, 2009 at 7:20 PM, David Lyon<david.lyon@preisshare.net> wrote:

My only point is that Windows ain't no embedded system. It's not short on memory or disk space. If a package manager is 5 megabytes extra say, with it's libraries.. what's the extra download time on that ? compared to three days+ stuffing around trying to find out how to install packages for a new user.

The problem is not so much the size by itself that more code means more maintenance burden for python developers. Including new code means it has to work everywhere where python works currently, and that other people can understand/support the related code. Adding code to a project is far from free from python maintainers POV. cheers, David

...

I'm a user, and personally I don't want Yet Another Integrated Package Management Tool. What I really want is the ability to install Python packages using the PM tool I already use, namely yum.

ok - but no alternative to that is available on windows.

For removal of packages, an alternative *is* available: APR.

pythonpkgmgr is not so different to that. And the idea behind it is to bring consistancy in package management across the different platforms.

At the cost of being inconsistent within a platform. Regards, Martin

On Mon, 27 Jul 2009 11:18:25 +0900, "Stephen J. Turnbull" <stephen@xemacs.org> wrote:

[1] on my part") and sysadmin goals ("something that works and plays nicely with the rest of the system").

pythonpkgmgr seems entirely oblivious to the latter issue, and not particularly compatible with the way package management works in *nix OSS distros (including Linux distros, but also *BSD, MacPorts, and Fink).

Well I'm a sysadmin on linux in my day job. I'm entirely wondering what you mean by that ? I'm oblivious how? What can a developer mode project, like pythonpkgmgr possibly do except to make it easier to manage local package space? A developer shouldn't need to do sudo every time they want to put or try some python package from pypi. imho that's wrong and I'm helping to reduce that need. If the sysadmin puts a package in (/usr/lib/pythonX.Y/site-packages) via the O/S package manager (ie synaptics/yum/apt) then that should stick over what a developer is doing. But the developer should be able to stick their own packages in their own local space. Correct me if my assertion is wrong. David

On Mon, 27 Jul 2009 20:12:54 +0900, "Stephen J. Turnbull" <stephen@xemacs.org> wrote:

Not you; pythonpkgmgr. You've said nothing about how pythonpkgmgr is supposed to deal with multiple installed versions of Python

Under windows it can deal with multiple versions of python. You just go to options and select which version of python you wish to manage. It's as simple as that.

or how it helps if you need a custom version of a module, etc., or how it can help the developer's "local" modules cooperate/not interfere with system versions, etc.

It manages local developer modules for python 2.6+. As for checks, it doesn't have so many at present. But it makes it a lot easier to see what you have installed. So if something needs to be removed and reinstalled - it can do that.

Pave the road to DLL Hell with good intentions, of course.

Well - there's plenty of that

...

A developer shouldn't need to do sudo every time they want to put or try some python package from pypi.

I don't remember *ever* sudo-ing to try a package from pypi.

ok. But if you were using an O/S level package manager you would have to enter a su password to start the tool. Because that is needed to load packages into /usr/lib/pythonX.y

I just don't understand what problem you're claiming to solve.

Making it simpler for new users to find and install packages from pypi. And manage the existing packages that are installed on their machine.

The problem of managing local package space is in some sense solved by virtualenv, but you haven't mentioned that at all.

I never heard of it till a few months ago. To my knowledge it doesn't offer a gui. I'm told it's good for certain things and I believe it. My problem was installing packages on windows boxes, not in the development.

Is pythonpkgmgr an alternative? A complement? to virtualenv or zc.buildout?

pythonpkgmgr is aimed at featherweight users. It could be highly complementory to both virtualenv or zc.buildout at some stage if it were customised to work with those. For example, it could work with virtualenv by having a a drop down list of the different 'environments'. I don't know how to do it right now - but I'm hoping I can figure it out sometime. With zc.buildout, there's no reason why it couldn't generate a package list in that format, or do an import from a manifest. pythonpkgmgr already can generate a list of local packages as a manifest - but not in the zc.buildout format. I'm sure in the future - I'll figure that one out as well. David

On Sun, 26 Jul 2009 19:33:37 +0200, "Martin v. Löwis" <martin@v.loewis.de> wrote:

...

pythonpkgmgr is not so different to that. And the idea behind it is to bring consistancy in package management across the different platforms.

At the cost of being inconsistent within a platform.

It has the most generic of user interfaces. On Windows, a command line interface is the most out-of-place thing anybody could ask for. 19/20 kids these days wouldn't even know what a command line interface is until you show them. The first thing that they would look for is a GUI tool in the Python X.Y programs menu... So I respectfully say that there couldn't be anything less true than the assertion that a GUI package manager is inconsistent within a modern GUI desktop environment. David

On Tue, 28 Jul 2009 07:12:25 +0200, "Martin v. Löwis" <martin@v.loewis.de> wrote:

So there are now two incompatible ways to install a package: either with the native manager, or with pythonpkgmgr. If you install them one way, and try to remove them the other way, you lose.

pythonpkgmgr is only a thin wrapper for easy_install/pip. If there is a problem, then it is already a pre-existing problem that is not of my doing. It's certainly true that if a python package is installed with a windows installer or as a .deb/.rpm, then that package should be uninstalled using the appropriate and corresponding tool. The same is true if the package was installed with either pip or easy_install. All I can do is put it in my tracker as a bug and deal with it later when I have time. Good point David

On Tue, 28 Jul 2009 07:55:11 +0200, "Martin v. Löwis" <martin@v.loewis.de> wrote:

Yes, eggs have the same problem. That's one of the reasons they don't get integrated into Python.

Yes but egg_info is included in python... and the egg is not.... Hence, what goes in and what doesn't isn't always that rational. I'm just accepting that for the moment. David

On Tue, 28 Jul 2009 11:50:00 +0100, Paul Moore <p.f.moore@gmail.com> wrote:

egg_info data is in to allow "standard" (setup.py install and hence OS package manager managed) packages to provide metadata in a discoverable way. Using a format that is (reasonably) compatible with setuptools is simply a matter of co-operating with existing de facto standards.

Eggs themselves (as a distribution format) are just zip files with a funny extension, and as such are supported by Python.

ok - I get it..... It's all one big monty python comedy thing where things are quirky and inconsistent - there's little continuety from one scene to the next but you have to stop sometimes and have a laugh... because to word quote from you - "funny" Your whole email whilst perphaps technically correct is terribly difficult for a software engineering person to follow. Monty python was only meant to be a funny film for entertainment, not a philosophy for building software from.... Why we can have the egg_info without allowing the egg is kind of like having a chicken and egg story without allowing the chicken.. or is it the egg.. oh I lost track... I think I need to watch the movie again to understand what's happening here... Let me go away confused... don't ask me any more questions or elaborate with more answers. Thanks. :-) David

David Lyon wrote:

Your whole email whilst perphaps technically correct is terribly difficult for a software engineering person to follow.

It made perfect sense to me. The words "eggs" brings with it a whole lot more baggage than just the sum of the technical parts in the language core that support them (primarily distutils and zipimport). I find it unfortunate that the name for the distutils metadata format contains the word "egg" because it implies much greater consensus around the philosophy behind eggs than actually exists. A lot of the baggage associated with the "eggs" concept is related to the inherent conflict between different approaches to dependency management: 1. Use the system package management system for everything (preferred by many, perhaps even most, *nix sysadmins, but not an option on Windows) 2. Create a Python specific package management system independent of the system package manager (an area dominated by setuptools, including both eggs and non-zipped package distributions) 3. Bundle everything into a monolithic application or framework (the typical approach on Windows with py2exe, but also the philosophy behind tools like virtualenv) All 3 of those philosophies have good arguments in their favour and they all have good arguments against them as well. Your comments about your package management system suggest that it is just yet another entrant in category 2 and you have said nothing to allay the concerns of those that despise that philosophy with a passion because of the problems it causes them when trying to manage their systems using the first philosophy. Since the Python constituency includes developers and system administrators that favour all 3 philosophies (and probably other variants that I haven't thought to describe), anything that makes it into the standard library will need to adequately balance the interests of all parties (e.g. as has occurred in the PEP 376 metadata enhancement discussions). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------

On Wed, 29 Jul 2009 10:56:20 +1000, Nick Coghlan <ncoghlan@gmail.com> wrote:

The words "eggs" brings with it a whole lot more baggage than just the sum of the technical parts in the language core that support them (primarily distutils and zipimport).

Well, in this case, (talking metaphorically) we have the ability to roll the eggs, crack a whole in them and suck out the contents (deal with a package in a zipped egg) So metaphorically we could say that python can work with egg shells.. As for what's in the egg... well lets just say that it's a bit floaty.. And perphaps that is the best way for things to be for a while.

I find it unfortunate that the name for the distutils metadata format contains the word "egg" because it implies much greater consensus around the philosophy behind eggs than actually exists.

I see it a different way. I thinks eggs are simple - as in like a java bean. A simple way to move a package from one place to another. What seems to have gone wrong is that there is a lot of pioneering and interconnected and interdependent technology within them. They're an egg.. but in the past they've had little monsters that bite your fingers when you try to put them in the pan... :-) I think "Egg" term is very good. But maybe we are best not trying to over-specify their contents. Maybe we should say it has certain things (EGG_INFO, PACKAGE DATA) as in (YOLK, WHITE) and pretty much leave it at that. If more detail is required - rtm.

A lot of the baggage associated with the "eggs" concept is related to the inherent conflict between different approaches to dependency management:

That's not an egg problem. That's a packaging/metadata problem. It's the package dependency issue that's the problem. That's a distutils problem.

1. Use the system package management system for everything (preferred by many, perhaps even most, *nix sysadmins, but not an option on Windows)

Yes, because the package dependency issues are just so great.

2. Create a Python specific package management system independent of the system package manager (an area dominated by setuptools, including both eggs and non-zipped package distributions)

More work needs to go into distutils. Not taking away from any existing work - but setuptools has relied on the deficiencies of distutils to gain a foothold. Fix distutils (give me time to think..) and the need for setuptools and any further fork is probably negated.

3. Bundle everything into a monolithic application or framework (the typical approach on Windows with py2exe, but also the philosophy behind tools like virtualenv)

Windows users are using py2exe and other products over distutils. To a normal windows developer, distutils makes imho no sense in the way that it goes about things now. For example, very simple concepts like "program directories", (where you stick the program) isn't an available option in distutils. But it is the first thing that you need to know in a windows program. So it's very hard to get to step 1...

Your comments about your package management system suggest that it is just yet another entrant in category 2 and you have said nothing to allay the concerns of those that despise that philosophy with a passion because of the problems it causes them when trying to manage their systems using the first philosophy.

I'm a windows user.. I can't be in category #2.. I'm in category #1, have nothing... (except now my own tool)

Since the Python constituency includes developers and system administrators that favour all 3 philosophies (and probably other variants that I haven't thought to describe), anything that makes it into the standard library will need to adequately balance the interests of all parties (e.g. as has occurred in the PEP 376 metadata enhancement discussions).

Well at least you are saying that there is some way of reconciling all the different options... There's an awful lot to take in, and there must be 20,000 lines of emails for every 1 line of python code that is required to fix this thing. Take care David

On Wed, Jul 29, 2009 at 2:36 AM, David Lyon<david.lyon@preisshare.net> wrote:

On Wed, 29 Jul 2009 10:56:20 +1000, Nick Coghlan <ncoghlan@gmail.com> wrote:

...

The words "eggs" brings with it a whole lot more baggage than just the sum of the technical parts in the language core that support them (primarily distutils and zipimport).

Well, in this case, (talking metaphorically) we have the ability to roll the eggs, crack a whole in them and suck out the contents (deal with a package in a zipped egg)

So metaphorically we could say that python can work with egg shells..

As for what's in the egg... well lets just say that it's a bit floaty..

And perphaps that is the best way for things to be for a while.

...

I find it unfortunate that the name for the distutils metadata format contains the word "egg" because it implies much greater consensus around the philosophy behind eggs than actually exists.

I see it a different way. I thinks eggs are simple - as in like a java bean. A simple way to move a package from one place to another.

What seems to have gone wrong is that there is a lot of pioneering and interconnected and interdependent technology within them. They're an egg.. but in the past they've had little monsters that bite your fingers when you try to put them in the pan... :-)

I think "Egg" term is very good. But maybe we are best not trying to over-specify their contents.

Maybe we should say it has certain things (EGG_INFO, PACKAGE DATA) as in (YOLK, WHITE) and pretty much leave it at that. If more detail is required - rtm.

...

A lot of the baggage associated with the "eggs" concept is related to the inherent conflict between different approaches to dependency management:

That's not an egg problem. That's a packaging/metadata problem.

It's the package dependency issue that's the problem. That's a distutils problem.

...

1. Use the system package management system for everything (preferred by many, perhaps even most, *nix sysadmins, but not an option on Windows)

Yes, because the package dependency issues are just so great.

...

2. Create a Python specific package management system independent of the system package manager (an area dominated by setuptools, including both eggs and non-zipped package distributions)

More work needs to go into distutils. Not taking away from any existing work - but setuptools has relied on the deficiencies of distutils to gain a foothold.

Fix distutils (give me time to think..) and the need for setuptools and any further fork is probably negated.

...

3. Bundle everything into a monolithic application or framework (the typical approach on Windows with py2exe, but also the philosophy behind tools like virtualenv)

Windows users are using py2exe and other products over distutils. To a normal windows developer, distutils makes imho no sense in the way that it goes about things now.

For example, very simple concepts like "program directories", (where you stick the program) isn't an available option in distutils. But it is the first thing that you need to know in a windows program.

So it's very hard to get to step 1...

...

Your comments about your package management system suggest that it is just yet another entrant in category 2 and you have said nothing to allay the concerns of those that despise that philosophy with a passion because of the problems it causes them when trying to manage their systems using the first philosophy.

I'm a windows user.. I can't be in category #2..

I'm in category #1, have nothing... (except now my own tool)

...

Since the Python constituency includes developers and system administrators that favour all 3 philosophies (and probably other variants that I haven't thought to describe), anything that makes it into the standard library will need to adequately balance the interests of all parties (e.g. as has occurred in the PEP 376 metadata enhancement discussions).

Well at least you are saying that there is some way of reconciling all the different options...

There's an awful lot to take in, and there must be 20,000 lines of emails for every 1 line of python code that is required to fix this thing.

Take care

David

I really do think this mail thread needs to move to disutils-sig or python-ideas. jesse

On Sat, 25 Jul 2009 10:40:52 +1000, Ben Finney <ben+python@benfinney.id.au> wrote:

[snip]

If that is not your intent, then your application shouldn't be mentioned in standard Python documentation.

Hm. But docutils isn't part of the standard library, and the documentation refers to it. And the docs link to ActivePython and Enthought's Python distribution. And they link to things like "Collin Winter’s functional module" and Django and PyOpenGL and so forth. I don't think anyone would seriously suggest removing all references to third-party software from the Python documentation. Perhaps the criterion that David is looking to have is explained is that people would probably prefer for his app to get a bit more popular before Python itself starts pointing people in its direction. Jean-Paul

On Fri, Jul 24, 2009 at 18:06, Jean-Paul Calderone <exarkun@divmod.com>wrote:

On Sat, 25 Jul 2009 10:40:52 +1000, Ben Finney <ben+python@benfinney.id.au<ben%2Bpython@benfinney.id.au>> wrote:

...

[snip]

If that is not your intent, then your application shouldn't be mentioned in standard Python documentation.

Hm. But docutils isn't part of the standard library, and the documentation refers to it.

Why do you think distutils is not part of the standard library? The official code location is Lib/distutils within the standard library. -Brett

On 2009-07-25 17:28, Brett Cannon wrote:

On Fri, Jul 24, 2009 at 18:06, Jean-Paul Calderone <exarkun@divmod.com <mailto:exarkun@divmod.com>> wrote:

On Sat, 25 Jul 2009 10:40:52 +1000, Ben Finney <ben+python@benfinney.id.au <mailto:ben%2Bpython@benfinney.id.au>> wrote:

[snip]

If that is not your intent, then your application shouldn't be mentioned in standard Python documentation.

Hm. But docutils isn't part of the standard library, and the documentation refers to it.

Why do you think distutils is not part of the standard library? The official code location is Lib/distutils within the standard library.

He said "docutils", not "distutils". -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco

That's a document describing how to use âdistutilsâ, which is what every recipient of Python will already have installed.

...

Can I ask that you also provide a link for windows users to my project:

http://sourceforge.net/projects/pythonpkgmgr/

That doesn't seem at all appropriate; promoting third-party packages isn't at all what the above document should be doing.

Distutils was once seperate and was then included in the standard python. So i guess that I am working with the same goal in mind.

If people want an alternative to the standard tools provided in Python, The Python Package Index <URL:http://pypi.python.org/> is the right place to be promoting (free-software) third-party tools.

Well I can sure try that and thank you for the tip. Btw, at the moment there exists no inbuilt mechanism within python for retrieving packages from pypi. Imho this is an omission for a so called 'batteries included' language. Distutils is a builtin module for 'pushing' a developer package 'to' pypi. But there is no corresponging mechanise for a user to 'pull' packages back. Surely this is a gap in the standard distro? So it is not inappropriate for me to ask about this on this list. Take care David

On Thu, 23 Jul 2009 18:30:58 +1000, Ben Finney <ben+python@benfinney.id.au> wrote:

In which case you should work to get it accepted into standard Python *before* asking for it to be promoted in the standard Python documentation.

I'm very interested in how I would go about doing that. Die-hard users probably know all the python issues relating to package installation and appear to have thick skins. An "easy-way" might not mean to much to them because they've settled down into their own comfort zone. But new users, who's first operating system comes complete with a polished GUI, have an expectation for having a polished GUI tool to download their python packages in. Under Windows, we get IDLE. And ActiveState as I understand are working on their own package manager for their own python distribution. So maybe it is appropriate to consider having a GUI Package Manager within at least the Windows distribution of Python. Yes, please tell me the process... I'm very interested. David

Raising it without at least glancing at the list archives which hold copious amounts of virtual text on that topic is somewhat inappropriate though :)

Well I have consulted every available expert on the distutils list to the point where I feel 'up' with the issues at hand. They're great people.. And as helpful as they possibly can be.. But surely Python isn't only about archives and age old problems. Sure they might be there.. So lets get in and fix them.. Which is totally what i'm attempting to do at the moment even if packaging isnt perceived as being the most interesting part of python develepment going on today. Inapropriate or not, i want to donate my time to it.. Because i think we need 'fresh' thinking - not archive regurgitation. David

On Thu, Jul 23, 2009 at 5:43 AM, <david.lyon@preisshare.net> wrote:

...

Raising it without at least glancing at the list archives which hold copious amounts of virtual text on that topic is somewhat inappropriate though :)

Well I have consulted every available expert on the distutils list to the point where I feel 'up' with the issues at hand.

They're great people.. And as helpful as they possibly can be..

But surely Python isn't only about archives and age old problems.

Sure they might be there.. So lets get in and fix them..

Which is totally what i'm attempting to do at the moment even if packaging isnt perceived as being the most interesting part of python develepment going on today.

Inapropriate or not, i want to donate my time to it.. Because i think we need 'fresh' thinking - not archive regurgitation.

David

Then why not include pip, easy_install, and this bash script I use to install packages into core? The more the merrier, right? Answer: None of these are standards, and as nick points out, there's issues with sysadmins, security, and other things. Not to mention they're fundamentally not part of the language. At the language summit, this was hashed out quite a bit. I think most everyone agreed that tools like easy_install, pip, virtualenv, zc.buildout, etc simply do not belong in core python. The "installation" landscape varies from platform to platform, it can run full in the face of people who just want to use apt or yum, etc. What *does* belong in core (distutils) however, and is something Tarek has been working on, are APIs/Hooks/standards for tools like the ones I've mentioned, and yours, to be able to do their job better, cleaner and easier. Standards on metadata, uninstall hooks, etc are what the stdlib should provide, not applications and tools built on top of those things. IMHO, the only "binary" python-core itself should "ship" is the python binary itself (and pydoc, maybe :-)) - everything else should be built with the idea of making integration with internals easier and standard. That way OS package maintainers can hook into these APIs (because they don't want to use something "non standard" to them), people such as yourself can build GUI apps for downloading and managing things, etc. I know you want to help, and I don't think anyone is discouraging that - but I think instead of solely focusing on your application is a mistake. The bulk of the effort should be spent helping Tarek and others "fix" distutils and spraying down bikeshedders so progress can be made. jesse

On Fri, Jul 24, 2009 at 11:47, Sridhar Ratnakumar <sridharr@activestate.com>wrote:

On Thu, 23 Jul 2009 06:11:38 -0700, Jesse Noller <jnoller@gmail.com> wrote:

Then why not include pip, easy_install, and this bash script I use to

...

install packages into core? The more the merrier, right? Answer: None of these are standards, and as nick points out, there's issues with sysadmins, security, and other things. Not to mention they're fundamentally not part of the language. At the language summit, this was hashed out quite a bit. I think most everyone agreed that tools like easy_install, pip, virtualenv, zc.buildout, etc simply do not belong in core python.

Once PEP 376 is implemented, my suggestion is to at least link to these tools (if not write a paragraph) as way of pointing users to alternative package managers that uses the 'uninstall' API. A mention of PyPI would also be helpful.

You can put that in the wiki, but it does not belong in the core documentation (except the PyPI mention as we do control that). That simply becomes a maintenance nightmare for use as we then have to keep up with the releases of external tools that we have no direct connection with. Plus it also give the illusion that Python has blessed these tools when in fact we have not. -Brett

On Fri, 24 Jul 2009 03:23:57 +0200, Christian Heimes <lists@cheimes.de> wrote:

I'm sorry to inform you that a wxWindows based solution has zero change to get into the Python standard library ever. We are not going to add another GUI toolkit to the core distribution.

In executable form, the Package Manager does not require wxWidgets to be installed. There is no dependency for this to be installed. I'm not requesting that wxPython be added to the standard python distribution.

We might even remove TK from the core and ship it as a separate package like bsddb3.

That doesn't affect the Python Package Manager in any way. David

2009/7/24 Georg Brandl <g.brandl@gmx.net>:

David Lyon schrieb:

...

On Fri, 24 Jul 2009 03:23:57 +0200, Christian Heimes <lists@cheimes.de> wrote:

...

I'm sorry to inform you that a wxWindows based solution has zero change to get into the Python standard library ever. We are not going to add another GUI toolkit to the core distribution.

In executable form, the Package Manager does not require wxWidgets to be installed.

There is no dependency for this to be installed.

What does "in exectuable form" mean?

I read this as meaning that David was proposing to ship a built application (on Windows, bundled up with something like py2exe, I guess) and any supporting DLLs such as the wxWindows ones would be bundled in, but the wxPython package would *not* be shipped as part of the standard library. If my assumption is right, that makes the package manager inappropriate for shipping with Python, whether or not there are other issues. In my view, for an "application" to be shipped with Python, whether it be Idle, or a package manager, or pydoc, or whatever, it must: - be written as a Python script - depend only on packages shipped with the standard library That is before any considerations of the value of the application itself. Given that David's package manager uses setuptools to do the installs (and now it appears it uses wxPython for its GUI) that immediately disqualifies it for inclusion in my view - regardless of its value as a package. Paul.

At 08:09 PM 7/24/2009 -0400, David Lyon wrote:

Presently it used pkg_resources to read the list of packages installed which is part of setuptools. I was told it was the "right" and only way to read a list of packages.

"Right" is relative, but right now it is certainly the *only* way to read a list of installed packages, that's compatible with both distutils and setuptools-based ways of installing packages (including of course pip, enstaller, and buildout). PEP 376 proposes a stdlib replacement for a portion of this functionality, but the status of whether it's actually going to support anything other than distutils, pip, and setuptools' backward-compatibility mode is apparently still up for grabs.

On Fri, 24 Jul 2009 17:54:09 +0200, Georg Brandl <g.brandl@gmx.net> wrote:

David Lyon schrieb:

...

In executable form, the Package Manager does not require wxWidgets to be installed.

There is no dependency for this to be installed.

What does "in exectuable form" mean?

Compiled with py2exe. An executable ".exe" file is created under windows. David

david.lyon@preisshare.net writes:

Distutils was once seperate and was then included in the standard python.

So i guess that I am working with the same goal in mind.

I interpret this as expressing your intent to (eventually) have your application included in standard Python. David Lyon <david.lyon@preisshare.net> writes:

On Fri, 24 Jul 2009 03:23:57 +0200, Christian Heimes <lists@cheimes.de> wrote:

...

I'm sorry to inform you that a wxWindows based solution has zero change to get into the Python standard library ever.

I wasn't suggesting including it in the standard library as I like others know how troublesome it can be.

This appears to contradict what you said above.

...

We are not going to add another GUI toolkit to the core distribution. We might even remove TK from the core and ship it as a separate package like bsddb3.

So is this a bias against native window applications?

No, it's a bias against adding things to Python that depend on things not already in Python. Are you, or are you not, intending for your application to be in standard Python? If that is your intent, I think Christian's answer is valid: applications, especially ones that depend on third-party widget libraries, shouldn't get added to Python. If that is not your intent, then your application shouldn't be mentioned in standard Python documentation. -- \ “Stop — Drive sideways.” —detour sign, Kyushu, Japan | `\ | _o__) | Ben Finney

David Lyon <david.lyon@preisshare.net> writes:

Not at all. In source form the pythonpkgmgr requires wx package. In executable form it does not.

The only way it could be added is in source form; that's essential for free software like Python. So, if it's not suitable for adding to Python in source form, it's not suitable for adding to Python.

...

No, it's a bias against adding things to Python that depend on things not already in Python.

That implies that nothing new can be added then.

You omit the important part: adding a new thing to Python *so long as it doesn't depend on anything outside Python*.

The Python Package Manager can be written to work in console mode.

I think this would be best. The functionality you often discuss around this tool would be best implemented independently of any UI. It would, in fact, be best to work with the team performing ongoing active standardisation of distutils functionality. -- \ “Consider the daffodil. And while you're doing that, I'll be | `\ over here, looking through your stuff.” —Jack Handey | _o__) | Ben Finney

On Sat, 25 Jul 2009 11:25:27 +1000, Ben Finney <ben+python@benfinney.id.au>

You omit the important part: adding a new thing to Python *so long as it doesn't depend on anything outside Python*.

I'm signing out on this silly discussion for now.... Any python program is dependant on things "outside" python. For example an operating system.. a computer... and a user perhaps for without such it wouldn't be possible to generate the all important 1Di0t errors that we so often have to deal with.

...

The Python Package Manager can be written to work in console mode.

I think this would be best.

Haha - I'm glad somebody took this seriously... It was a sort of a joke comment but it's a serious possibility.

The functionality you often discuss around this tool would be best implemented independently of any UI.

It is anyway. That's why doing an entirely different UI isn't any major issue. I have a class and a possible command line interface behind the scenes.

It would, in fact, be best to work with the team performing ongoing active standardisation of distutils functionality.

I am already doing that. But there is a bias against windows development and a bias against native applications. That's fine because I know they are using python on different platforms. I'm working on a proposal to make setup.py object oriented and "modern". http://wiki.python.org/moin/Distutils/Proposals So I'm doing as much as I can - really. David

On Sat, 25 Jul 2009 16:00:13 +1000, Ben Finney <ben+python@benfinney.id.au> wrote:

I think you know quite well what “depend on” means in this instance, so this is taking it to silly extremes.

haha - yes - no offence. It was just bad humour. Have a nice weekend David

On Sat, 25 Jul 2009 10:28:51 +0100, Paul Moore <p.f.moore@gmail.com> wrote:

??? I see no bias as you describe in the distutils enhancement work.

ok

Native applications are by definition not platform neutral. How does your proposal help Linux users? Mac OS? Solaris?

I'm doing a Linux/Solaris version. But I find issues every day that must be addressed.

If your concern is to make things easier for Windows users, then your application is worthwhile, but it should probably remain an external project. If it gets overwhelming support, maybe providing a standardised version with a simplified UI in the core would be an option.

That sounds ok.

How many users do you have currently?

Approx 250 downloads this (first) month for the windows version. I need to extend coverage to Linux and the Mac. Given that I'm relatively new to all this there's a learning curve. But so far so good. David

On Sat, 25 Jul 2009 12:47:21 -0400, Terry Reedy <tjreedy@udel.edu> wrote:

...

...

...

The Python Package Manager can be written to work in console mode. I think this would be best.

Haha - I'm glad somebody took this seriously... It was a sort of a joke comment but it's a serious possibility.

I took it seriously too ;-). It seems to me that you project can have at least 3 components.

It's logical and plausible.

1) the core library module, which might or might not be targeted at eventual stdlib inclusion.

I'm sure they'll fix distutils someday, and if they do, then I'll use that.

2) a small console driver script. 3) one or more GUI scripts.

It's possible. I'll give it serious consideration.

A TK version would be good from my viewpoint as it does not require download and installation of any other binaries.

Sure. Who cares if it's not as slick as something else. I'll try when I have time. David

On Sat, 25 Jul 2009 11:42:13 +0200, "Martin v. Löwis" <martin@v.loewis.de> wrote:

...

It's my intention to get a Package Manager included in standard python - yes.

In addition to the other constraints you'll have to meet for this to happen, you also have to wait a rather long time (several years) before inclusion becomes possible. This time is necessary for the community to accept your tool, and evaluate it. Ideally, the request for inclusion should not come from you, but from your users.

Yes. But users have been asking for a tool and complaining loudly about the lack of such a tool. I know you're an extremely competent and those emails you perphaps floss over. But the pleas are many and when we compare python to other languages, python rates towards at the low end of the spectrum its third party-package management facilities. You can't seriously expect users to wait for years for an integrated package management tool. Especially on Windows - that's cruel :-) To date, there just hasn't been any movement on the building of such a tool due to whatever reasons. A Package Manager isn't a frivolous "nice-to-have" tool. It's essential for a new user. (if you want I can sift the last twelve months worth of ML archives and report on how many times easier package management has been asked for. And how difficult people are finding it)

No. It implies that any addition could only depend on Python (and OS API) - so things *can* be added. For example, setuptools could be added by this principle. OTOH, if your tool depends on setuptools, you'll have to wait for setuptools to get included before inclusion of your tool can be considered.

Let's get precise. It doesn't "depend" on setuptools. But it will install setuptools if the user requests to use setuptools/easy_install. So we should only be back to the lack of a TK interface and the fact that the Package Manager Project is a new project, and needs to be working properly on more platforms. Thanks for your email. I appreciate the feedback, from everyone. Take care. David

On Sun, 26 Jul 2009 08:43:07 +0200, "Martin v. Löwis" <martin@v.loewis.de> wrote:

Hmm. I'm -0 on providing a tool whose only purpose is to download files from a web server. I always use a web browser for that...

It does a lot more than that. Firstly it shows what packages you already have installed and lets you manage them. Namely, deinstall, (manually) upgrade, view documentation and examples. In any case, even if it were only a tool to download packages it's still consistant with modern appliance design (mobile phones, programming languages, dedicated hardware) to have an internal application where a user can pretty easily download "checked" apps to their device/system. Nobody gives people just a browser to do that... and tell them "go browse". It isn't being done like that - except in python.

Hmm. I would expect that a new user is faced with the challenge of finding out what packages to install more so than with actually installing them.

Yes. That's exactly the point. You're 100% right. That's why the pythonpkgmgr provides a useful search capability for pypi packages. It's faster and more natural to use a native app than do it all in the browser. That's true in Perl (cpan) as in Python and I'm honestly saying that I can't find any noteable faults in pypi the way I find it.

If they read examples, they will see import statements, and then they have to find out how to make those work. Does your tool help with that?

Yes. It will open the website or homepage to the project/package in question. And find any documentation that might be stored in the package directory. If you only have "import interbasedb" it's a complicated process for a new user to located the home page or project documentation without pythonpkgmgr. Using pythonpkgmgr, they look on their list of installed packages, find the "interbasedb" or whatever module, and can get quick access to the pypi page or the package homepage. It's much easier. Much faster.. especially if the new user doesn't know the internals of package storage, pypi and the like.

When the user is not so new anymore, I seriously doubt that they still ask for a package management tool - except perhaps for dependencies, and here they use easy_install.

Well, that might be one use-case. It's true that many python programmers will just want to stick to their own "known" packages. However, this tool aids people with more curiosity. Because users can so easily (and safely) install and deinstall anything. It's only a click to install a package, and another to deinstall. At the moment, pythonpkgmgr doesn't handle dependencies except that which is provided internally by easy_install/pip. I'm hoping to change this as my experience and understanding grows. David

On Sun, 26 Jul 2009 19:31:40 +0200, "Martin v. Löwis" <martin@v.loewis.de> wrote:

...

...

If they read examples, they will see import statements, and then they have to find out how to make those work. Does your tool help with that?

Yes. It will open the website or homepage to the project/package in question.

How does it know the project in question?

Because it is either in the pypi search results or list of installed packages.

How do they find interbasedb in the list of installed packages if they haven't installed interbasedb yet?

They'd already installed it. But perphaps through a .exe installer. David

Steven D'Aprano wrote:

I'm a user, and personally I don't want Yet Another Integrated Package Management Tool. What I really want is the ability to install Python packages using the PM tool I already use, namely yum. (And I'd like a pony.)

Picking up on this point... out of curiousity, I threw a couple of package names at "apt-get -s" on my main (Ubuntu 8.04) PC to see what worked. The first two names I tried (numpy, scipy) didn't work, but the next four (python-numpy, python-scipy, python-django, python-turbogears) all had hits. Because Turbo Gears incorporates so many other projects, the simulated run at installing it provided a nice overview of a variety of other Python packages in the Ubuntu repositories as well. No doubt a search in Adept would reveal a whole lot more "python-*" packages that are just a few clicks away for me. </tangent> Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------

David Lyon wrote:

On Sun, 26 Jul 2009 18:05:07 +1000, Steven D'Aprano <steve@pearwood.info> wrote:

...

But you shouldn't expect the Python dev team to accept an unproven tool into the official library before demonstrating both the need and the solution.

Of course... that's why I started off by asking what the process is.

I am accepting that the current status of pythonpkgmgr is "unproven" so it will need some time for things to run their course.

It would be great to have a decent visual package manager for Python. It needs to be built on top of the work that Tarek is doing with distutils (and be compatible with his Distribute fork of setuptools) and it also needs to be established. The good news is that there is time for both... Aren't there other tools that have similar goals? PyPI browser, Yolk etc It is a shame that it needs to be built on Tk if it is to stand a chance of ever being included in the standard library - but with the right expertise it is *possible* to create decent looking UIs with Tk. Michael

...

(Just because users say they want something, doesn't mean they'll actually use the tool you provide -- perhaps they don't know what they really need, and perhaps your tool doesn't match their needs.)

Conversely, the process of satisfying needs is to discuss them. And people have been ever so helpful in that regard.

...

I'm a user, and personally I don't want Yet Another Integrated Package Management Tool. What I really want is the ability to install Python packages using the PM tool I already use, namely yum.

ok - but no alternative to that is available on windows.

pythonpkgmgr is not so different to that. And the idea behind it is to bring consistancy in package management across the different platforms.

It does have configuration that are specific to python, and will continue to do so. Like being able to specify where one wants the package installed.

...

Putting your software on the Cheeseshop, and making regular announcements to the Python community (e.g. on comp.lang.python) will be a good way to publicise the tool, and if does meet a need, people will use it, and then, if it's good enough and popular enough and supported, it may be blessed by inclusion in the standard library.

Thanks very much. Take care.

David _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.u...

-- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog

On Mon, Jul 27, 2009 at 7:20 PM, David Lyon<david.lyon@preisshare.net> wrote:

My only point is that Windows ain't no embedded system. It's not short on memory or disk space. If a package manager is 5 megabytes extra say, with it's libraries.. what's the extra download time on that ? compared to three days+ stuffing around trying to find out how to install packages for a new user.

The problem is not so much the size by itself that more code means more maintenance burden for python developers. Including new code means it has to work everywhere where python works currently, and that other people can understand/support the related code. Adding code to a project is far from free from python maintainers POV. cheers, David

...

I'm a user, and personally I don't want Yet Another Integrated Package Management Tool. What I really want is the ability to install Python packages using the PM tool I already use, namely yum.

ok - but no alternative to that is available on windows.

For removal of packages, an alternative *is* available: APR.

pythonpkgmgr is not so different to that. And the idea behind it is to bring consistancy in package management across the different platforms.

At the cost of being inconsistent within a platform. Regards, Martin

On Mon, 27 Jul 2009 11:18:25 +0900, "Stephen J. Turnbull" <stephen@xemacs.org> wrote:

[1] on my part") and sysadmin goals ("something that works and plays nicely with the rest of the system").

pythonpkgmgr seems entirely oblivious to the latter issue, and not particularly compatible with the way package management works in *nix OSS distros (including Linux distros, but also *BSD, MacPorts, and Fink).

Well I'm a sysadmin on linux in my day job. I'm entirely wondering what you mean by that ? I'm oblivious how? What can a developer mode project, like pythonpkgmgr possibly do except to make it easier to manage local package space? A developer shouldn't need to do sudo every time they want to put or try some python package from pypi. imho that's wrong and I'm helping to reduce that need. If the sysadmin puts a package in (/usr/lib/pythonX.Y/site-packages) via the O/S package manager (ie synaptics/yum/apt) then that should stick over what a developer is doing. But the developer should be able to stick their own packages in their own local space. Correct me if my assertion is wrong. David

On Mon, 27 Jul 2009 20:12:54 +0900, "Stephen J. Turnbull" <stephen@xemacs.org> wrote:

Not you; pythonpkgmgr. You've said nothing about how pythonpkgmgr is supposed to deal with multiple installed versions of Python

Under windows it can deal with multiple versions of python. You just go to options and select which version of python you wish to manage. It's as simple as that.

or how it helps if you need a custom version of a module, etc., or how it can help the developer's "local" modules cooperate/not interfere with system versions, etc.

It manages local developer modules for python 2.6+. As for checks, it doesn't have so many at present. But it makes it a lot easier to see what you have installed. So if something needs to be removed and reinstalled - it can do that.

Pave the road to DLL Hell with good intentions, of course.

Well - there's plenty of that

...

A developer shouldn't need to do sudo every time they want to put or try some python package from pypi.

I don't remember *ever* sudo-ing to try a package from pypi.

ok. But if you were using an O/S level package manager you would have to enter a su password to start the tool. Because that is needed to load packages into /usr/lib/pythonX.y

I just don't understand what problem you're claiming to solve.

Making it simpler for new users to find and install packages from pypi. And manage the existing packages that are installed on their machine.

The problem of managing local package space is in some sense solved by virtualenv, but you haven't mentioned that at all.

I never heard of it till a few months ago. To my knowledge it doesn't offer a gui. I'm told it's good for certain things and I believe it. My problem was installing packages on windows boxes, not in the development.

Is pythonpkgmgr an alternative? A complement? to virtualenv or zc.buildout?

pythonpkgmgr is aimed at featherweight users. It could be highly complementory to both virtualenv or zc.buildout at some stage if it were customised to work with those. For example, it could work with virtualenv by having a a drop down list of the different 'environments'. I don't know how to do it right now - but I'm hoping I can figure it out sometime. With zc.buildout, there's no reason why it couldn't generate a package list in that format, or do an import from a manifest. pythonpkgmgr already can generate a list of local packages as a manifest - but not in the zc.buildout format. I'm sure in the future - I'll figure that one out as well. David

On Sun, 26 Jul 2009 19:33:37 +0200, "Martin v. Löwis" <martin@v.loewis.de> wrote:

...

pythonpkgmgr is not so different to that. And the idea behind it is to bring consistancy in package management across the different platforms.

At the cost of being inconsistent within a platform.

It has the most generic of user interfaces. On Windows, a command line interface is the most out-of-place thing anybody could ask for. 19/20 kids these days wouldn't even know what a command line interface is until you show them. The first thing that they would look for is a GUI tool in the Python X.Y programs menu... So I respectfully say that there couldn't be anything less true than the assertion that a GUI package manager is inconsistent within a modern GUI desktop environment. David

On Tue, 28 Jul 2009 07:12:25 +0200, "Martin v. Löwis" <martin@v.loewis.de> wrote:

So there are now two incompatible ways to install a package: either with the native manager, or with pythonpkgmgr. If you install them one way, and try to remove them the other way, you lose.

pythonpkgmgr is only a thin wrapper for easy_install/pip. If there is a problem, then it is already a pre-existing problem that is not of my doing. It's certainly true that if a python package is installed with a windows installer or as a .deb/.rpm, then that package should be uninstalled using the appropriate and corresponding tool. The same is true if the package was installed with either pip or easy_install. All I can do is put it in my tracker as a bug and deal with it later when I have time. Good point David

On Tue, 28 Jul 2009 07:55:11 +0200, "Martin v. Löwis" <martin@v.loewis.de> wrote:

Yes, eggs have the same problem. That's one of the reasons they don't get integrated into Python.

Yes but egg_info is included in python... and the egg is not.... Hence, what goes in and what doesn't isn't always that rational. I'm just accepting that for the moment. David

On Tue, 28 Jul 2009 11:50:00 +0100, Paul Moore <p.f.moore@gmail.com> wrote:

egg_info data is in to allow "standard" (setup.py install and hence OS package manager managed) packages to provide metadata in a discoverable way. Using a format that is (reasonably) compatible with setuptools is simply a matter of co-operating with existing de facto standards.

Eggs themselves (as a distribution format) are just zip files with a funny extension, and as such are supported by Python.

ok - I get it..... It's all one big monty python comedy thing where things are quirky and inconsistent - there's little continuety from one scene to the next but you have to stop sometimes and have a laugh... because to word quote from you - "funny" Your whole email whilst perphaps technically correct is terribly difficult for a software engineering person to follow. Monty python was only meant to be a funny film for entertainment, not a philosophy for building software from.... Why we can have the egg_info without allowing the egg is kind of like having a chicken and egg story without allowing the chicken.. or is it the egg.. oh I lost track... I think I need to watch the movie again to understand what's happening here... Let me go away confused... don't ask me any more questions or elaborate with more answers. Thanks. :-) David

David Lyon wrote:

Your whole email whilst perphaps technically correct is terribly difficult for a software engineering person to follow.

It made perfect sense to me. The words "eggs" brings with it a whole lot more baggage than just the sum of the technical parts in the language core that support them (primarily distutils and zipimport). I find it unfortunate that the name for the distutils metadata format contains the word "egg" because it implies much greater consensus around the philosophy behind eggs than actually exists. A lot of the baggage associated with the "eggs" concept is related to the inherent conflict between different approaches to dependency management: 1. Use the system package management system for everything (preferred by many, perhaps even most, *nix sysadmins, but not an option on Windows) 2. Create a Python specific package management system independent of the system package manager (an area dominated by setuptools, including both eggs and non-zipped package distributions) 3. Bundle everything into a monolithic application or framework (the typical approach on Windows with py2exe, but also the philosophy behind tools like virtualenv) All 3 of those philosophies have good arguments in their favour and they all have good arguments against them as well. Your comments about your package management system suggest that it is just yet another entrant in category 2 and you have said nothing to allay the concerns of those that despise that philosophy with a passion because of the problems it causes them when trying to manage their systems using the first philosophy. Since the Python constituency includes developers and system administrators that favour all 3 philosophies (and probably other variants that I haven't thought to describe), anything that makes it into the standard library will need to adequately balance the interests of all parties (e.g. as has occurred in the PEP 376 metadata enhancement discussions). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------

On Wed, 29 Jul 2009 10:56:20 +1000, Nick Coghlan <ncoghlan@gmail.com> wrote:

The words "eggs" brings with it a whole lot more baggage than just the sum of the technical parts in the language core that support them (primarily distutils and zipimport).

Well, in this case, (talking metaphorically) we have the ability to roll the eggs, crack a whole in them and suck out the contents (deal with a package in a zipped egg) So metaphorically we could say that python can work with egg shells.. As for what's in the egg... well lets just say that it's a bit floaty.. And perphaps that is the best way for things to be for a while.

I find it unfortunate that the name for the distutils metadata format contains the word "egg" because it implies much greater consensus around the philosophy behind eggs than actually exists.

I see it a different way. I thinks eggs are simple - as in like a java bean. A simple way to move a package from one place to another. What seems to have gone wrong is that there is a lot of pioneering and interconnected and interdependent technology within them. They're an egg.. but in the past they've had little monsters that bite your fingers when you try to put them in the pan... :-) I think "Egg" term is very good. But maybe we are best not trying to over-specify their contents. Maybe we should say it has certain things (EGG_INFO, PACKAGE DATA) as in (YOLK, WHITE) and pretty much leave it at that. If more detail is required - rtm.

A lot of the baggage associated with the "eggs" concept is related to the inherent conflict between different approaches to dependency management:

That's not an egg problem. That's a packaging/metadata problem. It's the package dependency issue that's the problem. That's a distutils problem.

1. Use the system package management system for everything (preferred by many, perhaps even most, *nix sysadmins, but not an option on Windows)

Yes, because the package dependency issues are just so great.

2. Create a Python specific package management system independent of the system package manager (an area dominated by setuptools, including both eggs and non-zipped package distributions)

More work needs to go into distutils. Not taking away from any existing work - but setuptools has relied on the deficiencies of distutils to gain a foothold. Fix distutils (give me time to think..) and the need for setuptools and any further fork is probably negated.

3. Bundle everything into a monolithic application or framework (the typical approach on Windows with py2exe, but also the philosophy behind tools like virtualenv)

Windows users are using py2exe and other products over distutils. To a normal windows developer, distutils makes imho no sense in the way that it goes about things now. For example, very simple concepts like "program directories", (where you stick the program) isn't an available option in distutils. But it is the first thing that you need to know in a windows program. So it's very hard to get to step 1...

Your comments about your package management system suggest that it is just yet another entrant in category 2 and you have said nothing to allay the concerns of those that despise that philosophy with a passion because of the problems it causes them when trying to manage their systems using the first philosophy.

I'm a windows user.. I can't be in category #2.. I'm in category #1, have nothing... (except now my own tool)

Since the Python constituency includes developers and system administrators that favour all 3 philosophies (and probably other variants that I haven't thought to describe), anything that makes it into the standard library will need to adequately balance the interests of all parties (e.g. as has occurred in the PEP 376 metadata enhancement discussions).

Well at least you are saying that there is some way of reconciling all the different options... There's an awful lot to take in, and there must be 20,000 lines of emails for every 1 line of python code that is required to fix this thing. Take care David

On Wed, Jul 29, 2009 at 2:36 AM, David Lyon<david.lyon@preisshare.net> wrote:

On Wed, 29 Jul 2009 10:56:20 +1000, Nick Coghlan <ncoghlan@gmail.com> wrote:

...

The words "eggs" brings with it a whole lot more baggage than just the sum of the technical parts in the language core that support them (primarily distutils and zipimport).

Well, in this case, (talking metaphorically) we have the ability to roll the eggs, crack a whole in them and suck out the contents (deal with a package in a zipped egg)

So metaphorically we could say that python can work with egg shells..

As for what's in the egg... well lets just say that it's a bit floaty..

And perphaps that is the best way for things to be for a while.

...

I find it unfortunate that the name for the distutils metadata format contains the word "egg" because it implies much greater consensus around the philosophy behind eggs than actually exists.

I see it a different way. I thinks eggs are simple - as in like a java bean. A simple way to move a package from one place to another.

What seems to have gone wrong is that there is a lot of pioneering and interconnected and interdependent technology within them. They're an egg.. but in the past they've had little monsters that bite your fingers when you try to put them in the pan... :-)

I think "Egg" term is very good. But maybe we are best not trying to over-specify their contents.

Maybe we should say it has certain things (EGG_INFO, PACKAGE DATA) as in (YOLK, WHITE) and pretty much leave it at that. If more detail is required - rtm.

...

A lot of the baggage associated with the "eggs" concept is related to the inherent conflict between different approaches to dependency management:

That's not an egg problem. That's a packaging/metadata problem.

It's the package dependency issue that's the problem. That's a distutils problem.

...

1. Use the system package management system for everything (preferred by many, perhaps even most, *nix sysadmins, but not an option on Windows)

Yes, because the package dependency issues are just so great.

...

2. Create a Python specific package management system independent of the system package manager (an area dominated by setuptools, including both eggs and non-zipped package distributions)

More work needs to go into distutils. Not taking away from any existing work - but setuptools has relied on the deficiencies of distutils to gain a foothold.

Fix distutils (give me time to think..) and the need for setuptools and any further fork is probably negated.

...

3. Bundle everything into a monolithic application or framework (the typical approach on Windows with py2exe, but also the philosophy behind tools like virtualenv)

Windows users are using py2exe and other products over distutils. To a normal windows developer, distutils makes imho no sense in the way that it goes about things now.

For example, very simple concepts like "program directories", (where you stick the program) isn't an available option in distutils. But it is the first thing that you need to know in a windows program.

So it's very hard to get to step 1...

...

Your comments about your package management system suggest that it is just yet another entrant in category 2 and you have said nothing to allay the concerns of those that despise that philosophy with a passion because of the problems it causes them when trying to manage their systems using the first philosophy.

I'm a windows user.. I can't be in category #2..

I'm in category #1, have nothing... (except now my own tool)

...

Since the Python constituency includes developers and system administrators that favour all 3 philosophies (and probably other variants that I haven't thought to describe), anything that makes it into the standard library will need to adequately balance the interests of all parties (e.g. as has occurred in the PEP 376 metadata enhancement discussions).

Well at least you are saying that there is some way of reconciling all the different options...

There's an awful lot to take in, and there must be 20,000 lines of emails for every 1 line of python code that is required to fix this thing.

Take care

David

I really do think this mail thread needs to move to disutils-sig or python-ideas. jesse

On Sat, 25 Jul 2009 10:40:52 +1000, Ben Finney <ben+python@benfinney.id.au> wrote:

[snip]

If that is not your intent, then your application shouldn't be mentioned in standard Python documentation.

Hm. But docutils isn't part of the standard library, and the documentation refers to it. And the docs link to ActivePython and Enthought's Python distribution. And they link to things like "Collin Winter’s functional module" and Django and PyOpenGL and so forth. I don't think anyone would seriously suggest removing all references to third-party software from the Python documentation. Perhaps the criterion that David is looking to have is explained is that people would probably prefer for his app to get a bit more popular before Python itself starts pointing people in its direction. Jean-Paul

On Fri, Jul 24, 2009 at 18:06, Jean-Paul Calderone <exarkun@divmod.com>wrote:

On Sat, 25 Jul 2009 10:40:52 +1000, Ben Finney <ben+python@benfinney.id.au<ben%2Bpython@benfinney.id.au>> wrote:

...

[snip]

If that is not your intent, then your application shouldn't be mentioned in standard Python documentation.

Hm. But docutils isn't part of the standard library, and the documentation refers to it.

Why do you think distutils is not part of the standard library? The official code location is Lib/distutils within the standard library. -Brett

On 2009-07-25 17:28, Brett Cannon wrote:

On Fri, Jul 24, 2009 at 18:06, Jean-Paul Calderone <exarkun@divmod.com <mailto:exarkun@divmod.com>> wrote:

On Sat, 25 Jul 2009 10:40:52 +1000, Ben Finney <ben+python@benfinney.id.au <mailto:ben%2Bpython@benfinney.id.au>> wrote:

[snip]

If that is not your intent, then your application shouldn't be mentioned in standard Python documentation.

Hm. But docutils isn't part of the standard library, and the documentation refers to it.

Why do you think distutils is not part of the standard library? The official code location is Lib/distutils within the standard library.

He said "docutils", not "distutils". -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco