Bootstrap script for package management tool in Python 2.7 (Was: Re: [Python-Dev] At least one package management tool for 2.7)
So, there won't be any package management tool shipped with Python 2.7 and users will have to download and install `setuptools` manually as before: "search" -> "download" -> "unzip" -> "cmd" -> "cd" -> "python setup.py install" Therefore I still propose shipping bootstrap package that instruct user how to download and install an actual package management tool when users tries to use it. So far I know only one stable tool - `easy_install` - a part of `setuptools` package. The required behavior for very basic user friendliness: 1. user installs Python 2.7 2. user issues `python -m easy_install something` 3. user gets message 'easy_install' tool is not installed on this system. To make it available, download and install `setuptools` package from http://pypi.python.org/pypi/setuptools/ 4. the screen is paused before exit (for windows systems) Other design notes: 1. if package tries to import `easy_install` module used for bootstrap, it gets the same ImportException as if there were no `easy_install` at all 2. bootstrap module is overwritten by actual package when users installs it So, do we need a PEP for that? How else can I know if consensus is reached? Anybody is willing to elaborate on implementation? P.S. Please be careful to reply to relevant lists -- anatoly t. On Mon, Mar 29, 2010 at 9:37 AM, Tarek Ziadé <ziade.tarek@gmail.com> wrote:
2010/3/29 anatoly techtonik <techtonik@gmail.com>: [..]
It is really hard to follow. You should at least change subjects when switching topic.
I was talking about the work going on and the decisions taken lately.
I never change topics of threads mails when there's less than 100 mails, because I find it way more confusing :)
So, what is the verdict? Will there be a package management tool or bootstrap package for it shipped with Python 2.7 or not?
As I said in the mail you've quoted, all improvements are made in Distutils2. So the answer is no. Python 2.7b1 is due in less than a week anyways, so any new development on this topic will happen after.
Basically Python 2.7 == Python 2.6 in term of packaging.
Regards Tarek
-- Tarek Ziadé | http://ziade.org
On Mon, Mar 29, 2010 at 9:30 AM, anatoly techtonik <techtonik@gmail.com> wrote:
So, there won't be any package management tool shipped with Python 2.7 and users will have to download and install `setuptools` manually as before:
"search" -> "download" -> "unzip" -> "cmd" -> "cd" -> "python setup.py install"
Therefore I still propose shipping bootstrap package that instruct user how to download and install an actual package management tool when users tries to use it. So far I know only one stable tool - `easy_install` - a part of `setuptools` package.
There are other tools to install something in a vanilla Python : - pip (which also have an uninstall feature) - distutils - distribute
The required behavior for very basic user friendliness: 1. user installs Python 2.7 2. user issues `python -m easy_install something` 3. user gets message 'easy_install' tool is not installed on this system. To make it available, download and install `setuptools` package from http://pypi.python.org/pypi/setuptools/
Are you thinking about something generic ? Like, being able to call : $ python -m ANYTHING And get an error message saying that the ANYTHING script is not installed ? Then have a registry somewhere to get the name of the project that owns the ANYTHING script ? [..]
So, do we need a PEP for that? How else can I know if consensus is reached? Anybody is willing to elaborate on implementation?
I see two paths here: 1 - do you want to define a general mechanism for Python to install scripts ? 2 - are you just suggesting to have this mechanism only for Package Managers ? In case of 2), what you would need to do is propose an extension to PEP 376, we are currently working on, then we can add it there once people have discussed it on distutils-SIG, or maybe create a new PEP if the topic is too big to fit in 376.
P.S. Please be careful to reply to relevant lists
Yes, so if it's 2) let's keep this thread in Distutils-SIG. Cross-posting like this makes it hard to follow. Tarek -- Tarek Ziadé | http://ziade.org
On Mon, Mar 29, 2010 at 09:30, anatoly techtonik <techtonik@gmail.com> wrote:
Therefore I still propose shipping bootstrap package that instruct user how to download and install an actual package management tool when users tries to use it. So far I know only one stable tool - `easy_install` - a part of `setuptools` package.
We can't ship *A* bootstrap script until there is *A* package management tool in the Python world. Currently there are three. :) In short: The tools that exist are not ready yet. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64
On Mon, Mar 29, 2010 at 10:55 AM, Tarek Ziadé <ziade.tarek@gmail.com> wrote:
Therefore I still propose shipping bootstrap package that instruct user how to download and install an actual package management tool when users tries to use it. So far I know only one stable tool - `easy_install` - a part of `setuptools` package.
There are other tools to install something in a vanilla Python :
- pip (which also have an uninstall feature) - distutils - distribute
distutils is not a `package management` tool, because it doesn't know anything even about installed packages, not saying anything about dependencies. `pip` and `distribute` are unknown for a vast majority of Python users, so if you have a perspective replacement for `easy_install` - it can be said in bootstrap package message. There is no problem with packages that require `setuptools` either - they will require `setuptools` as dependency anyway. For now there are two questions: 1. Are they stable enough for the replacement of user command line `easy_install` tool? 2. Which one is the recommended? P.S. Should there be an accessible FAQ in addition to ML?
The required behavior for very basic user friendliness: 1. user installs Python 2.7 2. user issues `python -m easy_install something` 3. user gets message 'easy_install' tool is not installed on this system. To make it available, download and install `setuptools` package from http://pypi.python.org/pypi/setuptools/
Are you thinking about something generic ?
Like, being able to call :
$ python -m ANYTHING
And get an error message saying that the ANYTHING script is not installed ? Then have a registry somewhere to get the name of the project that owns the ANYTHING script ?
No. To get something in 2.7 I would refrain from developing into this direction for now.
[..]
So, do we need a PEP for that? How else can I know if consensus is reached? Anybody is willing to elaborate on implementation?
I see two paths here: 1 - do you want to define a general mechanism for Python to install scripts ? 2 - are you just suggesting to have this mechanism only for Package Managers ?
3 - I want current "best practice" for installing Python modules with dependencies from PyPI to be shipped with Python 2.7 by default Answering your questions: 1. I want to have some user-friendly way for installing Python scripts, but I am more concerned that I will miss `easy_install` in Python 2.7 2. Bootstrap script is aimed at users. Package managers are assumed to already have their package at PyPI and provide install instructions that involve `easy_install`, so nothing is required to be done on their part.
In case of 2), what you would need to do is propose an extension to PEP 376, we are currently working on, then we can add it there once people have discussed it on distutils-SIG, or maybe create a new PEP if the topic is too big to fit in 376.
PEP 376 is completely irrelevant to user side boot package proposal. This proposal is to recommend whatever package managing tool you think user need and instruct how to get the tool. The boot package doesn't know anything at all about how packages are managed.
P.S. Please be careful to reply to relevant lists
Yes, so if it's 2) let's keep this thread in Distutils-SIG. Cross-posting like this makes it hard to follow.
My vision is that decision about having bootstrap package or not in 2.7 should be in python-dev and specific packaging, implementation and pip/distutils/distribute questions in distutils-sig. -- anatoly t.
On Mon, Mar 29, 2010 at 11:02 AM, anatoly techtonik <techtonik@gmail.com> wrote: [..]
distutils is not a `package management` tool, because it doesn't know anything even about installed packages, not saying anything about dependencies.
At this point, no one knows anything about installed packages at the Python level. Keeping track of installed projects is a feature done within each package managment system. And the whole purpose of PEP 376 is to define a database of what's installed, for the sake of interoperability.
`pip` and `distribute` are unknown for a vast majority of Python users, so if you have a perspective replacement for `easy_install` -
Depending on how you call a Python user, I disagree here. Many people use pip and distribute. The first one because it has an uninstall feature among other things. The second one because it fixes some bugs of setuptools and provides Python 3 support
For now there are two questions: 1. Are they stable enough for the replacement of user command line `easy_install` tool? 2. Which one is the recommended?
P.S. Should there be an accessible FAQ in addition to ML?
This FAQ work has been started in th "HitchHicker's guide to Packaging" you can find here: http://guide.python-distribute.org [..]
No. To get something in 2.7 I would refrain from developing into this direction for now.
[..]
So, do we need a PEP for that? How else can I know if consensus is reached? Anybody is willing to elaborate on implementation?
I see two paths here: 1 - do you want to define a general mechanism for Python to install scripts ? 2 - are you just suggesting to have this mechanism only for Package Managers ?
3 - I want current "best practice" for installing Python modules with dependencies from PyPI to be shipped with Python 2.7 by default
Again, any new code work will not happen because 2.7 is due in less than a week. Things are happening in Distutils2. Now, for the "best practice" documentation, I think the guide is the best plce to look at. [..]
PEP 376 is completely irrelevant to user side boot package proposal.
It is, since it proposes an uninstall script called via -m. So having a install script called by -m is definitely its business.
My vision is that decision about having bootstrap package or not in 2.7 should be in python-dev and specific packaging, implementation and pip/distutils/distribute questions in distutils-sig.
If the mentioned bootstrap script is about a package managment system, you should keep the discussion in distutils-SIG I think. Regards Tarek -- Tarek Ziadé | http://ziade.org
2010/3/29 Lennart Regebro <regebro@gmail.com>:
On Mon, Mar 29, 2010 at 09:30, anatoly techtonik <techtonik@gmail.com> wrote:
Therefore I still propose shipping bootstrap package that instruct user how to download and install an actual package management tool when users tries to use it. So far I know only one stable tool - `easy_install` - a part of `setuptools` package.
We can't ship *A* bootstrap script until there is *A* package management tool in the Python world. Currently there are three. :)
In short: The tools that exist are not ready yet.
Ok. How about shipping bootstrap script only for `easy_install` tool for now? -- anatoly t.
2010/3/29 anatoly techtonik <techtonik@gmail.com>:
2010/3/29 Lennart Regebro <regebro@gmail.com>:
On Mon, Mar 29, 2010 at 09:30, anatoly techtonik <techtonik@gmail.com> wrote:
Therefore I still propose shipping bootstrap package that instruct user how to download and install an actual package management tool when users tries to use it. So far I know only one stable tool - `easy_install` - a part of `setuptools` package.
We can't ship *A* bootstrap script until there is *A* package management tool in the Python world. Currently there are three. :)
In short: The tools that exist are not ready yet.
Ok. How about shipping bootstrap script only for `easy_install` tool for now?
-1 I don't see how this will improve the current state. It will make it worse since the future of packaging for the stldib is being built in distutils2, and also in Pip. People that want to use easy_install can install it. It's very simple and doesn't require more work than what you have described through a -m call. The instructions you want to add in that bootstrap script to inform people belong to the documentation imho. And that's one of the goal of the packaging guide we are building. Another suggestion could be to add in the Python installer a notice about the guide, saying : "Hey, we are working on packaging right now. Python currently doesn't ship with a full-featured package manager, you can have a look in this guide though, for guidance" I still think you should remove python-dev from the cc list ;) Tarek -- Tarek Ziadé | http://ziade.org
On 29 March 2010 10:38, Tarek Ziadé <ziade.tarek@gmail.com> wrote:
I still think you should remove python-dev from the cc list ;)
I don't know what happened with the change of subject and crossposting, but I'm now seeing 2 copies of every mail, and both copies are appearing in both python-dev and distutils-sig (probably, some of that is gmail's fault, but other crosspostings don't make things this bad...) Given that nothing will happen in Python 2.7, I agree that python-dev should be removed. Given that easy_install is NOT the "one, true answer" that Anatoly seems to think it is, I'm dubious that anything can or should be done until distutils2 is published, so I'd suggest that the whole subject is dropped until distutils2 is available. Paul.
anatoly techtonik wrote:
So, there won't be any package management tool shipped with Python 2.7 and users will have to download and install `setuptools` manually as before:
Until the discussed package management tools support a robust inventory and uninstallation system that plays well with directly installed Python packages, you won't find widespread support on python-dev for endorsing any of them. Yes, the people who use them love them for good and valid reasons, but those who absolutely detest them also do so for good and valid reasons. While this is still the case, it would be highly inappropriate for python-dev to include bootstrap scripts that direct users to these systems, as they're all flawed in their current incarnations (this isn't the packaging systems' fault - the flaws are largely due to a lack of supporting infrastructure in the standard library). The distutils2 work and the various metadata PEPs that have been approved recently are all about addressing those limitations in the infrastructure. With those in place and flowing through the Python package management ecosystem, bootstrapping interoperable package management tools is likely to become a reasonable option in the future. But we aren't there yet, and won't be for 2.7 or 3.2. From an outsider's perspective, the 3.3 time frame appears to be very possible though. Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
On Mon, Mar 29, 2010 at 12:15 PM, Tarek Ziadé <ziade.tarek@gmail.com> wrote:
[..]
distutils is not a `package management` tool, because it doesn't know anything even about installed packages, not saying anything about dependencies.
At this point, no one knows anything about installed packages at the Python level.
Users do not care about this, and `distutils` doesn't know this even at package level.
Keeping track of installed projects is a feature done within each package managment system.
And the whole purpose of PEP 376 is to define a database of what's installed, for the sake of interoperability.
That's great. When it will be ready everybody would be happy to make their package management tool compliant.
`pip` and `distribute` are unknown for a vast majority of Python users, so if you have a perspective replacement for `easy_install` -
Depending on how you call a Python user, I disagree here. Many people use pip and distribute.
The first one because it has an uninstall feature among other things. The second one because it fixes some bugs of setuptools and provides Python 3 support
I do not mind if we can distribute three stubs, they will also serve as pointers for a better way of packaging when an ultimate tool is finally born. Me personally is willing to elaborate for `easy_install` stub in 2.7.
For now there are two questions: 1. Are they stable enough for the replacement of user command line `easy_install` tool? 2. Which one is the recommended?
P.S. Should there be an accessible FAQ in addition to ML?
This FAQ work has been started in th "HitchHicker's guide to Packaging" you can find here:
I can see any FAQ. To me the FAQ is something that could be posted to distutils ML once a month to reflect current state of packaging. It should also carry version number. So anybody can comment on the FAQ, ask another question or ask to make a change.
Again, any new code work will not happen because 2.7 is due in less than a week. Things are happening in Distutils2.
That doesn't solve the problem. Bootstrap script can be written in one day. What we need is a consensus whatever this script is welcomed in 2.7 or not? Who is the person to make the decision?
Now, for the "best practice" documentation, I think the guide is the best plce to look at.
Let's refer to original user story: "I installed Python and need a quick way to install my packages on top of it." "I execute `easy_install something` as said in installation manual, but nothing/error happens."
[..]
PEP 376 is completely irrelevant to user side boot package proposal.
It is, since it proposes an uninstall script called via -m. So having a install script called by -m is definitely its business.
The scope of my proposal is a bootstrap script that instructs user how to install the package management tool of user's choice if this tool is not yet installed on user system (this is the case with new Python installation). If you'll come up with a better way of package management - you will update this bootstrap script with relevant information in future Python releases. Right now it is essential to get this _feature_ in Python 2.7 until 2.7 is frozen for new features. The script that shows message upon invocation and is overwritten by the version of the package it proposes to install. Is it hard to do?
My vision is that decision about having bootstrap package or not in 2.7 should be in python-dev and specific packaging, implementation and pip/distutils/distribute questions in distutils-sig.
If the mentioned bootstrap script is about a package managment system, you should keep the discussion in distutils-SIG I think.
If there will be no bootstrap script in 2.7, I won't have any motivation to continue discussion until 2.8 deadline is near. Of course I will be disappointed, because I will have to explain everything to 2.7 users again and again instead of letting them execute one command and follow its instructions. -- anatoly t.
anatoly techtonik wrote:
So, there won't be any package management tool shipped with Python 2.7 and users will have to download and install `setuptools` manually as before:
"search" -> "download" -> "unzip" -> "cmd" -> "cd" -> "python setup.py install"
Therefore I still propose shipping bootstrap package that instruct user how to download and install an actual package management tool when users tries to use it. So far I know only one stable tool - `easy_install` - a part of `setuptools` package.
The required behavior for very basic user friendliness: 1. user installs Python 2.7 2. user issues `python -m easy_install something` 3. user gets message 'easy_install' tool is not installed on this system. To make it available, download and install `setuptools` package from http://pypi.python.org/pypi/setuptools/
4. the screen is paused before exit (for windows systems)
Other design notes: 1. if package tries to import `easy_install` module used for bootstrap, it gets the same ImportException as if there were no `easy_install` at all 2. bootstrap module is overwritten by actual package when users installs it
So, do we need a PEP for that? How else can I know if consensus is reached? Anybody is willing to elaborate on implementation?
P.S. Please be careful to reply to relevant lists
An even lighter option would be to add an item to pythons 'help' feature. Currently help("PACKAGES") == help("import") It may be enough at this time to add a "PACKAGES" help entry that gives an overview of packages and hints on installing them. Then "import" can be a related help topic for "PACKAGES". Ron
On Mon, Mar 29, 2010 at 11:02, anatoly techtonik <techtonik@gmail.com> wrote:
distutils is not a `package management` tool, because it doesn't know anything even about installed packages
With that definition, there are no packaga management tools for Python. So it's going to be pretty hard to ship one with 2.7.
`pip` and `distribute` are unknown for a vast majority of Python users, so if you have a perspective replacement for `easy_install` - it can be said in bootstrap package message. There is no problem with packages that require `setuptools` either - they will require `setuptools` as dependency anyway.
For now there are two questions: 1. Are they stable enough for the replacement of user command line `easy_install` tool?
Yes. But that's the wrong question. The correct question is: Are they stable enough to be included in standard library. And the answer is "no".
2. Which one is the recommended?
Both. -- Lennart Regebro: http://regebro.wordpress.com/ Python 3 Porting: http://python-incompatibility.googlecode.com/ +33 661 58 14 64
On Mon, Mar 29, 2010 at 10:45 PM, anatoly techtonik <techtonik@gmail.com> wrote:
On Mon, Mar 29, 2010 at 12:15 PM, Tarek Ziadé <ziade.tarek@gmail.com> wrote:
[..]
distutils is not a `package management` tool, because it doesn't know anything even about installed packages, not saying anything about dependencies.
At this point, no one knows anything about installed packages at the Python level.
Users do not care about this, and `distutils` doesn't know this even at package level.
Keeping track of installed projects is a feature done within each package managment system.
And the whole purpose of PEP 376 is to define a database of what's installed, for the sake of interoperability.
That's great. When it will be ready everybody would be happy to make their package management tool compliant.
`pip` and `distribute` are unknown for a vast majority of Python users, so if you have a perspective replacement for `easy_install` -
Depending on how you call a Python user, I disagree here. Many people use pip and distribute.
The first one because it has an uninstall feature among other things. The second one because it fixes some bugs of setuptools and provides Python 3 support
I do not mind if we can distribute three stubs, they will also serve as pointers for a better way of packaging when an ultimate tool is finally born. Me personally is willing to elaborate for `easy_install` stub in 2.7.
For now there are two questions: 1. Are they stable enough for the replacement of user command line `easy_install` tool? 2. Which one is the recommended?
P.S. Should there be an accessible FAQ in addition to ML?
This FAQ work has been started in th "HitchHicker's guide to Packaging" you can find here:
I can see any FAQ. To me the FAQ is something that could be posted to distutils ML once a month to reflect current state of packaging. It should also carry version number. So anybody can comment on the FAQ, ask another question or ask to make a change.
Again, any new code work will not happen because 2.7 is due in less than a week. Things are happening in Distutils2.
That doesn't solve the problem. Bootstrap script can be written in one day. What we need is a consensus whatever this script is welcomed in 2.7 or not? Who is the person to make the decision?
Now, for the "best practice" documentation, I think the guide is the best plce to look at.
Let's refer to original user story: "I installed Python and need a quick way to install my packages on top of it."
python setup.py install works well, and has for almost a decade. If you need setuptools, you can include ez_setup.py, which does exactly what you want, without adding a hugely controversial feature to python proper. You do something like: try: import setuptools except ImportError: print "Run ez_setup.py first" .... And you're done, cheers, David
2010/3/29 anatoly techtonik <techtonik@gmail.com>:
Ok. How about shipping bootstrap script only for `easy_install` tool for now?
Since there are many who are of the opinion that easy_install isn't very good, and pip should be used instead, that would be a bad idea. Just drop it. 2.7 will not have a package management tool or bootstrap script to one, because there is currently not one that is good enough to be the standard. People are working on making the situation good enough, this debate is wasting their time. The end. -- Lennart Regebro: http://regebro.wordpress.com/ Python 3 Porting: http://python-incompatibility.googlecode.com/ +33 661 58 14 64
On Mon, Mar 29, 2010 at 1:42 PM, Paul Moore <p.f.moore@gmail.com> wrote:
I still think you should remove python-dev from the cc list ;)
I don't know what happened with the change of subject and crossposting, but I'm now seeing 2 copies of every mail, and both copies are appearing in both python-dev and distutils-sig (probably, some of that is gmail's fault, but other crosspostings don't make things this bad...)
The same here. This time the thread was hijacked by software. That's why I prefer reading Google Groups from the web with optional subscriptions.
Given that nothing will happen in Python 2.7, I agree that python-dev should be removed.
Given that easy_install is NOT the "one, true answer" that Anatoly seems to think it is, I'm dubious that anything can or should be done until distutils2 is published, so I'd suggest that the whole subject is dropped until distutils2 is available.
It is NOT about UPPMS (Universal Python Package Management Solution) - it is about small bootstrap Script that Guides User how to install the package management system the user is trying to execute. -- anatoly t.
On Mon, Mar 29, 2010 at 3:45 PM, anatoly techtonik <techtonik@gmail.com> wrote: [..]
I can see any FAQ. To me the FAQ is something that could be posted to distutils ML once a month to reflect current state of packaging. It should also carry version number. So anybody can comment on the FAQ, ask another question or ask to make a change.
The guide was built by people in the community, and contains a "current state of packaging" section. We are trying to keep it accurate with what we are doing. You have a mailing list where you can ask question, and you are welcome to contribute changes. Adding a "FAQ" containing links to sections in the guide, or a summarized answered is a good idea though !
Again, any new code work will not happen because 2.7 is due in less than a week. Things are happening in Distutils2.
That doesn't solve the problem. Bootstrap script can be written in one day. What we need is a consensus whatever this script is welcomed in 2.7 or not? Who is the person to make the decision?
Guido makes all final decisions. But before he does, in general, we work all together to find a consensus, because he can't take part in all discussions. In this process, I am the guy in charge for packaging matters in the stdlib, and I am trying to write the PEPs that reflects those consensus. But anyone is welcome to champion a PEP as long as we discuss it a bit here, and we agree that a new PEP is required. Feel free to make more points here in distutils-SIG about your idea, to explain the advantages of doing it and gain traction. But there are no chance at all this will be accepted for 2.7. It's a long process (not writing a script, but deciding if it's the best way to go)
Now, for the "best practice" documentation, I think the guide is the best plce to look at.
Let's refer to original user story: "I installed Python and need a quick way to install my packages on top of it." "I execute `easy_install something` as said in installation manual, but nothing/error happens."
Have you read : http://guide.python-distribute.org/installation.html ? How could we change it to help this user story ?
[..] The scope of my proposal is a bootstrap script that instructs user how to install the package management tool of user's choice if this tool is not yet installed on user system (this is the case with new Python installation). If you'll come up with a better way of package management - you will update this bootstrap script with relevant information in future Python releases.
Sounds like a simple installation instruction, I can't see the benefit of adding a script for that. Plus, we can't make a choice for Python 2.7. Some people use easy_install, some people use pip, some use another tool. You use easy_install, fine, install it. Once PEP 376 is finished and accepted, we might provide a basic script in the stdlib though. [..]
If there will be no bootstrap script in 2.7, I won't have any motivation to continue discussion until 2.8 deadline is near. Of course I will be disappointed, because I will have to explain everything to 2.7 users again and again instead of letting them execute one command and follow its instructions.
Frankly, I am not understanding why you are pushing so hard to include this script one week before 2.7b1 without any previous discussion. I am sorry if you feel disappointed about this. Regards Tarek -- Tarek Ziadé | http://ziade.org
2010/3/29 Nick Coghlan <ncoghlan@gmail.com>:
anatoly techtonik wrote:
So, there won't be any package management tool shipped with Python 2.7 and users will have to download and install `setuptools` manually as before:
Until the discussed package management tools support a robust inventory and uninstallation system that plays well with directly installed Python packages, you won't find widespread support on python-dev for endorsing any of them.
This wasn't the question, but the summary. I would greatly appreciate if you could provide your feedback on the second part of my letter that started with "Therefore..."
Yes, the people who use them love them for good and valid reasons, but those who absolutely detest them also do so for good and valid reasons. While this is still the case, it would be highly inappropriate for python-dev to include bootstrap scripts that direct users to these
Scripts do not _direct_ users - they _help_ users, already directed by some installation instruction or tutorial, to find and install package management system they are _trying_ to use.
The distutils2 work and the various metadata PEPs that have been approved recently are all about addressing those limitations in the infrastructure. With those in place and flowing through the Python package management ecosystem, bootstrapping interoperable package management tools is likely to become a reasonable option in the future.
Bootstrap tools evolve together with packaging situation. You may deprecate package management tool in your future bootstrap scripts if it is "incompatible" with this Python release. It is just user message that is a flexible as the mind of its author.
But we aren't there yet, and won't be for 2.7 or 3.2. From an outsider's perspective, the 3.3 time frame appears to be very possible though.
Just a thought about user story my customers would likely write if I shipped Python as a product: "As a user, I think Python is suxx, because it makes its users suffer for a long time from packaging disorder". -- anatoly t.
On Mon, Mar 29, 2010 at 16:21, anatoly techtonik <techtonik@gmail.com> wrote:
Just a thought about user story my customers would likely write if I shipped Python as a product: "As a user, I think Python is suxx, because it makes its users suffer for a long time from packaging disorder".
So fix it. Python is open source. It's created by it's users. If there is something you think it lacking, you step up and help make it better. But the packaging solution will not be ready for 2.7. It's a really hard problem. Your help is appreciated. Complaints are not. -- Lennart Regebro: http://regebro.wordpress.com/ Python 3 Porting: http://python-incompatibility.googlecode.com/ +33 661 58 14 64
On Mon, Mar 29, 2010 at 5:18 PM, Tarek Ziadé <ziade.tarek@gmail.com> wrote:
Again, any new code work will not happen because 2.7 is due in less than a week. Things are happening in Distutils2.
That doesn't solve the problem. Bootstrap script can be written in one day. What we need is a consensus whatever this script is welcomed in 2.7 or not? Who is the person to make the decision?
Guido makes all final decisions.
But before he does, in general, we work all together to find a consensus, because he can't take part in all discussions. In this process, I am the guy in charge for packaging matters in the stdlib, and I am trying to write the PEPs that reflects those consensus. But anyone is welcome to champion a PEP as long as we discuss it a bit here, and we agree that a new PEP is required.
Feel free to make more points here in distutils-SIG about your idea, to explain the advantages of doing it and gain traction.
So far I haven't seen any counter arguments against _bootstrap script_ itself. I see that the difference between "package management tool" and "bootstrap script" should be defined more clearly. But I do not know how.
But there are no chance at all this will be accepted for 2.7. It's a long process (not writing a script, but deciding if it's the best way to go)
Ok. It seemed so simple, so that I do not even know if I able to propose anything at all. It takes an enormous amount of time and effort to keep discussion focused.
Now, for the "best practice" documentation, I think the guide is the best plce to look at.
Let's refer to original user story: "I installed Python and need a quick way to install my packages on top of it." "I execute `easy_install something` as said in installation manual, but nothing/error happens."
Have you read : http://guide.python-distribute.org/installation.html ? How could we change it to help this user story ?
1. Provide a less "boring" version for users who are not interested in details of package management history 2. It doesn't describe what command on windows should I use (and I am on windows) to "wget" distribute_setup.py 3. Too complicated for a user To completely get the picture - imagine a typical student who learns English as a second language. If the person has weak language skill - he will skip directly to examples, believe me. Even I do this. And your examples are not crossplatform, and this complicates things even more. "packaging ecosystem manipulation", "combination of tool used for ecosystem isolation and two other tools", argumentation that "Distribute was created because the Setuptools package is no longer maintained". It is all aimed at Python developers, probably with bachelor's degree. I speak on behalf of user that doesn't know anything about Python besides that it should be installed along with some API package to turn on support for youtube video sharing.
The scope of my proposal is a bootstrap script that instructs user how to install the package management tool of user's choice if this tool is not yet installed on user system (this is the case with new Python installation). If you'll come up with a better way of package management - you will update this bootstrap script with relevant information in future Python releases.
Sounds like a simple installation instruction, I can't see the benefit of adding a script for that.
Of course! Because you are a developer. You need to be a user or at least QA engineer to see it. =) For now in its simplest minimal form that is possible to create in a day for 2.7 it is an simple text instruction. This will surely can be ready for 2.7. But in future distributions, i.e. in Python 2.7 non-beta, it may be able to download required package management tool automatically if user agrees, unpack it to %TEMP% and execute "setup.py install".
Plus, we can't make a choice for Python 2.7. Some people use easy_install, some people use pip, some use another tool. You use easy_install, fine, install it.
Some people just blindly follow the instructions and they are disappointed if: 1. it doesn't work 2. if the thing need to be done quickly and instruction is too long 3. if instruction is too long and it still doesn't work
Once PEP 376 is finished and accepted, we might provide a basic script in the stdlib though.
I can see immediate benefit in this script now, but I can not predict if it will be actual 5 years later. =)
[..]
If there will be no bootstrap script in 2.7, I won't have any motivation to continue discussion until 2.8 deadline is near. Of course I will be disappointed, because I will have to explain everything to 2.7 users again and again instead of letting them execute one command and follow its instructions.
Frankly, I am not understanding why you are pushing so hard to include this script one week before 2.7b1 without any previous discussion. I am sorry if you feel disappointed about this.
I can't comment anything about it. It is too subjective, like no time, no Python 2.7 installation until alpha 4 (because I want to see yet another trivial patch with hijacked discussion included there), not being able to get back to problem, because installing `setuptools` for me as a developer is faster than describing user problems. I could also add that mailing lists are archaic to my taste, Python lists are too flooded, it is hard to follow up on threads, that there are always much more ideas that I able to type at any given moment. -- anatoly t.
On Mon, Mar 29, 2010 at 17:32, anatoly techtonik <techtonik@gmail.com> wrote:
So far I haven't seen any counter arguments against _bootstrap script_ itself. I see that the difference between "package management tool" and "bootstrap script" should be defined more clearly. But I do not know how.
What difference? The bootstrap script would bootstrap a package management tool. But non exists that is good enough for everyone to agree on at the moment. So there is nothing to bootstrap. You are asking for inclusion of something that does not yet exist. It's self-evident that this is impossible. -- Lennart Regebro: http://regebro.wordpress.com/ Python 3 Porting: http://python-incompatibility.googlecode.com/ +33 661 58 14 64
On Tue, Mar 30, 2010 at 12:32 AM, anatoly techtonik <techtonik@gmail.com> wrote:
On Mon, Mar 29, 2010 at 5:18 PM, Tarek Ziadé <ziade.tarek@gmail.com> wrote:
Again, any new code work will not happen because 2.7 is due in less than a week. Things are happening in Distutils2.
That doesn't solve the problem. Bootstrap script can be written in one day. What we need is a consensus whatever this script is welcomed in 2.7 or not? Who is the person to make the decision?
Guido makes all final decisions.
But before he does, in general, we work all together to find a consensus, because he can't take part in all discussions. In this process, I am the guy in charge for packaging matters in the stdlib, and I am trying to write the PEPs that reflects those consensus. But anyone is welcome to champion a PEP as long as we discuss it a bit here, and we agree that a new PEP is required.
Feel free to make more points here in distutils-SIG about your idea, to explain the advantages of doing it and gain traction.
So far I haven't seen any counter arguments against _bootstrap script_ itself.
You certainly have not provided any argument *for* one. Again, what's the point compared to adding the bootstrap script ez_setup.py in your package. I also find the easiness argument rather dubious: especially on windows, the thing to give a user is an installer (.exe/.msi), not sources. That will solve more problems than any buggy bootstrap script requiring a working internet connection will ever do, cheers, David
On Mon, Mar 29, 2010 at 5:32 PM, anatoly techtonik <techtonik@gmail.com> wrote: [..]
Sounds like a simple installation instruction, I can't see the benefit of adding a script for that.
Of course! Because you are a developer. You need to be a user or at least QA engineer to see it. =)
For now in its simplest minimal form that is possible to create in a day for 2.7 it is an simple text instruction. This will surely can be ready for 2.7. But in future distributions, i.e. in Python 2.7 non-beta, it may be able to download required package management tool automatically if user agrees, unpack it to %TEMP% and execute "setup.py install".
You are completely right about the fact that our tools and guide are aiming to developers. The CPython distribution comes with the stdlib, no more no less. So there's no point to launch the installation of a third party project when you install CPython. And until the stdlib contains what it takes to have a full-feature package manager, you will need to do an extra manual installation to get one, or use another Python distribution that contains one already. Maybe you should use Python distributions like Enthought's or ActiveState's ones, that are both including a package manager when you install them. Regards Tarek -- Tarek Ziadé | http://ziade.org
anatoly techtonik wrote:
So, there won't be any package management tool shipped with Python 2.7 and users will have to download and install `setuptools` manually as before:
"search" -> "download" -> "unzip" -> "cmd" -> "cd" -> "python setup.py install"
Therefore I still propose shipping bootstrap package that instruct user how to download and install an actual package management tool when users tries to use it.
For what it's worth, Guido prototyped something similar in March of 2008, but his was an actual bootstrapping tool for package management: http://mail.python.org/pipermail/python-dev/2008-March/077837.html His tool knew how to download a tar file, untar it, and run "python setup.py install" on it. No version numbers, no dependency management, simple enough that it should be easy to get right. Only appropriate for bootstrapping into a real package management tool. The thread ends with him saying "I don't have time to deal with this further this week", and I dunno, maybe it just fell off the radar? I'd been thinking about resurrecting the discussion but I didn't have time either. I fear it is too late for 2.7, //larry//
On 29 March 2010 16:32, anatoly techtonik <techtonik@gmail.com> wrote:
So far I haven't seen any counter arguments against _bootstrap script_ itself. I see that the difference between "package management tool" and "bootstrap script" should be defined more clearly. But I do not know how.
You haven't been reading closely, I guess: 1. The script doesn't exist and no-one has stepped up to write it. 2. Three scripts would be required not one (pip, distribute, setuptools were I think the 3 options mentioned) - and that goes against the Zen "There should be one obvious way". 3. You have no consensus that this is a good idea. 4. There is no clear target version - you're insisting that this go in 2.7, even though it has been repeatedly and clearly stated that this is impossible (3.3 is the earliest possible target for something like this). And one which hasn't been mentioned yet (that I've noticed), but I'll add for good measure: 5. You haven't explained what will happen if the target of the bootstrap (easy_install, or whatever) is removed or otherwise becomes unavailable, or isn't supported under whatever version of Python is running (remember, your bootstrap script will need to still be available at least until Python 3.6, for backward compatibility reasons if nothing else). I could come up with more, but these should do. If you want this to go further, I suggest that you do the following (all of these, don't expect anyone else to do any of them for you): 1. Write a PEP, proposing the feature, summarising the arguments for and against as expressed in this thread, addressing all of the objections (start with the 5 above) and suggesting a target version for inclusion. That must be 3.3 or later. 2. Write a patch which implements the feature, including tests and documentation. This can be against trunk. Be prepared to maintain it over an extended period (i.e. updating it to apply cleanly as needed). 3. Persuade python-dev that the idea has merit - get consensus that the feature should go in, and get the PEP accepted (I'd be surprised if you can get this without Tarek's support, which you don't have yet, so there's your starting point once 1 and 2 are done). It might be better spending your time on these activities rather than fanning the flames on distutils-sig in the short term... Paul.
anatoly techtonik wrote:
So, there won't be any package management tool shipped with Python 2.7 and users will have to download and install `setuptools` manually as before: [...] So, do we need a PEP for that? How else can I know if consensus is reached? Anybody is willing to elaborate on implementation?
Notice that the first beta release of Python 2.7 will be made on April 3. Any new feature that you propose must be committed by then. I suggest focusing on Python 3.2 instead. Regards, Martin
On Mon, Mar 29, 2010 at 11:26 AM, Larry Hastings <larry@hastings.org> wrote:
anatoly techtonik wrote:
So, there won't be any package management tool shipped with Python 2.7 and users will have to download and install `setuptools` manually as before:
"search" -> "download" -> "unzip" -> "cmd" -> "cd" -> "python setup.py install"
Therefore I still propose shipping bootstrap package that instruct user how to download and install an actual package management tool when users tries to use it.
For what it's worth, Guido prototyped something similar in March of 2008, but his was an actual bootstrapping tool for package management:
http://mail.python.org/pipermail/python-dev/2008-March/077837.html
His tool knew how to download a tar file, untar it, and run "python setup.py install" on it. No version numbers, no dependency management, simple enough that it should be easy to get right. Only appropriate for bootstrapping into a real package management tool.
The thread ends with him saying "I don't have time to deal with this further this week", and I dunno, maybe it just fell off the radar? I'd been thinking about resurrecting the discussion but I didn't have time either.
I would consider this bootstrap to be quite workable, though I would add that any extra option to the bootstrap script should be passed to setup.py install, and the download should be cached (so you can do -h and not have to re-download the package once you figure out the extra options -- at least a --user option is reasonable here for people without root). Specifically targeting this bootstrap for tools like pip and virtualenv is no problem. I think looking around PyPI etc is kind of more than I'd bother with. Those things change, this bootstrap code won't change, it could cause unnecessary future pain. Maybe (*maybe*) it could look in http://pypi.python.org/well-known-packages/PACKAGE_NAME and so we can have it install a certain small number of things quickly that way -- if the URL it looks to is targeted only for the bootstrap script itself then we don't have to worry about compatibility problems as much. Oh... then i can think of a half dozen other options it could take, and then it becomes an installer. Blech. OK, I'd be willing to cut off the options at --user (which I think is a minimum... maybe --prefix too), and maybe some simple package detection so people could write "python -m boostrap Setuptools --user" -- entirely based on some well-known URL baked into bootstrap.py, where the URL is independent of any other service (and so is least likely to cause future problems or ambiguities). An advantage to this kind of bootstrapper is that as future packaging systems are developed there's a clear way to get started with them, without prematurely baking anything in to Python. -- Ian Bicking | http://blog.ianbicking.org | http://twitter.com/ianbicking
anatoly techtonik wrote:
Let's refer to original user story: ... "I execute `easy_install something` as said in installation manual, but nothing/error happens."
Which installation manual did the user see that in? It can't be anything that came with Python, because easy_install is not a standard part of Python. If a package is telling the user to install it with a non-standard installation tool, without explaining this fact and how to get the tool, then the package's documentation is lacking. -- Greg
On Mon, Mar 29, 2010 at 6:52 PM, Tarek Ziadé <ziade.tarek@gmail.com> wrote:
[..]
Sounds like a simple installation instruction, I can't see the benefit of adding a script for that.
Of course! Because you are a developer. You need to be a user or at least QA engineer to see it. =)
For now in its simplest minimal form that is possible to create in a day for 2.7 it is an simple text instruction. This will surely can be ready for 2.7. But in future distributions, i.e. in Python 2.7 non-beta, it may be able to download required package management tool automatically if user agrees, unpack it to %TEMP% and execute "setup.py install".
You are completely right about the fact that our tools and guide are aiming to developers.
The CPython distribution comes with the stdlib, no more no less. So there's no point to launch the installation of a third party project when you install CPython.
Not when you (as a user) install, but when you try to execute `python -m easy_install` or `python -m pip` or `python -m distribute`, i.e. as advised in Packaging Guide.
And until the stdlib contains what it takes to have a full-feature package manager, you will need to do an extra manual installation to get one, or use another Python distribution that contains one already.
That's suxx that I need to prepend every simple tutorial for users of my packages with an obligatory notice how to install setuptools instead of just writing:
python -m easy_install review
With installing easy_install it is not as easy to install as it could be. -- anatoly t.
On Tue, Mar 30, 2010 at 12:45 AM, Ian Bicking <ianb@colorstudy.com> wrote:
On Mon, Mar 29, 2010 at 11:26 AM, Larry Hastings <larry@hastings.org> wrote:
anatoly techtonik wrote:
So, there won't be any package management tool shipped with Python 2.7 and users will have to download and install `setuptools` manually as before:
"search" -> "download" -> "unzip" -> "cmd" -> "cd" -> "python setup.py install"
Therefore I still propose shipping bootstrap package that instruct user how to download and install an actual package management tool when users tries to use it.
For what it's worth, Guido prototyped something similar in March of 2008, but his was an actual bootstrapping tool for package management:
http://mail.python.org/pipermail/python-dev/2008-March/077837.html
His tool knew how to download a tar file, untar it, and run "python setup.py install" on it. No version numbers, no dependency management, simple enough that it should be easy to get right. Only appropriate for bootstrapping into a real package management tool.
The thread ends with him saying "I don't have time to deal with this further this week", and I dunno, maybe it just fell off the radar? I'd been thinking about resurrecting the discussion but I didn't have time either.
I would consider this bootstrap to be quite workable, though I would add that any extra option to the bootstrap script should be passed to setup.py install, and the download should be cached (so you can do -h and not have to re-download the package once you figure out the extra options -- at least a --user option is reasonable here for people without root). Specifically targeting this bootstrap for tools like pip and virtualenv is no problem.
I think looking around PyPI etc is kind of more than I'd bother with. Those things change, this bootstrap code won't change, it could cause unnecessary future pain. Maybe (*maybe*) it could look in http://pypi.python.org/well-known-packages/PACKAGE_NAME and so we can have it install a certain small number of things quickly that way -- if the URL it looks to is targeted only for the bootstrap script itself then we don't have to worry about compatibility problems as much.
Oh... then i can think of a half dozen other options it could take, and then it becomes an installer. Blech. OK, I'd be willing to cut off the options at --user (which I think is a minimum... maybe --prefix too), and maybe some simple package detection so people could write "python -m boostrap Setuptools --user" -- entirely based on some well-known URL baked into bootstrap.py, where the URL is independent of any other service (and so is least likely to cause future problems or ambiguities).
An advantage to this kind of bootstrapper is that as future packaging systems are developed there's a clear way to get started with them, without prematurely baking anything in to Python.
The bootstrap should only be a temporary package that will propose user to automatically install package management tool user is trying to use if he doesn't have any. It is not Ubuntu-like "missing package autodetection" - only few most popular non-conflicting packaging solutions should be bootstrapped in this way:
python -m easy_install review `easy_install` is not installed on this system. Do you want to fetch it to proceed (Y/n)?
Think of it as of one used oriented "1-Click" technology. -- anatoly t.
participants (11)
-
"Martin v. Löwis" -
anatoly techtonik -
David Cournapeau -
Greg Ewing -
Ian Bicking -
Larry Hastings -
Lennart Regebro -
Nick Coghlan -
Paul Moore -
Ron Adam -
Tarek Ziadé