Binary CPython distribution for Linux
I'm an advocate of getting users and projects to move to modern Python versions. I believe dropping support for end-of-lifed Python versions is important for the health of the Python community. If you've done any amount of Python 3 porting work, you know things get much harder the more 2.x legacy versions you need to support. I led the successful charge to drop support for Python 2.6 and below from Firefox's build system. I failed to win the argument that Mercurial should drop 2.4 and 2.5 [1]. A few years ago, I started a similar conversation with the LLVM project [2]. I wrote a blog post on the subject [3] that even got Slashdotted [4] (although I don't think that's the honor it was a decade ago). While much of the opposition to dropping Python <2.7 stems from the RHEL community (they still have 2.4 in extended support and 2.7 wasn't in a release until a few weeks ago), a common objection from the users is "I can't install a different Python" or "it's too difficult to install a different Python." The former is a legit complaint - if you are on shared hosting and don't have root, as easy as it is to add an alternate package repository that provides 2.7 (or newer), you don't have the permissions so you can't do it. This leaves users with attempting a userland install of Python. Personally, I think installing Python in userland is relatively simple. Tools like pyenv make this turnkey. Worst case you fall back to configure + make. But I'm an experienced developer and have a compiler toolchain and library dependencies on my machine. What about less experienced users or people that don't have the necessary build dependencies? And, even if they do manage to find or build a Python distribution, we all know that there's enough finicky behavior with things like site-packages default paths to cause many headaches, even for experienced Python hackers. I'd like to propose a solution to this problem: a pre-built distribution of CPython for Linux available via www.python.org in the list of downloads for a particular release [5]. This distribution could be downloaded and unarchived into the user's home directory and users could start running it immediately by setting an environment variable or two, creating a symlink, or even running a basic installer script. This would hopefully remove the hurdles of obtaining a (sane) Python distribution on Linux. This would allow projects to more easily drop end-of-life Python versions and would speed adoption of modern Python, including Python 3 (because porting is much easier if you only have to target 2.7). I understand there may be technical challenges with doing this for some distributions and with producing a universal binary distribution. I would settle for a binary distribution that was targeted towards RHEL users and variant distros, as that is the user population that I perceive to be the most conservative and responsible for holding modern Python adoption back. [1] http://permalink.gmane.org/gmane.comp.version-control.mercurial.devel/68902 [2] http://lists.cs.uiuc.edu/pipermail/llvmdev/2012-December/056545.html [3] http://gregoryszorc.com/blog/2014/01/08/why-do-projects-support-old-python-r... [4] http://developers.slashdot.org/story/14/01/09/1940232/why-do-projects-contin... [5] https://www.python.org/download/releases/2.7.7/
Le 26/06/2014 20:34, Gregory Szorc a écrit :
I'm an advocate of getting users and projects to move to modern Python versions. I believe dropping support for end-of-lifed Python versions is important for the health of the Python community. If you've done any amount of Python 3 porting work, you know things get much harder the more 2.x legacy versions you need to support.
I led the successful charge to drop support for Python 2.6 and below from Firefox's build system. I failed to win the argument that Mercurial should drop 2.4 and 2.5 [1]. A few years ago, I started a similar conversation with the LLVM project [2]. I wrote a blog post on the subject [3] that even got Slashdotted [4] (although I don't think that's the honor it was a decade ago).
While much of the opposition to dropping Python <2.7 stems from the RHEL community (they still have 2.4 in extended support and 2.7 wasn't in a release until a few weeks ago), a common objection from the users is "I can't install a different Python" or "it's too difficult to install a different Python." The former is a legit complaint - if you are on shared hosting and don't have root, as easy as it is to add an alternate package repository that provides 2.7 (or newer), you don't have the permissions so you can't do it.
This leaves users with attempting a userland install of Python. Personally, I think installing Python in userland is relatively simple. Tools like pyenv make this turnkey. Worst case you fall back to configure + make. But I'm an experienced developer and have a compiler toolchain and library dependencies on my machine. What about less experienced users or people that don't have the necessary build dependencies? And, even if they do manage to find or build a Python distribution, we all know that there's enough finicky behavior with things like site-packages default paths to cause many headaches, even for experienced Python hackers.
I'd like to propose a solution to this problem: a pre-built distribution of CPython for Linux available via www.python.org in the list of downloads for a particular release [5]. This distribution could be downloaded and unarchived into the user's home directory and users could start running it immediately by setting an environment variable or two, creating a symlink, or even running a basic installer script. This would hopefully remove the hurdles of obtaining a (sane) Python distribution on Linux. This would allow projects to more easily drop end-of-life Python versions and would speed adoption of modern Python, including Python 3 (because porting is much easier if you only have to target 2.7).
I understand there may be technical challenges with doing this for some distributions and with producing a universal binary distribution. I would settle for a binary distribution that was targeted towards RHEL users and variant distros, as that is the user population that I perceive to be the most conservative and responsible for holding modern Python adoption back.
[1] http://permalink.gmane.org/gmane.comp.version-control.mercurial.devel/68902 [2] http://lists.cs.uiuc.edu/pipermail/llvmdev/2012-December/056545.html [3] http://gregoryszorc.com/blog/2014/01/08/why-do-projects-support-old-python-r...
[4] http://developers.slashdot.org/story/14/01/09/1940232/why-do-projects-contin...
Just today I installed Anaconda (https://store.continuum.io/cshop/anaconda/) on Linux servers running CentOS 6.4. It installs in a directory anywhere in the filesystem (no need to be root), and using it globally is just a matter of prepending a folder to the PATH and it was done. Of course Anaconda is oriented towards scientific applications but it is a proof that a pre-build binary installer works and can be simple to use. If someone wants to try it without all scientific libraries they provide Miniconda (http://conda.pydata.org/miniconda.html) which contains only python and the python package manager conda. Joseph
I have a little pet project for building rpm of python 2.7 (it should be trivial to port to 3.x): https://build.opensuse.org/project/show/home:cavallo71:opt-python-modules If there's enough interest I can help to integrate with python.org.
I understand there may be technical challenges with doing this for some distributions and with producing a universal binary distribution.
Opensuse provides the vm to build binaries for multiple platforms already since a very long time.
Of course Anaconda is oriented towards scientific applications but it is a proof that a pre-build binary installer works and can be simple to use.
Rpm are the "blessed" way to instal software on linux: it supports what most sysadmin expect (easy to list the installed packages, easy to validate if tampering to a package occurred, which file belongs to a package? etc..). Anaconda might appeal some group of user, but for deployment company-wide rpm is the best technical solution given its integration in linux. I hope this helps, Antonio
Le 26/06/2014 22:00, Antonio Cavallo a écrit :
Of course Anaconda is oriented towards scientific applications but it is a proof that a pre-build binary installer works and can be simple to use.
Rpm are the "blessed" way to instal software on linux: it supports what most sysadmin expect (easy to list the installed packages, easy to validate if tampering to a package occurred, which file belongs to a package? etc..).
Anaconda might appeal some group of user, but for deployment company-wide rpm is the best technical solution given its integration in linux.
1. Not all Linux distros use rpm (Debian, Ubuntu, Arch Linux...) 2. rpm need to be root to be installed. Btw, Anaconda is multiplatform and can be installed on Linux, Windows and Mac. Joseph
----- Original Message -----
While much of the opposition to dropping Python <2.7 stems from the RHEL community (they still have 2.4 in extended support and 2.7 wasn't in a release until a few weeks ago), a common objection from the users is "I can't install a different Python" or "it's too difficult to install a different Python." The former is a legit complaint - if you are on shared hosting and don't have root, as easy as it is to add an alternate package repository that provides 2.7 (or newer), you don't have the permissions so you can't do it.
It's not true that 2.7 wasn't released until few weeks ago. It was released few weeks ago as part of RHEL 7, but Red Hat has been shipping Red Hat Software Collections (RHSCL) 1.0, that contain Python 2.7 and Python 3.3, for almost a year now [1] - RHSCL is installable on RHEL 6; RHSCL 1.1 (also with 2.7 and 3.3) has been released few weeks ago and is supported on RHEL 6 and 7. Also, these collections now have their community rebuilds at [2], so you can just download them without needing to talk to Red Hat at all. But yeah, these are all RPMs, so you have to be root to install them.
I'd like to propose a solution to this problem: a pre-built distribution of CPython for Linux available via www.python.org in the list of downloads for a particular release [5]. This distribution could be downloaded and unarchived into the user's home directory and users could start running it immediately by setting an environment variable or two, creating a symlink, or even running a basic installer script. This would hopefully remove the hurdles of obtaining a (sane) Python distribution on Linux. This would allow projects to more easily drop end-of-life Python versions and would speed adoption of modern Python, including Python 3 (because porting is much easier if you only have to target 2.7).
I understand there may be technical challenges with doing this for some distributions and with producing a universal binary distribution. I would settle for a binary distribution that was targeted towards RHEL users and variant distros, as that is the user population that I perceive to be the most conservative and responsible for holding modern Python adoption back.
Speaking with my Fedora/RHEL/RHSCL Python maintainer's hat on, prebuilding Python is not as easy task as it may seem :) Someone has to write the build scripts (e.g. sort of specfile, but rpm/specfile wouldn't really work for you, since you want to install in user's home dirs). Someone has to update them when new Python comes out, so in the worst case you end up with slightly different build scripts for different versions of Python. Someone has to do rebuilds when there is CVE. Or a bug. Or a user requests a feature that makes sense. Someone has to do that for *each packaged version* - and each packaged version needs to be maintained for some amount of time so that it all actually makes sense. Maintaining a prebuilt distribution of Python is a time consuming task even if you do it just for one Linux distro. If you want to maintain a *universal* prebuilt Python distribution, then you'll find out that it's a) undoable b) consumes so many resources and it's so fragile, that it's probably not worth it. You could just bundle all Python dependencies into your distribution to make it "easier", but that would just make the result grow in size (perhaps significantly) and you would then also need to update/bugfix/securityfix the bundled dependencies (which would consume even more time). Please don't take this as a criticism of your ideas, I see what you're trying to solve. I just think the way you're trying to solve it is unachievable or would consume so much community resources, that it would end up unmaintained and buggy most of the time. -- Regards, Bohuslav "Slavek" Kabrda. [1] http://developerblog.redhat.com/2013/09/12/rhscl1-ga/ [2] https://www.softwarecollections.org/en/scls/
On 27 Jun 2014 17:33, "Bohuslav Kabrda"
It's not true that 2.7 wasn't released until few weeks ago. It was
Please don't take this as a criticism of your ideas, I see what you're
released few weeks ago as part of RHEL 7, but Red Hat has been shipping Red Hat Software Collections (RHSCL) 1.0, that contain Python 2.7 and Python 3.3, for almost a year now [1] - RHSCL is installable on RHEL 6; RHSCL 1.1 (also with 2.7 and 3.3) has been released few weeks ago and is supported on RHEL 6 and 7. Also, these collections now have their community rebuilds at [2], so you can just download them without needing to talk to Red Hat at all. But yeah, these are all RPMs, so you have to be root to install them. Indeed, while there are still some rough edges, software collections look like the best approach to doing maintainable system installs of Python runtimes other than the system Python into Fedora/RHEL/CentOS et al (and I say that while wearing both my upstream and downstream hats). Collections solve this problem in a general (rather than CPython specific) way, since they can be used to get upgraded versions of language runtimes, databases, web servers, etc, all without risking the stability of the OS itself. I hope to see someone put together collections for PyPy and PyPy3 as well. The approaches used for runtime isolation of software collections should also be applicable to Debian systems, but (as far as I am aware) the tooling to build them as debs rather than RPMs doesn't exist yet. trying to solve. I just think the way you're trying to solve it is unachievable or would consume so much community resources, that it would end up unmaintained and buggy most of the time. For prebuilt userland installs on Linux, I think "miniconda" is the current best available approach. It has its challenges (especially around its handling of security concerns), but it's designed to offer a full cross platform package management system that makes it well suited to the task of managing prebuilt language runtimes in user space. Cheers, Nick.
-- Regards, Bohuslav "Slavek" Kabrda.
[1] http://developerblog.redhat.com/2013/09/12/rhscl1-ga/ [2] https://www.softwarecollections.org/en/scls/ _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
participants (5)
-
Antonio Cavallo
-
Bohuslav Kabrda
-
Gregory Szorc
-
Joseph Martinot-Lagarde
-
Nick Coghlan