Python 2.7.7. on Windows
Greetings, I've just woken up and noticed Python 2.7.7 is on track to be released, and in a rather unique event contains a few security enhancements in addition to the usual fixes: http://legacy.python.org/dev/peps/pep-0466/ I thought this might be a good time to make a final plea to fix a long-standing security issue in the installer on Windows. By default it installs Python to the root folder, thereby bypassing filesystem permissions: http://bugs.python.org/issue1284316 The main rationale given (for not using the standard %ProgramFiles%) has been that the full path to python is too long to type, and ease of use is more important than the security benefits given by following Windows conventions. However, adding python to the PATH variable is an alternative solution that would result in even fewer keystrokes needing to be typed at a console, while maintaining system security. As this is an installer setting and not a code change, I gather that the opportunities for code breakage should be fewer. Please consider updating this setting to a more secure and friendly default, for 2.7.7 and 3.5+ as well. -Mike
Mike Miller wrote:
This would be an incredibly painful change that would surprise and hurt a lot of people. Mark Hammond's last comment on the linked issue sums up my view as well: "My take is still that Python is a tool, not an app. People writing an app they wish to distribute using Python should include Python in their package (ie, not rely on an installed version) and these apps should conform with the guidelines." Yes, it is possible for a non-admin user to install arbitrary packages into a place where an admin user may inadvertently run it, thereby providing escalation of privilege. On the other hand, that applies to a lot of development tools, especially since most users on Windows these days are actually limited administrators - ANYTHING they install could run when they elevate a certain process. I'm all for encouraging apps to redistribute Python (as most do on Windows). In this case, they can easily apply the correct security settings, typically by installing pythonxy.dll with their application and ignoring any global settings (PYTHONPATH, registry keys, etc.). On the other hand, Python from python.org is a tool that should only be installed by those who are prepared to manage it. On Windows it is easy enough to have a second, secured copy for your admin scripts and an unsecured copy for non-admin tasks.
I'm not sure what change you are proposing here... doesn't the installer already have an option to add to PATH? I'm sure I keep disabling it. I don't want "python.exe" on my PATH because I have 10+ versions installed at any one time. I have my own set of aliases because even the py.exe launcher can't handle my setup :) The info we've gotten from our users (Python Tools for Visual Studio) shows that most have two or more versions of Python installed, so having one or both of them in PATH or PATHEXT just adds ambiguity. The other rationales are .pyc generation and package installation. Unlike PATH, these are not red herrings, but they do only apply to developers and not end users (where the developer has done his/her job properly). Speaking as the one who is likely to take over from Martin as the Windows build manager (the only people arguing so far are my employer's lawyers, but I'm winning that argument), I don't see any change that can be made to 2.7.7 apart from adding a link in the installer to a documentation page on how to secure the Python installation. At this point, 2.7.6->2.7.7 should be an incredibly safe upgrade, and there's no way to safely change the default installation location or the ACLs on the install directory. As for 3.5, I have some ideas that I will raise with python-dev once I've had a chance to think through all the issues (think proper per-user installs, redist modules, etc.). More secure installations would certainly be on the table, but practicality very easily beats purity here IMHO. Cheers, Steve
On 04/29/2014 05:12 AM, Steve Dower wrote:
This would be an incredibly painful change that would surprise and hurt a lot of people.
Hi, I think "incredibly painful" is overstating the case a bit. ;) We're talking about an installer default, a setting that would still be changeable as it always has, that by definition only will affect brand new installs.
None of Microsoft's Dev tools install to C:\, rather to ProgramFiles. The fact that another security issue may exist is not a good justification for creating additional.
This sounds like the perspective of someone highly technical, forgetting that new users will be installing python as well and vastly outnumber us. "Normal people" need help to avoid security issues. Microsoft's guidelines on where to install software are clear, and don't make exceptions that "tools" should be installed to the root of the drive to bypass file system permissions, for convenience.
I'm not sure what change you are proposing here... doesn't the installer already have an option to add to PATH? I'm sure I keep disabling it.
No, it does not. Unless it got slipped in when I wasn't looking. It should be an option though, I think everyone would agree.
"python.exe" on my PATH because I have 10+ versions installed at any one time. I
Remember, python-dev's are not the target users of this package, and are a rather minuscule fraction of the user base.
Python installation. At this point, 2.7.6->2.7.7 should be an incredibly safe upgrade, and there's no way to safely change the default installation location
This would continue to be the case, as the installer will recognize the previously installed Python 2.7 and use its setting. This should affect only *brand new installs.*
or the ACLs on the install directory.
No ACLs would be affected or changed or even thought about. Simply installing to the correct folder (on new installs) will accomplish the goal. In short, this design of restricted permissions (read-only) for binaries and PATH conveniences goes back decades under Unix and other OS's. MS Windows has finally caught up in the security department in the last few releases. Please don't keep us back in the "bad old days" of DOS where everything was installed to the root folder. -- -Mike
On Mon, Apr 28, 2014 at 3:07 PM, Mike Miller <python-dev@mgmiller.net> wrote:
The option to add the current install to your path was added 3.3.
Knowing which Python you want on your path and that you want it on your path at all is somewhat of an advanced usage. While beginners do want to just open up cmd and type "python" and have it work, there are better ways to accomplish that which don't involve system-wide path manipulation or installation changes. Several PC manufacturers have been known to use Python for various system utilities, which is how Python sometimes ends up in the path on your grandma's Dell*. Even for a beginner who just wants "python" to work, we need to be careful to not wreck their entire system by helping them get our fresh Python install to show up. A more reasonable way to help beginners would be to create a shortcut somewhere that starts up cmd with a modified path. They can do whatever they want to do within that context without modifying their entire system. If they learn that they later want their system-wide path manipulated, they can do that within the installer or will known how to do that themselves. * watch Dave Beazley's PyCon 2014 talk for a good story involving one of those manufacturer installed Pythons: https://www.youtube.com/watch?v=RZ4Sn-Y7AP8
On 04/29/2014 08:38 AM, Brian Curtin wrote:
The option to add the current install to your path was added 3.3.
Ok, thanks. So there is some precedent it would be useful.
Installing python at all is a "system-wide" change. Why not default to a known safe, ready to use configuration, rather than a "tool-box" that needs assembly?
Thanks, I'm trying to get thru all the talk will watch that shortly. ;) Still, edge cases regarding manufactures should not override the needs of the majority of users. -Mike
On Mon, Apr 28, 2014 at 1:56 PM, Mike Miller <python-dev@mgmiller.net>wrote:
fun talk -- but start at about 18:00 minutes if you want the relevant bit... By the way, some people (or at least ESRI, with ArcMap) Install and use their own python in: C:\python27 -- if you dont want to interat with that, you need to put it somewhere else! I'd argue that any OEM or third party app that uses the standard location or puts it on the PATH be default is in error, but what can you so? Redhat used to have #!/usr/bin/env python on their admin scripts, which was really painful if you wanted to upgrade anything. Don't know it they've fixed that.... -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
Good point about it only affecting new installs, though that still constitutes a lot of installs.
The fact that the mitigations are well known by the people who have to worry about them is a good justification for not creating a compatibility issue. It's easy for IT admins to install Python in a way that the files are read-only, the .pyc and .pyo files are already there, and user site packages will be used by default (I think that last one is easy?). The good IT admins even know that they need to do this - perhaps we can help educate the bad admins? (FWIW, if you have admin privileges on your own machine, YOU are the IT admin. Are you a good one or a bad one?)
One place where mitigations are added are the distributions (Canopy, Anaconda, etc.). These packages redistribute Python and install to different locations with different permissions (I'm not promising that they all do it properly, but they have the opportunity to do so). The reference implementation of CPython is typically not the best option for "normal people", who are much better served by one of these bundles.
I'm well aware of the guidelines, hence the practicality vs. purity comment. I'm fairly certain that the installation to the root was originally about ease of command-line access rather than bypassing permissions - the permissions probably didn't exist for the first few versions of Python (Python for DOS obviously didn't care... maybe it's always been about backwards compatibility?) At this point, installing into the root is about not breaking everyone who *knows* that Python installs into the root directory and always has.
Thanks Brett for pointing out that it arrived in Python 3. I'm sure it would be an acceptable addition to 2.7.7, though you'd need to get it in before RC and you'd also need to find someone who is keen and able to keep making 2.7 installers for Windows. Right now, we don't have anyone who is both.
My day job revolves around making Python easier to use for non "python-dev"s, especially those who have multiple versions of Python installed simultaneously (as I mentioned, over 50% of our user base). Having PATH or PATHEXT set globally is a source of much confusion. (Though a global PYTHONPATH causes more problems, but that isn't part of this discussion.)
For example, next time you redeploy your hosted service on a clean VM with the latest security patches? The next time someone does a clean uninstall-reinstall because they've been burned by upgrades before? The next time someone arrives at work and IT has given them a new image and Python is "gone"? I can guarantee that you will break someone
The effective permissions change, and now commands that didn't require elevation do. Tools that didn't need to be manifested for UAC have to be recompiled. Scripts that used to work will simply break. This is the problem, not whether the permissions are explicitly set as part of the install process.
My employer would be most unhappy if I wasn't trying to encourage good Windows support :) At the same time, my employer, colleagues, friends, collaborators and most people on python-dev would all be unhappy if I made everyone angry at Python. At risk of saying it too often, practicality beats purity. Cheers, Steve
-- -Mike
On Mon, Apr 28, 2014 at 7:04 PM, Steve Dower <Steve.Dower@microsoft.com> wrote:
If it's an acceptable change to the release manager (Benjamin?), and if there's actually time before the RC (I don't know when it is planned), I am willing to backport my 3.3 change to get this in the 2.7 installer. However, I'm not currently setup to make release installers -- I think I need a signing certificate or something like that.
On Mon, Apr 28, 2014, at 17:14, Brian Curtin wrote:
That's fine.
However, I'm not currently setup to make release installers -- I think I need a signing certificate or something like that.
Apparently the current PSF one just expired, so we're in the process of procuring a new one.
Hi, Stepping back a bit... I doubt you'd take the idea this far, but that Python should need assembly by professionals before use doesn't match its "Batteries Included" spirit, nor the PC revolution for that matter. The reason I brought up the subject at 2.7.7 is because there are greater changes than normal in this release, and people are expecting security fixes. This is realistically the last chance to fix this vulnerability in 2.X. We should help people do the right thing and be safe, while keeping the possibility to customize. Yes, I can imagine a few people in the world unhappy with any change, but a large majority of "millions" will not notice and reap the benefits. There are also quite a few people unhappy the change hasn't been made (see the bug). However, those that want a less safe setting on brand new installs may still change it manually, script it with the command-line parameter, or could stay with 2.7.6 until they do. The porting effort is tiny and would effect few compared to those who'd benefit. -Mike p.s. I'm available to help with maintaining the 2.7 installer, unfortunately I've only used Inno so not sure how long it would take me to ramp up. On 04/29/2014 12:04 PM, Steve Dower wrote:
Mike Miller writes:
But there's the rub. In this case, Microsoft doesn't have *security*, it has "guidelines". They are *still* guidelines, not security, *exactly* because it's convenient for somebody. The fact that taking advantage of that convenience has the side effect of bypassing filesystem permissions is unfortunate (and a bug in Windows IMO). Note that if users actually paid attention to these guidelines, we'd be getting complaints from *them*, not from you. I don't recall ever seeing that. That implies that "normal users" will install anything anwhere anyway. If it's that unimportant to Microsoft, I see insufficient reason why we should risk confusing those "normal users" who already have Python 2.7 installed (and as pointed out, they *are* at risk precisely because the proposal changes the default install location).
On Tue, Apr 29, 2014 at 12:07:00PM +0900, Stephen J. Turnbull wrote:
I don't think that argument is terribly useful. If people waited for "normal users" to complain before doing something about Heartbeat, we'd be in a pretty pickle. "Normal users" don't understand the technology well enough to have a valid threat model or judge the consequences, and they are confused by a mixture of ignorance, misinformation and hype. It's up to technical users to lead, not to follow.
If it's that unimportant to Microsoft,
I think that's unfair. I'm not a MS fan, not even close. I think their business practices in the past have been reprehensible. But if there is anyone who takes backwards-compatibility even more seriously than Python-Dev, it is them. You should give them the courtesy of assuming that their decision is not based on apathy, but on *exactly* the same reasoning that *you* apply below:
And thus security vulnerabilities never get fixed :-) I don't have an opinion on the importance or magnitude of this security vulnerability, the risk of confusion, or whether it should be fixed or not. But I wonder why the installer is ignoring the OS's guidelines for where software should be installed? If this were Apple we were talking about, would we ignore their guidelines? Or on Linux, would we blithly install Python in / instead of (say) /usr/local/bin? I don't think so. I would have thought that the mere fact that Microsoft disapproves of installing applications into the root should be good enough reason to not do it. In the absense of an extremely compelling reason not to do so, we should be a "good citizen" regardless of the OS. -- Steven
On 4/28/2014 8:52 PM, Steven D'Aprano wrote:
I have perfectly good DOS and Windows programs that no longer run on supported Windows versions. XP->Vista was a bigger break than Python 2->3, from my perspective. And then if you get into the command line programs that have vanished or take different options, breaking batch files, and similar facilities, it gets even worse.
Steven D'Aprano writes:
On Tue, Apr 29, 2014 at 12:07:00PM +0900, Stephen J. Turnbull wrote:
I'm not suggesting that we wait for such reports in any case. My point is that the "security hole" is apparently wide open and nobody, not Microsoft and not corporate IT, is doing anything to close it. If it were, it would inconvenience users and we'd hear about it. I infer they assess the threat to be essentially zero.
If it's that unimportant to Microsoft,
I think that's unfair.
What's unfair? Mr. Miller is evidently concerned about users who have completely left their security up to their OS vendor. Isn't reference to the importance of an alleged security hole to that vendor both fair and relevant to a decision to make a backwards-incompatible change in a Python bugfix release, even in the installer?
You should give them the courtesy of assuming that their decision is not based on apathy,
My point is that Microsoft has *not* made a decision, but left it up to anybody who has software they hope Windows users will install -- both Python-Dev and crackers. I infer they do not consider this a security issue worthy of notice.
And thus security vulnerabilities never get fixed :-)
I have no objection *at all* to making the change in the next feature release. I think the "good citizenship" argument is more than sufficient, but of course I'll leave it up to the release manager. As for bugfix releases, given the arguments above, I want a stronger argument than "Microsoft guidelines", that's all. I don't even ask for a CVE. :-)
I'm not defending the failure to follow the guideline in the Python 2.x.0 releases (IIUC the guideline pre-existed 2.7). I'm questioning whether it is a sufficient reason to make a backwards-incompatible change in a bugfix release. My take is that Microsoft itself doesn't think it's very important. Regards,
On 04/29/2014 03:07 PM, Stephen J. Turnbull wrote:
Normally I would completely agree. However, this bug has been shitcanned for a decade. This is the last chance to fix this bug in a branch that's going to be supported until 2020! An unknown portion of Python 2.x users will never upgrade to 3. Do we continue on this path in good conscience for at least six more years? -Mike
Quoting "Stephen J. Turnbull" <stephen@xemacs.org>:
*My* silence is easy to explain: I don't maintain Python 2.7 any longer. So my view doesn't really matter (although I do support Steve's position). In any case, I think Mike is following a lost cause. If the maintainer in charge (i.e. Steve) doesn't want to make a specific change, the only way to force him would be from "higher up", which could be the release manager, the BDFL, or a PEP. Regards, Martin
On Mon, Apr 28, 2014 at 11:07 PM, Mike Miller <python-dev@mgmiller.net> wrote:
You don't take into account many tutorials and internal docs that make the job of new users easier by using this default. Human processes will be broken, newbies will suffer.
Hi, Every change has pluses and minuses. I can't guarantee 100% benefits, only trying to make the case that the benefits here outweigh them. Since we are talking about humans, I'd gather most of them trying to install something on Windows will have heard about ProgramFiles and not be too bothered at its inclusion in the path. -Mike On 04/29/2014 06:31 PM, anatoly techtonik wrote:
Mike Miller wrote:
Every change has pluses and minuses. I can't guarantee 100% benefits, only trying to make the case that the benefits here outweigh them.
If this is your case about the benefits, it's a weak case. Feel free to blog about how to secure a Python installation in multi-user environments, or contribute docs to python.org, or just go ahead and make your own secure installer (seriously - there are enterprises/IT departments that will prefer a "secure by default" version rather than having to secure it themselves). A change that will break many existing users is a huge minus. Even putting this into 3.5 would worry me unless there is a *much* higher-than-usual usage of the prereleases. Here are some more minuses beyond those listed on the issue: * Longer install time (due to having to compile the stdlib to pyc and pyo eagerly) * Larger installed size (same reason) * Incompatibility with existing non-elevating package installers (every bdist_wininst is in this category, I believe) * Incompatibility with tools that don't recognize user site-packages Incidentally, forcing normal users to elevate to install packages is a bigger security risk than we currently have, since installers (or setup.py's) do not run elevated. Once everyone "knows" that you need to elevate for this (and trust me, people already expect it), *any* package could become the delivery mechanism for malicious code that will be run with admin privileges - no exploits, just social engineering. Right now, we are actually more secure for the vast majority of single-user environments, since people aren't expecting a UAC prompt and will view it with suspicion.
Modifying PATH is not recommended by Microsoft (App Paths are - http://msdn.microsoft.com/en-US/ee872121.aspx), and your entire argument in support of using Program Files is that it is Microsoft's recommendation. It's only a minor inconsistency, but be aware that you are weakening your own argument here :) "Heard about ProgramFiles" and "inclusion in the path" are two very different ideas. My grandmother knows about Program Files (though the "x86" still confuses her), but doesn't know anything about PATH. If someone installed Python for her and her "trick" of typing "news.txt" into the Run dialog* suddenly broke, I'm sure she would be unhappy. (*She has no such trick, to my knowledge, but this is one of the side-effects of modifying PATH.) Or if someone dropped a "sol.py" ahead of sol.exe (the user profile directory will work, provided the file is not in App Paths) and the PATHEXT change caused the Python file to be run instead of the EXE. The unintended consequences are significant, which is why I argue that python.org Python should not be installed on my grandmother's computer - there is a difference between the Python development environment and the runtime environment. My basic argument is not that you are wrong, but that it is too late to change. If we were starting from scratch right now, the reference installers from python.org would absolutely install Python into Program Files. But it is far too late to change it for 2.7.x, and it would require a serious education campaign for 3.5 to make sure as many users as possible are not burned. Now is the time to start publicly making a noise about the risks of the current installer and how to mitigate them (the second part is important - otherwise you are just making noise). Cheers, Steve
On 04/30/2014 04:14 AM, Steve Dower wrote:
Here are some more minuses beyond those listed on the issue:
I have to say I'm a bit baffled. I expected disagreement, but didn't expect that multiple reasons against would be made up seemingly at random? I and a company I work for (that distributes Py) have been installing Python to ProgramFiles for almost a decade, and can assure that none of those things you mention have yet happened. Also, your security point contradicts industry best practice. Mac and Linux require sudo to install system packages, with user packages and venv as alternatives.
My basic argument is not that you are wrong, but that it is too late to change.
That's a fine argument. The problem is that we and others like us are not able to move to Py3 (for work) and won't be able to for a long-enough time that it could be considered to be "never". -Mike
Mike Miller <python-dev@mgmiller.net> wrote:
Relax, I don't think Steve is making things up. That said, I can confirm what you wrote: I've always installed Python to "Program Files" and I've never had any issues (then again, I'm mostly using Linux). Stefan Krah
On 29 April 2014 17:02, Stefan Krah <stefan-usenet@bytereef.org> wrote:
It's important to note that the feature backport exceptions in the network security enhancements PEP were granted specifically because they had security implications *beyond* the specific systems and applications still running Python 2.7. Unfortunately, I lost some of that rationale when I trimmed it down to the more specific proposal: ============================== The key requirement for a feature to be considered for inclusion in this policy is that it must have security implications *beyond* the specific application that is written in Python and the system that application is running on. Thus the focus on network security protocols, password storage and related cryptographic infrastructure - Python is a popular choice for the development of web services and clients, and thus the capabilities of widely used Python versions have implications for the security design of other services that may themselves be using newer versions of Python or other development languages, but need to interoperate with clients or servers written using older versions of Python. The intent behind this requirement is to minimise any impact that the introduction of this policy may have on the stability and compatibility of maintenance releases. It would be thoroughly counterproductive if end users became as cautious about updating to new Python 2.7 maintenance releases as they are about updating to new feature releases within the same release series. ============================== I'll find a place to add that back in (not tonight, though), since it's an important part of the reason Mike's suggested installer changes are *not* remotely in scope for 2.7.7. However, that's currently not obvious to folks that have only read the final version of the PEP, and didn't see the earlier more open ended versions that included that text. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Mike Miller wrote:
This would be an incredibly painful change that would surprise and hurt a lot of people. Mark Hammond's last comment on the linked issue sums up my view as well: "My take is still that Python is a tool, not an app. People writing an app they wish to distribute using Python should include Python in their package (ie, not rely on an installed version) and these apps should conform with the guidelines." Yes, it is possible for a non-admin user to install arbitrary packages into a place where an admin user may inadvertently run it, thereby providing escalation of privilege. On the other hand, that applies to a lot of development tools, especially since most users on Windows these days are actually limited administrators - ANYTHING they install could run when they elevate a certain process. I'm all for encouraging apps to redistribute Python (as most do on Windows). In this case, they can easily apply the correct security settings, typically by installing pythonxy.dll with their application and ignoring any global settings (PYTHONPATH, registry keys, etc.). On the other hand, Python from python.org is a tool that should only be installed by those who are prepared to manage it. On Windows it is easy enough to have a second, secured copy for your admin scripts and an unsecured copy for non-admin tasks.
I'm not sure what change you are proposing here... doesn't the installer already have an option to add to PATH? I'm sure I keep disabling it. I don't want "python.exe" on my PATH because I have 10+ versions installed at any one time. I have my own set of aliases because even the py.exe launcher can't handle my setup :) The info we've gotten from our users (Python Tools for Visual Studio) shows that most have two or more versions of Python installed, so having one or both of them in PATH or PATHEXT just adds ambiguity. The other rationales are .pyc generation and package installation. Unlike PATH, these are not red herrings, but they do only apply to developers and not end users (where the developer has done his/her job properly). Speaking as the one who is likely to take over from Martin as the Windows build manager (the only people arguing so far are my employer's lawyers, but I'm winning that argument), I don't see any change that can be made to 2.7.7 apart from adding a link in the installer to a documentation page on how to secure the Python installation. At this point, 2.7.6->2.7.7 should be an incredibly safe upgrade, and there's no way to safely change the default installation location or the ACLs on the install directory. As for 3.5, I have some ideas that I will raise with python-dev once I've had a chance to think through all the issues (think proper per-user installs, redist modules, etc.). More secure installations would certainly be on the table, but practicality very easily beats purity here IMHO. Cheers, Steve
On 04/29/2014 05:12 AM, Steve Dower wrote:
This would be an incredibly painful change that would surprise and hurt a lot of people.
Hi, I think "incredibly painful" is overstating the case a bit. ;) We're talking about an installer default, a setting that would still be changeable as it always has, that by definition only will affect brand new installs.
None of Microsoft's Dev tools install to C:\, rather to ProgramFiles. The fact that another security issue may exist is not a good justification for creating additional.
This sounds like the perspective of someone highly technical, forgetting that new users will be installing python as well and vastly outnumber us. "Normal people" need help to avoid security issues. Microsoft's guidelines on where to install software are clear, and don't make exceptions that "tools" should be installed to the root of the drive to bypass file system permissions, for convenience.
I'm not sure what change you are proposing here... doesn't the installer already have an option to add to PATH? I'm sure I keep disabling it.
No, it does not. Unless it got slipped in when I wasn't looking. It should be an option though, I think everyone would agree.
"python.exe" on my PATH because I have 10+ versions installed at any one time. I
Remember, python-dev's are not the target users of this package, and are a rather minuscule fraction of the user base.
Python installation. At this point, 2.7.6->2.7.7 should be an incredibly safe upgrade, and there's no way to safely change the default installation location
This would continue to be the case, as the installer will recognize the previously installed Python 2.7 and use its setting. This should affect only *brand new installs.*
or the ACLs on the install directory.
No ACLs would be affected or changed or even thought about. Simply installing to the correct folder (on new installs) will accomplish the goal. In short, this design of restricted permissions (read-only) for binaries and PATH conveniences goes back decades under Unix and other OS's. MS Windows has finally caught up in the security department in the last few releases. Please don't keep us back in the "bad old days" of DOS where everything was installed to the root folder. -- -Mike
On Mon, Apr 28, 2014 at 3:07 PM, Mike Miller <python-dev@mgmiller.net> wrote:
The option to add the current install to your path was added 3.3.
Knowing which Python you want on your path and that you want it on your path at all is somewhat of an advanced usage. While beginners do want to just open up cmd and type "python" and have it work, there are better ways to accomplish that which don't involve system-wide path manipulation or installation changes. Several PC manufacturers have been known to use Python for various system utilities, which is how Python sometimes ends up in the path on your grandma's Dell*. Even for a beginner who just wants "python" to work, we need to be careful to not wreck their entire system by helping them get our fresh Python install to show up. A more reasonable way to help beginners would be to create a shortcut somewhere that starts up cmd with a modified path. They can do whatever they want to do within that context without modifying their entire system. If they learn that they later want their system-wide path manipulated, they can do that within the installer or will known how to do that themselves. * watch Dave Beazley's PyCon 2014 talk for a good story involving one of those manufacturer installed Pythons: https://www.youtube.com/watch?v=RZ4Sn-Y7AP8
On 04/29/2014 08:38 AM, Brian Curtin wrote:
The option to add the current install to your path was added 3.3.
Ok, thanks. So there is some precedent it would be useful.
Installing python at all is a "system-wide" change. Why not default to a known safe, ready to use configuration, rather than a "tool-box" that needs assembly?
Thanks, I'm trying to get thru all the talk will watch that shortly. ;) Still, edge cases regarding manufactures should not override the needs of the majority of users. -Mike
On Mon, Apr 28, 2014 at 1:56 PM, Mike Miller <python-dev@mgmiller.net>wrote:
fun talk -- but start at about 18:00 minutes if you want the relevant bit... By the way, some people (or at least ESRI, with ArcMap) Install and use their own python in: C:\python27 -- if you dont want to interat with that, you need to put it somewhere else! I'd argue that any OEM or third party app that uses the standard location or puts it on the PATH be default is in error, but what can you so? Redhat used to have #!/usr/bin/env python on their admin scripts, which was really painful if you wanted to upgrade anything. Don't know it they've fixed that.... -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
Good point about it only affecting new installs, though that still constitutes a lot of installs.
The fact that the mitigations are well known by the people who have to worry about them is a good justification for not creating a compatibility issue. It's easy for IT admins to install Python in a way that the files are read-only, the .pyc and .pyo files are already there, and user site packages will be used by default (I think that last one is easy?). The good IT admins even know that they need to do this - perhaps we can help educate the bad admins? (FWIW, if you have admin privileges on your own machine, YOU are the IT admin. Are you a good one or a bad one?)
One place where mitigations are added are the distributions (Canopy, Anaconda, etc.). These packages redistribute Python and install to different locations with different permissions (I'm not promising that they all do it properly, but they have the opportunity to do so). The reference implementation of CPython is typically not the best option for "normal people", who are much better served by one of these bundles.
I'm well aware of the guidelines, hence the practicality vs. purity comment. I'm fairly certain that the installation to the root was originally about ease of command-line access rather than bypassing permissions - the permissions probably didn't exist for the first few versions of Python (Python for DOS obviously didn't care... maybe it's always been about backwards compatibility?) At this point, installing into the root is about not breaking everyone who *knows* that Python installs into the root directory and always has.
Thanks Brett for pointing out that it arrived in Python 3. I'm sure it would be an acceptable addition to 2.7.7, though you'd need to get it in before RC and you'd also need to find someone who is keen and able to keep making 2.7 installers for Windows. Right now, we don't have anyone who is both.
My day job revolves around making Python easier to use for non "python-dev"s, especially those who have multiple versions of Python installed simultaneously (as I mentioned, over 50% of our user base). Having PATH or PATHEXT set globally is a source of much confusion. (Though a global PYTHONPATH causes more problems, but that isn't part of this discussion.)
For example, next time you redeploy your hosted service on a clean VM with the latest security patches? The next time someone does a clean uninstall-reinstall because they've been burned by upgrades before? The next time someone arrives at work and IT has given them a new image and Python is "gone"? I can guarantee that you will break someone
The effective permissions change, and now commands that didn't require elevation do. Tools that didn't need to be manifested for UAC have to be recompiled. Scripts that used to work will simply break. This is the problem, not whether the permissions are explicitly set as part of the install process.
My employer would be most unhappy if I wasn't trying to encourage good Windows support :) At the same time, my employer, colleagues, friends, collaborators and most people on python-dev would all be unhappy if I made everyone angry at Python. At risk of saying it too often, practicality beats purity. Cheers, Steve
-- -Mike
On Mon, Apr 28, 2014 at 7:04 PM, Steve Dower <Steve.Dower@microsoft.com> wrote:
If it's an acceptable change to the release manager (Benjamin?), and if there's actually time before the RC (I don't know when it is planned), I am willing to backport my 3.3 change to get this in the 2.7 installer. However, I'm not currently setup to make release installers -- I think I need a signing certificate or something like that.
On Mon, Apr 28, 2014, at 17:14, Brian Curtin wrote:
That's fine.
However, I'm not currently setup to make release installers -- I think I need a signing certificate or something like that.
Apparently the current PSF one just expired, so we're in the process of procuring a new one.
Hi, Stepping back a bit... I doubt you'd take the idea this far, but that Python should need assembly by professionals before use doesn't match its "Batteries Included" spirit, nor the PC revolution for that matter. The reason I brought up the subject at 2.7.7 is because there are greater changes than normal in this release, and people are expecting security fixes. This is realistically the last chance to fix this vulnerability in 2.X. We should help people do the right thing and be safe, while keeping the possibility to customize. Yes, I can imagine a few people in the world unhappy with any change, but a large majority of "millions" will not notice and reap the benefits. There are also quite a few people unhappy the change hasn't been made (see the bug). However, those that want a less safe setting on brand new installs may still change it manually, script it with the command-line parameter, or could stay with 2.7.6 until they do. The porting effort is tiny and would effect few compared to those who'd benefit. -Mike p.s. I'm available to help with maintaining the 2.7 installer, unfortunately I've only used Inno so not sure how long it would take me to ramp up. On 04/29/2014 12:04 PM, Steve Dower wrote:
Mike Miller writes:
But there's the rub. In this case, Microsoft doesn't have *security*, it has "guidelines". They are *still* guidelines, not security, *exactly* because it's convenient for somebody. The fact that taking advantage of that convenience has the side effect of bypassing filesystem permissions is unfortunate (and a bug in Windows IMO). Note that if users actually paid attention to these guidelines, we'd be getting complaints from *them*, not from you. I don't recall ever seeing that. That implies that "normal users" will install anything anwhere anyway. If it's that unimportant to Microsoft, I see insufficient reason why we should risk confusing those "normal users" who already have Python 2.7 installed (and as pointed out, they *are* at risk precisely because the proposal changes the default install location).
On Tue, Apr 29, 2014 at 12:07:00PM +0900, Stephen J. Turnbull wrote:
I don't think that argument is terribly useful. If people waited for "normal users" to complain before doing something about Heartbeat, we'd be in a pretty pickle. "Normal users" don't understand the technology well enough to have a valid threat model or judge the consequences, and they are confused by a mixture of ignorance, misinformation and hype. It's up to technical users to lead, not to follow.
If it's that unimportant to Microsoft,
I think that's unfair. I'm not a MS fan, not even close. I think their business practices in the past have been reprehensible. But if there is anyone who takes backwards-compatibility even more seriously than Python-Dev, it is them. You should give them the courtesy of assuming that their decision is not based on apathy, but on *exactly* the same reasoning that *you* apply below:
And thus security vulnerabilities never get fixed :-) I don't have an opinion on the importance or magnitude of this security vulnerability, the risk of confusion, or whether it should be fixed or not. But I wonder why the installer is ignoring the OS's guidelines for where software should be installed? If this were Apple we were talking about, would we ignore their guidelines? Or on Linux, would we blithly install Python in / instead of (say) /usr/local/bin? I don't think so. I would have thought that the mere fact that Microsoft disapproves of installing applications into the root should be good enough reason to not do it. In the absense of an extremely compelling reason not to do so, we should be a "good citizen" regardless of the OS. -- Steven
On 4/28/2014 8:52 PM, Steven D'Aprano wrote:
I have perfectly good DOS and Windows programs that no longer run on supported Windows versions. XP->Vista was a bigger break than Python 2->3, from my perspective. And then if you get into the command line programs that have vanished or take different options, breaking batch files, and similar facilities, it gets even worse.
Steven D'Aprano writes:
On Tue, Apr 29, 2014 at 12:07:00PM +0900, Stephen J. Turnbull wrote:
I'm not suggesting that we wait for such reports in any case. My point is that the "security hole" is apparently wide open and nobody, not Microsoft and not corporate IT, is doing anything to close it. If it were, it would inconvenience users and we'd hear about it. I infer they assess the threat to be essentially zero.
If it's that unimportant to Microsoft,
I think that's unfair.
What's unfair? Mr. Miller is evidently concerned about users who have completely left their security up to their OS vendor. Isn't reference to the importance of an alleged security hole to that vendor both fair and relevant to a decision to make a backwards-incompatible change in a Python bugfix release, even in the installer?
You should give them the courtesy of assuming that their decision is not based on apathy,
My point is that Microsoft has *not* made a decision, but left it up to anybody who has software they hope Windows users will install -- both Python-Dev and crackers. I infer they do not consider this a security issue worthy of notice.
And thus security vulnerabilities never get fixed :-)
I have no objection *at all* to making the change in the next feature release. I think the "good citizenship" argument is more than sufficient, but of course I'll leave it up to the release manager. As for bugfix releases, given the arguments above, I want a stronger argument than "Microsoft guidelines", that's all. I don't even ask for a CVE. :-)
I'm not defending the failure to follow the guideline in the Python 2.x.0 releases (IIUC the guideline pre-existed 2.7). I'm questioning whether it is a sufficient reason to make a backwards-incompatible change in a bugfix release. My take is that Microsoft itself doesn't think it's very important. Regards,
On 04/29/2014 03:07 PM, Stephen J. Turnbull wrote:
Normally I would completely agree. However, this bug has been shitcanned for a decade. This is the last chance to fix this bug in a branch that's going to be supported until 2020! An unknown portion of Python 2.x users will never upgrade to 3. Do we continue on this path in good conscience for at least six more years? -Mike
Quoting "Stephen J. Turnbull" <stephen@xemacs.org>:
*My* silence is easy to explain: I don't maintain Python 2.7 any longer. So my view doesn't really matter (although I do support Steve's position). In any case, I think Mike is following a lost cause. If the maintainer in charge (i.e. Steve) doesn't want to make a specific change, the only way to force him would be from "higher up", which could be the release manager, the BDFL, or a PEP. Regards, Martin
On Mon, Apr 28, 2014 at 11:07 PM, Mike Miller <python-dev@mgmiller.net> wrote:
You don't take into account many tutorials and internal docs that make the job of new users easier by using this default. Human processes will be broken, newbies will suffer.
Hi, Every change has pluses and minuses. I can't guarantee 100% benefits, only trying to make the case that the benefits here outweigh them. Since we are talking about humans, I'd gather most of them trying to install something on Windows will have heard about ProgramFiles and not be too bothered at its inclusion in the path. -Mike On 04/29/2014 06:31 PM, anatoly techtonik wrote:
Mike Miller wrote:
Every change has pluses and minuses. I can't guarantee 100% benefits, only trying to make the case that the benefits here outweigh them.
If this is your case about the benefits, it's a weak case. Feel free to blog about how to secure a Python installation in multi-user environments, or contribute docs to python.org, or just go ahead and make your own secure installer (seriously - there are enterprises/IT departments that will prefer a "secure by default" version rather than having to secure it themselves). A change that will break many existing users is a huge minus. Even putting this into 3.5 would worry me unless there is a *much* higher-than-usual usage of the prereleases. Here are some more minuses beyond those listed on the issue: * Longer install time (due to having to compile the stdlib to pyc and pyo eagerly) * Larger installed size (same reason) * Incompatibility with existing non-elevating package installers (every bdist_wininst is in this category, I believe) * Incompatibility with tools that don't recognize user site-packages Incidentally, forcing normal users to elevate to install packages is a bigger security risk than we currently have, since installers (or setup.py's) do not run elevated. Once everyone "knows" that you need to elevate for this (and trust me, people already expect it), *any* package could become the delivery mechanism for malicious code that will be run with admin privileges - no exploits, just social engineering. Right now, we are actually more secure for the vast majority of single-user environments, since people aren't expecting a UAC prompt and will view it with suspicion.
Modifying PATH is not recommended by Microsoft (App Paths are - http://msdn.microsoft.com/en-US/ee872121.aspx), and your entire argument in support of using Program Files is that it is Microsoft's recommendation. It's only a minor inconsistency, but be aware that you are weakening your own argument here :) "Heard about ProgramFiles" and "inclusion in the path" are two very different ideas. My grandmother knows about Program Files (though the "x86" still confuses her), but doesn't know anything about PATH. If someone installed Python for her and her "trick" of typing "news.txt" into the Run dialog* suddenly broke, I'm sure she would be unhappy. (*She has no such trick, to my knowledge, but this is one of the side-effects of modifying PATH.) Or if someone dropped a "sol.py" ahead of sol.exe (the user profile directory will work, provided the file is not in App Paths) and the PATHEXT change caused the Python file to be run instead of the EXE. The unintended consequences are significant, which is why I argue that python.org Python should not be installed on my grandmother's computer - there is a difference between the Python development environment and the runtime environment. My basic argument is not that you are wrong, but that it is too late to change. If we were starting from scratch right now, the reference installers from python.org would absolutely install Python into Program Files. But it is far too late to change it for 2.7.x, and it would require a serious education campaign for 3.5 to make sure as many users as possible are not burned. Now is the time to start publicly making a noise about the risks of the current installer and how to mitigate them (the second part is important - otherwise you are just making noise). Cheers, Steve
On 04/30/2014 04:14 AM, Steve Dower wrote:
Here are some more minuses beyond those listed on the issue:
I have to say I'm a bit baffled. I expected disagreement, but didn't expect that multiple reasons against would be made up seemingly at random? I and a company I work for (that distributes Py) have been installing Python to ProgramFiles for almost a decade, and can assure that none of those things you mention have yet happened. Also, your security point contradicts industry best practice. Mac and Linux require sudo to install system packages, with user packages and venv as alternatives.
My basic argument is not that you are wrong, but that it is too late to change.
That's a fine argument. The problem is that we and others like us are not able to move to Py3 (for work) and won't be able to for a long-enough time that it could be considered to be "never". -Mike
Mike Miller <python-dev@mgmiller.net> wrote:
Relax, I don't think Steve is making things up. That said, I can confirm what you wrote: I've always installed Python to "Program Files" and I've never had any issues (then again, I'm mostly using Linux). Stefan Krah
On 29 April 2014 17:02, Stefan Krah <stefan-usenet@bytereef.org> wrote:
It's important to note that the feature backport exceptions in the network security enhancements PEP were granted specifically because they had security implications *beyond* the specific systems and applications still running Python 2.7. Unfortunately, I lost some of that rationale when I trimmed it down to the more specific proposal: ============================== The key requirement for a feature to be considered for inclusion in this policy is that it must have security implications *beyond* the specific application that is written in Python and the system that application is running on. Thus the focus on network security protocols, password storage and related cryptographic infrastructure - Python is a popular choice for the development of web services and clients, and thus the capabilities of widely used Python versions have implications for the security design of other services that may themselves be using newer versions of Python or other development languages, but need to interoperate with clients or servers written using older versions of Python. The intent behind this requirement is to minimise any impact that the introduction of this policy may have on the stability and compatibility of maintenance releases. It would be thoroughly counterproductive if end users became as cautious about updating to new Python 2.7 maintenance releases as they are about updating to new feature releases within the same release series. ============================== I'll find a place to add that back in (not tonight, though), since it's an important part of the reason Mike's suggested installer changes are *not* remotely in scope for 2.7.7. However, that's currently not obvious to folks that have only read the final version of the PEP, and didn't see the earlier more open ended versions that included that text. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
participants (14)
-
anatoly techtonik -
Antoine Pitrou -
Benjamin Peterson -
Brian Curtin -
Chris Barker -
Glenn Linderman -
martin＠v.loewis.de -
Mike Miller -
Nick Coghlan -
Stefan Krah -
Stephen J. Turnbull -
Steve Dower -
Steven D'Aprano -
Sturla Molden