On Mon, Apr 28, 2014 at 3:07 PM, Mike Miller wrote:

On 04/29/2014 05:12 AM, Steve Dower wrote:

...

This would be an incredibly painful change that would surprise and hurt a lot of people.

Hi, I think "incredibly painful" is overstating the case a bit. ;) We're talking about an installer default, a setting that would still be changeable as it always has, that by definition only will affect brand new installs.

...

Yes, it is possible for a non-admin user to install arbitrary packages into a place where an admin user may inadvertently run it, thereby providing escalation of privilege. On the other hand, that applies to a lot of development tools, especially since most users on Windows these days are actually limited administrators - ANYTHING they install could run when they elevate a certain process.

None of Microsoft's Dev tools install to C:\, rather to ProgramFiles. The fact that another security issue may exist is not a good justification for creating additional.

...

On the other hand, Python from python.org is a tool that should only be installed by those who are prepared to manage it. On Windows it is easy enough to have a second, secured copy for your admin scripts and an unsecured copy for non-admin tasks.

This sounds like the perspective of someone highly technical, forgetting that new users will be installing python as well and vastly outnumber us. "Normal people" need help to avoid security issues.

Microsoft's guidelines on where to install software are clear, and don't make exceptions that "tools" should be installed to the root of the drive to bypass file system permissions, for convenience.

...

I'm not sure what change you are proposing here... doesn't the installer already have an option to add to PATH? I'm sure I keep disabling it.

No, it does not. Unless it got slipped in when I wasn't looking.

It should be an option though, I think everyone would agree.

The option to add the current install to your path was added 3.3.

...

"python.exe" on my PATH because I have 10+ versions installed at any one time. I

Remember, python-dev's are not the target users of this package, and are a rather minuscule fraction of the user base.

Knowing which Python you want on your path and that you want it on your path at all is somewhat of an advanced usage. While beginners do want to just open up cmd and type "python" and have it work, there are better ways to accomplish that which don't involve system-wide path manipulation or installation changes. Several PC manufacturers have been known to use Python for various system utilities, which is how Python sometimes ends up in the path on your grandma's Dell*. Even for a beginner who just wants "python" to work, we need to be careful to not wreck their entire system by helping them get our fresh Python install to show up. A more reasonable way to help beginners would be to create a shortcut somewhere that starts up cmd with a modified path. They can do whatever they want to do within that context without modifying their entire system. If they learn that they later want their system-wide path manipulated, they can do that within the installer or will known how to do that themselves. * watch Dave Beazley's PyCon 2014 talk for a good story involving one of those manufacturer installed Pythons: https://www.youtube.com/watch?v=RZ4Sn-Y7AP8

On Mon, Apr 28, 2014 at 1:56 PM, Mike Miller wrote:

* watch Dave Beazley's PyCon 2014 talk for a good story involving one

...

of those manufacturer installed Pythons: https://www.youtube.com/watch?v=RZ4Sn-Y7AP8

Thanks, I'm trying to get thru all the talk will watch that shortly. ;)

fun talk -- but start at about 18:00 minutes if you want the relevant bit... By the way, some people (or at least ESRI, with ArcMap) Install and use their own python in: C:\python27 -- if you dont want to interat with that, you need to put it somewhere else! I'd argue that any OEM or third party app that uses the standard location or puts it on the PATH be default is in error, but what can you so? Redhat used to have #!/usr/bin/env python on their admin scripts, which was really painful if you wanted to upgrade anything. Don't know it they've fixed that.... -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov

On Mon, Apr 28, 2014 at 7:04 PM, Steve Dower wrote:

...

Mike Miller wrote: On 04/29/2014 05:12 AM, Steve Dower wrote:

...

This would be an incredibly painful change that would surprise and hurt a lot of people.

Hi, I think "incredibly painful" is overstating the case a bit. ;) We're talking about an installer default, a setting that would still be changeable as it always has, that by definition only will affect brand new installs.

Good point about it only affecting new installs, though that still constitutes a lot of installs.

...

...

Yes, it is possible for a non-admin user to install arbitrary packages into a place where an admin user may inadvertently run it, thereby providing escalation of privilege. On the other hand, that applies to a lot of development tools, especially since most users on Windows these days are actually limited administrators - ANYTHING they install could run when they elevate a certain process.

None of Microsoft's Dev tools install to C:\, rather to ProgramFiles. The fact that another security issue may exist is not a good justification for creating additional.

The fact that the mitigations are well known by the people who have to worry about them is a good justification for not creating a compatibility issue. It's easy for IT admins to install Python in a way that the files are read-only, the .pyc and .pyo files are already there, and user site packages will be used by default (I think that last one is easy?). The good IT admins even know that they need to do this - perhaps we can help educate the bad admins? (FWIW, if you have admin privileges on your own machine, YOU are the IT admin. Are you a good one or a bad one?)

...

...

On the other hand, Python from python.org is a tool that should only be installed by those who are prepared to manage it. On Windows it is easy enough to have a second, secured copy for your admin scripts and an unsecured copy for non-admin tasks.

This sounds like the perspective of someone highly technical, forgetting that new users will be installing python as well and vastly outnumber us. "Normal people" need help to avoid security issues.

One place where mitigations are added are the distributions (Canopy, Anaconda, etc.). These packages redistribute Python and install to different locations with different permissions (I'm not promising that they all do it properly, but they have the opportunity to do so). The reference implementation of CPython is typically not the best option for "normal people", who are much better served by one of these bundles.

...

Microsoft's guidelines on where to install software are clear, and don't make exceptions that "tools" should be installed to the root of the drive to bypass file system permissions, for convenience.

I'm well aware of the guidelines, hence the practicality vs. purity comment. I'm fairly certain that the installation to the root was originally about ease of command-line access rather than bypassing permissions - the permissions probably didn't exist for the first few versions of Python (Python for DOS obviously didn't care... maybe it's always been about backwards compatibility?)

At this point, installing into the root is about not breaking everyone who *knows* that Python installs into the root directory and always has.

...

...

I'm not sure what change you are proposing here... doesn't the installer already have an option to add to PATH? I'm sure I keep disabling it.

No, it does not. Unless it got slipped in when I wasn't looking.

It should be an option though, I think everyone would agree.

Thanks Brett for pointing out that it arrived in Python 3. I'm sure it would be an acceptable addition to 2.7.7, though you'd need to get it in before RC and you'd also need to find someone who is keen and able to keep making 2.7 installers for Windows. Right now, we don't have anyone who is both.

If it's an acceptable change to the release manager (Benjamin?), and if there's actually time before the RC (I don't know when it is planned), I am willing to backport my 3.3 change to get this in the 2.7 installer. However, I'm not currently setup to make release installers -- I think I need a signing certificate or something like that.

Hi, Stepping back a bit... I doubt you'd take the idea this far, but that Python should need assembly by professionals before use doesn't match its "Batteries Included" spirit, nor the PC revolution for that matter. The reason I brought up the subject at 2.7.7 is because there are greater changes than normal in this release, and people are expecting security fixes. This is realistically the last chance to fix this vulnerability in 2.X. We should help people do the right thing and be safe, while keeping the possibility to customize. Yes, I can imagine a few people in the world unhappy with any change, but a large majority of "millions" will not notice and reap the benefits. There are also quite a few people unhappy the change hasn't been made (see the bug). However, those that want a less safe setting on brand new installs may still change it manually, script it with the command-line parameter, or could stay with 2.7.6 until they do. The porting effort is tiny and would effect few compared to those who'd benefit. -Mike p.s. I'm available to help with maintaining the 2.7 installer, unfortunately I've only used Inno so not sure how long it would take me to ramp up. On 04/29/2014 12:04 PM, Steve Dower wrote:

On Tue, Apr 29, 2014 at 12:07:00PM +0900, Stephen J. Turnbull wrote:

Mike Miller writes:

...

Microsoft's guidelines on where to install software are clear, and don't make exceptions that "tools" should be installed to the root of the drive to bypass file system permissions, for convenience.

But there's the rub. In this case, Microsoft doesn't have *security*, it has "guidelines". They are *still* guidelines, not security, *exactly* because it's convenient for somebody. The fact that taking advantage of that convenience has the side effect of bypassing filesystem permissions is unfortunate (and a bug in Windows IMO).

Note that if users actually paid attention to these guidelines, we'd be getting complaints from *them*, not from you. I don't recall ever seeing that. That implies that "normal users" will install anything anwhere anyway.

I don't think that argument is terribly useful. If people waited for "normal users" to complain before doing something about Heartbeat, we'd be in a pretty pickle. "Normal users" don't understand the technology well enough to have a valid threat model or judge the consequences, and they are confused by a mixture of ignorance, misinformation and hype. It's up to technical users to lead, not to follow.

If it's that unimportant to Microsoft,

I think that's unfair. I'm not a MS fan, not even close. I think their business practices in the past have been reprehensible. But if there is anyone who takes backwards-compatibility even more seriously than Python-Dev, it is them. You should give them the courtesy of assuming that their decision is not based on apathy, but on *exactly* the same reasoning that *you* apply below:

I see insufficient reason why we should risk confusing those "normal users" who already have Python 2.7 installed (and as pointed out, they *are* at risk precisely because the proposal changes the default install location).

And thus security vulnerabilities never get fixed :-) I don't have an opinion on the importance or magnitude of this security vulnerability, the risk of confusion, or whether it should be fixed or not. But I wonder why the installer is ignoring the OS's guidelines for where software should be installed? If this were Apple we were talking about, would we ignore their guidelines? Or on Linux, would we blithly install Python in / instead of (say) /usr/local/bin? I don't think so. I would have thought that the mere fact that Microsoft disapproves of installing applications into the root should be good enough reason to not do it. In the absense of an extremely compelling reason not to do so, we should be a "good citizen" regardless of the OS. -- Steven

Steven D'Aprano writes:

On Tue, Apr 29, 2014 at 12:07:00PM +0900, Stephen J. Turnbull wrote:

...

Note that if users actually paid attention to these guidelines, we'd be getting complaints from *them*, not from you. I don't recall ever seeing that. That implies that "normal users" will install anything anwhere anyway.

I don't think that argument is terribly useful. If people waited for "normal users"

I'm not suggesting that we wait for such reports in any case. My point is that the "security hole" is apparently wide open and nobody, not Microsoft and not corporate IT, is doing anything to close it. If it were, it would inconvenience users and we'd hear about it. I infer they assess the threat to be essentially zero.

...

If it's that unimportant to Microsoft,

I think that's unfair.

What's unfair? Mr. Miller is evidently concerned about users who have completely left their security up to their OS vendor. Isn't reference to the importance of an alleged security hole to that vendor both fair and relevant to a decision to make a backwards-incompatible change in a Python bugfix release, even in the installer?

You should give them the courtesy of assuming that their decision is not based on apathy,

My point is that Microsoft has *not* made a decision, but left it up to anybody who has software they hope Windows users will install -- both Python-Dev and crackers. I infer they do not consider this a security issue worthy of notice.

And thus security vulnerabilities never get fixed :-)

I have no objection *at all* to making the change in the next feature release. I think the "good citizenship" argument is more than sufficient, but of course I'll leave it up to the release manager. As for bugfix releases, given the arguments above, I want a stronger argument than "Microsoft guidelines", that's all. I don't even ask for a CVE. :-)

I would have thought that the mere fact that Microsoft disapproves of installing applications into the root should be good enough reason to not do it.

I'm not defending the failure to follow the guideline in the Python 2.x.0 releases (IIUC the guideline pre-existed 2.7). I'm questioning whether it is a sufficient reason to make a backwards-incompatible change in a bugfix release. My take is that Microsoft itself doesn't think it's very important. Regards,

Mike Miller writes:

However, this bug has been shitcanned for a decade. This is the last chance to fix this bug in a branch that's going to be supported until 2020!

Probably. I'm not convinced. But that doesn't really matter. Your bigger concern is the deafening silence from the senior committers, which suggests you're going to need to come up with a stronger argument. Maybe there's a relevant CVE?

On Mon, Apr 28, 2014 at 11:07 PM, Mike Miller wrote:

On 04/29/2014 05:12 AM, Steve Dower wrote:

...

This would be an incredibly painful change that would surprise and hurt a lot of people.

Hi, I think "incredibly painful" is overstating the case a bit. ;) We're talking about an installer default, a setting that would still be changeable as it always has, that by definition only will affect brand new installs.

You don't take into account many tutorials and internal docs that make the job of new users easier by using this default. Human processes will be broken, newbies will suffer.

Mike Miller wrote:

Every change has pluses and minuses. I can't guarantee 100% benefits, only trying to make the case that the benefits here outweigh them.

If this is your case about the benefits, it's a weak case. Feel free to blog about how to secure a Python installation in multi-user environments, or contribute docs to python.org, or just go ahead and make your own secure installer (seriously - there are enterprises/IT departments that will prefer a "secure by default" version rather than having to secure it themselves). A change that will break many existing users is a huge minus. Even putting this into 3.5 would worry me unless there is a *much* higher-than-usual usage of the prereleases. Here are some more minuses beyond those listed on the issue: * Longer install time (due to having to compile the stdlib to pyc and pyo eagerly) * Larger installed size (same reason) * Incompatibility with existing non-elevating package installers (every bdist_wininst is in this category, I believe) * Incompatibility with tools that don't recognize user site-packages Incidentally, forcing normal users to elevate to install packages is a bigger security risk than we currently have, since installers (or setup.py's) do not run elevated. Once everyone "knows" that you need to elevate for this (and trust me, people already expect it), *any* package could become the delivery mechanism for malicious code that will be run with admin privileges - no exploits, just social engineering. Right now, we are actually more secure for the vast majority of single-user environments, since people aren't expecting a UAC prompt and will view it with suspicion.

Since we are talking about humans, I'd gather most of them trying to install something on Windows will have heard about ProgramFiles and not be too bothered at its inclusion in the path.

Modifying PATH is not recommended by Microsoft (App Paths are - http://msdn.microsoft.com/en-US/ee872121.aspx), and your entire argument in support of using Program Files is that it is Microsoft's recommendation. It's only a minor inconsistency, but be aware that you are weakening your own argument here :) "Heard about ProgramFiles" and "inclusion in the path" are two very different ideas. My grandmother knows about Program Files (though the "x86" still confuses her), but doesn't know anything about PATH. If someone installed Python for her and her "trick" of typing "news.txt" into the Run dialog* suddenly broke, I'm sure she would be unhappy. (*She has no such trick, to my knowledge, but this is one of the side-effects of modifying PATH.) Or if someone dropped a "sol.py" ahead of sol.exe (the user profile directory will work, provided the file is not in App Paths) and the PATHEXT change caused the Python file to be run instead of the EXE. The unintended consequences are significant, which is why I argue that python.org Python should not be installed on my grandmother's computer - there is a difference between the Python development environment and the runtime environment. My basic argument is not that you are wrong, but that it is too late to change. If we were starting from scratch right now, the reference installers from python.org would absolutely install Python into Program Files. But it is far too late to change it for 2.7.x, and it would require a serious education campaign for 3.5 to make sure as many users as possible are not burned. Now is the time to start publicly making a noise about the risks of the current installer and how to mitigate them (the second part is important - otherwise you are just making noise). Cheers, Steve

Mike Miller wrote:

I have to say I'm a bit baffled. I expected disagreement, but didn't expect that multiple reasons against would be made up seemingly at random? I and a company I work for (that distributes Py) have been installing Python to ProgramFiles for almost a decade, and can assure that none of those things you mention have yet happened.

Relax, I don't think Steve is making things up. That said, I can confirm what you wrote: I've always installed Python to "Program Files" and I've never had any issues (then again, I'm mostly using Linux). Stefan Krah

On 04/30/2014 04:14 AM, Steve Dower wrote:

...

Since we are talking about humans, I'd gather most of them trying to install something on Windows will have heard about ProgramFiles and not be too bothered at its inclusion in the path.

Modifying PATH is not recommended by Microsoft...

Sorry, I meant the install path in that sentence, not the environment var. -Mike