[issue12632] Windows GPF with Code Page 65001

[Bruce Ferris]

Bruce Ferris <bferris57 at bferris.co.uk> added the comment:

I disagree with the "it's not really a GPF since it calls Abort".

Consider the following cmd.exe session...

  Microsoft Windows [Version 6.0.6002]
  Copyright (c) 2006 Microsoft Corporation.  All rights reserved.

  D:\>chcp 65001
  Active code page: 65001

  D:\>python >t.txt
  Fatal Python error: Py_Initialize: can't initialize sys standard streams
  LookupError: unknown encoding: cp65001

  This application has requested the Runtime to terminate it in an unusual way.
  Please contact the application's support team for more information.

  D:\>type t.txt

  D:\>dir t.txt
  Volume in drive D is DATA
  Volume Serial Number is 2E61-626C

  Directory of D:\

  25/07/2011  06:10 PM                 0 t.txt
                 1 File(s)              0 bytes
                 0 Dir(s)  16,768,655,360 bytes free

  D:\>

This means that, even if it was "intentional", from a programatic point of view. the Python process in this case leaves no other indication other than transient bytes in the transient cmd.exe console buffer.  No way of redirecting the output and examining it.

I strongly disagree with the statement "(If it were a true segfault-like error, there would be no message from Python itself.)"

The "no message from Python itself" case is shown above.

My application handles code page 65001 just fine, no problems.  If it attempts to use Windows WriteConsole function and it fails, it tries using WriteFile instead.  So, when my application fails and output is redirected, it produces output.

But, Python 3.1 doesn't.  See the following Microsoft MSDN link, it states the WriteConsole point explicitly...

  http://msdn.microsoft.com/en-us/library/ms687401%28v=VS.85%29.aspx

So, if Python doesn't like Code Page 65001, for whatever reason, it can simply save it on startup, and change it to whatever makes it happy.  Then, upon Python exit (including Abort), change it back to 65001 before calling Abort.

I'm sorry, but the following is "easy" in my book...

  1) At Startup...  Call GetConsoleOutputCP and save that somewhere.

                    If code page is 65001, change it to something that
                    doesn't cause problems by calling SetConsoleOutputCP

  2) On Write... If WriteConsole fails, try calling WriteFile instead.

  3) At Abort or Exit... Call SetConsoleOutputCP to set it back
                         to whatever it was on Startup.

I don't care if your app (Python) can display UTF-8 on Microsoft's cmd.exe console or if it can't. 

All I'm trying to do is point out a bit of misbehaviour that CAN be easily changed and will make your product more resilient.

I don't know the details of how Python deals with character encoding and, quite honestly, I shouldn't need to since it's not my product.  however, I DO know how I handle a similiar scenario in my own app.

Microsoft made it complicated, not me.  But, I can "easily" get around the problem using the above scenario.  If Python can't do it just as "easily", then it tells me more about Python's implementation and the people behind Python then it tells me about Microsoft and the people behind Windows.

Don't get me wrong, I love Python as a tool for solving certain classes of problems and, please, keep up the good work.  It's appreciated.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue12632>
_______________________________________