[Python-bugs-list] [ python-Bugs-633480 ] IOError w/ large string to CMD shell
noreply@sourceforge.net
noreply@sourceforge.net
Tue, 05 Nov 2002 20:55:34 -0800
Bugs item #633480, was opened at 2002-11-04 16:06
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=633480&group_id=5470
Category: Windows
>Group: 3rd Party
Status: Open
Resolution: None
Priority: 4
Submitted By: Michael McCandless (mikemccand)
>Assigned to: Tim Peters (tim_one)
Summary: IOError w/ large string to CMD shell
Initial Comment:
If I try to run this program with "python2.2.1 -u":
# 61409 works!!
print ' ' * 61410
(which tries to print a string of 61,410 space characters),
I get this Traceback on Windows 2000 SP2:
Traceback (most recent call last):
File "write.py", line 4, in ?
print ' ' * 61410
IOError: [Errno 12] Not enough space
Without "-u", this runs fine, and if I change "print"
to "import sys; sys.stdout.write" instead, it still fails.
I think this is happening in Objects/fileobject.c -- the
calls to fwrite apparently don't write all bytes in win32
when the number of bytes is large -- seems like it may
be necessary to parcel the data up into smaller pieces
or something?
NOTE: it's possible that the exact length of the string
that fails may be larger on other OS's...
----------------------------------------------------------------------
>Comment By: Tim Peters (tim_one)
Date: 2002-11-05 23:55
Message:
Logged In: YES
user_id=31435
The thing is, if you want to do this, you're going to *have* to
worm around it. I've got no magic to offer -- it's a vanilla C
library call, and there's no sign that Python has anything to
do with the failure. Try doing a staight fwrite() to stdout,
unbuffered, in C, and I expect you'll see the same thing (or
if not, then *maybe* Python had something to do with it).
If it does fail in straight C, then filing a bug report with
Microsoft is appropriate. Or you could upgrade to Win98
<wink>.
Assigning to me and marking 3rdParty; I expect to close it
as WontFix unless evidence appears that not a Windows
bug.
BTW, you may have better luck installing the Python Win32
extensions and using the Win32 file API directly.
Sometimes that works better than MS's std-C I/O
implementation.
----------------------------------------------------------------------
Comment By: Michael McCandless (mikemccand)
Date: 2002-11-05 08:29
Message:
Logged In: YES
user_id=323786
Thanks for looking into this Tim...
True, we can workaround it -- either don't run UNBUFFERED,
or break up the strings before printing (override sys.stdout w/
our own file object that tries to break things up before
printing), or don't run under "CMD" or other platform-specific
shells that have this issue.
But it's rather unexpected and to a "Python newbie" would
be "surprising"; though I suppose most would not run with "-
u".
Also, from the IOError that's raised I don't know how many
bytes were actually successfully written (apparently 0 in this
case).
NOTE: I'm able to reproduce on Windows XP Professional
(no SP), running "cmd.exe", Python 2.2.2. I've also tried
changing the "screen buffer" sizes on the CMD and it does
not seem to change things.
----------------------------------------------------------------------
Comment By: Tim Peters (tim_one)
Date: 2002-11-04 16:34
Message:
Logged In: YES
user_id=31435
Couldn't reproduce under Win98SE and 2.2.2 even
boosting the size to a million. It *may* have to do with
console-size settings under cmd.exe.
Reduced the priority. "So don't do that" comes to mind
<wink>. Seriously, if the platform C fwrite chokes, it's not
Python's job to fix it -- it's Microsoft's. The interpreter isn't
dying here, it's raising a well-behaved exception, just
passing on the error the platform fwrite() raised. To me it's
the same as the system malloc complaining that you don't
have enough memory to satisfy a request: you've
exceeded a platform limitation. If this is important to you,
you can presumably fix it yourself by chunking up your
writes in a loop.
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=633480&group_id=5470