[Python-bugs-list] [ python-Bugs-440707 ] doctest crashes on cross-platform .pyc

[noreply@sourceforge.net]

Bugs item #440707, was opened at 2001-07-12 08:51
You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=105470&aid=440707&group_id=5470

Category: Windows
Group: Irreproducible
Status: Closed
Resolution: Works For Me
Priority: 5
Submitted By: David Goodger (goodger)
Assigned to: Nobody/Anonymous (nobody)
Summary: doctest crashes on cross-platform .pyc

Initial Comment:
doctest.py raises an exception when run on true cross-
platform .pyc files (i.e., MacOS .py compiled with 
compileall.compile_dir on Windows). The problem is, 
_extract_examples splits up __doc__ strings with:

    lines = s.split("\n")

MacOS-compiled-on-Windows .pyc __doc__ strings contain 
\r, not \n. The fix is to replace the line above (line 
5 of _extract_examples) with:

    lines = s.splitlines() + ['']

The reason for the "+ ['']" is that the behavior of 
s.splitlines() is subtly different from s.split("\n"): 
splitlines() doesn't return a final null string at the 
end of, say, "one\ntwo\n", whereas split("\n") does. 
(Is this difference of behavior a bug? If not, should 
it be described in the docs?)

The "+ ['']" also circumvents errors when the 
docstring doesn't end with a newline, ie:

    """
    >>> print 'hi'
    hi"""


----------------------------------------------------------------------

>Comment By: David Goodger (goodger)
Date: 2001-07-13 10:41

Message:
Logged In: YES 
user_id=7733

GvR> Curious: how do you transfer files between Mac & Win?

The old-fashioned way: by floppy disk.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2001-07-13 09:29

Message:
Logged In: YES 
user_id=6380

OK, closing this bug.

Curious: how do you transfer files between Mac & Win?


----------------------------------------------------------------------

Comment By: David Goodger (goodger)
Date: 2001-07-13 09:26

Message:
Logged In: YES 
user_id=7733

I'm using Python 2.1 final on Win NT 4.0. Try as I might, I 
can't reproduce yesterday's behavior. (Although you may 
want to mention in the docs that "Docstrings must end with 
a newline.") Thanks for your attention, and apologies for 
time wasted. I was either totally confused, the victim of 
an environment glitch, or a target of the PSU's time ma

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2001-07-13 02:30

Message:
Logged In: YES 
user_id=31435

Which version of Python are you using?

Guido is baffled (as am I) because a modern py_compile.py 
(which produces the .pyc files) contains these lines:

f = open(file)
codestring = f.read()
codestring = codestring.replace("\r\n","\n")
codestring = codestring.replace("\r","\n")

That is, there's simply no way a carriage return can 
survive.  The checkin comment that introduced this hack 
(it's "a hack" because Python doesn't *generally* try to 
support non-native line conventions -- you're skating on 
the edge here at best) reads:

revision 1.16
date: 2000/09/15 06:57:26;  author: loewis;  state: Exp;  
lines: +5 -0
Support \r in source files. Closes bug #101425.


----------------------------------------------------------------------

Comment By: David Goodger (goodger)
Date: 2001-07-12 21:29

Message:
Logged In: YES 
user_id=7733

I'm doing MacOS-based cross-platform development. I 
like to keep the working files with Mac line-endings. So I 
run this "test_all" script on Windows:

    python -c 'import compileall; 
compileall.compile_dir(".")'
    python test_utils.pyc
    ...

If test_utils.py has \r line endings, docstrings in 
test_utils.pyc also end up with \r line endings. (The 
.pyc's are generated on Windows, from Mac-format .py 
files.) This breaks doctest. Perhaps compileall.py is the 
module with the bug? Or is my approach flawed or too 
rare to merit attention? :-)

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2001-07-12 13:54

Message:
Logged In: YES 
user_id=6380

I experimented (on Windows) and looked at the source some
more, and I cannot see how a \r could ever have ended up in
the .pyc file for a newline, no matter how or where the .pyc
file was written.

The only other theory that might explain what you see is
that when you transfered the .pyc file from Windows to Mac,
\n got translated into \r.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2001-07-12 13:27

Message:
Logged In: YES 
user_id=6380

I'll have to look into this more, but I strongly recommend
against fixing doctest in the way you suggest.  The bug must
somehow be in the portability of .pyc files.  I'm surprised
that string literals compiled on Windows contain \r in the
.pyc file.  Are you *sure* this is what causes your problem?

----------------------------------------------------------------------

You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=105470&aid=440707&group_id=5470