[Python-bugs-list] [ python-Bugs-464405 ] freeze doesn't like DOS files on Linux

noreply@sourceforge.net noreply@sourceforge.net
Mon, 18 Nov 2002 00:29:52 -0800


Bugs item #464405, was opened at 2001-09-24 15:41
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=464405&group_id=5470

Category: Demos and Tools
Group: Python 2.3
Status: Closed
Resolution: Fixed
Priority: 5
Submitted By: Nobody/Anonymous (nobody)
Assigned to: Neal Norwitz (nnorwitz)
Summary: freeze doesn't like DOS files on Linux

Initial Comment:
I've got a perfectly running Python program on Linux, 
importing some modules that were apparently written 
on Windows, since the files are in DOS format 
(carriage return + line feed at the end of lines).

When I try to freeze this program, the module finder 
crashes on every module in DOS format claiming that 
there's a syntx error at the end of each line.

This is not really serious, but quite annoying...

Thanks.


----------------------------------------------------------------------

>Comment By: Thomas Heller (theller)
Date: 2002-11-18 09:29

Message:
Logged In: YES 
user_id=11105

Jack, this looks much nicer. Thanks.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2002-11-16 00:34

Message:
Logged In: YES 
user_id=45365

Just a quick note on testing for universal newlines: the idiom I use is "hasattr(sys.stdout, 'newlines')". Cleaner may be hasattr(file, 'newlines'), but this won't work as far back as 1.5.2, I think.

----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2002-11-14 20:43

Message:
Logged In: YES 
user_id=11105

Yes, I requested to include modulefinder in PEP 291.
There's no need to discuss this again, but I'll try to 
summarize the discussion we had in June:

I use modulefinder in py2exe, a freeze like tool. Py2exe 
(currently) supports all python versions from 1.5.2 up to 2.2. 
Modulefinder is distributed together with py2exe, since it is 
not easily accessible in the Tools directory.
So I requested it to be compatible with all these Python 
versions, but you doubted it would work because of byte-code 
changes. My experience is that it works well.
Another approach would have been to move modulefinder into 
the standard lib, but you didn't want this because it is only 
useful for the freeze utility.
So it finally ended up been listed in the pep, with me as the 
maintainer.

Neal: I'll think about the Python versions which I want to be 
supported, and contact you again (or update the pep myself).

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-11-14 20:28

Message:
Logged In: YES 
user_id=6380

I have no need for modulefinder so I'm not deciding
anything. It's up to Thomas.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-11-14 20:26

Message:
Logged In: YES 
user_id=6380

That still doesn't explain to me *why* you wanted
modulefinder.py listed. (Maybe my memory is bad.) If you
were the source of that request and you're now revising the
request, please go ahead.

----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2002-11-14 20:25

Message:
Logged In: YES 
user_id=11105

Neal: In this case (remove modulefinder from PEP 291) I won't 
insist on it. Let the BDFL decide, or has he already?

----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2002-11-14 20:18

Message:
Logged In: YES 
user_id=11105

Guido: We had some discussion about this when neal wrote 
the pep, and I suggested to list modulefinder there. Neal 
finally wrote the requirement into the PEP, and it was my 
impression that you accepted it, although you were not very 
happy with it.
I will insert a comment into modulefinder.py listing the 
requirement.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2002-11-14 20:11

Message:
Logged In: YES 
user_id=33168

Hmmm, I have an outstanding change to PEP 291 to remove
modulefinder.  Is there a requirement or does the PEP need
to be updated?

How's the following addition to the PEP.  I put it as the
last para in the Rationale section.

    Even if a package or module is listed in this PEP, authors
    should document a compatibility requirement in each file
    with a comment at the top of the file.


----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2002-11-14 20:01

Message:
Logged In: YES 
user_id=33168

I can update the PEP if you want, or you can do it yourself.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-11-14 19:56

Message:
Logged In: YES 
user_id=6380

I don't remenber whence the 1.5.2 requirement comes.

I propose that all such requirements should be documented in
a COMMENT at the TOP OF THE FILE, not just in PEP 291.


----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2002-11-14 19:53

Message:
Logged In: YES 
user_id=11105

I like the first way better.
Re the string methods: I'm getting tired fixing these, so I'd like 
to suggest to raise the PEP 291 version requirement for 
modulefinder to 2.0 (or 2.2?). What should I do? Contact the 
author (Neal, in this case)?

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-11-14 19:31

Message:
Logged In: YES 
user_id=6380

try:
 f = open(filename, "U")
except IOError:
 f = open(filename, "r")

You can also check for sys.version >= "2.3"

----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2002-11-14 19:28

Message:
Logged In: YES 
user_id=11105

Modulefinder is listed in PEP 0291 as being required to stay 
compatible with Python 1.5.2.
A string method has crept in in line 167 which is easy to 
remove.
Not so easy is this change for universal newline support.
Since I am also listed as the maintainer, I would like to fix 
this.
So, any idea how to detect if the current Python has universal 
newline support?

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2002-11-13 00:21

Message:
Logged In: YES 
user_id=33168

I don't know if there's an easy way to fix this for 2.2.x,
so I'm fixing this for 2.3 with universal newlines.

Checked in as:
 * Tools/freeze/modulefinder.py 1.22

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2002-10-18 22:56

Message:
Logged In: YES 
user_id=33168

I added "U" for the mode flag to open() for universal
newlines.  This fixed the problem.  I modified
Tools/freeze/modulefinder.py in run_script() and load_file()
about lines 120 and 127.  I didn't see any other open()s
which seemed to need "U".

Is that sufficient to close this bug?

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2001-09-27 20:02

Message:
Logged In: YES 
user_id=6380

I can reproduce this with Python 2.2 quite easily: on Linux,
in the Tools/freeze directory, use the ../scripts/lfcr.py
script to give hello.py CRLF line endings.  Then try to
freeze it.

The cause is the code in the tokenizer that parses from
strings, which doesn't like the \r\n line endings and
doesn't throw the \r out like the code that parses from a
file does.

I'm tempted to request a feature that the tokenizer should
be smarter when it finds a \r (e.g. just ignore it).

----------------------------------------------------------------------

Comment By: Martin v. L÷wis (loewis)
Date: 2001-09-26 21:51

Message:
Logged In: YES 
user_id=21627

What Python version? Could you please submit an example of 
one of the files it is complaining about? Since you cannot 
attach it to this report, it would be good if you could 
open a new report. When doing so, please identify 
yourself. 


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=464405&group_id=5470