Just made an unpleasant discovery: if a Python source file has CR-LF line-endings, you can import it just fine under Unix. But attempting to 'py_compile.compile()' it fails with a SyntaxError at the first line-ending. Arrghh! This means that Distutils will either have to check/convert line-endings at build-time (hey, finally, a good excuse for the "build_py" command), or implicitly compile modules by importing them (instead of using 'py_compile.compile()'). Perhaps I should "build" modules by line-at-a-time copying -- currently it copies them in 16k chunks, which would make it hard to fix line endings. Hmmm. Greg
Greg> Arrghh! This means that Distutils will either have to Greg> check/convert line-endings at build-time (hey, finally, a good Greg> excuse for the "build_py" command), or implicitly compile modules Greg> by importing them (instead of using 'py_compile.compile()'). I don't think you can safely compile modules by importing them. You have no idea what the side effects of the import might be. How about fixing py_compile.compile() instead? -- Skip Montanaro, skip@mojam.com, http://www.mojam.com/, http://www.musi-cal.com/ "We have become ... the stewards of life's continuity on earth. We did not ask for this role... We may not be suited to it, but here we are." - Stephen Jay Gould
On 26 May 2000, Skip Montanaro said:
I don't think you can safely compile modules by importing them. You have no idea what the side effects of the import might be.
Yeah, that's my concern.
How about fixing py_compile.compile() instead?
Would be a good thing to do this for Python 1.6, but I can't go back and fix all the Python 1.5.2 installations out there. Does anyone know if any good reasons why 'import' and 'py_compile.compile()' are different? Or is it something easily fixable? Greg
On Fri, 26 May 2000, Greg Ward wrote:
On 26 May 2000, Skip Montanaro said:
I don't think you can safely compile modules by importing them. You have no idea what the side effects of the import might be.
Yeah, that's my concern.
I agree. You can't just import them.
How about fixing py_compile.compile() instead?
Would be a good thing to do this for Python 1.6, but I can't go back and fix all the Python 1.5.2 installations out there.
You and your 1.5 compatibility... :-)
Does anyone know if any good reasons why 'import' and 'py_compile.compile()' are different? Or is it something easily fixable?
I seem to recall needing to put an extra carriage return on the file, but that the Python parser was fine with the different newline concepts. Guido explained the difference once to me, but I don't recall offhand -- I'd have to crawl back thru the email. Just yell over the cube at him to find out. *ponder* Well, assuming that it is NOT okay with \r\n in there, then read the whole blob in and use string.replace() on it. Cheers, -g -- Greg Stein, http://www.lyra.org/
Greg> Well, assuming that it is NOT okay with \r\n in there, then read Greg> the whole blob in and use string.replace() on it. I thought of that too, but quickly dismissed it. You may have a CRLF pair embedded in a triple-quoted string. Those should be left untouched. -- Skip Montanaro, skip@mojam.com, http://www.mojam.com/, http://www.musi-cal.com/ "We have become ... the stewards of life's continuity on earth. We did not ask for this role... We may not be suited to it, but here we are." - Stephen Jay Gould
On Fri, 26 May 2000, Skip Montanaro wrote:
I thought of that too, but quickly dismissed it. You may have a CRLF pair embedded in a triple-quoted string. Those should be left untouched.
No, it would be OK to do the replacement; source files are supposed to be treated as text, meaning that lineends should be represented as \n. We're not talking about changing the values of the strings, which will still be treated as \n and that's what will be incorporated in the value of the string. This has no impact on the explicit inclusion of \r or \r\n in strings. -Fred -- Fred L. Drake, Jr. <fdrake at acm.org>
[GregS]
Well, assuming that it is NOT okay with \r\n in there, then read the whole blob in and use string.replace() on it.
[Skip Montanaro]
I thought of that too, but quickly dismissed it. You may have a CRLF pair embedded in a triple-quoted string. Those should be left untouched.
Why? When Python compiles a module "normally", line-ends get normalized, and the CRLF pairs on Windows vanish anyway. For example, here's cr.py: def f(): s = """a b c d """ for ch in s: print ord(ch), print f() import dis dis.dis(f) I'm running on Win98 as I type, and the source file has CRLF line ends. C:\Python16>python misc/cr.py 97 10 98 10 99 10 100 10 That line shows that only the LFs survived. The rest shows why: 0 SET_LINENO 1 3 SET_LINENO 2 6 LOAD_CONST 1 ('a\012b\012c\012d\012') 9 STORE_FAST 0 (s) etc That is, as far as the generated code is concerned, the CRs never existed. 60-years-of-computers-and-we-still-can't-agree-on-how-to-end-a-line-ly y'rs - tim
participants (5)
-
Fred L. Drake -
Greg Stein -
Greg Ward -
Skip Montanaro -
Tim Peters