Indentation oddness...

Consider the code: code = "def Foo():\n\n pass\n\n " This code is malformed in that the final indentation (2 spaces) does not agree with the previous indentation of the pass statement (4 spaces). Or maybe it's just fine if you take the blank lines should be ignored statement from the docs to be true. So let's look at different ways I can consume this code. If I use compile to compile this: compile(code, 'foo', 'single') I get an IndentationError: unindent does not match any outer indentation level But if I put this in a file: f= file('indenttest.py', 'w') f.write(code) f.close() import indenttest It imports just fine. If I run it through the tokenize module it also tokenizes just fine:
And if it fails anywhere it would seem tokenization is where it should fail - especially given that tokenize.py seems to report this error on other occasions. And stranger still if I add a new line then it will even compile fine: compile(code + '\n', 'foo', 'single') Which seems strange because in either case all of the trailing lines are blank lines and as such should basically be ignored according to the documentation. Is there some strange reason why compile rejects what everything else agrees is perfectly valid code?

On 2009-05-29 19:08, Dino Viehland wrote:
The 'single' mode, which is used for the REPL, is a bit different than 'exec', which is used for modules. This difference lets you insert "blank" lines of whitespace into a function definition without exiting the definition. Ending with a truly empty line does not cause the IndentationError, so the REPL can successfully compile the code, signaling that the user has finished typing the function. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco

I usually append some extra newlines before passing a string to compile(). That's the usual work-around. There's probably a subtle bug in the tokenizer when reading from a string -- if you find it, please upload a patch to the tracker! --Guido On Fri, May 29, 2009 at 5:52 PM, Dino Viehland <dinov@microsoft.com> wrote:
-- --Guido van Rossum (home page: http://www.python.org/~guido/)

Unfortunately my problem is the opposite one - trying to emulate what compile does for IronPython rather than just trying to make some code compile. So adding newlines doesn't help me. But this case isn't really that important - it was just a wacky corner case I ran into while trying to get other behavior right. I think I can safely ignore this one especially if it's just a bug.

On 2009-05-30 21:02, Greg Ewing wrote:
Not true. The REPL will keep reading lines until compile(code,'<string>','single') passes without a SyntaxError. You can put in blank lines when line continuation is implicit, like in the middle of a list. This is the reason that there is a 'single' mode in the first place, to determine when you've stopped typing. It's easier to add the grammar rule that a block does not end with a line of whitespace to the compiler than to implement all of the context-specific special cases for pure empty lines outside of the compiler. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco

On 2009-05-29 19:08, Dino Viehland wrote:
The 'single' mode, which is used for the REPL, is a bit different than 'exec', which is used for modules. This difference lets you insert "blank" lines of whitespace into a function definition without exiting the definition. Ending with a truly empty line does not cause the IndentationError, so the REPL can successfully compile the code, signaling that the user has finished typing the function. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco

I usually append some extra newlines before passing a string to compile(). That's the usual work-around. There's probably a subtle bug in the tokenizer when reading from a string -- if you find it, please upload a patch to the tracker! --Guido On Fri, May 29, 2009 at 5:52 PM, Dino Viehland <dinov@microsoft.com> wrote:
-- --Guido van Rossum (home page: http://www.python.org/~guido/)

Unfortunately my problem is the opposite one - trying to emulate what compile does for IronPython rather than just trying to make some code compile. So adding newlines doesn't help me. But this case isn't really that important - it was just a wacky corner case I ran into while trying to get other behavior right. I think I can safely ignore this one especially if it's just a bug.

On 2009-05-30 21:02, Greg Ewing wrote:
Not true. The REPL will keep reading lines until compile(code,'<string>','single') passes without a SyntaxError. You can put in blank lines when line continuation is implicit, like in the middle of a list. This is the reason that there is a 'single' mode in the first place, to determine when you've stopped typing. It's easier to add the grammar rule that a block does not end with a line of whitespace to the compiler than to implement all of the context-specific special cases for pure empty lines outside of the compiler. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
participants (4)
-
Dino Viehland
-
Greg Ewing
-
Guido van Rossum
-
Robert Kern