On Wed, Oct 31, 2012 at 8:52 AM, Eli Bendersky
Also, I'd avoid the term "context-dependent". It sounds too similar to "context-sensitive" !
I use the two as rough synonyms. Shouldn't I?
"context sensitive" has a technical meaning, in the same way that "regular" or "recursively enumerable" does. In this particular case, the technical meaning doesn't align very well with the lay / intuitive meaning, but gets used in the same place as where one might use the phrase in the lay / intuitive sense -- if you'd said "context sensitive" I would've assumed you meant it in the technical sense. I guess I can't say that you should avoid the term unless I have a replacement. Maybe just using more words would help, like saying "then the actions of the tokenizer would depend on the context"?
Anyway, it looks like this isn't how the tokenizer treats braces/brackets (it ignores indent/dedent, but not newlines (I guess the grammar handles those)). What I meant to suggest was, treat "with ... :" similarly to how the OP suggests treating "with (...) :".
If this gets accepted, then, is there a reason to stop at "with"? Why not ignore newlines between "if" and its trailing ":" as well? [playing devil's advocate here]
I'd be very confused if newlines were acceptable inside `with` but not `if` and those. I'm not seeing a downside to changing them as well, except that it makes the workload (maybe significantly?) larger. I'm not sure if it's made that much larger. In the tokenizer it's easy, maybe in the grammar it's not so easy, and I don't know if this has to be in the grammar. The last time I ever tried editing python's parsing rules it ended very very poorly. -- Devin