My first Python program -- a lexer

Thomas Mlynarczyk thomas at
Mon Nov 10 14:06:45 CET 2008

Robert Lehmann schrieb:

> You don't have to introduce a `next` method to your Lexer class. You 
> could just transform your `tokenize` method into a generator by replacing 
> ``self.result.append`` with `yield`. It gives you the just in time part 
> for free while not picking your algorithm into tiny unrelated pieces.

> Python generators recently (2.5) grew a `send` method. You could use 
> `next` for unconditional tokenization and ``mytokenizer.send("expected 
> token")`` whenever you expect a special token.

 > See for details.

I will try this. Thank you for the suggestion.


Ce n'est pas parce qu'ils sont nombreux à avoir tort qu'ils ont raison!

More information about the Python-list mailing list