python lexer

[Alessio Pace]

Alessio Pace wrote:

> Hi, I was wondering which is a good lexer written in Python, I heard about
> SPARK and PLY. I need only to tokenize texts, not to build trees on their
> structure.
> string.split() I guess it is not enough, because I can't use reg exp as
> token separators..How could I do otherwise?
> Thanks
> 

Ehm.... it seems like I solved...

import re

tokens = re.split("reg_exp", string_text)

:-)



-- 
bye
Alessio Pace