[Python-Dev] RE: [Python-checkins] python/dist/src/Lib pyclbr.py,1.26,1.27
Thu, 22 Aug 2002 23:29:59 -0400
> Update of /cvsroot/python/python/dist/src/Lib
> In directory usw-pr-cvs1:/tmp/cvs-serv15469
> Modified Files:
> Log Message:
> Rewritten using the tokenize module, which gives us a real tokenizer
> rather than a number of approximating regular expressions.
> Alas, it is 3-4 times slower. Let that be a challenge for the
> tokenize module.
Was this just for purity, or did it fix a bug? The regexps there were close
to being heroically careful, and even so it was somtimes uncomfortably slow
using the class browser in IDLE (based on pyclbr), and even on a fast
machine. A factor of 3 or 4 might make that unbearable.
If it was for purity, note that tokenize is also based on mounds of regexp