Good String Tokenizer

Duncan Booth duncan.booth at invalid.invalid
Tue Jul 24 21:28:20 CEST 2007


JamesHoward <James.w.Howard at gmail.com> wrote:

> I have searched the board
what board? I don't see any boards here.

> 
> And my splitting points are comma, and exclamation point then I would
> expect to get back.
> 
> ["Hello", ",", " World", "!", "  How are you?"]
> 
> Does anyone know of a tokenizer that will allow for this sort of use?
> 

>>> import re
>>> re.split("([!,])", "Hello, World!  How are you?")
['Hello', ',', ' World', '!', '  How are you?']



More information about the Python-list mailing list