[Python-Dev] "".tokenize() ?

[Fredrik Lundh]

mal wrote:


> Gustavo Niemeyer submitted a patch which adds a tokenize like
> method to strings and Unicode:
>
> "one, two and three".tokenize([",", "and"])
> -> ["one", " two ", "three"]
>
> I like this method -- should I review the code and then check it in ?

-1.  method bloat.  not exactly something you do every day, and
when you do, it's a one-liner:

def tokenize(string, ignore):
    [word for word in re.findall("\w+", string) if not word in ignore]

> PS: Haven't gotten any response regarding the .decode() method yet...
> should I take this as "no objections" ?

-0.  method bloat.  we don't have asfloat methods on integers and
asint methods on strings either...

Cheers /F