Decorator Pattern with Iterator

[Ian Kelly]

On Mon, Jun 11, 2012 at 1:51 AM, Tom Harris <celephicus at gmail.com> wrote:
> Greetings,
>
> I have a class that implements the iterator protocol, and tokenises a string
> into a series of tokens. As well as the token, it keeps track of some
> information such as line number, source file, etc.

So each processor needs to be able to access that information?  A
decorator pattern for the processors to propagate that information
down might look like this:

class TokenProcessor(object):
    def __init__(self, processor):
        self._processor = processor
    def __call__(self, tokens):
        self._tokens = tokens
        return self._processor(tokens)
    @property
    def line_number(self):
        return self._tokens.line_number
    @property
    def source_file(self):
        return self._tokens.source_file

@TokenProcessor
def processor(tokens):
    for token in tokens:
        line_number = tokens.line_number
        do_stuff(token, line_number)
        yield token

for token in processor_1(processor_2(tokeniser)):
    do_more_stuff(token)