how can this iterator be optimized?

[josh logan]

On Feb 11, 8:22 pm, Basilisk96 <basilis... at gmail.com> wrote:
> Hello all,
>
> I have the following function that uses an intermediate iterator
> "rawPairs":
>
> def MakePairs(path):
>     import os
>     import operator
>     join = os.path.join
>     rawPairs = (
>         (join(path, s), func(s))
>         for s in os.listdir(path)
>         if func(s) is not None and func(s).endswith("some criterion")
>     )
>     #Use the second item in the pair as the sort criterion
>     result = sorted(rawPairs, key=operator.itemgetter(1))
>     return result
>
> where "func" is a single-argument function that returns either a
> string or None, but is an expensive call.
> I am pretty sure that the sorted() construct cannot be improved much
> further, but...
> ...does anyone have ideas on improving the "rawPairs" iterator so that
> it calls "func(s)" only once per iteration?  Perhaps a lambda
> construct, but I am not sure how to go about it...?
>
> Cheers,
> Basilisk96

Hi,
Try something like this:

import os
from os.path import join
from itertools import ifilter #assuming 2.5 and earlier, for 3.0 just
use the filter builtin
from operator import itemgetter

def isvalid(pair):
    return (s[1] is not None) and s[1].endswith('some criteria')

def makepairs(path):
    pair_iter = ((join(path,s), func(s)) for s in os.listdir(path))
    pair_iter = ifilter(isvalid, pair_iter)
    result = sorted(pair_iter, key=itemgetter(1))

    return result