[Python-Dev] Idea: __length_hint__ wrapper in itertools

[Nick Coghlan]

On 20 February 2015 at 03:15, Carlos Pita <carlosjosepita at gmail.com> wrote:
> Hi all,
>
> python now supports the __length_hint__ method but not for every use
> case that could potentially benefit from it, v.g.:
>
> 1) Some common builtins like map don't set the hint.
> 2) Generators.
> 3) Application specific hints that, by definition, can't be predicted
> by a general strategy.
>
> I know 1 and 2 were discussed at some length in the past, but still
> there is no agreement about what to do with them.
>
> In the meantime, and because of 3, what do you think about adding a
> simple iterator wrapper to itertools which would allow to provide the
> hint by the user without any significant performance lost (__next__ in
> the wrapper could be just the original __next__). AFAIK there is no
> such thing in the standard library or anywhere else and it seems
> pretty easy to implement, although maybe I'm completely wrong.

Seamless wrappers for arbitrary types are actually fairly tricky to
implement, but Graham Dumpleton's wrapt library hides most of those
messy details: https://pypi.python.org/pypi/wrapt

A pure iterator-only wrapper would be a fair bit easier to implement,
but it's not clear it's worth it for cases like this where data
processing and memory reallocation time dominates your runtime costs
sufficiently for __length_hint__ to make a big difference. Writing
your own wrapper using a generic wrapper library like wrapt may be a
better choice (and has the advantage of working today rather than only
in 3.5+)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia