Idea: __length_hint__ wrapper in itertools
Hi all, python now supports the __length_hint__ method but not for every use case that could potentially benefit from it, v.g.: 1) Some common builtins like map don't set the hint. 2) Generators. 3) Application specific hints that, by definition, can't be predicted by a general strategy. I know 1 and 2 were discussed at some length in the past, but still there is no agreement about what to do with them. In the meantime, and because of 3, what do you think about adding a simple iterator wrapper to itertools which would allow to provide the hint by the user without any significant performance lost (__next__ in the wrapper could be just the original __next__). AFAIK there is no such thing in the standard library or anywhere else and it seems pretty easy to implement, although maybe I'm completely wrong. Say: hinted_iter = it.length_hint(iter, hint) Cheers -- Carlos
On 20 February 2015 at 03:15, Carlos Pita <carlosjosepita@gmail.com> wrote:
Hi all,
python now supports the __length_hint__ method but not for every use case that could potentially benefit from it, v.g.:
1) Some common builtins like map don't set the hint. 2) Generators. 3) Application specific hints that, by definition, can't be predicted by a general strategy.
I know 1 and 2 were discussed at some length in the past, but still there is no agreement about what to do with them.
In the meantime, and because of 3, what do you think about adding a simple iterator wrapper to itertools which would allow to provide the hint by the user without any significant performance lost (__next__ in the wrapper could be just the original __next__). AFAIK there is no such thing in the standard library or anywhere else and it seems pretty easy to implement, although maybe I'm completely wrong.
Seamless wrappers for arbitrary types are actually fairly tricky to implement, but Graham Dumpleton's wrapt library hides most of those messy details: https://pypi.python.org/pypi/wrapt A pure iterator-only wrapper would be a fair bit easier to implement, but it's not clear it's worth it for cases like this where data processing and memory reallocation time dominates your runtime costs sufficiently for __length_hint__ to make a big difference. Writing your own wrapper using a generic wrapper library like wrapt may be a better choice (and has the advantage of working today rather than only in 3.5+) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
participants (2)
-
Carlos Pita
-
Nick Coghlan