[Python-Dev] [Python-ideas] itertools.chunks(iterable, size, fill=None)

[Stefan Behnel]

Hi Raymond,

Raymond Hettinger, 01.07.2012 09:07:
> On Jun 30, 2012, at 10:44 PM, Stefan Behnel wrote:
>>> Another addition could be a new subsection on grouping (chunking) that
>>> would discuss post-processing of grouper (as discussed above), as well as
>>> other recipes, including ones specific to strings and sequences. It would
>>> essentially be a short how-to. Call it 9.1.3 "Grouping, Blocking, or
>>> Chunking Sequences and Iterables". The synonyms will help external
>>> searching. A toc would let people who have found this doc know to look for
>>> this at the bottom.
>>
>> If it really is such an important use case for so many people, I agree that
>> it's worth special casing it in the docs. It's not a trivial algorithmic
>> step from a sequential iterable to a grouped iterable.
> 
> I'm not too keen on adding a section like this to the itertools docs.

I've only just seen that the recipes section is part of the same page since
the 2.6 documentation was sphinxified. I had remembered it being on a
separate page before. That resolves most of my original concerns. Sorry,
should have looked earlier.

To address the main problem of users not finding what they need, what about
simply extending the docstring of the grouper() function with a sentence
like this:

"This functionality is also called 'chunking' or 'blocking' and can be used
for load distribution and sharding."

That would make it easy for users to find what they are looking for when
they search the page for "chunk". I find that a much more common and less
ambiguous name than "grouping", which reminds me more of "group by".

It might be a good idea in general to add a short comment on a use case to
each recipe where it's not immediately obvious or where there is a use case
with a well-known name, simply to aid in text searches over the page.


> Instead, I would be open adding "further reading" section with external links 
> to interesting iterator writeups in blogs, cookbooks, stack overflow answers, wikis, etc.
> 
> If one of you wants to craft an elegant blog post on "Grouping, Blocking, or
> Chunking Sequences and Iterables", I would be happy to link to it.

That could be done in addition, but it bares the risk of bit rotting the
documentation by links dying, blogs moving or texts changing.

Stefan