[Python-ideas] itertools.chunks()

[Mathias Panzenböck]

A function like this is useful, but I don't agree with the name. This
name implies to me that it actually yields chunks and not chunk sizes.
Maybe call it chunk_sizes? I don't know.

Also I find myself often writing helper functions like these:

def chunked(sequence,size):
	i = 0
	while True:
		j = i
		i += size
		chunk = sequence[j:i]
		if not chunk:
			return
		yield chunk

def chunked_stream(stream,size):
	while True:
		chunk = stream.read(size)
		if not chunk:
			return
		yield chunk

Maybe these functions should be in the stdlib? Too trivial?

On 04/06/2013 02:50 PM, Giampaolo Rodolà wrote:
> def chunks(total, step):
>      assert total >= step
>      while total > step:
>          yield step;
>          total -= step;
>      if total:
>          yield total
>
>>>> chunks(12, 4)
> [4, 4, 4]
>>>> chunks(13, 4)
> [4, 4, 4, 1]
>
>
> I'm not sure how appropriate "chunks" is as a name for such a function.
> Anyway, I wrote that because in a unit test I had to create a file of
> a precise size, like this:
>
> FILESIZE = (10 * 1024 * 1024) + 423  # 10MB and 423 bytes
> with open(TESTFN, 'wb') as f:
>       for csize in chunks(FILESIZE, 262144):
>           f.write(b'x' * csize)
>
> Now I wonder, would it make sense to have something like this into
> itertools module?
>
>
> --- Giampaolo
> https://code.google.com/p/pyftpdlib/
> https://code.google.com/p/psutil/
> https://code.google.com/p/pysendfile/