2011-04-11

A long list, and you want to process its items n at a time; easy, but how to split that list in sublists of n elements (except the last one, of course)?

I looked a bit into the stdlib but it doesn't seem to exist anything I could use (oh, did I say I'm still on 2.4?) so I directed my research to Google, and found a nice recipe at ActiveState, but it has the problem it discards the last list, if it has less than n items.

Searching again, I got more lucky with this article: it's a generator of tuples from a list, splitting every n elements and optionally return the last semi-full tuple. I slightly modified it to obtain:

def group_iter(iterator, n=2):
""" Given an iterator, it returns sub-lists made of n items
(except the last that can have len < n)
inspired by http://countergram.com/python-group-iterator-list-function"""
accumulator = []
for item in iterator:
accumulator.append(item)
if len(accumulator) == n: # tested as fast as separate counter
yield accumulator
accumulator = [] # tested faster than accumulator[:] = []
# and tested as fast as re-using one list object
if len(accumulator) != 0:
yield accumulator

My favorite solution is zip(*[iter(input)]*N), where input is your input list and N is how many elements you want per sub-list. It's not exactly the same as your solution, since it drops dangling items instead of giving you back a short sublist. Replacing zip() with map(None gives you a solution that None-pads the last sublist if necessary, instead. However, this is "favorite" in a kind of perl golf way, not a use it in real software sort of way.