Check Size of Object in memcached

I've been working with a data set where a single object won't fit into memcached's 1 Mb slab limit. The two functions have been useful to me for debugging the size of the data structure once pickled, and if said pickled data structure is greater than 1 Mb.

These functions assume CACHE_BACKEND is memcached, obviously.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

importmemcachetry:fromcStringIOimportStringIOexceptImportError:fromStringIOimportStringIOtry:importcPickleaspickleexceptImportError:importpickledefdata_size(data):"""Returns the size in bytes of the data passed in."""fp=StringIO()pickler=pickle.Pickler(fp,protocol=0)pickler.dump(data)val=fp.getvalue()returnlen(val)defdata_too_large(data):"""Returns True if the data passed in is too large for memcached."""returndata_size(data)>=memcache.SERVER_MAX_VALUE_LENGTH