That means that Calibre can't find a point at which to perform the split into smaller files. It tries to do this at "natural" split points like chapter breaks and such, as far as I know.
Increasing that size may horribly break stuff on readers especially with ADE, like the Sony and Bookeen devices- they are not designed to load chunks larger than 260 kB into memory, so most often, they just hang. You may have to reset that setting to its standard.

As for how to solve that problem, I'm not sure. Does the conversion fail entirely or is this just printed as an error, but an ePub is produced anyway?