I've been unable to solve this problem and this component has been causing problems in other situations. As such I decided to rewrite the component to use a little custom functionality as possible.

My solution has been to create a custom LinearLayout which contains a nested LinearLayout. The outer layout is responsible for drawing the background, and applies sufficient padding to allow space to draw the arrows. All children get passed to the inner layout. This solution isn't perfect as there is often excess padding and so wasted space, but it is sufficient for my purposes.

So, what I ended up with is copying LinearLayoutICS code (from here, hope it's the latest version) and some of the original LinearLayout code, to make something that does work. I hope it doesn't have any bugs.