Update: After spending the afternoon stripping everything out of the fragment, I confirmed that it's our setup that is causing the problem. Even so, it's still strange that setting the child view to VIEW.Gone works, but doing so to the swipe refresh layout doesn't hide the child.