Temporarily allowing a physics object to pass through another

In the game there are ladders which a user can climb to get from one level to another. While the user is on the ladder they can pass through the floor/roof but when they're off they can't and walk on the floor.