Dave, you're right. The "colors" are flags, and a ship from one country that emerged victorious in a sea battle with a ship from another country would have its own colors "flying." The colors of the losing ship would have been confiscated by the winner. Or so I understand.