The fruit to my mind is like a gravestone of somebody you loved who has since passed. Just because you loved that person for who they were the gravestone stands in for them/has emotional significance...you wouldn't want somebody pissing all over it. Apply that logic to the DF and I can see Luffy wanting to protect it/say fuck you to the person trying to mess with him through Ace.