Just sneaking it without asking? Yep, I think it's wrong. I used to take my halloween candy to school, the whole bag because I knew if i left it home, my dad would steal some - he worked nights. If the child identifies the candy as thiers and you take it, it becomes a breech of trust.

In our house, we mix DD1's trick or treat candy with the leftover candy from shelling out and it becomes family candy and we all share. DD2 is obviously too young to trick or treat.