Yeah, but NonCommutativeMultiply[a] and NonCommutativeMultiply[] do not simplify like you want them to. I'm not sure how Plus and Times do it... they might use a slightly nonstandard evaluation order - maybe you could customize the evaluation of your p expressions.
The only (simple) automated solution I can think of at the moment is to use
$Post = (# /. {p[a_] :> a, p[] -> 1} &); (* order is important *)
When I implemented some fermionic algebra (arXiv:1102.3043), I actually included these types of rules in my NonCommutativeMultiply sorting algorithm. It's not perfect (like a lot of my code), but since I always needed to canonicalize my expressions and this could be considered part of the canonicalization procedure, it seemed like an acceptable compromise.