This paper addresses the problem of fault detection in the context of a collection of agents performing a shared task and exchanging relative information over a communication network. We resort to techniques in the literature to construct a meaningful observable system and overcome the issue that the system of systems is not observable. A solution involving Set-Valued Observers (SVOs) is proposed to estimate the state in a distributed fashion and a proof of convergence of the estimates is given under mild assumptions. The performance of the proposed algorithm is assessed through simulations.