It appears from discussions held with practitioners on forums and conferences that hedging products with CDSs is often considered to be an easy task. For instance, a bond-CDS position is frequently seen to be a perfect hedge as is the index CDS and its corresponding single-name counterparts. On the other hand, theoreticians claim that a perfect hedge is impossible to achieve with credit derivatives: currently it seems there is no model from which a perfect hedge of credit derivatives can be set up. So, who's right? This paper aims to show that, surprisingly, both are right, in some sense. The misunderstanding comes from a misspecification of the “hedge” term, together with a confusion between hedging and arbitraging. We focus on Credit Default Swaps and explain why, among other things, the “Bond-CDS on the Bond” position does not result in a perfect hedge (except in a very specific case, which is not market standard). By contrast, partial hedge (that is, hedging some parts of the risk) can be obtained and, of course, arbitrage opportunities could be found. We then explain, adopting a philosophical point of view, why it is so problematic to set up a credit portfolio being perfectly hedged (even in a first-order approximation sense). It appears that the nature of the derivative's underlying prevents us reaching this with the currently available financial instruments.