Can someone please explain how it is possible to have a heterozygous knock out mouse? If a gene is "knocked-out," how can the mouse be heterozygous for the gene? Will the mouse still produce express this "knocked-out" gene?

If one parent is a wild-type mouse and the other is a homozygous knockout mouse, their offspring will be heterozygous at the knockout gene. The mouse will likely produce the protein from the wild-type copy of the gene, but depending on how the gene is regulated it is likely that expression of the protein will be below wild-type levels.

Does this imply that all the somatic cells in the mice contain 1 copy of the functional gene and 1 copy of the non-functional gene? Would this then mean that each cell is expressing the protein to some extent?

What confuses me is that if this is the case, then what's the difference between knock-out and knock-down mice?

"Does this imply that all the somatic cells in the mice contain 1 copy of the functional gene and 1 copy of the non-functional gene? "

Yes.

"Would this then mean that each cell is expressing the protein to some extent?"

Likely yes, though this can be complicated by the biology, especially if the gene is on the X sex chromosome (where chromosome silencing can complicate expression).

"What confuses me is that if this is the case, then what's the difference between knock-out and knock-down mice?"

The difference is whether the homozygote is expressing some of the gene product. If there is some of the protein made, the mouse is a knock-down and not a knock-out.

Watch out for the other meaning of knockdown: in addition to the meaning of having a genetically altered promoter to decrease expression of a protein, it is also used to describe inhibition at the RNA level using antisense oligos or siRNA.