The situation in Missouri in early summer in 1861 was chaotic. Gov. Claiborne Jackson and his prosecession legislature had been driven out of his state capital at Jefferson City and were trying to stay ahead of pursuing Union army, led chiefly by Brig. Gen. Nathaniel Lyon. Jackson withdrew to the southwest of the state, rallying secessionists as he went. At Lamar, he was joined by Brig. Gen. James S. Rains and about 4,000 ill-equipped militia. Together, they continued their retreat.

Meanwhile, Col. Franz Sigel commanded a army of about 1,100 into southwest Missouri in search of the Jackson and his troops.. By July 4, Sigel had reached Carthage camped for the night. When Jackson learned this, he immediately took command of the small Confederate army. Even though he held no military commision and there were several generals present, he announced they would attack.

On July 5, the Confederates approached Sigel in the morning with not much of a battle plan. Sigel himself was little better prepared. Jackson drew up his army on a ridge, awaiting a Union attack, and Sigel obliged, even though he knew he was outnumbered and did not have any cavalry to protect his flanks. He opened fire first with his artillery and sent enough metal into Jackson's line to shake it a bit but did not have enough antipersonnel charges to do real damage. Jackson, meanwhile, ordered 2,000 of his unarmed state-guard cavalry toward the shelter of woods on his right. Sigel misinterpreted this move as an attempt to turn his flank, decided that he had done enough fighting, and began to retreat.
After a deultory pursuit, halted frequently by able rearguard actions from Sigel's artillery, the Federals held Carthage that evening. In darkness, they continued the retreat to Sarcoxie.

Carthage was not much of a battle. The battle had little meaning, but the pro-Southern elements in Missouri, anxious for any good news, championed their first victory after the humiliating defeat at the Battle of Bonnville a few weeks earlier. This renewed hope that they could still lead Missouri out of the Union.

The Battle of Carthage was the first victory for the Confederate Army in Missouri.