At the start of the Civil War, McPherson was stationed in San Francisco, California, but requested a transfer to the Corps of Engineers, rightly thinking that a transfer to the East would further his career. He departed California on August 1, 1861, and arrived soon after in New York. He requested a position on the staff of Maj. Gen.Henry W. Halleck, one of the senior Western commanders. He received this (while a captain in the Corps of Engineers), and was sent to St. Louis, Missouri.

Sherman planned to have the bulk of his forces feint toward Dalton, Georgia, while McPherson would bear the brunt of Confederate General Joseph E. Johnston's attack, and attempt to trap them. However, the Confederate forces eventually escaped, and Sherman blamed McPherson (for being "slow"), although it was mainly faulty planning on Sherman's part that led to the escape.[citation needed] McPherson's troops followed the Confederates "vigorously", and were resupplied at Kingston, Georgia. The troops drew near Pumpkinvine Creek, where they attacked and drove the Confederates from Dallas, Georgia, even before Sherman's order to do so. Johnston and Sherman maneuvered against each other, until the Union disaster at the Battle of Kennesaw Mountain. McPherson then tried a flanking maneuver at the Battle of Marietta, but that failed as well.

On July 17, Confederate PresidentJefferson Davis became frustrated with Johnston's strategy of maneuver and retreat, and replaced him with Lt. Gen.John Bell Hood. With the Union armies closing in on Atlanta, Hood first attacked George Henry Thomas's Army of the Cumberland north of the city on July 20, at Peachtree Creek, hoping to drive Thomas back before other forces could come to his aid. The attack failed. Then Hood's cavalry reported that the left flank of McPherson's Army of the Tennessee, east of Atlanta, was unprotected. Hood visualized a glorious replay of Jackson's famous flank attack at Chancellorsville and ordered a new attack. McPherson had advanced his troops into Decatur, Georgia, and from there, they moved onto high ground on Bald Hill overlooking Atlanta. Sherman believed that the Confederates had been defeated and were evacuating; however, McPherson rightly believed that they were moving to attack the Union left and rear. On July 22, while they were discussing this new development, however, four Confederate divisions under Lt. Gen. William J. Hardee flanked Union Maj. Gen. Grenville Dodge's XVI Corps. While McPherson was riding his horse toward his old XVII Corps, a line of Confederate skirmishers appeared, yelling "Halt!". McPherson raised his hand to his head as if to remove his hat, but suddenly wheeled his horse, attempting to escape. The Confederates opened fire and mortally wounded McPherson. This was early in the one-day Battle of Atlanta, part of the Atlanta Campaign that led to the surrender of Atlanta a month later.

His adversary, John Bell Hood, wrote,

I will record the death of my classmate and boyhood friend, General James B. McPherson, the announcement of which caused me sincere sorrow. Since we had graduated in 1853, and had each been ordered off on duty in different directions, it has not been our fortune to meet. Neither the years nor the difference of sentiment that had led us to range ourselves on opposite sides in the war had lessened my friendship; indeed the attachment formed in early youth was strengthened by my admiration and gratitude for his conduct toward our people in the vicinity of Vicksburg. His considerate and kind treatment of them stood in bright contrast to the course pursued by many Federal officers.[5]

A monument marking the death of McPherson was established at the location of his death in East Atlanta, at the intersection of McPherson Avenue and Monument Avenue.[7] McPherson Avenue in Atlanta was named for him. The spot is marked by a Union cannon once placed at Glenwood Road and Flat Shoals Road to protect the flank of the front line and return fire against the defensive positions built by Lemuel P. Grant.[8]

A distinctive engraved portrait of McPherson appeared on U.S. paper money in 1890 and 1891. The bills are called "treasury notes" or "coin notes" and are widely collected today because of their fine, detailed engraving. The $2 McPherson "fancyback" note of 1890, with an estimated 600-900 in existence relative to the 4.9 million printed, ranks as number 15 in the "100 Greatest American Currency Notes" compiled by Bowers and Sundman (2006).[9]

The James B. McPherson Elementary School in the Ravenswood area of Chicago, Illinois, was named for McPherson.[10]

In his home town of Clyde, Ohio, James B. McPherson Highway (US-20), McPherson Middle School and McPherson Cemetery are named for him.

^Woodworth, p. 167. Eicher, pp. 383–84, 477–78: John Sedgwick, a Union officer who was also killed in battle, was promoted to major general of volunteers on July 4, 1862, almost three months before McPherson, therefore technically had a higher rank. However, unlike McPherson, Sedgwick never commanded an army.