In 1949 post-WWII Los Angeles, gangster Mickey Cohen (Sean Penn) has become the most powerful figure in the California criminal underworld, and intends to continue to expand his criminal enterprise across the Western United States, forcing out his rivals in Chicago. The police have not been able to stop Cohen’s ruthless rise, as Cohen has eliminated witnesses and bribed both the courts and the police, controlling both illegal and legal sides of L.A.

After saving a young girl from Cohen’s thugs, Sgt. John O’Mara (Josh Brolin) is personally selected by Chief Bill Parker (Nick Nolte) due to O’Mara’s special operations background and training at Camp X during World War II to wage guerilla warfare on Cohen, dismantling his crime empire and forcing him out of Los Angeles. With the help of his reluctant and pregnant wife, Connie (Mireille Enos), O’Mara recruits fellow officers in his squad. Initially he is joined by hard-headed detective Coleman Harris (Anthony Mackie), wire-tapper Conway Keeler (Giovanni Ribisi), and legendary gangster-killer Max Kennard (Robert Patrick). Kennard’s partner, Navidad Ramirez (Michael Peña), follows the squad and they reluctantly allow him to join. Sgt. Jerry Wooters (Ryan Gosling), O’Mara’s close friend, turns down the offer, as he is disillusioned from the war and his time on the police force.

Wooters has been keeping close to his old friend Jack Whalen (Sullivan Stapleton), who is his informant from within Cohen’s operations. After meeting for drinks, Wooters becomes infatuated with Cohen’s etiquette coach and girlfriend Grace Faraday (Emma Stone). Wooters and Grace begin a secret romantic relationship. After Cohen orders a hit on rival Jack Dragna (Jon Polito) that results in the death of Pete (Austin Adams), an innocent boy Wooters knew, Wooters becomes motivated to take down Cohen. He saves O’Mara’s squad after an unsuccessful hit on one of Cohen’s casinos, and later joins them.

With organization and better planning the unit strikes several successful blows at the heart of Cohen’s criminal organization. The media refers to them as “The Gangster Squad”. After wire-tapping Cohen’s house, Keeler deduces the central location of Cohen’s wire gambling business, the heart of his empire. The squad successfully burns down the business, but Cohen’s men inform him that they didn’t take any of his money. Cohen believes they are a group of honest cops, and believes someone ratted him out and tapped his house. As his men sweep the house for bugs, Grace runs away, fearing Cohen knows of her relationship with Wooters.

The Gangster Squad is lured into a trap in Los Angeles’ Chinatown by Cohen’s head hitman Wrevock (Troy Garity), who was also responsible for Pete’s death. Wooters interrupts the ambush just in time to save O’Mara, but Wrevock escapes. Karl Lennox (Holt McCallany), Cohen’s right-hand man, kills Keeler in the unit’s operation. Wooters asks Whalen to get Grace out of town, but Cohen reaches him first, looking for Grace, and kills him. O’Mara’s house is later hit by a drive-by, and Connie gives birth to their son under the stress. Chief Parker tells O’Mara, as a result of their failure, that their careers with the LAPD are over. O’Mara gets Connie out of town, but refuses to leave with the job unfinished. Grace tells Wooters she is willing to testify against Cohen for Whalen’s murder, prompting O’Mara, along with the remaining members of his unit, to go to the Park Plaza Hotel to arrest Cohen.

Cohen and his men engage in a firefight with the police unit, with Wooters being wounded in the process after killing Wrevock. Cohen and Lennox escape, but O’Mara pursues them, which results in their vehicle crashing into a fountain. Kennard gets shot by Lennox, but manages to kill him with help from Navidad, before dying from his wounds. O’Mara prepares to arrest Cohen, but Cohen tempts him into fighting each other in a bare-knuckle boxing match, while a crowd of onlookers and journalists gather. O’Mara finally beats Cohen and has him arrested, ending his reign as a crime boss and the spread of the mafia into the Los Angeles area. The Gangster Squad was never mentioned in taking down Cohen, their surviving members remaining a secret. Cohen is sentenced to 25 to life, and is welcomed violently by Whalen’s friends. Grace and Wooters continue their relationship, while Navidad and Harris become partners on the force. O’Mara quits to live a quiet life with his wife and son