For a second time this year, a federal appeals court rejected efforts by investors with convicted Ponzi scheme operator Bernard Madoff to sue the U.S. Securities and Exchange Commission for not rooting out his fraud.

On Wednesday, the U.S. Second Circuit Court of Appeals in New York rejected a bid by former Madoff investors to sue the SEC for negligence despite a finding by the regulator's inspector general that the agency missed...