The trading boom that helped reshape global investment banks over the past decade is sputtering, raising fears that one of Wall Street's biggest profit engines is in peril.

Executives have warned that lackluster markets could lead to year-over-year declines in fixed-income, commodities and currency trading revenue when banks begin reporting fourth-quarter results next week. That would mark the fourth consecutive drop and the 11th in...