The big U.S. banking firms are expected to present a plan that identifies where improvements to the pay structure are needed, and failure to submit or implement a satisfactory plan may lead to enforcement action, Tarullo said.

"Our supervisory approach to compensation practices at LCBOs" — large, complex banking organizations — "will include an extensive horizontal review process, in keeping with our increased emphasis on this approach for the supervision of the largest institutions," Tarullo said.

The Fed and Treasury Department unveiled last month a set of curbs and rules for executive compensation at banks, aimed at reducing the excessive risk-taking that contributed to the global financial crisis.