While Trump has condemned the CFPB as a "disaster" that has "devastated" banks, he signaled Friday that Wells Fargo is still very much in the regulator's crosshairs. The president tweeted that fines and penalties "will be pursued and, if anything, substantially increased."

Reuters reported that Richard Cordray, who set off a CFPB power struggle when he stepped down last month, approved a possible Wells Fargo settlement that called for the bank to pay tens of millions of dollars.

Asked about Wells Fargo at his first press conference as acting director, Mulvaney said he remains "disturbed" by the scandals at the bank.

The CFPB did not respond to a request for comment. Wells Fargo declined to comment.

Fines and penalties against Wells Fargo Bank for their bad acts against their customers and others will not be dropped, as has incorrectly been reported, but will be pursued and, if anything, substantially increased. I will cut Regs but make penalties severe when caught cheating!

Wells Fargo has previously said the CFPB is investigating the alleged mortgage lending abuse. In October, Wells Fargo said some mortgage borrowers were inappropriately charged for missing a deadline to lock in promised interest rates, even though the delays were the bank's fault.

Wells Fargo promised refunds for any of the 110,000 customers who were wrongly charged so-called "mortgage rate lock extension fees" between September 2013 and this February.

In his Wells Fargo tweet on Friday, Trump said that while he wants to "cut" regulations, he will "make penalties severe when caught cheating!"

Wells Fargo's (WFC)stock was mostly unfazed by the Trump threats. After initially dipping slightly on the tweet, Wells Fargo stock quickly recovered and was trading slightly higher on the day.

"Wells Fargo will remain a political punching bag ... until the bank makes additional changes at the very top," Jaret Seiberg, financial services analyst at Cowen Research Group, wrote in a note on Friday.

Tim Sloan, a Wells Fargo veteran, was promoted to CEO last year to clean up the bank's scandals. Democratic Senator Elizabeth Warren has called on Wells Fargo to further clean house by firing Sloan.

Earlier this year, Trump took a softer tone towards banks.

Bank CEOs are "petrified of the regulators. They're petrified. They can't move," Trump said at an April town hall event with Citigroup(C) boss Michael Corbat and other business leaders. The president also promised to do a "very major haircut on Dodd Frank," the Wall Street reform law enacted in 2010.