Quantitative easing—the Federal Reserve‘s program of buying long term government and mortgage debt known as QE—is one of the more controversial policies practiced today. While there is evidence that it has successfully lowered interest rates, and therefore put more money in the pockets of creditworthy businesses and Americans, opponents of the policy have argued that the risks associated with the policy far outweigh the benefits

Andrew Huszar, a senior fellow at Rutgers Business School and former manager of the Fed’s $1.25 trillion agency mortgage-backed security purchase program, has now thrown his hat in the ring with those critics, penning an article in the Wall Street Journal in which he apologized for his role in the policy and called for its reversal.

TIME spoke with Huszar and asked him to explain and defend this point of view. The interview has been edited for length and clarity.

TIME: Why did you write the op-ed?Huszar: The reason I wrote this piece is because I believe America had a significant wake up call with the financial crisis, yet five years later the country’s economy looks eerily similar to the way it did then. My belief is that the Fed is a significant reason why we haven’t reformed, and I wanted to try to start a conversation.

TIME: You argue that QE helps Wall Street banks but not the real economy. How does QE help Wall Street banks?Huszar: QE had two goals, but one of them was originally highlighted as the primary goal by Fed Chairman Ben Bernanke: to make credit more accessible to more Americans. QE aimed to achieve this through lowering the wholesale cost for banks to make loans, and we were actually successful at doing this. For example, my program – which was buying mortgage backed securities – drove down the cost for banks to make mortgage loans. But the banks weren’t fully passing on the benefits to their customers — they were pocketing a lot of the extra profit.

In addition, though we lowered the cost for banks to make mortgages, the banks didn’t actually start making more mortgages. If you look at the first day of trading in my program which was in January of 2009, and you look at the last day of trading fifteen months later, the overall amount of U.S. mortgage lending had actually decreased despite the fact that the Fed had spent $1.25 trillion trying to stimulate mortgage lending. In fact, if you look as late as 2012, U.S. mortgage lending was at a fifteen year low. The banks weren’t making more loans. Instead, they were often investing their extra money into securities to take advantage of the rising tide of asset prices in the market.

The third point to make is that the Fed was actually buying and continues to buy all of these mortgage bonds through the network of primary dealers – banks. The commissions that the banks were generating off these purchases were also significant.

TIME: But there are surely benefits to the real economy, and not just Wall Street, from QE. For instance, many regular people were able to refinance their mortgages and spend less each month on housing. Now that money can be spent elsewhere, correct?Huszar: Of course. You can’t spend $4 trillion in the financial markets and not see benefits. First of all I should say that while I feel the Fed has lost perspective, I believe the people working there are smart, well-intentioned people. There is a public policy argument to be made for this program. I just don’t believe it flies, and whatever benefit you get from this program is being outweighed dramatically by the negative distortions that QE is creating. Pushing down interest rates to help people refinance is going to help. Unfortunately, when banks aren’t lending, there is no transmission mechanism to help most people. Another argument for the program is that by raising asset prices the Fed is generating wealth and stimulating economic activity. But, for the amount of money that has been spent and the amount of risk the Fed is taking for itself and for the U.S. economy, it’s just not worth it.

TIME: Let’s talk about the risks because if you admit that there are some benefits to program, however small they are, it would make sense to continue it unless there are negative side effects that outweigh the benefits. What are those negative effects?Huszar: I think one thing we’re seeing is that even the gains that are being made in the markets may be unsustainable; that the Fed is creating new bubbles. You saw Larry Fink of BlackRock making that argument recently, and there are many people who are now expressing concern about this. Also, QE has required the Fed to expand its interaction with the US economy dramatically. From a monetary policy perspective, the Fed used to only determine short term interest rates. Now it’s experimenting and expanding the way in which it interacts with the US economy. The potential for the Fed to distort the financial markets and the overall U.S. economy – the potential to ultimately get things wrong – is unprecedented and immense.

For example, there’s the question of how the Fed unwinds its unprecedented operational expansion. This is an unwinding that will have to be invented on the fly, and it could have huge downside risks for the US economy. Note that even the Fed’s suggestion of a minor taper this summer led to the beginning of a substantial stock market sell off. What if the Fed ever actually has to sell off bonds? You can imagine far greater risk and volatility in the markets. The longer the Fed waits, the greater the risk gets.

Also, the Federal Reserve now owns more or less 10% of the overall U.S. housing market, and the equivalent of 30% of the U.S. federal government debt. There may be some minor benefit to QE, but you can’t look at those figures and not think that such heavy government involvement in the economy doesn’t lead to serious distortions.

TIME: One of the arguments you’ve made is that the Fed, through pumping up the stock market, is masking the problems in the economy and helping Congress avoid taking steps that would help the economy on a fundamental level. But isn’t this backwards? Shouldn’t an independent Fed behave the way it sees best, and then we can criticize and hold Congress accountable for its mistakes?Huszar: Conceptually, I’d like to agree with you. Unfortunately, Congress only seems to be able to act during crises. And I’m not suggesting the Fed shouldn’t have acted during the original financial crisis, but I do think that it went way too far. After the Fed saw that QE didn’t work as it had intended, it should have stopped what it was doing. Probably after QE1 would have been the right time. Perhaps there would have been more volatility in the market afterwards, but that could have put pressure on Congress to pursue more fundamental change in the U.S. banking sector and to try to enhance the overall conditions for U.S. economic growth like taking steps to improve competitiveness, infrastructure, education, among other things.

TIME: If the Fed decides tomorrow to begin to wind down this program, wouldn’t the effect be higher interest rates and lower asset values? How will this help the economy?Huszar: I agree with you. We could have some pain associated with Fed tapering. But each day that QE continues, that potential pain increases because we’re distorting markets even more. We need to get to a point where we can get the size and the role of the Fed and the size and role of Wall Street back to where they belong. We have a structurally unsound economy where the conditions for growth are diminishing. If you look at global competitiveness of the U.S. economy, five years ago we were first in the world, today we’re seventh. If you look at our education system, forty years ago we were first in the world in college graduation rates, and today we’re fourteenth. You have a real decay in the conditions for growth in this country. There’s a question as to how much the government should be trying to stimulate economic activity and how much the government should be laying the groundwork for that activity to happen on its own. I worked in the government for nine years, and I believe it has an important role to play, but I also believe that the overall perspective is now entirely off.