The high frequency trading battle between exchanges and market makers is resulting in big losses not just for Wall Street, but, likely, for us too.

FORTUNE -- In life there are few coincidences, and this one probably isn't either: The day Knight Capital Group's computers nearly blew up the market and lost the firm $440 million in 45 minutes is the same day that the New York Stock Exchange (NYX) launched a new trading system that was, in part, meant to take business away from Knight (KCG).

For the past half decade or so, there has been a tug of war over who completes the buy and sell orders for stocks that average investors like you and I make. It used to happen in the pits of the NYSE. These days, almost none of the trades that folks like you and I make ever get to the exchange. Instead, they get cut off, diverted into the computer systems of Knight or its main competitors Citadel, Citigroup and UBS, which match those with the millions of other orders they collect.

And the pace at which these firms have been able to divert traffic from the NYSE has been accelerating. In 2009, about 15% of all trades took place away from the NYSE. Now about a third of all the trades in NYSE-listed shares happen elsewhere.
It's not clear why this battle over individual stock trades is so pitched. Knight pays brokers for its so-called order flow. And it guarantees that individuals get a slightly better price than what they would get at the exchange. Those stock trades get fed into Knight's computers, which use lightning fast trading algorithms to figure out how to make money off the orders the firm has just paid up for. This is, in part, the high frequency trading that you have heard about.

Some say that market makers provide a service. Others say Knight and others seek out the orders of individual investors because they view those orders as so-called dumb flow and easier to trade against. What is clear is that Knight and others have figured out how to make money off the stock trades of you and me in ways that we can't detect but we probably pay for somehow. Eric Scott Hunsader, who runs trading research firm Nanex, estimates market makers have been able to generate $5 billion in profits rapidly trading the orders of individual investors and others in the past seven years.
On Wednesday, the same day that Knight lost $440 million, the NYSE launched its own computer driven trading system, called the Retail Liquidity Program, that the exchange hopes will reclaim some of the trading volume it has lost to market makers. NYSE hopes RLP will create more competition among traders and brokers and market makers so that more of those orders get filled at better prices on the exchange. The new system also offers financial incentives for brokers to complete their orders on the exchange, similar to the payments long made by Knight and others that lured trades away from NYSE.

Knight says the computer problems it ran into had to do with NYSE's new trading system, but it didn't say what. Tellingly, all of the stocks that Knight's computers did bogus trades in were listed on the NYSE. It's likely that Knight tried to upgrade its own algorithm to allow its computers to do an end around the NYSE's new system. But it messed up somehow. Instead, Knight's computer system, launched on the same day as the NYSE's, went on a trading frenzy, buying and selling millions of shares on its own shortly after both systems were switched on when the market opened at 9:30 Wednesday morning.

Normally that shouldn't have produced any real losses. These weren't actual orders, so Knight's system should have just been buying and selling to itself. But that's not how the world of high frequency trading works. When other traders, i.e. computer systems, saw the spike in activity, they jumped in too.

Knight disabled the faulty algorithm by 10:15. But by then the damage was done. Knight was out $440 million. Dozens of stocks, including Warren Buffett's Berkshire Hathaway (BRKB), had gyrated up and down, and our faith in the market was shaken once again.

In theory, we should all benefit from this competition, being able to trade at cheaper and cheaper prices. But in practice the "price improvements" that Knight and now NYSE offer are fractions of a fraction of a penny. At best, what we are getting in return is a market that is less stable. At worst, we are getting a system that is picking our pockets.

If this isn't a clear case where we need regulators to step in, I don't know what is.

Firstly, here's BRKA chart. Notice the big jolt on 1st Aug?

What's most interesting were the comments made by Eric Scott Hunsader, who runs trading research firm Nanex . Here's Nanex posting on 1st Aug 2012.

1. EXC One second interval chart. Circles are trades, the blue coloring is the NYSE bid and ask which is mostly covered by gray lines that connect the trades.
If we zoom in and look at what happens under one second, then a clear pattern emerges. We think it's important to note that the SEC claimed there is no value to be gained from looking at data in time resolutions under a second "because it is just noise". We strongly disagree.

2. A 25 millisecond interval chart that zooms into a 27 second period of the chart above. Now the gray lines connecting trades are more clearly visible. NYSE's bid/ask is the blue shaded area (the bid price is the bottom of the shading, and the ask is the top).

The next 2 charts illustrate trade executions that ping-pong between hitting the ask and hitting the bid. As if someone is buying at the offer and then, almost immediately, selling at the bid, then buying at the offer, then selling at the bid and so forth. It turns out, the gray shading you see in charts above are the zig-zag lines connecting the alternating buy and sell executions. That's right, almost all these trades alternate between buying at the offer and selling at the bid, which means losing the difference in price. In the case of EXC, that means losing about 15 cents on every pair of trades. Do that 40 times a second, 2400 times a minute, and you now have a system that's very efficient at burning money.

3. Zooming in to a 1 millisecond interval chart, we can see one second of data which shows 39 trades.

4. A 25 millisecond interval chart of Nokia (NOK) showing the same pattern as above. Note: this chart shows bids and offers from NYSE as blue triangles instead of shading. There are so many trades that they appear as a coiling black mass. In Nokia, the trade rate was a steady 200 per second for many minutes.

We do hear some traders who trades the 5 mins and even the 1 min interval but with these machines, trades can be done 200 times per second for many minutes! These machines, they are trading the milliseconds time frame!

“On days like yesterday and today, we have a lot of work to do. We have to make sure we work with our counterparties, our clients to get the answers they want and we are pursuing that as we speak and we are also exploring other alternatives such as strategic investments or investors and other financing alternatives. We have work to do and we’re doing it now.”

On what he means by strategic investments:
“Obviously, if we were having specific conversations I wouldn’t be able to tell you that… We know we have some work ahead of us and we are diligently pursuing that, and we’re staying in close contact with our clients and counterparties as we get through the situation.”

On whether these other financing alternatives will keep Knight Capital in business:
“That’s what our goal is, and we’re working hard to accomplish that. We have all hands on deck and we understand what the issues are, we are talking to a lot of capable people, people who are in touch with situations like this. So, we’re working hard and we have all hands moving forward to address this and resolve this.”

On whether any credit lines have been pulled:
“As you might imagine, during the day to day activity, it’s hard to comment. Our general counsel would prefer I don’t go too deeply into what’s going on with the day to day action. So it’s one of those things I don’t think I can really comment on right now.”

On whether there are any firms with which Knight Capital had been trading that are not right now:
“All our clients respect what we did yesterday. They were very happy with us because once we realized we had a problem, we alerted them and got them out of the way. Our clients have a great deal of confidence in us, but much like the questions that were just asked about day to day activity, I can assure you our general counsel would prefer that I not get into any specific details.”

On what happened yesterday:
“We put in a new bit of software the night before because we were getting ready to trade the NYSE’s RLP program. This has nothing to do with the stock exchange. It had to do with our readiness to trade it. Unfortunately, the software had a fairly major bug in it. It sent into the market a ton of orders, all erroneous, so we ended up with a large error position which we had to sort through the balance of the day. It was a software bug, except it happened to be a very large software bug, as soon as we realized what we had we got it out of the code and it is gone now. The code has been restored. We feel very confident in the current operating environment we’ve reestablished.”

On why it took so long to not only fix the problem, but to identify and take responsibility for it:
“We were talking to our clients right away and got they got out of the way, which is great because nobody else except for us was wounded by this activity. So, we don’t think we actually acted in a slow fashion at all because our primary focus was on us alerting our clients and keeping them out of harm’s way.”

On how Knight Capital’s situation compares to the Facebook IPO:
“I know we’re in a similar situation, but I would argue that it’s apples and baseballs when you compare them because as I said at the time, technology breaks. It ain’t good. We don’t look forward to it, but technology breaks. What happens next is how you escalate it. We’re very proud of the fact that we escalated to it directly to our clients and got them out of harm’s way. There are two things over here that we take great pride in which is our client focus and our culture which is so involved with compliance. We have a culture of compliance and client focus and we asserted both of those things yesterday.”

On what Knight Capital’s situation says about the integrity of the U.S. equity markets:
“If you get involved in the day to take minutia and not invest for the long term, this will give you a headache, no question about it. We’re not happy that we added to that pile around days there were difficult for the individual investor. But it is also just affecting us. It does not affect the individual investor and we did not harm any investors and got them out of the web.”

On whether he’d buy shares of a company that within a day or two could be wiped out, like Knight Capital:
“Of course not, this was an anomaly. You cannot immunize people from making mistakes. You cannot keep people from doing stupid things whether it is writing some imperfect code or buying the wrong stock at the wrong time. That is what happens when you have a culture of risk. If you do your homework and do the right job, you’ll be rewarded over time. If you stay away from the day-to-day minutia and look for the long term, as an equity investor, it will work out.”

On whether Knight is still steering clients to other places:
“As you can see behind me, we are open for business today. We are reasonably busy and keeping active. By the end of the day yesterday, many clients had started to route back. We’re open for business. We got rid of the bad trade and that freed up a lot of capital. We actually have excess capital right now. We have worked to get ourselves in good shape and communicated that to our clients. Clients are making their decisions with full transparency as to how to interact with us.”

On whether voice trading should be brought back:
“This had very little to do with voice trading versus electronic trading. This software problem was an infrastructure problem. It had nothing to do with our quantitative models and nothing to do with our market-making models. This was something that was separate and distinct from trading. It was more of a networking problem as opposed to using quantitative tools to trade.”

On Knight Capital’s search for more capital:
“It is hard to give anybody a timeline when you are working through it, but we are focused and doing all the work I believe need to be done as we speak. We’re kind of pushing ahead and we hope to have the news and then we will share it immediately with our shareholders, our regulators and stakeholders.”

On what kind of investor Knight Capital is looking for:
“It is kind of hard to isolate what kind of structure we prefer. We’re keeping an open mind and talking to some counterparties and relationships as we speak.”

On how to restore confidence in the company’s employees:
“We tried very hard over the last decade to build a culture here that people admire and enjoy coming to work and people are supportive and there’s teamwork. During the trials and tribulations of the last 48 hours, this team has been nothing short of remarkable and the people at Knight Capital group are a team I am incredibly proud to be a part of.” (transcript taken from: http://wallstreetpit.com/94521-knight-capital-kcg-ceo-says-all-hands-on-deck )