Mysterious and possibly nefarious trading algorithms are operating every minute of every day in the nation's stock exchanges.

What they do doesn't show up in Google Finance, let alone in the pages of the Wall Street Journal. No one really knows how they operate or why. But over the past few weeks, Nanex, a data services firm has dragged some of the odder algorithm specimens into the light.

The trading bots visualized in the stock charts in this story aren't doing anything that could be construed to help the market. Unknown entities for unknown reasons are sending thousands of orders a second through the electronic stock exchanges with no intent to actually trade. Often, the buy or sell prices that they are offering are so far from the market price that there's no way they'd ever be part of a trade. The bots sketch out odd patterns with their orders, leaving patterns in the data that are largely invisible to market participants.

In fact, it's hard to figure out exactly what they're up to or gauge their impact. Are they doing something illicit? If so, what? Or do the patterns emerge spontaneously, a kind of mechanical accident? If so, why? No matter what the answers to these questions turn out to be, we're witnessing a market phenomenon that is not easily explained. And it's really bizarre.

It's thanks to Nanex, the data services firm, that we know what their handiwork looks like at all. In the aftermath of the May 6 "flash crash,"
which saw the Dow plunge nearly 1,000 points in just a few minutes, the company spent weeks digging into their market recordings, replaying the day's trades and trying to understand what happened. Most stock charts show, at best, detail down to the one-minute scale, but Nanex's data shows much finer slices of time. The company's software engineer Jeffrey Donovan stared and stared at the data. He began to think that he could see odd patterns emerge from the numbers. He had a hunch that if he plotted the action around a stock sequentially at the millisecond range, he'd find something. When he tried it, he was blown away by the pattern. He called it "The Knife." This is what he saw:

"When I pulled up that first chart, we saw 'the knife,' we said, that's certainly algorithmic and that is weird. We continued to refine our software, honing the algorithms we use to find this stuff," Donovan told me. Now that he knows where and how to look, he could spend all day for weeks just picking out these patterns in the market data. The examples that he posts online are just the ones that look the most interesting, but at any given moment, some kind of bot is making moves like this in the stock exchange.

"We probably get 10 stocks in any 10 minutes where we see something like this," Donovan said. "It's happening all the time."

These odd bots don't really make sense within the normal parameters of the high-frequency trading business. High-frequency traders do employ algorithms to look for patterns in the market
and exploit them, but their goal is making winning trades, not simply sending quotes into the financial ether.

Here's the way a stock trade is supposed to work: a buyer says they'll pay some amount for 100 shares of a company, a seller makes an ask for slightly more money, and the two of them usually meet in the middle. Perhaps a middle man (no joke intended) helps match buyer and seller and takes a cut. That's the role that a lot of high-frequency traders play: they help make markets work. Regulatory changes over the past several years have extended their usefulness and provided a nice business model for those that can move quickly to provide options for buyers and sellers.

"Under the maker-taker model, market participants
that offer to provide, or make, liquidity by posting an order to buy or
sell a certain number of shares at a particular price receive a rebate," explained Michael Peltz in a June feature for Institutional Investor.
"Those that execute against that order -- that is, take the liquidity --
have to pay a fee. Exchanges earn the difference between the rebate they
pay and the fee they charge. The SEC limits taker fees to 0.30 cents a
share; rebates tend to be lower for economic reasons, but for high
frequency firms trading millions of shares a day, they can make for a
pretty good living."

In a sense, they take nickel-and-diming down an order of magnitude or two. The advantage is that their trades are low-risk: they rarely hold positions for very long and any individual stock, future, or currency can't really sink the boat. High-frequency traders have become a target for all kinds of people, but most of them appear to make their money being a little faster and little smarter than their competitors. And if they are playing by the rules, they improve the quality of markets by minuscule amounts trade after trade after trade.

But the algorithms we see at work here are different. They don't serve any function in the market. University of Pennsylvania finance professor, Michael Kearns, a specialist in algorithmic trading, called the patterns "curious," and noted that it wasn't immediately apparent what such order placement strategies might do.

Donovan thinks that the odd algorithms are just a way of introducing noise into the works. Other firms have to deal with that noise, but the originating entity can easily filter it out because they know what they did. Perhaps that gives them an advantage of some milliseconds. In the highly competitive and fast HFT world, where even one's physical proximity to a stock exchange matters, market players could be looking for any advantage.

"They are moving the high-frequency services as close to the exchanges as possible because even the speed of light matters," in such a competitive market, said Stanford finance professor Peter Hansen.

Given Nanex's data, let's say that these algorithms are being run each and every day, just about every minute. Are they really a big deal? Donovan said that quote stuffing or market spoofing played a role in the Flash Crash, but that event appears to have had so many causes and failures that it's nearly impossible to apportion blame. (It is worth noting that European markets are largely protected from a similar event by volatility interruption auctions.)

But already since the May event, Nanex's monitoring turned up another potentially disastrous situation. On July 16 in a quiet hour before the market opened, suddenly they saw a huge spike in bandwidth. When they looked at the data, they found that 84,000 quotes for each of 300 stocks had been made in under 20 seconds.

"This all happened pre-market when volume is low, but if this kind of burst had come in at a time when we were getting hit hardest, I guarantee it would have caused delays in the [central quotation system]," Donovan said. That, in turn, could have become one of those dominoes that always seem to present themselves whenever there is a catastrophic failure of a complex system.

There are ways to prevent quote stuffing, of course, and at least one of the members of the Commodity Futures Trading Commission's Technology Advisory Committee thinks it should be outlawed.

"Algorithms that might be spoofing the market are something that should be made illegal," said John Bates, a former Cambridge professor and the CTO of Progress Software. But he didn't want this presumably negative practice to color the more mundane competitive practices of high-frequency traders.

"There is algorithmic terrorism and then there is reverse engineering, which is probably just part of good business practice," Bates said.

For now, Donovan plans to keep putting out the charts, which he calls "crop circles," of the odd trading algorithms at work. That's an apt name for the visualizations we see of this alien world of bot trading. And it certainly gets at a central mystery surrounding them: if trading firms aren't sending out these orders, how are they getting into the market?

On the quantitative trading forum, Nuclear Phynance, the consensus on the patterns seemed to be that they simply just emerged. They were the result of "a dynamical system that can enter oscillatory/unstable modes of behaviour," as one member put it. If so, what you see here really is just the afterscent of robot traders gliding through the green-on-black darkness of the financial system on their way from one real trade to another.

No matter why the bots end up executing these behaviors, the Nanex charts offer a window onto a kind of market behavior that's fascinating and oddly beautiful. And we may never have seen them, if not for the mildly obsessive behavior of one dedicated nerd.

"Who looks at millisecond charts?" Donovan said. "You'd never see those patterns in any other fashion. The SEC and CFTC certainly weren't."

Here are a few more bots at work with explanations of what's going on.

Here we see a "flag repeater" being executed on the BATS Exchange, the third-largest equity market after the NYSE and NASDAQ.
15,000 quote requests were made in 11 seconds in a repeating pattern. Each iteration upped the quote a penny until $9.36, and then the algorithm went down the same way, a penny at a time.

This is an extreme closeup of just one second of trading of the stock SHG, the Shinhan Financial Group. This is 760 quotes from a total of 10,000 made in 12 seconds.

This chart shows a different kind of strategy. It represents 56,000 quotes in one second all at the same price (the top chart) but with the size of the order increasing by one (i.e. 100 shares) all the way up to 40,000.

Finally, we see what Donovan calls the "stubby triangles" chart. It shows high quotes being made and then immediately followed by a stub order of $0.01 (basically canceled in most contexts). The quote is then remade at a lower price and followed with another stub quote. This cycle happened at the rate of 380 quotes a second. [This last description was clarified thanks to the kindness of author Joe Flood.]

Most Popular

Writing used to be a solitary profession. How did it become so interminably social?

Whether we’re behind the podium or awaiting our turn, numbing our bottoms on the chill of metal foldout chairs or trying to work some life into our terror-stricken tongues, we introverts feel the pain of the public performance. This is because there are requirements to being a writer. Other than being a writer, I mean. Firstly, there’s the need to become part of the writing “community”, which compels every writer who craves self respect and success to attend community events, help to organize them, buzz over them, and—despite blitzed nerves and staggering bowels—present and perform at them. We get through it. We bully ourselves into it. We dose ourselves with beta blockers. We drink. We become our own worst enemies for a night of validation and participation.

Even when a dentist kills an adored lion, and everyone is furious, there’s loftier righteousness to be had.

Now is the point in the story of Cecil the lion—amid non-stop news coverage and passionate social-media advocacy—when people get tired of hearing about Cecil the lion. Even if they hesitate to say it.

But Cecil fatigue is only going to get worse. On Friday morning, Zimbabwe’s environment minister, Oppah Muchinguri, called for the extradition of the man who killed him, the Minnesota dentist Walter Palmer. Muchinguri would like Palmer to be “held accountable for his illegal action”—paying a reported $50,000 to kill Cecil with an arrow after luring him away from protected land. And she’s far from alone in demanding accountability. This week, the Internet has served as a bastion of judgment and vigilante justice—just like usual, except that this was a perfect storm directed at a single person. It might be called an outrage singularity.

Forget credit hours—in a quest to cut costs, universities are simply asking students to prove their mastery of a subject.

MANCHESTER, Mich.—Had Daniella Kippnick followed in the footsteps of the hundreds of millions of students who have earned university degrees in the past millennium, she might be slumping in a lecture hall somewhere while a professor droned. But Kippnick has no course lectures. She has no courses to attend at all. No classroom, no college quad, no grades. Her university has no deadlines or tenure-track professors.

Instead, Kippnick makes her way through different subject matters on the way to a bachelor’s in accounting. When she feels she’s mastered a certain subject, she takes a test at home, where a proctor watches her from afar by monitoring her computer and watching her over a video feed. If she proves she’s competent—by getting the equivalent of a B—she passes and moves on to the next subject.

The Wall Street Journal’s eyebrow-raising story of how the presidential candidate and her husband accepted cash from UBS without any regard for the appearance of impropriety that it created.

The Swiss bank UBS is one of the biggest, most powerful financial institutions in the world. As secretary of state, Hillary Clinton intervened to help it out with the IRS. And after that, the Swiss bank paid Bill Clinton $1.5 million for speaking gigs. TheWall Street Journal reported all that and more Thursday in an article that highlights huge conflicts of interest that the Clintons have created in the recent past.

The piece begins by detailing how Clinton helped the global bank.

“A few weeks after Hillary Clinton was sworn in as secretary of state in early 2009, she was summoned to Geneva by her Swiss counterpart to discuss an urgent matter. The Internal Revenue Service was suing UBS AG to get the identities of Americans with secret accounts,” the newspaper reports. “If the case proceeded, Switzerland’s largest bank would face an impossible choice: Violate Swiss secrecy laws by handing over the names, or refuse and face criminal charges in U.S. federal court. Within months, Mrs. Clinton announced a tentative legal settlement—an unusual intervention by the top U.S. diplomat. UBS ultimately turned over information on 4,450 accounts, a fraction of the 52,000 sought by the IRS.”

There’s no way this man could be president, right? Just look at him: rumpled and scowling, bald pate topped by an entropic nimbus of white hair. Just listen to him: ranting, in his gravelly Brooklyn accent, about socialism. Socialism!

And yet here we are: In the biggest surprise of the race for the Democratic presidential nomination, this thoroughly implausible man, Bernie Sanders, is a sensation.

He is drawing enormous crowds—11,000 in Phoenix, 8,000 in Dallas, 2,500 in Council Bluffs, Iowa—the largest turnout of any candidate from any party in the first-to-vote primary state. He has raised $15 million in mostly small donations, to Hillary Clinton’s $45 million—and unlike her, he did it without holding a single fundraiser. Shocking the political establishment, it is Sanders—not Martin O’Malley, the fresh-faced former two-term governor of Maryland; not Joe Biden, the sitting vice president—to whom discontented Democratic voters looking for an alternative to Clinton have turned.

During the multi-country press tour for Mission Impossible: Rogue Nation, not even Jon Stewart has dared ask Tom Cruise about Scientology.

During the media blitz for Mission Impossible: Rogue Nation over the past two weeks, Tom Cruise has seemingly been everywhere. In London, he participated in a live interview at the British Film Institute with the presenter Alex Zane, the movie’s director, Christopher McQuarrie, and a handful of his fellow cast members. In New York, he faced off with Jimmy Fallon in a lip-sync battle on The Tonight Show and attended the Monday night premiere in Times Square. And, on Tuesday afternoon, the actor recorded an appearance on The Daily Show With Jon Stewart, where he discussed his exercise regimen, the importance of a healthy diet, and how he still has all his own hair at 53.

Stewart, who during his career has won two Peabody Awards for public service and the Orwell Award for “distinguished contribution to honesty and clarity in public language,” represented the most challenging interviewer Cruise has faced on the tour, during a challenging year for the actor. In April, HBO broadcast Alex Gibney’s documentary Going Clear, a film based on the book of the same title by Lawrence Wright exploring the Church of Scientology, of which Cruise is a high-profile member. The movie alleges, among other things, that the actor personally profited from slave labor (church members who were paid 40 cents an hour to outfit the star’s airplane hangar and motorcycle), and that his former girlfriend, the actress Nazanin Boniadi, was punished by the Church by being forced to do menial work after telling a friend about her relationship troubles with Cruise. For Cruise “not to address the allegations of abuse,” Gibney said in January, “seems to me palpably irresponsible.” But in The Daily Show interview, as with all of Cruise’s other appearances, Scientology wasn’t mentioned.

An attack on an American-funded military group epitomizes the Obama Administration’s logistical and strategic failures in the war-torn country.

Last week, the U.S. finally received some good news in Syria:.After months of prevarication, Turkey announced that the American military could launch airstrikes against Islamic State positions in Syria from its base in Incirlik. The development signaled that Turkey, a regional power, had at last agreed to join the fight against ISIS.

The announcement provided a dose of optimism in a conflict that has, in the last four years, killed over 200,000 and displaced millions more. Days later, however, the positive momentum screeched to a halt. Earlier this week, fighters from the al-Nusra Front, an Islamist group aligned with al-Qaeda, reportedly captured the commander of Division 30, a Syrian militia that receives U.S. funding and logistical support, in the countryside north of Aleppo. On Friday, the offensive escalated: Al-Nusra fighters attacked Division 30 headquarters, killing five and capturing others. According to Agence France Presse, the purpose of the attack was to obtain sophisticated weapons provided by the Americans.

The Islamic State is no mere collection of psychopaths. It is a religious group with carefully considered beliefs, among them that it is a key agent of the coming apocalypse. Here’s what that means for its strategy—and for how to stop it.

What is the Islamic State?

Where did it come from, and what are its intentions? The simplicity of these questions can be deceiving, and few Western leaders seem to know the answers. In December, The New York Times published confidential comments by Major General Michael K. Nagata, the Special Operations commander for the United States in the Middle East, admitting that he had hardly begun figuring out the Islamic State’s appeal. “We have not defeated the idea,” he said. “We do not even understand the idea.” In the past year, President Obama has referred to the Islamic State, variously, as “not Islamic” and as al-Qaeda’s “jayvee team,” statements that reflected confusion about the group, and may have contributed to significant strategic errors.

Some say the so-called sharing economy has gotten away from its central premise—sharing.

This past March, in an up-and-coming neighborhood of Portland, Maine, a group of residents rented a warehouse and opened a tool-lending library. The idea was to give locals access to everyday but expensive garage, kitchen, and landscaping tools—such as chainsaws, lawnmowers, wheelbarrows, a giant cider press, and soap molds—to save unnecessary expense as well as clutter in closets and tool sheds.

The residents had been inspired by similar tool-lending libraries across the country—in Columbus, Ohio; in Seattle, Washington; in Portland, Oregon. The ethos made sense to the Mainers. “We all have day jobs working to make a more sustainable world,” says Hazel Onsrud, one of the Maine Tool Library’s founders, who works in renewable energy. “I do not want to buy all of that stuff.”

A controversial treatment shows promise, especially for victims of trauma.

It’s straight out of a cartoon about hypnosis: A black-cloaked charlatan swings a pendulum in front of a patient, who dutifully watches and ping-pongs his eyes in turn. (This might be chased with the intonation, “You are getting sleeeeeepy...”)

Unlike most stereotypical images of mind alteration—“Psychiatric help, 5 cents” anyone?—this one is real. An obscure type of therapy known as EMDR, or Eye Movement Desensitization and Reprocessing, is gaining ground as a potential treatment for people who have experienced severe forms of trauma.

Here’s the idea: The person is told to focus on the troubling image or negative thought while simultaneously moving his or her eyes back and forth. To prompt this, the therapist might move his fingers from side to side, or he might use a tapping or waving of a wand. The patient is told to let her mind go blank and notice whatever sensations might come to mind. These steps are repeated throughout the session.