Looking for general advice about capturing more volume in a liquidity providing HFT strategy. One thing I’m working on is a system doing this at a relatively niche venue. The operation is doing well in terms of performance - high sharpe, consistent daily profitability, high turnover, etc. But because the volume’s are small achieving high market share is important. Right now we’re trading about 0.5% of the ADV, and I’d like to be hitting 10% or more.

Currently using an ensemble of pretty good unconditional directional alphas. But the toxicity/adversity/impact model is basic. The only feature is queue position. Obviously that biases to small clips on new levels. Which perform great, but are a limited subset of the volume. The market’s also pretty thick-book, wide-tick, so a high proportion of executions occur when price levels stabilize.

If the adversity model relies on queue alone, you mostly miss out on these fills. You never get to the front of the queue, because you have to work your way from the back of an already long queue, that’s overly penalized. Even if you join when alpha’s in your favor, you have to rest for a while to get to the front. Over that long lifetime, alpha’s pretty much guaranteed to revert back, and the model cancels.

My kneejerk is that what’s required is a more feature-rich adversity model. Particularly some metrics indicative of a stable market, like rolling volatility, recent trade intensity, age of the orders in the level, etc. But with some (admittedly shallow) investigation, it seems like queue position has so far been overwhelmingly predictive compared to other indicators.

Maybe I’m barking up the wrong tree, and effort would be better spend on some avenue. Like just improving the alphas, or using longer-biased alphas when evaluating long queues. Or just experimenting with different quoting mechanics.

Obviously every market and strategy is different. Not expecting to hear any easy answers. But if any other NPers who’ve faced similar dilemmas can point me in the general direction of something that worked, offer advice, or even just randomly speculate, I’d be very appreciative.

It sounds to me like you need more short term alpha. I think what you mean by an adversity model is effectively short term alpha since you're literally looking for when price is going to move. So for example, if you have an order out then when your short term alpha reverses (what for that order is adverse selection) then you cancel. And this is where speed comes into play since the short term alpha reversals are extremely quick and event driven, so unless you're extremely fast then you'll be one of the last set of non-informed orders that are adversely selected.

As far as when placing the order, if you're placing the order after a new queue has already formed likewise need to have shorter term alpha in order to have a signal to send the order earlier, ideally as the queue is forming and so gotta be fast.

A good market impact model is definitely useful since it'll help determine when and how much to trade on the alpha.

So I think what you're looking for is different horizon alphas from the ultra short term to whatever horizon you're currently trading, then one way or another perform a multi-period optimization taking into account market impact and trading costs to then end up with your final decisions as far as when to place orders and cancel. Lots of different ways to skin that cat though and generally not the easiest problem to solve and thus lots of hackish solutions out there. Also, depending who the other participants are then could be hard to capture a significant portion of the ADV without a significant investment in tech/research.

It sounds to me that you're complaining that your adversity model is too sensitive. A naive example with similar properties is if you just used book imbalance for your adversity model, so your order gets pulled too frequently. Just like the book imbalance case, you're right that more dimensions in your model will help. But I've also found that simple base features like queue position tend to have more nonlinear properties that you need to capture, so what you're lacking may be more structure in your model rather than just more feature-richness. It also sounds to me that you're aware of some time-evolution properties of your adversity target function and level replenishment properties that you're not explicitly capturing, there's several state space models that are well-known for these.

You haven't complained about the trigger rate and/or fill rate of your joining condition(s), but something also sounds strange to me since it sounds to me that you're always waiting at the back of a long queue. Are you not participating inside-spread when the book dislocates?

I'm assuming your strategy isn't parameterized in a way that order size ("small clips") is what's limiting the capacity. This seems to be the case since you're not exclusively hitting. It's definitely possible to hit 10% ADV with small clips. In certain venues even where modify is not atomic and you lose priority on down modify, you do want to splice your large order into small clips anyway.

> It also sounds to me that you're aware of some time-evolution properties of your adversity target function and level replenishment properties that you're not explicitly capturing, there's several state space models that are well-known for these.

Care to share some references?

@EL: Is your alpha ever large enough to cross the spread? The easiest way to get queue priority (and increase ADV) is to trigger a new price level yourself.

Also, I might be misunderstanding something, but it sounds like you're saying that a short queue means high adversity, so you wait for the queue to be long enough before you join. But joining a short queue also means high chance of good fills. If you take the net effect, do you still see a large penalty for joining a short queue?

Guys, thanks so much each one of you for the thoughtful and thought-provoking replies! This definitely helped clarify my thinking and gave me a lot of momentum this past week to tackle the problem.

@anonq

> I think what you mean by an adversity model is effectively short term alpha

That’s a fair point. I guess I’d call “alpha” the unconditional directional forecast, whereas adverse selection is the forecast conditional on a given order being filled. (I.e. at a given time open orders all have the same alpha, but different adverse selection scores.) This is really just semantics, but wanted to clarify any ambiguity from the original post.

> unless you're extremely fast

Should have clarified. I basically have at or near the lowest latency in the market for the time being.

> to have a signal to send the order earlier, ideally as the queue is forming and so gotta be fast.

You’re definitely right. I have the alpha and the latency, so should be leaning on this more. It’s pretty low-hanging fruit.

The challenge is that the market has very thick tick-sizes, and queue formation doesn’t occur that often. Even if you’re always ahead of the curve on queue formation, there’s still a lot of missing ADV, where from resting order entry to execution, the book never ticks.

> So I think what you're looking for is different horizon alphas from the ultra short term to whatever horizon you're currently trading

Great point. Started mining a little bit in this direction, and it already looks promising.

@prikolno

Yeah, I think you hit the nail on the head regarding over-sensitivity. The state-space model is an interesting point. Currently I’m just training with supervised learning, which doesn’t do a good job penalizing position loss on cancels. Looking into re-fitting some of the models with reinforcement learning.

> Are you not participating inside-spread when the book dislocates?

Yes, I am participating in these conditions. And those trades tend to be consistently profitable.But this particular market has very thick tick sizes, so the book doesn’t dislocate that frequently. A high proportion of the ADV in the market comes from orders that join an already long queue on a stable book, and sit for a while before being filled. It’s on that volume, where participation is a challenge.

> I'm assuming your strategy isn't parameterized in a way that order size ("small clips") is what's limiting the capacity… It's definitely possible to hit 10% ADV with small clips.

Thanks. It’s very helpful to hear this perspective in terms of focusing the direction of the research.

> Is your alpha ever large enough to cross the spread? The easiest way to get queue priority (and increase ADV) is to trigger a new price level yourself.

Sometimes, but the spreads are very thick. Also should have mentioned the fee-structure is taker-taker, so colonizing a new level isn’t a free lunch like in maker-taker. Overall though, you’re right, this is low-hanging fruit and I should be utilizing it.

> it sounds like you're saying that a short queue means high adversity, so you wait for the queue to be long enough before you join.

Sorry for not being clear in the original post. Definitely meant the opposite: shorter queues have much lower adversity (at least in this market). So much so, that I’m heavily biased towards only participating when touch size is small. The problem is that I’m missing out on all the volume that occurs when queues are long.

Some of that can be mitigated by getting in when queues are small, settling into a good position, and staying in as the queues grow large. But the market has pretty large spreads and doesn’t tick that often. So a high percentage of time the queues are long, and even the orders at the front joined an already large touch.

@ronin

Thanks, that’s a good perspective. You may very well might be right about the targets being unrealistic. It’s good to keep this hypothesis on the table. No point trying to squeeze more juice from a spent lemon.

But there are some reasons why I think 10% ADV is feasible. That’s basically what Virtu does as a market maker on lit US equities. It’s also about the collective market share of passive HFTs in ES on the CME. Obviously Virtu has a lot more resources, but this market’s also a lot less competitive than US equities. As of now I have a pretty substantial infrastructure and research advantage over the other participants.

To clarify, I’m not trying to build up positions with successive trades in a single direction. Just trying to opportunistically trade on fast alpha, with O(100X) daily turnover and capturing a few tenths a tick per trade. Not overly concerned with the impact from my trading activity, since the autocorrelation of the trades are effectively zero.

> To clarify, I’m not trying to build up positions with successive trades in a single direction. Just trying to opportunistically trade on fast alpha, with O(100X) daily turnover and capturing a few tenths a tick per trade. Not overly concerned with the impact from my trading activity, since the autocorrelation of the trades are effectively zero.

That doesn't make your impact zero. It means half your impact is on the bid, half on the ask.

And you are saying that they are so perfectly balanced that they cancel each other out on every single fill?

@EspressoLoverTLDR: Profitable high volume trading in low vol contracts for me took mostly changing my methods rather than adjusting and adding features and using an adjusted parameter set from a similar HFT MM model I had been trading.

I worked on a similar project in what now was many years ago (like, 0.05 worth of edge in cross exchange CL@NYMEX vs WTI@ICE arb without hardware acceleration or microwave lines, long ago). And although I've since stopped trading in what'd be considered UHFT, I hope my dated observations might be of some value to you.

I started on the project for multiple reasons. Most of which had significant direct and indirect incentives (daily/monthly fee tier breaks, quoting new products or look-alike contracts for rebates or free/reduced fee agreements, in order to join exchange MM programs, to get API access to a mostly API restricted exchange, and also to improve quote to trade ratios in particular products to allow us to run more quote intensive strategies) from the exchanges and from my firm as well.

While we had had success in high volume strategies in vanilla arbitrage and HFT stat arb, it was clear that in order to hit some of the volume requirements (ex. trade >10% ADV of the entire curve volume for a product) we'd have to trade more than our usual outright futures and trade significantly more calendar spreads and low vol, highly liquid products, and manage our resulting curve books (which we did in OTC energy, but not in an automated fashion).

We initially discovered most of our features weren't well suited for slow, liquid order books as our strength of avoiding toxic flow in other product/contracts just meant we were losing queue priority far too frequently to have any success. For us it took looking at the problem in a different manor than we were accustomed to. The "view from 20,000 ft" explanation of our trading was that we managed our quoting behavior based on some regression methods with a bit of model position and performance feedback thrown in as many other MMs were doing as well. In order to trade some these products that sometimes took days/weeks/months to make it through the top of book queue we determined we either had to stay in the book longer without the added adverse selection becoming more costly than the expected value of the additional trades we'd make from remaining in the queue longer. The other goal was to better our ability to identify when to start a new price level and when to add/cancel orders as the new price level built up.

We found the former task significantly easier and started having luck when we added classifier methods (classes: 'up tick', 'down tick', 'no movement'; or dual classifiers 'uptick' vs 'no movement or down tick' and 'down tick' vs 'no movement or uptick'). I'm afraid I shouldn't get to into the methods we used to assign various states to the classification types as I'd rather not have someone confuse it for IP. Though I assure you that you can likely find more robust methods than we did without too much of a time investment.

The task of determining when to start a new price level was significantly more difficult. For one, the dynamics of a new price level being built up in near impossible to model as there are a significant number of players reacting to a very large number of messages during this type of event and a new price level can bring in strategies with very diverse intentions (lead-lag, arbitrage, stat-arb, position puking, momentum, and any mix of the bunch could be triggered to act in a new price level event). So while we had success using the former method to meet many of our obligations, and we added a little performance to our hedging methods indirectly from research related to the latter, we were unable to achieve a solution that successfully used both as we had hoped. It may be a bit of a unicorn, but I've seen at least one desk over the years that had focused significant research on when to add to new price levels and manage positions accordingly and they are among the most successful I've met. Best of luck hunting down your answers.

@djfostner: just curious, but were/are you solely doing intermarket arb, or did you focus on intramarket MM strategy as well? For instance, fut spreads in STIRs where the order matching engine is size Pro Rata instead of FIFO?

"How dreadful...to be caught up in a game and have no idea of the rules." - C.S.

@JTDerp: The intermarket arbs were just one strategy we ran for awhile early on. Despite the strategy not being too significant to our PnL or total volumes, looking back it was probably pretty important in the grand scheme of things as its consistent PnL early on allowed for our young/inexperienced group to learn and pay some market tuition without the worry of our daily PnL reports going into the red. We never got into IR products or currencies unfortunately (for some contractual reasons probably best left unexplained), but we did a fair amount of trading in most everything else including: Ags, energy, softs, metals, and indices. As a result we did end up building out a fair amount of order management logic that was matching engine type specific (FIFO, ProRata, K-Algorithm, TOP, etc, and also some exchange specific stuff for products like Kospi options).

@djfostner: For Pro Rata, did you build your own backtest/sim environment for working with it, or lean on a platform vendor? From what I know thus far, Stellar Trading Systems has provision for Pro Rata in their backtest/sim...still looking into other options.

"How dreadful...to be caught up in a game and have no idea of the rules." - C.S.