Chase Gets High-Performance Analytics

Nobody wants to be in the position of trading speed for accuracy of analytical work, but sometimes, you don't have much choice. The business has questions it wants answered, and you need to deliver -- the sooner the better.

Problem is, "as you're enabling more and more data to become usable, there are more and more questions being asked," Chris Gifford, senior vice president of customer analytics at JPMorgan Chase, recounted during yesterday's "The Big Reveal: What's Your Data Telling You?" session at SAS Global Forum Executive Conference 2013. Maybe the organization no longer wants to know about customer behavior in aggregate, for example, but on an individualized basis -- taking into account all touch points.

Chris Gifford,
Chase

And so, you may well find yourself waiting... and waiting... and waiting for your answers as your models chug through all those bits and bytes. And then you may find yourself doing things that go against best-practices wisdom.

You might, as did the Chase customer analytics group Gifford heads, decide to select only some rows in your database to model rather than using all rows. Or, you might only look at some attributes and restrict others, or settle on one algorithm rather than test multiple algorithms to figure out the best solution to a problem. And maybe you wouldn't do quite as much testing of your model as you really ought, explained Gifford, who also told some of the Chase story during Sunday night's combined opening session for the SAS Global Forum 2013 and the executive conference, both taking place in San Francisco this week (watch the session here on demand).

At Chase, Gifford said he knew he needed to figure out a way that would let his analytics team stop cutting corners.

With a goal of increasing the speed and accuracy of the customer analytics models, Gifford said he began investigating high-performance computing solutions, like SAS High-Performance Analytics (HPA), that would streamline the time required for modeling runs. When first seeing SAS HPA in action, Gifford admitted he was a bit skeptical. "Could it really run fast with our models and our data?" he recalled thinking.

The proof-of-concept testing answered with a definitive, "Yes."

During that testing, his team ran a number of its traditional models. One, a risk model for its mortgage business, had been taking about 160 hours -- nearly one full week -- to complete. HPA testing showed the same model, with the same data, running in about 84 seconds. A credit risk model, which took 14 hours using the traditional SAS approach, ran in 180 seconds on HPA, Gifford said.

Needless to say -- although he did -- "those kinds of improvements are fantastic." And with them, we suddenly have options again, he added.

You can continue to chase the efficiencies. You can do more models per statistician per day or week. But you can improve your accuracy if you're willing to trade off some of that newfound speed. You can test additional algorithms or increase the sample counts -- more rows, more columns, perform more tests.

HPA has significantly improved the speed -- 200 times and 100 times performance improvements in the case of the two examples Gifford cited -- while improving accuracy. That's of no small significance for a financial institution the size of Chase. As he noted, "as we reduce the type one and type two errors for this kind of use case, it turns us into being able to say yes to more of the good guys and no to more of the bad guys -- so, yes, there are very big impacts."

At the end of the day, bringing data down to all these servers gives modelers the chance to rethink how they do things, too, he added. "The speed gets them ahead."

In fact, traditional business analytics speeds aren't going to cut it in the future. "As you keep moving more and more data to the edge of real-time decision making, you have to engage in this process."

Beth Schultz, Editor in Chief

Beth Schultz has more than two decades of experience as an IT writer and editor. Most recently, she brought her expertise to bear writing thought-provoking editorial and marketing materials on a variety of technology topics for leading IT publications and industry players. Previously, she oversaw multimedia content development, writing and editing for special feature packages at Network World. In particular, she focused on advanced IT technology and its impact on business users and in so doing became a thought leader on the revolutionary changes remaking the corporate datacenter and enterprise IT architecture. Beth has a keen ability to identify business and technology trends, developing expertise through in-depth analysis and early adopter case studies. Over the years, she has earned more than a dozen national and regional editorial excellence awards for special issues from American Business Media, American Society of Business Press Editors, Folio.net, and others.

@Waqas, while waiting for a SAS Global Forum session on HPA to begin, i did overhear a couple of gentleman from a satellite TV company say they thought HPA was great, and would be useful, but at "a half a million bucks" unreasonable for its environment. I asked SAS CTO Keith Collins about this when I met with him the next day. He agreed that, yes, the initial offering was not affordable at large. But a June release changes that, eliminating the need for the pricey, dedicated appliance plus slicing up the functionality so you can buy only what you need and grow from there. Plus, companies will be able to take the SAS procedures they have running on a single server and move them over to an HPA cluster as needed.

Kicheko, you have raised a good point. Completeness is the thing to compromise on rather than the accuracy and that is what practically happens in practical life. Misleading information and delayed information; both are not acceptable.

Beth, true. Budgeting is not a factor to be ignored esp for a SME however large corporations may agree to pay as much as a high quality solution costs because of the economies of large scale they benefit from.

waqasaltaf, - If this was a debate i would be on the other side saying compromise on report if you have to. Speed is important..as they say late information is uselsss. I like to hope that if a report has to suffer from having been quick it would be on the completeness as opposed tp the accuracy.

There is certainly truth in that statement, as it could be applied to many considerations. As I read your implication, you hold that the context of modeling and analytics are essential to determining what is acceptable, in terms of both the nature of the results, and the means of achieving them - the principles to which I adhere are in agreement.

The problem with such an endorsement, in many cases (though I can't say it applies in this case), is that the very context which the principle requires has been stripped from the data on which the processes operate, or even the processes themselves. The effect ends up being like some food being deemed both beneficial and detrimental to good health; without an understanding of the particulars of how these contradictory conclusions were derived, how can anyone make an informed decision as to what to eat or not eat? I think a question similar in nature is faced by those considering analytics strategies. With mushrooms, the safe assumption is to assume they are unsafe, unless proved otherwise - pretty much the same with speed vs. accuracy.

@waqasaltaf, while I agree solutions are available I don't think it's that simple. High-performance analytics/computing doesn't necessarily come cheap, for example, so there is budgeting to consider, for one.