Advocate Health Care, a large integrated delivery network in Chicago, and Blue Cross Blue Shielf of Illinois (BCBSI), the largest health plan in that metro area by far, announced last week that their collaboration on an ‘accountable care organization’ framework where medical management was fully delegated to the delivery system yielded medical trends that were 6.1% lower than the rest of the market – based on data from the first six months of claims.

I’m optimistic that this kind of collaboration can yield significant cost savings, and I’m convinced that doing medical management at the provider entity, rather than at the health plan, will yield to better patient engagement and improved results. I’ve seen in my experience that physician groups and physician hospital dyads which accept capitation are much more creative about how to improve care while lowering overall costs. Advocate has also been forward-looking in its data collection, and has excellent dedicated physician leadership. So – I’m happy to see this report.

However, it’s early –and the short Kaiser Health Network report leaves a lot of questions.

Was the ACO group comparable?It appears that the Advocate ACO had a limited network –whereas the standard BCBSI product has a very wide network. This could discourage those with more illness (and more established relationships outside of Advocate) to stay out of the ACO product. It would be nice to be sure that at least the demographics were not different for the Advocate group between the baseline year and the observed period.

Was there any change in benefit design? Was the evaluation adjusted for richness of benefit design (or change of benefit design.)Companies that choose the limited network might have also trimmed benefits further – in which case an evaluation should include out-of-pocket costs.

How much of the claimed cost savings was utilization, and how much was cost per unit?It’s likely that Advocate agreed to steeper discounts to establish this limited network.It would be nice to see a decrease in the resource cost of care – not just a decrease in trend due to lower stated prices.

How will any ‘side payments’ or reconciliation affect the overall cost savings?Note that for BCBS Massachusetts evaluation of the first year of its Alternative Quality Contract, the cost of paid claims went down, but the total cost of care went up due to bonus payments to providers. BCBSMA has done the right thing by hiring reputable independent researchers to review its data and publish the results.

Were there any claims payment issues that could lead to differential ‘incurred but not received’ (IBNR) amounts?It’s more likely that there would be some delay in payments in the ACO arrangement in the first few months, especially if there are any “subcapitations” or different contracts that help Advocate manage care. If there is a different IBNR amount, that could eat into the stated savings.

It’s great to see early reports of success in lowering cost through collaborations between payers and providers. However, early reports sometimes don’t tell the whole story, so we should look for future reports that answer the questions above before accepting claims of cost savings.