31110https://www.ibm.com/developerworks/community/forums/atom/replies?topicUuid=76c48645-6b95-482c-ba3b-1c76a9b5034cVirtual Roundtable-Agile Metrics Breakout--Enterprise Level measures [Event closed] Replies2013-01-07T22:34:08.823ZIBM Connections - Discussion Forumc914709e-8097-4537-92ef-8982fc416138urn:lsid:ibm.com:forum:6beb73c4-ac1f-4d3b-b4a0-c3077d7af93dRe: Virtual Roundtable-Agile Metrics Breakout--Enterprise Level measures2012-11-02T12:36:21.978ZRob Suritis120000EYKGactive2012-11-02T16:29:24.659Z2012-11-02T16:29:24.659ZRob Suritis120000EYKGactive<div>Can we test another assertion:</div><div>For significantly large organizations, enterprise level measures are independent of development methodology.</div><div> </div><div>Large organizations are heterogeneous (e.g. banks, insurance companies, etc). Within IT there are different technologies, application life cycles, tools, cultures, vendors/partners. Decisions about which systems and applications to fund is business driven, often outside of IT. Decisions about who develops it, or how, is driven by past performance. These are different decisions, and therefore require different sets of metrics.</div><div> </div><div>For example: An enterprise wants to improve it's electronic filing/account capabilities. I believe the portfolio management decision is easier from a metrics standpoint because business value, market demands, strategy, etc appear more normalized across options. The decisions of who develops it, and how, are more difficult: continue on mainframe w/ mostly assembler, port to Java as new system, outsource to system integrator, stick with waterfall or move to iterative/agile, and a host of other decisions (a side conversation here is that this is an architecture driven decision). Here the enterprise has a normalization problem because the options are many and heterogeneous. I believe we come back to Walkers list of Predictability, Quality, and Time-to-value. Productivity is more difficult to normalize.</div><div> </div><div>So to me, enterprise level metrics are not tied to agile as a development methodology. It raises a question about the difference in &quot;agility&quot; between a development organization vs. an enterprise (they are increasingly tightly coupled).</div><div> </div><div> </div><div> </div>none, view_forum, view_categoryurn:lsid:ibm.com:forum:08f2d484-169c-4724-b251-db7375f6af5dRe: Virtual Roundtable-Agile Metrics Breakout--Enterprise Level measures2012-11-02T14:20:10.272ZReedy Feggins120000A43Dactive2012-11-02T14:20:10.272Z<div>Also we need to consider that most agile teams keep track the following data</div><div>- Story points completed (velocity) in Sprint</div><div>- Hours consumed (or planned hours for next sprint)</div><div>- Produc Backlog ( Story points)</div><div> </div><div>However in the Product backlog most teams only spend time developing &quot;good&quot; estimates for the high priority stories. The estimates for the medium and low priority stories usually are not valid and in some cases they are Epics where the scope have not clearly been defined by the team</div>none, view_forum, view_categoryurn:lsid:ibm.com:forum:6abe7063-95e2-41c6-be8d-370e447836b7Re: Virtual Roundtable-Agile Metrics Breakout--Enterprise Level measures2012-11-02T13:55:58.969ZCherifa270001GAPCactive2012-11-02T14:05:48.695Z2012-11-02T14:05:48.695ZCherifa270001GAPCactive Yes, when referring to an Enterprise Agile transformation and -watching- as you climb the ladder of your adoption, whether you are maturing in this agile transformation as an enterprise. In this particular case this is tied to the agile process/practices you are adopting.<br>none, view_forum, view_categoryurn:lsid:ibm.com:forum:06f5275f-5c14-410f-ac10-5eb68ef27558Re: Virtual Roundtable-Agile Metrics Breakout--Enterprise Level measures2012-11-02T13:19:08.155ZWalkerRoyce0600019HVSactive2012-11-02T13:19:08.155Z<div> My answer is yes. </div><div>The raw data should all be the same. There may be differences in how the data is synthesized, normalized or presented, but the data should be the same. And furthermore, all practitioner data sets and reports should be transparently viewable by organizational leaders and organizational data and reports should be viewable by practitioners. </div><div> </div><div>Now, there will be some hesitancy to share some financial data, quality trends and forecasts due to &quot;competitive paranoia&quot; but in general, we need bilateral transparency to ensure minimal gamesmanship and maximize trust.<br></div>none, view_forum, view_categoryurn:lsid:ibm.com:forum:2d55fcb6-a425-4a46-8c3f-f59467692d28Re: Virtual Roundtable-Agile Metrics Breakout--Enterprise Level measures2012-11-02T13:12:51.714ZMikeORourke060001EVM6active2012-11-02T13:12:51.714Z In general I agree, but lets be clear..... even though the actual metrics / reports are different at an enterprise level from a practitionerlevel, tha data underneath these different metrics must be the same. Right?<br>none, view_forum, view_categoryurn:lsid:ibm.com:forum:b3c3ed08-c6f9-4d03-b271-8bc1944b8e93Re: Virtual Roundtable-Agile Metrics Breakout--Enterprise Level measures2012-11-02T12:49:28.735ZWalkerRoyce0600019HVSactive2012-11-02T12:49:28.735Z<div>Good observations. And a relevant comment for the Enterprise level discussion thread. Organization level metrics should be independent of process and people and tooling and even architecture. organizations exist to serve a business purpose:</div><div>1) profit centers maximize profit, i.e., deliver the most value per unit cost.</div><div>2) cost centers to deliver a service for minimum cost .</div><div>Productivity (resource management), TTV (process management), quality (product management) and predictability (uncertainty management) are important in all of these. </div><div> </div><div>I hesitated to say that these metrics should be independent of architecture because I believe that organizations that are structured around architectures are much more likely to have better correlated metrics, opportunities for reuse, and commonality of processes, tooling and skills.<br></div>none, view_forum, view_categoryurn:lsid:ibm.com:forum:a1f10c73-98c4-45b0-9b9e-dd2a40d08683Re: Virtual Roundtable-Agile Metrics Breakout--Enterprise Level measures2012-11-01T22:19:06.360ZJimDensmore1200006V99active2012-11-01T22:19:06.360Z In the limit, is a new application completely worthless because we have not yet shipped it? Of course not. But there are FASDI rules that often make our clients think of unshipped apps as worthless. This makes portfolio management difficult to reason about in terms of value. Since the cost is often fairly well known, they turn to cost and try to drive the cost down. This is why you sometimes see valuable programs killed. &quot;They cost too much.&quot; Value should be in the prioritization criteria (or ROI - see discussion with me and Kevin above), not cost. Cost is a constraint, not a good prioritization criterion.<br>none, view_forum, view_categoryurn:lsid:ibm.com:forum:b2e88198-0046-4b89-a20a-da5902aafb3eRe: Virtual Roundtable-Agile Metrics Breakout--Enterprise Level measures2012-11-01T19:51:55.680ZMikeORourke060001EVM6active2012-11-01T19:51:55.680Z<div> Its not waste IMHO just because its not finished. It's just not &quot;there&quot; yet. You could potentially still test it, stub it out, try some of the scenarios against it.... so it has value... but notvalue from a customers perspective yet. The value is that its closer to being done than not. However, to your point, until it can get stakeholder feedback... we've got no understanding of how close it is. The point I was making was closer to where Jim discussed.... which is that we have to use the same &quot;data&quot; for burndown charts and other practitioner measurements but as an executive... I don't care about those measurements... I need measurements that translate to how I am being measured (Time to market, ROI, Costs, etc).</div><div> </div>none, view_forum, view_categoryurn:lsid:ibm.com:forum:ce801003-5f35-406e-b436-56290ac1290aRe: Virtual Roundtable-Agile Metrics Breakout--Enterprise Level measures2012-11-01T19:46:18.928ZCherifa270001GAPCactive2012-11-01T19:47:43.614Z2012-11-01T19:47:43.614ZCherifa270001GAPCactiveGood point Mike..in addition......if a story is worth 100 points (by the way we do not recommend such a big story)..and if we have delivered 90 points...meaning the story is not done yet...would not that be waste? This story worth of 100 points must have X acceptance criteria (AC) that have been identified by the product owner. Delivering on a certain number of acceptance criteria instead of story points ( in accordance with the PO decision) can be a way of having delivered value... some value not all!!! why not? We can track the number of AC and related tests coverage.<br>none, view_forum, view_categoryurn:lsid:ibm.com:forum:0a1f1d0a-4729-45b2-b51b-6f19e2665288Re: Virtual Roundtable-Agile Metrics Breakout--Enterprise Level measures2012-11-01T19:40:38.864ZJimDensmore1200006V99active2012-11-01T19:40:38.864Z<div>Mike, I was going to post this in business value ... and sorry it's long ... but your comment causes me to wish it posted here. </div><div> </div><div>As we've mentioned, story points generally correlate only to the amount of work to complete. I am not sure what the answer is yet but I have some thoughts on it based on the canonical (woo hoo!) ATM example. There is a mix here between trying to be efficient and productive (team) and trying to be effective, build the right thing, build the thing right (enterprise, and business alignment). We have to determine where to do each.<br> <br>Suppose we build an ATM with two use cases, (1) transfer funds and (2) get cash. Turns out the transfer funds user stories are complex to build and the cash user stories are easy. Lots of EV by SP measures for transfer funds, but not so much for get cash. Later we release our new ATM, first with only the transfer funds bits implemented. Customers already have ATM cards. But no one comes to the ATM. Guess there wasn't much <u><b>business</b></u> (&quot;biz&quot;) value in that. But then we release the next version of the ATM and it has get cash implemented. Whoa, customers flock to our bank, open a new account to have access to the cash option of the ATM and everything. BIG biz value in that.<br> <br>Ok that's fairly obvious as far as it goes, but how to measure it? We have to go back to the original biz plan (hopefully the biz planners have been keeping it up to date as things develop). And the result that we get, prior to our release, must be an estimate, possibly with large <u>uncertainty</u>. We must include the uncertainty in the estimate. Hindsight being what it is, wouldn't we work to get the Get Cash bits released first, if we had it to do over? Even at the expense of including the transfer funds in the first release? Surely Shirley. <br> <br>Next thought, suppose (as is probably true in real life) that the implementations of transfer funds and get cash use similar bits of the underlying architecture? As transfer funds is developed, some of the components under construction will later be used in get cash. What is the biz value of those components? Whatever it is, it's not just associated with the biz value of the transfer funds stories. It must also include the biz value of the get cash stories.<br> <br>So if we somehow manage to pick good biz value measures, then we automatically lean toward building the things first that will make us more revenue, and we also lean toward building the things right in the architectural sense, because our biz value measures do better under those conditions. Clearly the biz value measures should relate to revenue, or profit, or ROI, or something like that. Just as clearly, they're very uncertain numbers until after the fact, so we need to include that uncertainty. Unfortunately, uncertainty relates back to honesty - it's hard to stay honest. Long-term calibration helps.<br> <br>Now why am I bringing this up in an Agile Metrics context? The answer is that Mike took it up a level, up to the Enterprise level, where doing the work efficiently isn't any longer the only thing that 's important. At the enterprise level we need to consider managing the portfolio as well, that is, doing the right work, doing the work that will best align our development shop with the business. Your comment about becoming useless, Mike, applies. The <u>team</u> needs to be <i>responsible </i>for efficiency and productivity, and needs to be <i>responsive</i> to the enterprise portfolio management prioritization.<br></div>none, view_forum, view_category