We’ve kicked off a study on the cost impacts of various quality assurance standards, and this post gives our preliminary results for modeling DO-178c and DO-254 in TruePlanning®. DO-178c and DO-254 are standards that deal with the safety of software and electronics used in airborne systems. It began as a standard used predominately by the U.S. Federal Aviation Administration (FAA) for commercial aircraft, and its usage has spread significantly to the U.S. military and many other countries.
All of the software and electronics on-board an aircraft are categorized into 5 Design Assurance Levels (DAL) based on how failure of ...

Here’s something I’ve been thinking about a lot lately - technical debt and its relationships to software maintenance costs. Technical debt speaks to the structural quality of software applications. Technical debt is incurred for many different reasons; sometimes it is intentional when shortcuts are taken to meet a time to market requirement; sometimes it occurs because a development team gets sloppy about applying good coding practices (or a development team has not documented coding practices); sometimes it happens when the technology in an application is not kept up to date and it literally gets lapped by technology.
Not all technical ...

Here’s a great blog about estimates. The first thing I liked about it was the author managed to quote Lewis Caroll in a blog about software estimation. You can’t really go wrong if you can quote Lewis Caroll and invoke iconic memories about “Through the Looking Glass”. The specific quote refers to the word “Estimate” and goes like this - “When I use a word it means just what I choose it to mean – neither more nor less.”
The author posits that maybe before we perform an estimate, and certainly before we consume one – we should remind ourselves ...

The CRASH (CAST Research on Application Software Health) report for 2104/2015 is out and an Executive Summary can be downloaded for free from this link. This is the third biennial report produced by CAST based on an analysis of the data collected by their AppMarq static code analysis tool to develop a report on the health of software projects based on their structural quality. Structural quality speaks to the engineering goodness of the architecture and code for an application, rather than the functional quality that results by delivering software that solves users’ problems. CAST determines structural quality of code ...

Am I correct in understanding that in order to determine Payload PM, SE, MA [e.g., WBS 5.1, 5.2, 5.3], Payload I&T/GSE [e.g., WBS 5.5, 5.6], Spacecraft PM, SE, MA [e.g., WBS 6.1, 6.2, 6.3], and Spacecraft I&T/GSE [e.g., WBS 6.6] costs, each user must export TP results into an Excel spreadsheet, then apply their own factor to obtain the aforementioned WBS costs because TP lumps all of those costs into 1 PM bucket, 1 SE bucket, 1 MA bucket and 1 I&T bucket? And we have to do this every time we make a change to determine those costs?
Gosh ...

From the presentation, it looked like the TP risk area included the capability for percent inputs around the point inputs rather than having to enter value?
Yes, we’ve added Percent (as well as offset) to the FRISK input method to set pessimistic and optimistic values off the point values. In the 1st example below I’ve used +20% and -10% respectively, around the Weight of Structure’s point’s value. Per the 2nd example below, we can do likewise in our Monte Carlo companion applications, where our new custom logic satisfies NASA’s typical approach for mass growth-risk with Optimistic=CBE {i.e., your point value}, ...

Does any of the Space catalogs (objects) cover the launch vehicle and stage 1 & 2 engines? Do the defaults cover manned space?
Not, not soon at least. HOWEVER, we and multiple NASA Centers have used the True-Hardware/Software catalogs for many years to estimate launch vehicles and manned vehicles. Currently the Space Missions Catalog objects (i.e., estimating models) support robotic and unmanned missions. But again, in the TruePlanning catalogs (HW, SW, Systems, etc), there has always been an Operating Specification choice for manned space per below—
To watch the "Best Practices using the TruePlanning Space Missions" Webinar, click here.

Will other recent programs be added (to basis/analysis/CERs)?
Yes, typically within one year of launch. We don’t often need to make changes to the component estimates but we test them as new data comes in. The support functions get minor tweaks each time we add another data point.
To watch the "Best Practices using the TruePlanning Space Missions" Webinar, click here.

Do you plan to add MAVEN any time soon?
Just learned that yes, Maven will be added to the basis & analysis in the near future.
To watch the "Best Practices using the TruePlanning Space Missions" Webinar, click here.

In your webinar presentation--
Slide 21: NASA WBS mapping for the WBS 4.0 – this slide seems to indicate that this WBS is estimated by TP
Slide 35: Space Systems Object – this slide seems to indicate that this WBS is estimated by TP
Slide 48: Mapping Rules.. – this slide seems to indicate that this WBS is estimated by TP, but Notes column this very slide are contradicting this – by saying typically passed-thru what would be right approach; shall we have a pass-thru number for this WBS (4.0) or shall we assume that the TP estimate includes this WBS?
Good catch! ...

If you look at slide 68 of the presentation (Reference Mission Set Used in the Chicago Development Cost Model), it says that the data sets includes “actual costs for completed projects and projected costs for projects in development and near-term mission candidates”. Does this mean that historical actuals were used and there was no normalization of the data? Does the actual costs include contractor fee and any subcontractor burdens or were those stripped out? If contractor fees or burdens were removed, how do we go about adding that to the model estimate? Is there an input parameter for that ...

It is not evident how WBS 10.0 which is not just integration of all systems together but also the integration with launch vehicle as well as other related launch operations is calculated without knowing the complexities of the mission. Also, environmental I&T associated with observatory integration is in WBS 10.0.
WBS 10 covers all S/C and System-level I&T that is not specific to an individual instrument or S/C subsystem. It includes Assembly & Integration, System Test, and Ground Support Equipment. Costs for the prototype/EM hardware used in flight system testbeds are covered within each applicable component estimate (represented by the ...

Why is there no assembly level object at the s/c top level. All the subsystems within the s/c bus need to be assembled because all the s/c bus subsystems are very much intertwined together. So there has to be an assembly object at the top of the bus?
Assembly I&T is performed by the Space Subsystem object. Adding an additional Assembly object at the top of multiple subsystem-assembly pairings produces very high redundant costs. However, the estimator can certainly perform organizational calibration and use worksheet set multipliers to adjust outputs at this top level if an overall assembly is added.
...

What about software? I never heard it mentioned. How does the space catalog address software?
Actually, we did discuss software briefly. Since flight software is included with each electronic item, some portion of an independently estimated software cost should be removed from the result. The separate TruePlanning for Software catalog can be used to estimate development and COTS new/modified/reused/deleted code, sized via multiple approaches: SLOC, function point, use cases as well as via a Functional Sizing calculator.
To watch the "Best Practices using the TruePlanning Space Missions" Webinar, click here.

What kinds of software costs would you say are not included in the calibrated cost to the particular space component?
Flight software costs are included with each electronic component. Ground software costs are included with MOS. Ground data handling costs beyond the typical level used by most robotic space missions may not be adequately covered with the MOS factors.
To watch the "Best Practices using the TruePlanning Space Missions" Webinar, click here.

Some space components had an asterisk (modeled outside of PRICE TruePlanning)? What does that mean?
There are some components that are not found on the Component Type drop down but rather are modeled with CERs outside of True H (Space Parachute for example). These CERs are available via custom cost objects, per below, rather than the Space Component cost object.
To watch the "Best Practices using the TruePlanning Space Missions" Webinar, click here.

Is there an EM and prototype calculator, or do you treat every type or prototoype (breadboard, brassboard, mass model, etc) as a full prototype?
Fractional prototype quantities are acceptable. Regarding a calculator-- good idea. We will investigate feasibility.
To watch the "Best Practices using the TruePlanning Space Missions" Webinar, click here.

Can you mix catalog objects from different catalogs into the same assembly? (ex. a microcircuit with a space component)
The Space Catalog was designed to use only the components in that Catalog and some activities are different to other Catalogs. When using a Space System cost object, a Space Sub-assembly and a Space Assembly, do not mix regular System, Assembly and Hardware Cost Objects in the same Space System.
If Regular System, Assembly and Hardware items are included as component of a Space System, the results will be different to using only Space Catalog components. A Project may contain separate Space ...

On one of your slides you mention using Assembly at the subsystem level for HW to HW integration (I can see this for Payloads as those are usually stand alone), however, for spacecrafts a lot of subsystems aren't integrated as a stand alone, are you overstating possible costs by doing at each subsystem vs. a spacecraft level assembly which would encompass the integration of all the subsystems at the spacecraft level?
A bus (or payload) can have just one subsystem-assembly pairing, with all components listed below, rather than multiple subsystems (or instruments) each with multiple separate pairings.
In general, note that the ...

You mentioned using production qty vs. prototype, even with the Space Mission Objects, however typically Spacecrafts are very low qty and my experience from heavy PRICE H use is that the production qty uses a much steeper slope. Is it now more realistic with the industry data in the space objects that production qty is good to use now?
Yes, that is our recommendation. We have seen production first units higher in cost than prototypes. We recommend using the latter for engineering models and test units.
To watch the "Best Practices using the TruePlanning Space Missions" Webinar, click here.

Do you have capability to do CONOPS which drive the design?
Good idea! As demonstrated, we allow the user to model complexity (or any other input) as a function of requirements, specifications, etc which could include a CONOPS value as well. Also, another companion application is our linkage to the design-trade tools ModelCenter and i-Sight.
To watch the "Best Practices using the TruePlanning Space Missions" Webinar, click here.

Are you able to triangulate bounds on inputs, low -high-average, to get nominal values?
Yes, the risk tools above all allow for triangular distributions. Specifically in FRISK, the “Auto” selection (from the Method drop-down in the Risk Input Sheet) will generate pessimistic and optimistic values around the point estimate’s baseline. Or the inputs can be manually input.
To watch the "Best Practices using the TruePlanning Space Missions" Webinar, click here.

Do the cost models include SEER, USCM, OGC, NAFCOM, ACEIT?
Per our discussion, ACEIT is an export option from TruePlanning. And we are evaluating the feasibility of hosting PCEC (fka NAFCOM) as a custom catalog, available to NASA users.
To watch the "Best Practices using the TruePlanning Space Missions" Webinar, click here.

Spares may be a contract deliverable?
Yes. Quantities include flight units, spares and prototypes for Ems/proof-of-concept builds.
To watch the "Best Practices using the TruePlanning Space Missions" Webinar, click here.

How is cost uncertainty integrated into cost basis and subsequent roll-up into subsystems, etc?
Complexities are density functions that act as surrogates for combinations of CERs. Hence, overall model prediction error is reflected by the impact of complexity uncertainty on overall risk at subsystem and system levels.
To watch the "Best Practices using the TruePlanning Space Missions" Webinar, click here.

Can you provide a pointer to additional information regarding the ChiCoMo Model?
Here’s an earlier webinar by our Chief Scientist: http://www.pricesystems.com/Resources/Videos/tabid/487/VideoId/23/language/en-US/Default.aspx
To watch the "Best Practices using the TruePlanning Space Missions" Webinar, click here.

Whether a new cost estimator or a seasoned professional cost estimator, we all need to continuously seek to improve the credibility of our cost estimates. I can think of no better reference to assess our cost estimates than the March 2009 Government Accountability Office (GAO) Cost Estimating and Assessment Guide.
The basic characteristics of effective estimating from the guide are listed below, were first published in 1972, and have stood the test of time as the yard stick for measuring the credibility of our cost estimates done for both government and industry.
1. Clear identification of task
...

We’re often asked about how to model the cost impacts of various quality assurance standards like SEI Capability Maturity Model (CMM) and CMM-I for software, DO-178c and DO-254 for airborne software and electronics, various MilSpecs, and even process standards like ISO-9000. As a result, we’re kicking off a study to create new and updated modeling guidance for use with our cost models.
Many of these standards have significant overlap, and an implementation of one can give you a head start in implementing another. For example, DO-178c and DO-254 are design assurance standards, which basically says that if you follow these ...