Zen and the art of the accurate estimate

Schwable states that Estimates are difficult for the following Reasonsⁱ:

• Estimates are done too quickly
• Lack of Estimate Experience
• Humans are biased towards underestimation
• Management Desires Accuracy

In addition to these difficulties there are many methods that may be used to estimate the cost of a software project including:

1. Top-Down Estimatesⁱⁱ
2. Bottom Up Estimatesⁱⁱⁱ
3. Parametric Modeling ^iv
4. COCOMO^v
5. COSYSMO^vi
6. Event Chain Methodology^vii
7. Function Points^viii
8. Program Evaluation Review Technique (PERT)^ix
9. Proxy Based Estimation (PROBE)^x
10. The Planning Game^xi
11. Weighted Micro Function Points^xii
12. Wideband Delphi^xiii

Thus far within the confines of project estimation that we have examined the art of “triangulating” a project estimation method using a top-down and bottom up approach promises to be both the most efficent and accurate method; However Grimstad et al. State that one of the greatest risks of failure for any software project is a “Reliable cost estimate” as previously stated by various studies.^xiv

The etymology of the term to “Estimate” is from Latin, and it's origin is a “Builders statement of projected costs”, from 1796 OED. Builders have the luxury of using tangible assets to develop costs upon.

Software is intangible, that is to say ultimately we may use metrics to measure software but these standards include “Guessing” how many lines of code may be required for a non-existant function, or how many man hours may be derived using Bohem's COCOMO estimates. Since software is based in the art of describing functions in a language of Mathematica abstraction, this abstraction itself further lends to the difficulties of developing an “estimate” of how many lines of code may be needed in what programming language for which function.

Given that we may select a methodology and develop metrics based upon empirical data with a given project; we may assume that we must use estimates based on previous projects with the same group of people, that is to say we must have previous work completed to compare against to help in developing a more accurate method to estimate future projects; this is a common practice derived primarily from fiscal budgeting and actuarial standards ; ie if a given business wishes to develop estimates of income they will use the previous years sales as a starting point; so to would a software development house use the previous years number of lines of code vs profit gained and man-hours of work as metrics for the next or any other quoted project.

The biggest problem in software cost estimation is the simple fact that even considering all of the research, and formal methods we may use; it's still just a very educated and formalized guess at a given cost.

Just as the old proverb states; the only constant in life is change; that is next to death and taxes, so given that the industry of software development is that with the greatest risk and potential for profit the amount of change within the software market in a given year is gargantuan when compared with more traditional areas or segments of industry.

The core business of mining resources for use has not changed in a millennia, neither has farming or travel; all of these sectors realize greater efficiencies in the information age by using software and automated systems along with great leaps in technology.

In the information age the rate of change and evolution is increasing as stated previously by Ray Kurtzweil; although he predicts systems with greater intelligence than their human agents at some point in the near future until then we are the agents of change within the software industry.

The biggest problem in developing accurate cost estimates for any given software project is change; Change in the scope of the project, change in the market forces driving the products development, internal recursive changes in the code used by the programmers to develop the project and a change in the project environment and business including change in it's risk profile or even change composition of the development and testing team of resources for said project.

We can model projects with a fair degree of accuracy given a few constants; these include the teams of people involved in the project, their expected salary, the variance in the software market which is far more volatile than most commodities and securities markets and the expected revenues from the project given previous years market data. However if any of those constants change with respect to the project then we must revisit it's model and cost estimate to maintain an accurate view.

A simple analogy for project costing would include how economists model the prime interest rate for a country. The models used over time vary; but the goal of the rate remains the same to maintain a stable level of inflation. 

They may use any models at their disposal or even develop custom models using automated systems but ultimately at the highest levels of research the numbers used for CPI, GDP and the like are aggregates of averages; thus the amount of removal or abstraction in the process may not compensate for a change in the global economic environment. These models are slow to react to change.

Thus any methodology used to develop cost models may use aggregates of averages or even aggregates of aggregates to develop a high level idea of what an enterprise spends on all of it's projects but ultimately these models are all so completely abstracted from reality that they must be sanity checked against expenditures to ensure that they hold any ounce of accuracy. Each of these processes is sensitive to change and although we excel at placing variance in an automated system; we still require experience and previous data upon which to develop our cost models. 

 

This reason alone is why many software startups fail, even why many business fail.

iSchwable, Kathy (Cengage Learning, 2010) Information Technology Project Management P.263 ISBN: 978-0-324-78692-7

iiSchwable, Kathy (Cengage Learning, 2010) Information Technology Project Management P.264 ISBN: 978-0-324-78692-7

iiiSchwable, Kathy (Cengage Learning, 2010) Information Technology Project Management P.264 ISBN: 978-0-324-78692-7

ivSchwable, Kathy (Cengage Learning, 2010) Information Technology Project Management P.265 ISBN: 978-0-324-78692-7

vBohem, Barry; Clark, Bradford; Horowitz, Ellis; Westland, Chris (USC, USC Center for Software Engineering, Annals of Software Engineering, 1995, 57-94)

viValderi, Ricardo (University of Sothern California, August 2005) THE CONSTRUCTIVE SYSTEMS ENGINEERING COST MODEL [Online] PDF Document, Avaialble from: http://csse.usc.edu/csse/TECHRPTS/PhD_Dissertations/files/Valerdi_Dissertation.pdf (Accessed on August 17^th 2011)

viiAktasa , Nihat; de Bodta , Eric, Cousinb, Jean-Gabriel (IAG Louvain School of Management, Université catholique de Louvain, Belgium, November 13 2006) Event studies with a contaminated estimation period [Online] PDF Document, Avaialble from: http://www.sciencedirect.com/science/article/pii/S0929119906000290 (Accessed on August 17^th 2011)

viiiCapers, Jones (McGraw-Hill, 2007) Estimating software costs: bringing realism to estimating P.246 ISBN: 978-0-07-148300-1

ixKeefer, Donald L.; Verdini, William A.; (Arazona State University, Departemnt of Decition and Information Systems, Management Science, Vol 39. No. 9 September 1995) Better Estimation of PERT Activity Time Paramaters [Online] PDF Document, Available from: http://www.jstor.org/pss/2632814 (Accessed on August 17^th 2011)

xJohnson, Philip M.; Moore, Carleton A.; Dane, Joseph A.; Brewer , Robert S.; (Unversity of Hawaii, IEEE, 2000) Empirically-Guided Software Effort Guesstimation [Online] PDF Document, Avaialble from: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.103.5058&rep=rep1&type=pdf (Accessed on August 17^th 2011)

xiWilliam A. Wood, William L. Kleb (IEEE, NASA Langley Research Center, Issue 3, May / June 2003) Software Exploring XP for Scientific Research [Online] PDF Document, available from: http://doi.ieeecomputersociety.org/10.1109/MS.2003.1196317 (Accessed on August 17^th 2011)

xiiSymons, Charles; (Proc. of the 4th European Conf. on Software, 2001) Comeback Function Point Analisys (Modernized) All is Forgiven! [Online] PDF Document, Avaialble from: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.8.2999&rep=rep1&type=pdf (Accessed on August 17^th 2011)

xiiiWiegers, Karl E. (Process Impact, 2000) Stop Promising Miracles [Online] PDF Document, Available from: http://www.uml.org.cn/SoftWareProcess/pdf/delphi.pdf (Accessed on August 17^th 2011)

xivGrimstad, Stein; Jørgensen, Magne; Moløkken-Østvold, Kjetil; (Journal of Information and Software Technology 48(4):302-310 , 19 April 2005) Software Estimation Terminology: The Tower of Babel [Online] PDF Document, Availble from: http://simula.no/research/se/publications/Grimstad.2006.1 (Accessed on August 17^th 2011)