Project Management Estimation Methods – Delphi / Wideband Delphi

Posted by mgocean on May 27, 2009 in Estimation, Project Management |

The Delphi technique is an organized way to help a group of informed experts achieves a consensus on an estimated value. It is an iterative process consisting of a series of rounds. In each round, every expert in the group anonymously and independently estimates the value of an item. No person knows the values provided by other persons. This removes peer pressure and political bias. At the end of each round, the estimators receive feedback based on all of the estimated values for that round. The feedback tends to filter out extreme opinions and values, leading to convergence on the value.


Delphi technique can be used to estimate many kinds of quantities such as the sizes of software modules, the amount of effort required to perform a task, or the value for a model’s input parameters. Model developers sometimes use the Delphi technique to estimate the values of effort multipliers used in parametric models before any empirical data is available.


You implement the Delphi technique as follows: Select a group of knowledgeable experts and provide them with information relevant to the quantity to be estimated such as descriptions of software modules or project tasks. It is important to provide clear descriptions of the “nature and scope” of each item to be estimated.


Each expert in the group produces a series of estimated values for a particular quantity. Figure 4‑3 shows a form to submit estimated values for a set of items such as modules or tasks. The left column lists the items or activities to be estimated. The center column contains the estimated value for each item. (May be asked for multiple values [e.g., lowest, most likely, and highest].) The last column allows the estimator to submit written notes explaining his/her estimating rationale for the value recorded.

Figure 4‑3 Delphi form for individual estimator


A Facilitator summarizes the set of estimates received from all of the estimators for the round. Figure 4‑4 illustrates a typical tabular summary suitable for estimates of several items. The left column lists the items. The next four columns summarize the estimated values. This particular table shows the smallest value and the largest value reported by the estimators, as well as the average and the standard deviation. (Also the median, mode, or other statistics can be showed. The median is better than the average because it is less sensitive to extreme values. The utility of various statistical measures depends on the number of estimators.) The rightmost column references any notes or remarks that the Facilitator wants to share with the estimators. For example, the Facilitator could summarize comments or rationale provided by the various estimators.

Figure 4‑4 Summary of estimation round


The Facilitator then provides the summary data to the estimators prior to the start of the next estimation round. Each expert then independently estimates a new set of values. The feedback causes the estimated values to converge as the iteration continues. The values of a particular quantity typically stabilize in three or four iterations. There may still be some variation in the values.


To monitor convergence you could plot the mean value with error bars based on the standard deviation for each round. This becomes difficult if there are many items being estimated. If, however, you are estimating values of the same quantity for all the items, then you can total the data, and take the square root of the sum of the variances to get the standard deviation for the sum. Then plot this value. For example, the total size of the software versus iteration could be plotted to see whether convergence is occurring.


There are two versions of the Delphi technique, distinguished by the type of feedback provided. In both versions, the Facilitator prepares summaries such as the one shown in Figure 4‑4. The estimators remain anonymous. In Narrow-Band Delphi, the estimators never meet face to face. Instead they only receive copies of the summary for each estimation round. In Wide-Band Delphi, the Facilitator provides the summary information, and then the estimators meet face to face to discuss the results and their rationale. In both versions, the estimators individually estimate values for the next round in private. Narrow-Band Delphi is useful when the estimators are geographically dispersed. The Facilitator can use electronic mail or facsimile transmission to exchange data with the estimators. Wide-Band Delphi requires the estimators to be collocated.


The Delphi technique provides an important benefit: It sharpens the definitions of the “nature and scope” of the items being estimated. For example, in estimating the size of software modules, estimators may clarify where the error handling logic resides (some in each module or all in one module). The team revises the module descriptions appropriately, and the estimators use these for the next round. One benefit of good estimating techniques is improved understanding of the product, processes, and project. (Stutzke R. D., 2005)







The main disadvantage of the Delphi Method is listed as;

  • Discounting the future: Future (and past) happenings are not as important as the current ones; therefore one may have a tendency to discount the future events.
  • The simplification urge: Experts tend to judge the future of events in isolation from other developments. A holistic view of future events where change has had a pervasive influence cannot be visualized easily. At this point cross-impact analysis is of some help.
  • Illusory expertise: some of the experts may be poor forecasters. The expert tends to be a specialist and thus views the forecast in a setting which is not the most appropriate one.
  • Sloppy execution: there are many ways to do a poor job. Execution of the Delphi process may lose the required attention easily.
  • Format bias: it should be recognized that the format of the questionnaire may be unsuitable to some potential societal participants.
  • Manipulation of Delphi: The responses can be altered by the monitors in the hope of moving the next round responses in a desired direction.
  • Hard to document the factors used by the experts and does not define a process for individuals to follow when estimating.

Wideband Delphi


Another popular and simple technique for estimating size and for estimating effort is the Wideband Delphi group consensus approach. The Delphi technique originated at the Rand Corporation decades ago; the name was derived from the Oracle of Delphi in Greek mythology. It was used successfully at Rand to predict the future of major world technologies.


This method is a structured way of estimation based on collective expertise. Quite often the Wideband Delphi method is used to cross-validate estimations that have been done using other popular estimation methods. This is based on the recognition of the fact that when many experts independently arrive at the same estimate based on the same assumptions, the estimate is likely to be correct. The consensus approach helps eliminate bias in estimates produced by self-proclaimed experts, inexperienced estimators or influential individuals who have hidden agendas or divergent objectives.


This is a disciplined method of using the experience of several people to reach an estimate that incorporates all of their knowledge.


In software engineering circles, the original Delphi approach has been modified. The “pure” approach is to collect expert opinion in isolation, feedback anonymous summary results, and iterate until consensus is reached (without group discussion).

Guidelines for Conducting Wideband Delphi Group Consensus

Because the Delphi approach can take a very long time, the concept of Wideband Delphi was introduced to speed up the process. This improved approach uses group discussion.




There are six major steps in conducting Wideband Delphi:

1. Present experts with a problem and a response form.
2. Conduct a group discussion.
3. Collect expert opinion anonymously.
4. Feed back a summary of results to each expert.
5. Conduct another group discussion.
6. Iterate as necessary until consensus is reached.

Group discussions are the primary difference between pure Delphi and Wideband Delphi. The summary of results in Step 4 is presented in Figure 4‑5.

Figure 4‑5 Delphi Software Size Estimation Results Summary Form


Advantages of Wideband Delphi

The advantages of Wideband Delphi include the following:

  • Implementation is easy and inexpensive.
  • It takes advantage of the expertise of several people.
  • All participants become better educated about the software.
  • It does not require historical data, although it is useful if available.
  • It is used for high-level and detailed estimation.
  • Results are more accurate and less “dangerous” than LOC estimating.
  • It aids in providing a global view of project to team members.

The disadvantages of Wideband Delphi include the following:

  • It is difficult to repeat with a different group of experts.
  • You can reach consensus on an incorrect estimate. Because you all “buy in,” you may not be skeptical enough when actual data shows it is wrong.
  • You can develop a false sense of confidence.
  • You may fail to reach a consensus.
  • Experts may be all biased in the same subjective direction.

(Futrell R.T., Shafer D.F. & Safer L. I., 2002)

Tags: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

1 Comment

Leave a Reply

Your email address will not be published. Required fields are marked *

Copyright © 2006-2021 Güzel Blog All rights reserved.
This site is using the Desk Mess Mirrored theme, v2.0.3, from BuyNowShop.com.