Risk Analysis with Subjective EMAB Distributions

George Rebane

A few years back I introduced to our more technically oriented readers a new approach to capture and work with subjective random variables.  In ‘Predicting with Expressed Beliefs – a formal approach’ the Mode Augmented Boxcar (MAB) probability distribution was made available in a report which described its derivation and various useful applications.  Among the advantages of EMAB is its compatibility with Bayesian analysis.  In this piece I share with readers the Expanded MAB (EMAB), present its formal derivation (TR1908-1), and provide two tools that further explicate and support its uses.  You can download all three files here –

Download TR1908-1_EMABderivation

Download EMABworksheet

Download ProposalRiskAnalysis2

The following scenario illustrates but one of the many uses of EMAB, and assumes that the reader is generally familiar with the MAB distribution.

Suppose you head up an engineering company contemplating responding to an RFP, and want to know the financial risk in doing the job were you to win the contract.  The work will be done and billed on a ‘cost plus’ (cost plus fixed fee) basis.  There will be four separate deliverables D1, D2, D3, D4 that are to be produced in sequence – each succeeding deliverable must be tested with those previously completed.  Due to job particulars, the fee percentages F_i on related costs C_i of each Di (i = 1, … ,4) will be negotiated individually.  The customer also states that the entire project must be completed within a time interval T_C = 60 weeks of contract start date, and that 1% of the total contract price will be subtracted for every week late of all deliverables.

You call your team together for a bid meeting, and, using the best information and expertise at hand, you generate some ‘best estimates’ for the salient job parameters – i.e. all the C_i, F_i, and T_i, the time to complete and test each Di.  With a little algebra you calculate the first cut total bid price P_B for the contract as

with your margin amount to cover overhead, taxes, and profit computed from

Here the dollar amounts, percentages, time durations are the best guesses or estimates from your staff based on company history, experience, and current conditions.

The customer will make no progress payments and remit the full (delivery adjusted) amount at completion.  Your company will finance the work out of retained earnings.  Based on these inputs, should you continue to bid the job given that you have no idea of the financial risk you are taking – i.e. no idea of the chances and likelihoods for any of the critical parameters P_B, T_F, P_M, and possible penalties actually coming to pass?

Experienced managers know that there is much more information and knowledge available about the point estimates that staff generates to include in the draft work-up used to make the bid/no-bid decision, and select the bid parameters if the decision is to respond to the RFP.  Incorporating rigorous risk analysis in the work-up using EMAB is a productive approach that requires no additional work to what was expended in coming up with the point estimates.  What this means is that prudent professionals already went through the process of bracketing the most probable range for any required numerical quantity, along with its most likely value.  In this process the estimator naturally forms a measure of confidence that the realized value of the estimated (random) variable will come in near its most likely value, and a sense of the overall chance that it will be within the high/low bracket values that s/he considered.

This valuable information is never captured and therefore lost in the standard most likely point-estimate approach that attempts to deal with the ‘known unknowns’ derived from the staff’s experience, education, and corporate knowledge (i.e. historical record).  Using EMAB allows such subjective/expert uncertainties to be captured, communicated, and computed to produce a product that correctly and completely represents the quantitative nature of the involved uncertainties and therefore the related risk based on the best information available.  Additionally, such an EMAB-based work-up serves as a tool that can be used to refine the proposal while it answers a multitude of questions in explicit and understandable quantified formats.

An example of such a work-up in a spreadsheet is shown here and can be downloaded above.  This self-explanatory spreadsheet can also serve as a template for more complex scenarios than the admittedly ‘toy scenario’ example presented here.  Also available for download is the EMAB worksheet that summarizes the salient math model and contains an interactive implementation of it for use as a tool for any work-up using EMAB distributions.  We remind the reader that the distribution of a resulting expected value of a function of random variables (e.g. a sum of EMAB r.v.s) rapidly converges to the normal or Gaussian (or ‘bell curve’) distribution by virtue of the Central Limit Theorem of Probability (more here).  Hence the resulting expectation values and variances produced by such functions represent stochastic variables from such a bell curve, and can be treated accordingly.

Before concluding, here are just a few of the many questions that one can ask of an EMAB-based proposal worksheet shown below.  The proposal RFP was summarized above.

1. What is the chance that we’ll blow the 60-week deadline? Answer: 0.2214 or about 2 out of 9.
2. What should be our total cost bid so that we don’t have more than a 5% chance of overrunning that total cost. Answer: $1,381,500.
3. Joe, your work-up on D1 shows a 12.4% or 1 out of 8 chance of exceeding your expected cost by more than 5%. Have your team take another look and see what it takes to reduce that overrun probability to 10% max.  Answer: Joe comes back with ‘There’s a way we can bump our most likely cost up to $572,000 with an increased confidence of 0.7, and not affect the other cost bracket and time-to-complete parameters.  This would increase our D1 bid expected cost from $548,200 to $551,000.

I welcome questions and comments from readers wishing to use the EMAB distribution.  Please use the comment stream below for such requests.  I will respond to the email that you supply (not visible to other readers) when you post your comment.

Comments

[Paleo Lithic]

"our more technically oriented readers" -- who is that, Todd or Walt?

[George Rebane]

PaleoL 1136am - Ah yes, another worthy new to the internet. In the future you should try not to confuse the population of a website's commenters with that of its readers.

[Andy Hill]

"In the future you should try not to confuse the population of a website's commenters with that of its readers."

That's right, dang nabit. What reader's write doesn't necessarily mean what they meant to say. Obviously, those who post on this page are not its readers. Todd and Walt don't read, they just hum along.

Shame on you Lil Lithic, you should have known that. It's in the secret code.

[George Rebane]

AndyH 1217pm - You want to try that comment again, or are you among the cognitively disabled - commenters are a subset of readers. And in your attempt at snark, try to learn the difference between plural and singular possessive.