Abstract

A number of risk and resilience-based design methods have been put forward over the years that seek to provide designers the tools to reduce the effects of potential hazards in the early design phase. However, because of the associated high level of uncertainty and low-fidelity design representations, one might justifiably wonder if using a resilient design process in the early design phase will reliably produce useful results that would improve the realized design. This paper provides a testing framework for design processes that determines the validity of the process by quantifying the epistemic uncertainty in the assumptions used to make decisions. This framework uses this quantified uncertainty to test whether three metrics are within desirable bounds: the change in the design when uncertainty is considered, the increase in the expected value of the design, and the cost of choice-related uncertainty. This approach is illustrated using two examples to demonstrate how both discrete and continuous parametric uncertainty can be considered in the testing procedure. These examples show that early design process validity is sensitive to the level of uncertainty and magnitude of design changes, suggesting that while there is a justifiable decision-theoretic case to consider high-level, high-impact design changes during the early design phase, there is less of a case to choose between relatively similar design options because the cost of making the choice under high uncertainty is greater than the expected value improvement from choosing the better design.

References

1.
Raz
,
T.
,
Shenhar
,
A. J.
, and
Dvir
,
D.
,
2002
, “
Risk Management, Project Success, and Technological Uncertainty
,”
R&D Manage.
,
32
(
2
), pp.
101
109
.
2.
Tan
,
J. J.
,
Otto
,
K. N.
, and
Wood
,
K. L.
,
2017
, “
Relative Impact of Early Versus Late Design Decisions in Systems Development
,”
Des. Sci.
,
3
. doi:10.1017/dsj.2017.13
3.
Floricel
,
S.
, and
Miller
,
R.
,
2001
, “
Strategizing for Anticipated Risks and Turbulence in Large-Scale Engineering Projects
,”
Int. J. Project Manage.
,
19
(
8
), pp.
445
455
. 10.1016/S0263-7863(01)00047-3
4.
Allenby
,
K.
, and
Kelly
,
T.
,
2001
, “
Deriving Safety Requirements Using Scenarios
,”
Proceedings Fifth IEEE International Symposium on Requirements Engineering
,
Toronto, Canada
,
Aug. 27–31
, IEEE, pp.
228
235
.
5.
Anthony (Tony) Cox
,
L.
, Jr.,
2008
, “
What’s Wrong With Risk Matrices?
,”
Risk Anal.: An Int. J.
,
28
(
2
), pp.
497
512
. 10.1111/j.1539-6924.2008.01030.x
6.
Wright
,
M. K.
,
Stokes
,
L.
, and
Dyer
,
J. S.
,
1994
, “
Reliability and Coherence of Causal, Diagnostic, and Joint Subjective Probabilities
,”
Decis. Sci.
,
25
(
5–6
), pp.
691
709
. 10.1111/j.1540-5915.1994.tb01866.x
7.
Kreye
,
M. E.
,
Goh
,
Y. M.
, and
Newnes
,
L. B.
,
2011
, “
Manifestation of Uncertainty-a Classification
,”
DS 68-6: Proceedings of the 18th International Conference on Engineering Design (ICED 11), Impacting Society through Engineering Design, Vol. 6: Design Information and Knowledge
,
Lyngby/Copenhagen, Denmark
,
Aug. 15–19
.
8.
Khorsandi
,
J.
, and
Aven
,
T.
,
2017
, “
Incorporating Assumption Deviation Risk in Quantitative Risk Assessments: A Semi-Quantitative Approach
,”
Reliab. Eng. Syst. Saf.
,
163
(
1
), pp.
22
32
. 10.1016/j.ress.2017.01.018
9.
Fadel
,
G.
,
Summers
,
J.
,
Mocko
,
G.
, and
Paredis
,
C.
,
2015
, “
Nsf Sponsored Engineering Design and Systems Engineering Foundations Workshop Final Report
,”
Technical Report
,
Clemson University
.
10.
Chen
,
W.
,
Kesidis
,
G.
,
Morrison
,
T.
,
Oden
,
J. T.
,
Panchal
,
J. H.
,
Paredis
,
C.
,
Pennock
,
M.
,
Atamturktur
,
S.
,
Terejanu
,
G.
, and
Yukish
,
M.
,
2017
,
Uncertainty in Modeling and Simulation
,
Springer International Publishing
,
Cham
, pp.
75
86
.
11.
Frischknecht
,
B. D.
,
Whitefoot
,
K.
, and
Papalambros
,
P. Y.
,
2010
, “
On the Suitability of Econometric Demand Models in Design for Market Systems
,”
ASME J. Mech. Des.
,
132
(
12
), p.
121007
. 10.1115/1.4002941
12.
Owens
,
A.
, and
de Weck
,
O.
,
2016
, “
Sensitivity Analysis of the Advanced Missions Cost Model
,”
46th International Conference on Environmental Systems
,
Vienna, Austria
,
July 10–14
.
13.
Binder
,
W. R.
, and
Paredis
,
C. J.
,
2017
, “
Optimization Under Uncertainty Versus Algebraic Heuristics: A Research Method for Comparing Computational Design Methods
,”
ASME 2017 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference
,
Cleveland, OH
,
Aug. 6–9
, American Society of Mechanical Engineers, p. V02BT03A057.
14.
Prince
,
F. A.
,
2002
, “
Why Nasa’s Management Doesn’t Believe the Cost Estimate
,”
Eng. Manage. J.
,
14
(
1
), pp.
7
12
. 10.1080/10429247.2002.11415143
15.
Aven
,
T.
, and
Flage
,
R.
,
2009
, “
Use of Decision Criteria Based on Expected Values to Support Decision-Making in a Production Assurance and Safety Setting
,”
Reliab. Eng. Syst. Saf.
,
94
(
9
), pp.
1491
1498
. 10.1016/j.ress.2009.02.007
16.
Aughenbaugh
,
J. M.
, and
Paredis
,
C. J.
,
2006
, “
The Value of Using Imprecise Probabilities in Engineering Design
,”
ASME J. Mech. Des.
,
128
(
4
), pp.
969
979
. 10.1115/1.2204976
17.
Bjørnsen
,
K.
,
Selvik
,
J. T.
, and
Aven
,
T.
,
2019
, “
A Semi-Quantitative Assessment Process for Improved Use of the Expected Value of Information Measure in Safety Management
,”
Reliab. Eng. Syst. Saf.
,
188
(
1
), pp.
494
502
. 10.1016/j.ress.2019.03.035
18.
Owens
,
A.
,
de Weck
,
O.
,
Stromgren
,
C.
,
Goodliff
,
K.
, and
Cirillo
,
W.
,
2017
, “
Accounting for Epistemic Uncertainty in Mission Supportability Assessment: A Necessary Step in Understanding Risk and Logistics Requirements
,”
47th International Conference on Environmental Systems
,
Charleston, SC
,
July 16–20
.
19.
Stone
,
T.
,
Choi
,
S.-K.
, and
Amarchinta
,
H.
,
2013
, “
Structural Model Refinement Under Uncertainty Using Decision-Maker Preferences
,”
J. Eng. Des.
,
24
(
9
), pp.
640
661
. 10.1080/09544828.2013.824560
20.
Malak
,
R. J.
, and
Paredis
,
C. J.
,
2007
, “
Validating Behavioral Models for Reuse
,”
Res. Eng. Des.
,
18
(
3
), pp.
111
128
. 10.1007/s00163-007-0031-0
21.
Elele
,
J. N.
,
Hall
,
D. H.
,
Farid
,
A. R.
,
Turner
,
D. J.
,
Davis
,
M. E.
, and
Keyser
,
D. R.
,
2014
, “
Lessons Learned From Independent Verification and Validation of Models and Simulations (Work in Progress)
,”
Proceedings of the Symposium on Theory of Modeling & Simulation-DEVS Integrative
,
San Diego, CA
,
Apr.
, p.
17
.
22.
Thacker
,
B. H.
,
Doebling
,
S. W.
,
Hemez
,
F. M.
,
Anderson
,
M. C.
,
Pepin
,
J. E.
, and
Rodriguez
,
E. A.
,
2004
, “
Concepts of Model Verification and Validation
,”
Technical Report
,
Los Alamos National Lab
.
23.
Hazelrigg
,
G. A.
,
1998
, “
A Framework for Decision-Based Engineering Design
,”
ASME J. Mech. Des.
,
120
(
4
), pp.
653
658
. 10.1115/1.2829328
24.
Von Neumann
,
J.
, and
Morgenstern
,
O.
,
2007
,
Theory of Games and Economic Behavior
,
Princeton University Press
,
Princeton, NJ
.
25.
Thurston
,
D. L.
,
2006
, “Utility Function Fundamentals,”
Decision Making in Engineering Design
,
K.E.
Lewis
,
W.
Chen
,
L.C.
Schmidt
, eds.,
ASME Press
,
New York, NY
. doi.org/10.1115/1.802469
26.
Hazelrigg
,
G. A.
,
1999
, “
An Axiomatic Framework for Engineering Design
,”
ASME J. Mech. Des.
,
121
(
3
), pp.
342
347
. 10.1115/1.2829466
27.
Gu
,
X.
,
Renaud
,
J. E.
,
Ashe
,
L. M.
,
Batill
,
S. M.
,
Budhiraja
,
A. S.
, and
Krajewski
,
L. J.
,
2002
, “
Decision-Based Collaborative Optimization
,”
ASME J. Mech. Des.
,
124
(
1
), pp.
1
13
. 10.1115/1.1432991
28.
Collopy
,
P. D.
, and
Hollingsworth
,
P. M.
,
2011
, “
Value-Driven Design
,”
J. Aircraft
,
48
(
3
), pp.
749
759
. 10.2514/1.C000311
29.
Linkov
,
I.
,
Bridges
,
T.
,
Creutzig
,
F.
,
Decker
,
J.
,
Fox-Lent
,
C.
,
Kröger
,
W.
,
Lambert
,
J. H.
,
Levermann
,
A.
,
Montreuil
,
B.
,
Nathwani
,
J.
,
Nyer
,
R.
,
Renn
,
O.
,
Scharte
,
B.
,
Scheffler
,
A.
,
Schreurs
,
M.
, and
Thiel-Clemen
,
T.
,
2014
, “
Changing the Resilience Paradigm
,”
Nat. Clim. Change
,
4
(
6
), p.
407
. 10.1038/nclimate2227
30.
Stone
,
R. B.
,
Tumer
,
I. Y.
, and
Van Wie
,
M.
,
2004
, “
The Function-Failure Design Method
,”
ASME J. Mech. Des.
,
127
(
3
), pp.
397
407
. 10.1115/1.1862678
31.
Lough
,
K. G.
,
Stone
,
R.
, and
Tumer
,
I. Y.
,
2009
, “
The Risk in Early Design Method
,”
J. Eng. Des.
,
20
(
2
), pp.
155
173
. 10.1080/09544820701684271
32.
Ullman
,
D.
,
2017
,
The Mechanical Design Process
, 6th ed.,
Ingram Book Co.
,
La Vergne, TN
.
33.
Zhao
,
Y.
,
Jiao
,
J.
, and
Zhao
,
T.
,
2015
, “
A Functional Hazard Analysis Based on Multilevel Flow Modeling
,”
International Conference on Engineering Management and Industrial Engineering
,
Xiamen, China
,
Oct. 16–17
.
34.
Papadopoulos
,
Y.
,
Walker
,
M.
,
Parker
,
D.
,
Rüde
,
E.
,
Hamann
,
R.
,
Uhlig
,
A.
,
Grätz
,
U.
, and
Lien
,
R.
,
2011
, “
Engineering Failure Analysis and Design Optimisation With Hip-Hops
,”
Eng. Failure Anal.
,
18
(
2
), pp.
590
608
. 10.1016/j.engfailanal.2010.09.025
35.
Kurtoglu
,
T.
, and
Tumer
,
I. Y.
,
2008
, “
A Graph-Based Fault Identification and Propagation Framework for Functional Design of Complex Systems
,”
ASME J. Mech. Des.
,
130
(
5
), p.
051401
. 10.1115/1.2885181
36.
Jensen
,
D.
,
Tumer
,
I. Y.
, and
Kurtoglu
,
T.
,
2009
, “
Design of an Electrical Power System Using a Functional Failure and Flow State Logic Reasoning Methodology
,”
Annual Conference of the Prognostics and Health Management Society
,
San Diego, CA
.
37.
Short
,
A.-R.
,
Lai
,
A. D.
, and
Van Bossuyt
,
D. L.
,
2017
, “
Conceptual Design of Sacrificial Sub-Systems: Failure Flow Decision Functions
,”
Res. Eng. Des.
,
29
(
1
), pp.
23
38
. 10.1007/s00163-017-0258-3
38.
Kmenta
,
S.
, and
Ishii
,
K.
,
2000
, “
Scenario-Based FMEA: a Life Cycle Cost Perspective
,”
ASME Design Engineering Technical Conference
,
Baltimore, MD
,
Sept. 10–14
.
39.
Kmenta
,
S.
, and
Ishii
,
K.
,
2004
, “
Scenario-Based Failure Modes and Effects Analysis Using Expected Cost
,”
ASME J. Mech. Des.
,
126
(
6
), pp.
1027
1035
. 10.1115/1.1799614
40.
Sharma
,
N.
,
Tabandeh
,
A.
, and
Gardoni
,
P.
,
2018
, “
Resilience Analysis: A Mathematical Formulation to Model Resilience of Engineering Systems
,”
Sustainable Resilient Infrastructure
,
3
(
2
), pp.
49
67
. 10.1080/23789689.2017.1345257
41.
da Silva
,
F. S.
, and
Matelli
,
J. A.
,
2019
, “
Development of Metrics for Resilience Quantification in Energy Systems
,”
Proceedings of the Annual Conference of the PHM Society
,
Scottsdale, AZ
,
Sept. 23–26
, Vol.
11
.
42.
Mehrpouyan
,
H.
,
Haley
,
B.
,
Dong
,
A.
,
Tumer
,
I. Y.
, and
Hoyle
,
C.
,
2015
, “
Resiliency Analysis for Complex Engineered System Design
,”
AI EDAM
,
29
(
1
), pp.
93
108
. 10.1017/s0890060414000663
43.
Hosseini
,
S.
,
Barker
,
K.
, and
Ramirez-Marquez
,
J. E.
,
2016
, “
A Review of Definitions and Measures of System Resilience
,”
Reliab. Eng. Syst. Saf.
,
145
(
1
), pp.
47
61
. 10.1016/j.ress.2015.08.006
44.
Specking
,
E.
,
Cottam
,
B.
,
Parnell
,
G.
,
Pohl
,
E.
,
Cilli
,
M.
,
Buchanan
,
R.
,
Wade
,
Z.
, and
Small
,
C.
,
2019
, “
Assessing Engineering Resilience for Systems With Multiple Performance Measures
,”
Risk Anal.
,
39
(
9
), pp.
1899
1912
. 10.1111/risa.13395
45.
Matelli
,
J. A.
, and
Goebel
,
K.
,
2018
, “
Conceptual Design of Cogeneration Plants Under a Resilient Design Perspective: Resilience Metrics and Case Study
,”
Appl. Energy.
,
215
(
1
), pp.
736
750
. 10.1016/j.apenergy.2018.02.081
46.
Zhang
,
X.
,
Mahadevan
,
S.
,
Sankararaman
,
S.
, and
Goebel
,
K.
,
2018
, “
Resilience-Based Network Design Under Uncertainty
,”
Reliab. Eng. Syst. Saf.
,
169
(
1
), pp.
364
379
. 10.1016/j.ress.2017.09.009
47.
Madni
,
A. M.
, and
Jackson
,
S.
,
2009
, “
Towards a Conceptual Framework for Resilience Engineering
,”
IEEE Syst. J.
,
3
(
2
), pp.
181
191
. 10.1109/JSYST.2009.2017397
48.
Giahi
,
R.
,
MacKenzie
,
C. A.
, and
Hu
,
C.
,
2020
, “
Design Optimization for Resilience for Risk-Averse Firms
,”
Comput. Ind. Eng.
,
139
(
1
), pp.
106122
. 10.1016/j.cie.2019.106122
49.
Youn
,
B. D.
,
Hu
,
C.
, and
Wang
,
P.
,
2011
, “
Resilience-Driven System Design of Complex Engineered Systems
,”
ASME J. Mech. Des.
,
133
(
10
), p.
101011
. 10.1115/1.4004981
50.
Keshavarzi
,
E.
,
McIntire
,
M.
,
Goebel
,
K.
,
Tumer
,
I. Y.
, and
Hoyle
,
C.
,
2017
, “
Resilient System Design Using Cost-Risk Analysis With Functional Models
,”
ASME 2017 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference
,
Cleveland, OH
,
Aug. 6–9
.
51.
Hulse
,
D.
,
Hoyle
,
C.
,
Goebel
,
K.
, and
Tumer
,
I. Y.
,
2019
, “
Quantifying the Resilience-Informed Scenario Cost Sum: A Value-Driven Design Approach for Functional Hazard Assessment
,”
ASME J. Mech. Des.
,
141
(
2
), p.
021403
. 10.1115/1.4041571
52.
Rhee
,
S. J.
, and
Ishii
,
K.
,
2003
, “
Using Cost Based Fmea to Enhance Reliability and Serviceability
,”
Adv. Eng. Informatics
,
17
(
34
), pp.
179
188
. 10.1016/j.aei.2004.07.002
53.
Schlosser
,
J.
, and
Paredis
,
C. J.
,
2007
, “
Managing Multiple Sources of Epistemic Uncertainty in Engineering Decision Making
,”
Technical Report, SAE Technical Paper
.
54.
Bradley
,
S.
, and
Agogino
,
A. M.
,
1994
, “
An Intelligent Real Time Design Methodology for Component Selection: An Approach to Managing Uncertainty
,”
ASME J. Mech. Des.
,
116
(
4
), pp.
980
988
. 10.1115/1.2919508
55.
Wood
,
W. H.
, and
Agogino
,
A. M.
,
2005
, “
Decision-Based Conceptual Design: Modeling and Navigating Heterogeneous Design Spaces
,”
ASME J. Mech. Des.
,
127
(
1
), pp.
2
11
. 10.1115/1.1799612
56.
Takai
,
S.
, and
Ishii
,
K.
,
2010
, “
The Maximum Resource Allocation for Uncertainty Reduction in a Decision-Analytic Concept Selection
,”
Concurrent Eng.
,
18
(
1
), pp.
19
29
. 10.1177/1063293X09359551
57.
Hsiao
,
C.
, and
Malak
,
R.
,
2015
, “
Considering Risk Attitude in a Value of Information Problem
,”
DS 80-3 Proceedings of the 20th International Conference on Engineering Design (ICED 15) Vol 3: Organisation and Management
,
Milan, Italy
,
July 27–30
, pp.
93
102
.
58.
Messer
,
M.
,
Panchal
,
J. H.
,
Krishnamurthy
,
V.
,
Klein
,
B.
,
Yoder
,
P. D.
,
Allen
,
J. K.
, and
Mistree
,
F.
,
2010
, “
Model Selection Under Limited Information Using a Value-of-Information-Based Indicator
,”
ASME J. Mech. Des.
,
132
(
12
), p.
121008
. 10.1115/1.4002751
59.
Messer
,
M.
,
Panchal
,
J. H.
,
Allen
,
J. K.
, and
Mistree
,
F.
,
2011
, “
Model Refinement Decisions Using the Process Performance Indicator
,”
Eng. Optim.
,
43
(
7
), pp.
741
762
. 10.1080/0305215X.2010.512086
60.
Messer
,
M.
,
Panchal
,
J.
,
Allen
,
J.
,
Mistree
,
F.
,
Krishnamurthy
,
V.
,
Klein
,
B.
, and
Yoder
,
P.
,
2008
, “
Designing Embodiment Design Processes Using a Value-of-Information-Based Approach With Applications for Integrated Product and Materials Design
,”
ASME 2008 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference
,
Brooklyn, NY
,
Aug. 3–6
, American Society of Mechanical Engineers, pp.
823
840
.
61.
Panchal
,
J. H.
,
Paredis
,
C. J.
,
Allen
,
J. K.
, and
Mistree
,
F.
,
2008
, “
A Value-of-Information Based Approach to Simulation Model Refinement
,”
Eng. Optim.
,
40
(
3
), pp.
223
251
. 10.1080/03052150701690764
62.
Halbert
,
T. R.
,
2015
,
"An Improved Algorithm for Sequential Information-Gathering Decisions in Design under Uncertainty," Ph.D. Thesis, Department of Mechanical Engineering, Texas A&M University
.
63.
Thompson
,
S. C.
, and
Paredis
,
C. J.
,
2010
, “
An Investigation Into the Decision Analysis of Design Process Decisions
,”
ASME J. Mech. Des.
,
132
(
12
), p.
121009
. 10.1115/1.4002969
64.
Eckermann
,
S.
,
Karnon
,
J.
, and
Willan
,
A. R.
,
2010
, “
The Value of Value of Information
,”
Pharmacoeconomics
,
28
(
9
), pp.
699
709
. 10.2165/11537370-000000000-00000
65.
Chick
,
S. E.
,
2006
, “Subjective Probability and Bayesian Methodology,”
Handbooks Oper. Res. Manage. Sci.
, Vol.
13
: Simulation,
S. G.
Henderson
,
B. L.
Nelson
, eds.,
Elsevier
,
Amsterdam, The Netherlands
, pp.
225
257
. https://doi.org/10.1016/S0927-0507
66.
Wagenmakers
,
E.-J.
,
Lee
,
M.
,
Lodewyckx
,
T.
, and
Iverson
,
G.J.
,
2008
, “Bayesian Versus Frequentist Inference,”
Bayesian Evaluation of Informative Hypotheses
,
H.
Hoijtink
,
I.
Klugkist
,
P. A.
Boelen
, eds.,
Springer
,
New York, NY
, pp.
181
207
.
67.
Garthwaite
,
P. H.
,
Kadane
,
J. B.
, and
O’Hagan
,
A.
,
2005
, “
Statistical Methods for Eliciting Probability Distributions
,”
J. Am. Stat. Assoc.
,
100
(
470
), pp.
680
701
. 10.1198/016214505000000105
68.
Sharot
,
T.
,
2011
, “
The Optimism Bias
,”
Curr. Biol.
,
21
(
23
), pp.
R941
R945
. 10.1016/j.cub.2011.10.030
69.
Moskowitz
,
H.
, and
Bullers
,
W. I.
,
1979
, “
Modified Pert Versus Fractile Assessment of Subjective Probability Distributions
,”
Organizational Behav. Human Performance
,
24
(
2
), pp.
167
194
. 10.1016/0030-5073(79)90024-2
70.
Giudici
,
P.
,
Givens
,
G. H.
, and
Mallick
,
B. K.
,
2013
,
Computational Statistics
, 2nd ed.,
Wiley
,
Hoboken, NJ, United States
.
71.
Hulse
,
D.
,
Hoyle
,
C.
,
Goebel
,
K.
, and
Tumer
,
I.
,
2019
, “
Using Value Assessment to Drive PHM System Development in Early Design
,”
Proceedings of the Annual Conference of the PHM Society
,
Scottsdale, AZ
,
Sept. 23–26
, Vol.
11
.
72.
Kurtoglu
,
T.
,
Tumer
,
I. Y.
, and
Jensen
,
D. C.
,
2010
, “
A Functional Failure Reasoning Methodology for Evaluation of Conceptual System Architectures
,”
Res. Eng. Des.
,
21
(
4
), pp.
209
234
. 10.1007/s00163-010-0086-1
73.
McIntire
,
M. G.
,
Keshavarzi
,
E.
,
Tumer
,
I. Y.
, and
Hoyle
,
C.
,
2016
, “
Functional Models With Inherent Behavior: Towards a Framework for Safety Analysis Early in the Design of Complex Systems
,”
ASME 2016 International Mechanical Engineering Congress and Exposition
,
Phoenix, AZ
,
Nov. 11–17
, American Society of Mechanical Engineers, p. V011T15A035.
74.
Poll
,
S.
,
de Kleer
,
J.
,
Abreau
,
R.
,
Daigle
,
M.
,
Feldman
,
A.
,
Garcia
,
D.
, and
Sweet
,
A.
,
2011
, “
Third International Diagnostics Competition–dxc’11
,”
Proc. of the 22nd International Workshop on Principles of Diagnosis
, pp.
267
278
.
75.
Poll
,
S.
,
de Kleer
,
J.
,
Feldman
,
A.
,
Garcia
,
D.
,
Kurtoglu
,
T.
, and
Narasimhan
,
S.
,
2010
, “
Second International Diagnostics Competition–DXC’10
,”
Proceedings of the 21st International Workshop on Principles of Diagnosis
,
Portland, OR
,
Oct. 13–16
.
76.
Kurtoglu
,
T.
,
Narasimhan
,
S.
,
Poll
,
S.
,
Garcia
,
D.
,
Kuhn
,
L.
,
de Kleer
,
J.
,
van Gemund
,
A.
, and
Feldman
,
A.
,
2009
, “
First International Diagnosis Competition-dxc’09
,”
Proc. DX
’09, pp.
383
396
.
77.
Wang
,
Y.
,
2012
,
“On Rational Decision with Subjective Probability in Design,” Woodruff School of Mechanical Engineering, Georgia Institute of Technology
.
78.
Fischhoff
,
B.
, and
MacGregor
,
D.
,
1982
, “
Subjective Confidence in Forecasts
,”
J. Forecast.
,
1
(
2
), pp.
155
172
. 10.1002/for.3980010203
79.
Kattakuri
,
V.
, and
Panchal
,
J.H.
,
2019
, “
Spacecraft Failure Analysis From the Perspective of Design Decision-Making
,”
ASME 2019 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference
,
Anaheim, CA
,
Aug. 18–21
.
80.
Yaniv
,
I.
, and
Foster
,
D. P.
,
1997
, “
Precision and Accuracy of Judgmental Estimation
,”
J. Behav. Decis. Making
,
10
(
1
), pp.
21
32
. 10.1002/(SICI)1099-0771(199703)10:1<21::AID-BDM243>3.0.CO;2-G
81.
Seaver
,
D. A.
,
Von Winterfeldt
,
D.
, and
Edwards
,
W.
,
1978
, “
Eliciting Subjective Probability Distributions on Continuous Variables
,”
Organizational Behav. Human Performance
,
21
(
3
), pp.
379
391
. 10.1016/0030-5073(78)90061-2
You do not currently have access to this content.