Abstract

Traditional risk-based design processes seek to mitigate operational hazards by manually identifying possible faults and devising corresponding mitigation strategies—a tedious process which critically relies on the designer’s limited knowledge. In contrast, resilience-based design seeks to embody generic hazard-mitigating properties in the system to mitigate unknown hazards, often by modelling the system’s response to potential randomly generated hazardous events. This work creates a framework to adapt these scenario generation approaches to the traditional risk-based design process to synthetically generate fault modes by representing them as a unique combination of internal component fault states, which can then be injected and simulated in a model of system failure dynamics. Based on these simulations, the designer may then better understand the underlying failure mechanisms and mitigate them by design. The performance of this approach is evaluated in a model of an autonomous rover, where cluster analysis shows that elaborating the faulty state-space in the drive system uncovers a wider range of possible hazardous trajectories and failure consequences within each trajectory than would be uncovered from manual mode identification. However, this increase in hazard information gained from exhaustive mode sampling comes at a high computational expense, highlighting the need for advanced, efficient methods to search and sample the faulty state-space.

References

1.
Hajikazemi
,
S.
,
Ekambaram
,
A.
,
Andersen
,
B.
, and
Zidane
,
Y. J.
,
2016
, “
The Black Swan–Knowing the Unknown in Projects
,”
Procedia-Social and Behav. Sci.
,
226
(
1
), pp.
184
192
.
2.
Aven
,
T.
,
2013
, “
On the Meaning of a Black Swan in a Risk Context
,”
Saf. Sci.
,
57
(
1
), pp.
44
51
.
3.
Aven
,
T.
,
2015
, “
Implications of Black Swans to the Foundations and Practice of Risk Assessment and Management
,”
Reliab. Eng. Syst. Saf.
,
134
(
1
), pp.
83
91
.
4.
Woods
,
D. D.
,
2015
, “
Four Concepts for Resilience and the Implications for the Future of Resilience Engineering
,”
Reliab. Eng. Syst. Saf.
,
141
(
1
), pp.
5
9
.
5.
Pahl
,
G.
, and
Beitz
,
W.
,
2007
,
Engineering Design: A Systematic Approach
, 3rd ed.,
Springer-Verlag
,
London, UK
.
6.
Yodo
,
N.
, and
Wang
,
P.
,
2016
, “
Engineering Resilience Quantification and System Design Implications: A Literature Survey
,”
ASME J. Mech. Des.
,
138
(
11
), p.
111408
.
7.
MacKenzie
,
C. A.
, and
Hu
,
C.
,
2019
, “
Decision Making Under Uncertainty for Design of Resilient Engineered Systems
,”
Reliab. Eng. Syst. Saf.
,
192
(
1
), p.
106171
.
8.
Keshavarzi
,
E.
,
2018
, “
Resilient Design for Complex Engineered Systems in the Early Design Phase
,” Master’s thesis,
Oregon State University
,
Corvallis, OR
.
9.
Falco
,
G. J.
,
2015
, “
City Resilience Through Data Analytics: A Human-Centric Approach
,”
Procedia. Eng.
,
118
(
1
), pp.
1008
1014
.
10.
Garnier
,
E.
,
2019
, “
Lessons Learned From the Past for a Better Resilience to Contemporary Risks
,”
Disaster Prev. Manage.: An Int. J.
,
26
(
6
), pp.
786
803
.
11.
Wilhelm
,
B.
,
Ballesteros Cánovas
,
J. A.
,
Macdonald
,
N.
,
Toonen
,
W. H.
,
Baker
,
V.
,
Barriendos
,
M.
,
Benito
,
G.
,
Brauer
,
A.
,
Corella
,
J. P.
,
Denniston
,
R.
, and
Glaser
,
R.
,
2019
, “
Interpreting Historical, Botanical, and Geological Evidence to Aid Preparations for Future Floods
,”
Wiley Interdisciplinary Rev.: Water
,
6
(
1
), p.
e1318
.
12.
Walsh
,
H. S.
,
Dong
,
A.
,
Tumer
,
I. Y.
, and
Brat
,
G.
,
2020
, “
Detecting and Characterizing Archetypes of Unintended Consequences in Engineered Systems
,”
International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, Virtual, Online, Aug. 17–19
, Vol.
83976
,
American Society of Mechanical Engineers
, p.
V008T08A021
.
13.
Walsh
,
H. S.
,
2020
, “
The Structural Characteristics of Robustness in Large-Scale Complex Engineered Systems
,” Ph.D. thesis,
Oregon State University
,
OR
. https://ir.library.oregonstate.edu/concern/graduate˙thesis˙or˙dissertations/cj82kf537
14.
Paparistodimou
,
G.
,
Duffy
,
A.
,
Whitfield
,
R. I.
,
Knight
,
P.
, and
Robb
,
M.
,
2020
, “
A Network Tool to Analyse and Improve Robustness of System Architectures
,”
Des. Sci.
,
6
(
1
), p.
e8
.
15.
Markina-Khusid
,
A.
,
Jacobs
,
R. B.
,
Antul
,
L.
,
Cho
,
L.
, and
Tran
,
H. T.
,
2021
, “
A Complex Network Framework for Validated Assessments of Systems of Systems Robustness
,”
IEEE Syst. J.
,
16
(
1
), pp.
1092
1102
.
16.
Nafday
,
A. M.
,
2011
, “
Consequence-Based Structural Design Approach for Black Swan Events
,”
Struct. Saf.
,
33
(
1
), pp.
108
114
.
17.
Chopra
,
S. S.
,
Dillon
,
T.
,
Bilec
,
M. M.
, and
Khanna
,
V.
,
2016
, “
A Network-Based Framework for Assessing Infrastructure Resilience: A Case Study of the London Metro System
,”
J. R. Soc. Interface
,
13
(
118
), p.
20160113
.
18.
Zhang
,
D.-m.
,
Du
,
F.
,
Huang
,
H.
,
Zhang
,
F.
,
Ayyub
,
B. M.
, and
Beer
,
M.
,
2018
, “
Resiliency Assessment of Urban Rail Transit Networks: Shanghai Metro as an Example
,”
Saf. Sci.
,
106
(
1
), pp.
230
243
.
19.
Short
,
A.-R.
, and
DuPont
,
B. L.
,
2019
, “
Computational Cognition for Mission Command and Control Decisions Facing Risk in Unknown Environments
,”
International Design Engineering Technical Conferences and Computers and Information in Engineering Conference
,
Anaheim, CA
,
Aug. 18–21
, Vol.
59193
,
American Society of Mechanical Engineers
, p.
V02BT03A020
.
20.
Nguyen
,
T.
,
Wang
,
S.
,
Alhazmi
,
M.
,
Nazemi
,
M.
,
Estebsari
,
A.
, and
Dehghanian
,
P.
,
2020
, “
Electric Power Grid Resilience to Cyber Adversaries: State of the Art
,”
IEEE Access
,
8
(
1
), pp.
87592
87608
.
21.
Lee
,
R.
,
Mengshoel
,
O. J.
, and
Kochenderfer
,
M. J.
,
2019
, “Adaptive Stress Testing of Safety-Critical Systems,”
Safe, Autonomous and Intelligent Vehicles
,
H.
Yu
,
X.
Li
,
R.
Murray
, and
C.
Tomlin
, eds.,
Springer Nature
,
Cham, Switzerland
, pp.
77
95
.
22.
Kong
,
Z.
,
Guo
,
J.
,
Li
,
A.
, and
Liu
,
C.
,
2020
, “
Physgan: Generating Physical-World-Resilient Adversarial Examples for Autonomous Driving
,”
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
,
Virtual, Online
,
June 14–19
, pp.
14254
14263
.
23.
Stone
,
R. B.
,
Tumer
,
I. Y.
, and
Stock
,
M. E.
,
2005
, “
Linking Product Functionality to Historic Failures to Improve Failure Analysis in Design
,”
Res. Eng. Des.
,
16
(
1
), pp.
96
108
.
24.
Oman
,
S.
,
Koch
,
M.
,
Tumer
,
I. Y.
, and
Bohm
,
M.
,
2010
, “
Verifying the Usability of Failure-Based Computational Design Methods
,”
ASME International Mechanical Engineering Congress and Exposition
,
Vancouver, British Columbia, Canada
,
Nov. 12–18
, Vol.
44489
, pp.
329
337
.
25.
Budynas
,
R. G.
, and
Nisbett
,
J. K.
,
2011
,
Shigley’s Mechanical Engineering Design
, Vol.
9
.
McGraw-Hill
,
New York
.
26.
Bluvband
,
Z.
, and
Grabov
,
P.
,
2009
, “
Failure Analysis of FMEA
,”
2009 Annual Reliability and Maintainability Symposium
,
Fort Worth, TX
,
Jan. 26–29
,
IEEE
, pp.
344
347
.
27.
Montgomery
,
T. A.
,
Pugh
,
D. R.
,
Leedham
,
S. T.
, and
Twitchett
,
S. R.
,
1996
, “
FMEA Automation for the Complete Design Process
,”
Proceedings of 1996 Annual Reliability and Maintainability Symposium
,
Las Vegas, NV
,
Jan. 22–25 1996
,
IEEE
, pp.
30
36
.
28.
Price
,
C. J.
, and
Taylor
,
N. S.
,
2002
, “
Automated Multiple Failure FMEA
,”
Reliab. Eng. Syst. Saf.
,
76
(
1
), pp.
1
10
.
29.
Hulse
,
D.
,
Hoyle
,
C.
,
Tumer
,
I. Y.
,
Goebel
,
K.
, and
Kulkarni
,
C.
,
2020
, “
Temporal Fault Injection Considerations in Resilience Quantification
,”
International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, Virtual, Online, Aug. 17–19
, Vol.
84003
,
American Society of Mechanical Engineers
, p.
V11AT11A040
,
30.
Wang
,
J.
,
Zuo
,
W.
,
Rhode-Barbarigos
,
L.
,
Lu
,
X.
,
Wang
,
J.
, and
Lin
,
Y.
,
2019
, “
Literature Review on Modeling and Simulation of Energy Infrastructures From a Resilience Perspective
,”
Reliab. Eng. Syst. Saf.
,
183
(
1
), pp.
360
373
.
31.
Zhang
,
X.
,
Miller-Hooks
,
E.
, and
Denny
,
K.
,
2015
, “
Assessing the Role of Network Topology in Transportation Network Resilience
,”
J. Transp. Geogr.
,
46
(
1
), pp.
35
45
.
32.
Morozov
,
A.
,
Ding
,
K.
,
Steurer
,
M.
, and
Janschek
,
K.
,
2019
, “
Openerrorpro: A New Tool for Stochastic Model-Based Reliability and Resilience Analysis
,”
2019 IEEE 30th International Symposium on Software Reliability Engineering (ISSRE)
,
Berlin, Germany
,
Oct. 28–31
,
IEEE
, pp.
303
312
.
33.
Leveson
,
N.
,
Dulac
,
N.
,
Zipkin
,
D.
,
Cutcher-Gershenfeld
,
J.
,
Carroll
,
J.
, and
Barrett
,
B.
,
2017
, “Engineering Resilience into Safety-Critical Systems,”
Resilience Engineering
, 1st ed.,
E.
Hollnagel
, and
D.
Woods
, eds.,
CRC Press
,
Boca Raton, FL
, pp.
95
123
.
34.
Links
,
J. M.
,
Schwartz
,
B. S.
,
Lin
,
S.
,
Kanarek
,
N.
,
Mitrani-Reiser
,
J.
,
Sell
,
T. K.
,
Watson
,
C. R.
,
Ward
,
D.
,
Slemp
,
C.
,
Burhans
,
R.
, and
Gill
,
K.
,
2018
, “
COPEWELL: A Conceptual Framework and System Dynamics Model for Predicting Community Functioning and Resilience After Disasters
,”
Disaster Med. Public Health Preparedness
,
12
(
1
), pp.
127
137
.
35.
Miller-Hooks
,
E.
,
Zhang
,
X.
, and
Faturechi
,
R.
,
2012
, “
Measuring and Maximizing Resilience of Freight Transportation Networks
,”
Comput. Oper. Res.
,
39
(
7
), pp.
1633
1643
.
36.
Kurtoglu
,
T.
, and
Tumer
,
I. Y.
,
2008
, “
A Graph-Based Fault Identification and Propagation Framework for Functional Design of Complex Systems
,”
ASME J. Mech. Des.
,
130
(
5
), p.
051401
.
37.
McIntire
,
M. G.
,
Keshavarzi
,
E.
,
Tumer
,
I. Y.
, and
Hoyle
,
C.
,
2016
, “
Functional Models With Inherent Behavior: Towards a Framework for Safety Analysis Early in the Design of Complex Systems
,”
ASME International Mechanical Engineering Congress and Exposition
,
Phoenix, AZ
,
Nov. 11–17
, Vol.
50657
,
American Society of Mechanical Engineers
, p.
V011T15A035
.
38.
Krus
,
D.
, and
Lough
,
K. G.
,
2009
, “
Function-Based Failure Propagation for Conceptual Design
,”
AI EDAM
,
23
(
4
), pp.
409
426
.
39.
Hughes
,
N.
,
Chou
,
E.
,
Price
,
C. J.
, and
Lee
,
M. H.
,
1999
, “
Automating Mechanical FMEA Using Functional Models
,”
FLAIRS Conference
,
May 1–5
,
Orlando, FL
, pp.
394
398
.
40.
Hulse
,
D.
,
Walsh
,
H.
,
Dong
,
A.
,
Hoyle
,
C.
,
Tumer
,
I.
,
Kulkarni
,
C.
, and
Goebel
,
K.
,
2021
, “
Fmdtools: A Fault Propagation Toolkit for Resilience Assessment in Early Design
,”
Int. J. Prognostics Health Manage.
,
12
(
3
), p.
2954
.
41.
Allspaw
,
J.
,
2012
, “
Fault Injection in Production
,”
Commun. ACM
,
55
(
10
), pp.
48
52
.
42.
Natella
,
R.
,
Cotroneo
,
D.
, and
Madeira
,
H. S.
,
2016
, “
Assessing Dependability With Software Fault Injection: A Survey
,”
ACM Comput. Surv. (CSUR)
,
48
(
3
), pp.
1
55
.
43.
Soyturk
,
M. A.
,
Parasyris
,
K.
,
Salami
,
B.
,
Unsal
,
O.
,
Yalcin
,
G.
, and
Gomez
,
L. B.
,
2019
, “
Hardware Versus Software Fault Injection of Modern Undervolted SRAMs
,”
Computing Research Repository
, arxiv preprint. https://arxiv.org/abs/1912.00154
44.
Goldstein
,
B.
,
Srinivasan
,
S.
,
Mellempudi
,
N. K.
,
Das
,
D.
,
Santiago
,
L.
,
Ferreira
,
V. C.
,
Solon
,
N.
,
Kundu
,
S.
, and
França
,
F. M. G.
,
2020
, “
Reliability Evaluation of Compressed Deep Learning Models
,”
2020 IEEE 11th Latin American Symposium on Circuits Systems (LASCAS)
,
San José, Costa Rica
,
Feb. 25–28
, pp.
1
5
.
45.
Georgakoudis
,
G.
,
Laguna
,
I.
,
Vandierendonck
,
H.
,
Nikolopoulos
,
D. S.
, and
Schulz
,
M.
,
2019
, “
SAFIRE: Scalable and Accurate Fault Injection for Parallel Multithreaded Applications
,”
2019 IEEE International Parallel and Distributed Processing Symposium (IPDPS)
,
Rio de Janeiro, Brazil
,
May 20–24
,
IEEE
, pp.
890
899
.
46.
Engelmann
,
C.
, and
Naughton
,
T.
,
2013
, “
Toward a Performance/Resilience Tool for Hardware/Software Co-Design of High-Performance Computing Systems
,”
2013 42nd International Conference on Parallel Processing
,
Lyon, France
,
Oct. 1–4
,
IEEE
, pp.
960
969
.
47.
Martins
,
R.
,
Gandhi
,
R.
,
Narasimhan
,
P.
,
Pertet
,
S.
,
Casimiro
,
A.
,
Kreutz
,
D.
, and
Veríssimo
,
P.
,
2013
, “
Experiences With Fault-Injection in A Byzantine Fault-Tolerant Protocol
,”
Acm/ifip/usenix International Conference on Distributed Systems Platforms and Open Distributed Processing
,
Beijing, China
,
Dec. 9–13
,
Springer
, pp.
41
61
.
48.
Zhang
,
L.
,
Morin
,
B.
,
Haller
,
P.
,
Baudry
,
B.
, and
Monperrus
,
M.
,
2019
, “
A Chaos Engineering System for Live Analysis and Falsification of Exception-Handling in the JVM
,”
IEEE Trans. Softw. Eng.
,
47
(
11
), pp.
2534
2548
.
49.
Snooke
,
N. A.
,
2015
, “Automated Failure Effect Analysis for PHM of UAV,”
Handbook of Unmanned Aerial Vehicles
, 1st ed.,
K.
Valavanis
, and
G.
Vachtsevanos
, eds.,
Springer Dordrecht
,
Dordrecht, The Netherlands
, pp.
1027
1051
.
50.
Struss
,
P.
,
2006
, “
A Model-Based Methodology for the Integration of Diagnosis and Fault Analysis During the Entire Life Cycle
,”
IFAC Proc. Vol.
,
39
(
13
), pp.
1157
1162
.
51.
Liggesmeyer
,
P.
, and
Rothfelder
,
M.
,
1998
, “
Improving System Reliability With Automatic Fault Tree Generation
,”
Digest of Papers. Twenty-Eighth Annual International Symposium on Fault-Tolerant Computing (Cat. No. 98CB36224)
,
Munich, Germany
,
June 23–25
,
IEEE
, pp.
90
99
.
52.
Papazoglou
,
I. A.
,
1998
, “
Functional Block Diagrams and Automated Construction of Event Trees
,”
Reliab. Eng. Syst. Saf.
,
61
(
3
), pp.
185
214
.
53.
Sen
,
D. K.
,
Banks
,
J. C.
,
Maggio
,
G.
, and
Railsback
,
J.
,
2006
, “
Rapid Development of an Event Tree Modeling Tool Using COTS Software
,”
2006 IEEE Aerospace Conference
,
Big Sky, MT
,
Mar. 4–11
,
IEEE
, pp.
8
.
54.
Mercurio
,
D.
,
Podofillini
,
L.
,
Zio
,
E.
, and
Dang
,
V.
,
2009
, “
Identification and Classification of Dynamic Event Tree Scenarios Via Possibilistic Clustering: Application to a Steam Generator Tube Rupture Event
,”
Accid. Anal. Prev.
,
41
(
6
), pp.
1180
1191
.
55.
Smith
,
C.
,
Knudsen
,
J.
,
Kvarfordt
,
K.
, and
Wood
,
T.
,
2008
, “
Key Attributes of the SAPHIRE Risk and Reliability Analysis Software for Risk-Informed Probabilistic Applications
,”
Reliab. Eng. Syst. Saf.
,
93
(
8
), pp.
1151
1164
.
56.
Irshad
,
L.
,
Demirel
,
H. O.
, and
Tumer
,
I. Y.
,
2020
, “
Automated Generation of Fault Scenarios to Assess Potential Human Errors and Functional Failures in Early Design Stages
,”
J. Comput. Inf. Sci. Eng.
,
20
(
5
), p.
051009
.
57.
Papakonstantinou
,
N.
,
Sierla
,
S.
,
O’Halloran
,
B.
, and
Tumer
,
I. Y.
,
2013
, “
A Simulation Based Approach to Automate Event Tree Generation for Early Complex System Designs
,”
International Design Engineering Technical Conferences and Computers and Information in Engineering Conference
,
Portland, OR
,
Aug. 4–7
, Vol.
55867
,
American Society of Mechanical Engineers
, p.
V02BT02A008
.
58.
Nejad
,
H.
, and
Mosleh
,
A.
,
2005
, “
Automated Risk Scenario Generation Using System Functional and Structural Knowledge
,”
ASME 2005 International Mechanical Engineering Congress and Exposition
,
Orlando, FL
,
Nov. 5–11
,
American Society of Mechanical Engineers
, pp.
85
89
.
59.
Hilf
,
K.-D.
,
Matheis
,
I.
,
Mauss
,
J.
, and
Rauh
,
J.
,
2010
, “
Automated Simulation of Scenarios to Guide the Development of a Crosswind Stabilization Function
,”
IFAC Proc. Vol.
,
43
(
7
), pp.
768
772
.
60.
Junghanns
,
A.
,
Mauss
,
J.
, and
Tatar
,
M.
,
2008
, “
Testweaver: A Tool for Simulation-Based Test of Mechatronic Designs
,”
6th International Modelica Conference
,
Mar. 3
,
Bielefeld, Germany
,
Citeseer
, pp.
341
348
.
61.
Cunning
,
S. J.
, and
Rozenblit
,
J. W.
,
2005
, “
Automating Test Generation for Discrete Event Oriented Embedded Systems
,”
J. Intell. Rob. Syst.
,
41
(
2
), pp.
87
112
.
62.
Ester
,
M.
,
Kriegel
,
H.-P.
,
Sander
,
J.
, and
Xu
,
X.
,
1996
, “
A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases With Noise
,” Portland, OR, Aug. 2–4, Vol. 96, Paper No. 34, pp.
226
231
.
63.
Pedregosa
,
F.
,
Varoquaux
,
G.
,
Gramfort
,
A.
,
Michel
,
V.
,
Thirion
,
B.
,
Grisel
,
O.
,
Blondel
,
M.
,
Prettenhofer
,
P.
,
Weiss
,
R.
,
Dubourg
,
V.
, and
Vanderplas
,
J.
,
2011
, “
Scikit-Learn: Machine Learning in Python
,”
J. Mach. Learn. Res.
,
12
(
85
), pp.
2825
2830
.
64.
Hulse
,
D.
,
Hoyle
,
C.
,
Tumer
,
I. Y.
, and
Goebel
,
K.
,
2021
, “
How Uncertain Is Too Uncertain? Validity Tests for Early Resilient and Risk-Based Design Processes
,”
ASME J. Mech. Des.
,
143
(
1
), p.
011702
.
You do not currently have access to this content.