Thanks to recent advances in computing power and speed, engineers can now generate a wealth of data on demand to support design decision-making. These advances have enabled new approaches to search multidimensional trade spaces through interactive data visualization and exploration. In this paper, we investigate the effectiveness and efficiency of interactive trade space exploration strategies by conducting human subject experiments with novice and expert users. A single objective, constrained design optimization problem involving the sizing of an engine combustion chamber is used for this study. Effectiveness is measured by comparing the best feasible design obtained by each user, and efficiency is assessed based on the percentage of feasible designs generated by each user. Results indicate that novices who watch a 5-min training video before the experiment obtain results that are not significantly different from those obtained by expert users, and both groups are statistically better than the novices without the training video in terms of effectiveness and efficiency. Frequency and ordering of the visualization and exploration tools are also compared to understand the differences in each group’s search strategy. The implications of the results are discussed along with future work.
References
1.
Wang
, G. G.
, and Shan
, S.
, 2007, “Review of Metamodeling Techniques in Support of Engineering Design Optimization
,” ASME J. Mech. Des.
, 129
(4
), pp. 370
–380
.2.
Papalambros
, P. Y.
, and Wilde
, D. J.
, 2000. Principles of Optimal Design: Modeling and Computation
, Cambridge University
, New York
.3.
Balling
, R.
, 1999, “Design by Shopping: A New Paradigm?
,” Proceedings of the Third World Congress of Structural and Multidisciplinary Optimization (WCSMO-3)
, University at Buffalo
, Buffalo, NY
, pp. 295
–297
.4.
Stump
, G.
, Lego
, S.
, Yukish
, M.
, Simpson
, T. W.
, and Donndelinger
, J. A.
, 2009, “Visual Steering Commands for Trade Space Exploration: User-Guided Sampling With Example
,” ASME J. Comput. Inf. Sci. Eng.
, 9
(4
), p. 044501
.5.
Winer
, E. H.
, and Bloebaum
, C. L.
, 2001, “Visual Design Steering for Optimization Solution Improvement
,” Struct. Optim.
, 22
(3
), pp. 219
–229
.6.
Kesavadas
, T.
, and Sudhir
, A.
, 2000, “Computational Steering in Simulation of Manufacturing Systems
,” Proceedings of the 2000 IEEE International Conference on Robotics and Automation
, IEEE
, San Francisco, CA
, pp. 2654
–2658
.7.
Carlsen
, D.
, Malone
, M.
, Kollat
, J.
, and Simpson
, T. W.
, 2008, “Evaluating the Performance of Visual Steering Commands for User-Guided Pareto Frontier Sampling During Trade Space Exploration
,” ASME Design Engineering Technical Conferences - Design Automation Conference
, ASME
, New York
, Paper No. DETC2008/DAC-49681.8.
Stump
, G. M.
, Yukish
, M.
, Simpson
, T. W.
, and Bennett
, L.
, 2002, “Multidimensional Visualization and Its Application to a Design by Shopping Paradigm
,” AIAA Paper No. 2002
–5622
.9.
Stump
, G.
, Yukish
, M.
, Simpson
, T. W.
, and Harris
, E. N.
, 2003, “Design Space Visualization and Its Application to a Design by Shopping Paradigm
,” ASME Design Engineering Technical Conferences - Design Automation Conference
, ASME
, Chicago, IL
, Paper No. DETC2003/DAC-48785.10.
Simpson
, T. W.
, Spencer
, D. B.
, and Yukish
, M. A.
, 2008, “Visual Steering Commands and Test Problems to Support Research in Trade Space Exploration
,” AIAA Paper No. 2008–6085.11.
Seo
, J.
, and Shneiderman
, B.
, 2005, “A Rank-By-Feature Framework for Interactive Exploration of Multidimensional Data
,” Inf. Visualization
, 4
, pp. 96
–113
.12.
Klein
, G.
, 1998, Sources of Power: How People Make Decisions
, MIT
, Cambridge, MA
.13.
Randel
, J. M.
, Pugh
, H. L.
, and Reed
, S. K.
, 1996, “Differences in Expert and Novice Situation Awareness in Naturalistic Decision Making
,” Int. J. Hum.-Comput. Stud.
, 45
, pp. 579
–597
.14.
Jiang
, X.
, Gramopadhye
, A. K.
, and Melloy
, B. J.
, 2004, “Theoretical Issues in the Design of Visual Inspection Systems
,” Theor. Issues Ergon. Sci.
, 5
(3
), pp. 232
–247
.15.
Petre
, M.
, and Green
, T. R. G.
, 1993, “Learning to Read Graphics: Some Evidence That ‘Seeing’ an Information Display is an Acquired Skill
,” J. Visual Lang. Comput.
, 4
, pp. 55
–70
.16.
Zhu
, Y.
, 2007, “Measuring Effective Data Visualization
,” Advance in Visual Computing
, Springer
, Berlin
, pp. 652
–661
.17.
Simpson
, T. W.
, Spencer
, D. B.
, and Yukish
, M. A.
, 2008, “Visual Steering Commands and Test Problems to Support Research in Trade Space Exploration
,” 12th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference
, Victoria, British Columbia, Canada, AIAA
, September 10–12, Paper No. AIAA-2008-6085.18.
Orasanu
, J.
, and Connoly
, T.
, 1993, “The Reinvention of Decision Making
,” Decision Making in Action: Models and Methods
, G. A.
Klein
, J.
Orasanu
, R.
Calderwood
, and C. E.
Zsambok
, eds., Ablex Publishing
, Norwood, CT
, pp. 3
–20
.19.
Gilmour
, P.
, and Corner
, J.
, 1998, “The Role of the Expert’s Decision Making Skills in Management
,” 33rd Annual Operational Research Society of New Zealand
, Ablex Publishing
, Auckland
, pp. 195
–204
.20.
Bouwman
, M. J.
, 1984, “Expert vs Novice Decision Making in Accounting: A Summary
,” Account., Organ., Soc.
, 9
(3/4
), pp. 325
–327
.21.
Anderson
, J. R.
, 1982, “Acquisition of Cognitive Skill
,” Psychol. Rev.
, 89
, pp. 369
–406
.22.
Ritter
, F. E.
, and Bibby
, P. A.
, 2008, “Modeling How, When, and What is Learned in a Simple Fault-Finding Task
,” Cogn. Sci.
, 32
, pp. 862
–892
.23.
Wiedenbeck
, S.
, 1986, “Organization of Programming Knowledge of Novices and Experts
,” J. Am. Soc. Inf. Sci.
, 37
(5
), pp. 284
–299
.24.
Jones
, D. G.
, and Endsley
, M. R.
, 1996, “Sources of Situation Awareness Errors in Aviation
,” Aviat., Space Environ. Med.
, 67
(6
), pp. 507
–512
.25.
Shneiderman
, B.
, 1998, Designing the User Interface: Strategies for Effective Human-Computer Interaction
, Addison-Wesley, Reading.26.
Buja
, A.
, McDonald
, J. A.
, Michalak
, J.
, and Stuetzle
, W.
, 1991, “Interactive Data Visualization Using Focusing and Linking
,” Proceedings of the IEEE Conference on Visualization
, IEEE Society
, San Diego, CA
, pp. 156
–163
.27.
Becker
, R. A.
, and Cleveland
, W. S.
, 1987, “Brushing Scatterplots
,” Technometrics
, 29
(1
), pp. 127
–142
.28.
Stump
, G. M.
, Yukish
, M. A.
, Martin
, J. D.
, and Simpson
, T. W.
, 2004, “The ARL Trade Space Visualizer: An Engineering Decision-Making Tool
,” 10th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference
, AIAA
, Albany, NY
.29.
Law
, A. M.
, and Kelton
, W. D.
, 2000, Simulation Modeling and Analysis
, McGraw-Hill
, Boston, MA
.30.
Price
, K.
, Storn
, R.
, and Lampinen
, J.
, 2005, Differential Evolution - A Practical Approach to Global Optimization
, Springer-Verlag
, Berlin
.31.
Robic
, T.
, and Filipic
, B.
, 2005, “DEMO: Evolution for Multiobjective Optimization
,” Third International Conference on Evolutionary Multi-Criterion Optimization
, Springer
, Guanajuato, Mexico
, pp. 520
–533
.32.
Stump
, G. M.
, Simpson
, T. W.
, Yukish
, M.
, and O’Hara
, J. J.
, 2004, “Trade Space Exploration of Satellite Datasets Using a Design by Shopping Paradigm
,” 2004 IEEE Aerospace Conference Proceedings
, IEEE
, Big Sky, MT
.33.
Neilson
, J.
, and Mack
, R. L.
, 1994, Usability Inspection Methods
, Wiley
, New York
.34.
Wagner
, T. C.
, and Papalambros
, P. Y.
, 1991, “Optimal Engine Design Using nonlinear Programming and the Engine System Assessment Model
,” Ford Motor Company, Technical Report No. SR-91-154.35.
McAllister
, C. D.
, and Simpson
, T. W.
, 2001, “Multidisciplinary Robust Design Optimization of an Internal Combustion Engine
,” ASME J. Mech. Des.
, 125
(1
), pp. 124
–130
.36.
ANSI/ASME PTC 19.1-1998, Measurement Uncertainty, Part I, ASME, New York, NY.
37.
Wolf
, D.
, Simpson
, T. W.
, and Zhang
, X.
, 2009, “A Preliminary Study of Novice and Expert Users’ Decision-Making Procedures During Visual Trade Space Exploration
,” ASME Design Engineering Technical Conferences—Design Automation Conference
, ASME
, San Diego, CA
, Paper No. DETC2009/DAC-87294.38.
Devore
, J. L.
, 1995, Probability and Statistics for Engineering and the Sciences
, Wadsworth
, Belmont, CA
.39.
Booth
, T.
, 1967, Sequential Machines and Automata Theory
, Wiley
, New York
.40.
Wolf
, D. R.
, 2009, “An Assessment of Novice Users’ Decision-Making Strategies During Visual Trade Sapce Exploration
,” M.Sc. Thesis, Mechanical Engineering, Penn State University, University Park.41.
Wu
, J.
, and Azarm
, S.
, 2001, “Metrics for Quality Assessment of a Multiobjective Design Optimization Solution Set
,” ASME J. Mech. Des.
, 123
(1
), pp. 18
–25
.Copyright © 2011
by American Society of Mechanical Engineers
You do not currently have access to this content.
