Assessing similarity between design ideas is an inherent part of many design evaluations to measure novelty. In such evaluation tasks, humans excel at making mental connections among diverse knowledge sets and scoring ideas on their uniqueness. However, their decisions on novelty are often subjective and difficult to explain. In this paper, we demonstrate a way to uncover human judgment of design idea similarity using two dimensional idea maps. We derive these maps by asking humans for simple similarity comparisons of the form “Is idea A more similar to idea B or to idea C?” We show that these maps give insight into the relationships between ideas and help understand the domain. We also propose that the novelty of ideas can be estimated by measuring how far items are on these maps. We demonstrate our methodology through the experimental evaluations on two datasets of colored polygons (known answer) and milk frothers (unknown answer) sketches. We show that these maps shed light on factors considered by raters in judging idea similarity. We also show how maps change when less data is available or false/noisy ratings are provided. This method provides a new direction of research into deriving ground truth novelty metrics by combining human judgments and computational methods.
Skip Nav Destination
ASME 2018 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference
August 26–29, 2018
Quebec City, Quebec, Canada
Conference Sponsors:
- Design Engineering Division
- Computers and Information in Engineering Division
ISBN:
978-0-7918-5184-5
PROCEEDINGS PAPER
Unpacking Subjective Creativity Ratings: Using Embeddings to Explain and Measure Idea Novelty
Faez Ahmed,
Faez Ahmed
University of Maryland, College Park, MD
Search for other works by this author on:
Mark Fuge,
Mark Fuge
University of Maryland, College Park, MD
Search for other works by this author on:
Sam Hunter,
Sam Hunter
Pennsylvania State University, University Park, PA
Search for other works by this author on:
Scarlett Miller
Scarlett Miller
Pennsylvania State University, University Park, PA
Search for other works by this author on:
Faez Ahmed
University of Maryland, College Park, MD
Mark Fuge
University of Maryland, College Park, MD
Sam Hunter
Pennsylvania State University, University Park, PA
Scarlett Miller
Pennsylvania State University, University Park, PA
Paper No:
DETC2018-85470, V007T06A003; 12 pages
Published Online:
November 2, 2018
Citation
Ahmed, F, Fuge, M, Hunter, S, & Miller, S. "Unpacking Subjective Creativity Ratings: Using Embeddings to Explain and Measure Idea Novelty." Proceedings of the ASME 2018 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference. Volume 7: 30th International Conference on Design Theory and Methodology. Quebec City, Quebec, Canada. August 26–29, 2018. V007T06A003. ASME. https://doi.org/10.1115/DETC2018-85470
Download citation file:
33
Views
0
Citations
Related Proceedings Papers
Related Articles
In Search of Design Inspiration: A Semantic-Based Approach
J. Comput. Inf. Sci. Eng (September,2010)
The Relationship Between Design Outcomes and Mental States During Ideation
J. Mech. Des (May,2017)
Comparing Ideation Techniques for Beginning Designers
J. Mech. Des (October,2016)
Related Chapters
Processing Free Form Objects within a Product Development Process Framework
Advances in Computers and Information in Engineering Research, Volume 1
Engineering Consultancy Services
Marketing of Engineering Consultancy Services: A Global Perspective
Stimulating Creative Design Alternatives Using Customer Values
Decision Making in Engineering Design