Abstract

Additive manufacturing (AM) provides design flexibility and allows rapid fabrications of parts with complex geometries. The presence of internal defects, however, can lead to the deficit performance of the fabricated part. X-ray computed tomography (XCT) is a nondestructive inspection technique often used for AM parts. Although defects within AM specimens can be identified and segmented by manually thresholding the XCT images, the process can be tedious and inefficient, and the segmentation results can be ambiguous. The variation in the shapes and appearances of defects also poses difficulty in accurately segmenting defects. This article describes an automatic defect segmentation method using U-Net-based deep convolutional neural network (CNN) architectures. Several models of U-Net variants are trained and validated on an AM XCT image dataset containing pores and cracks, achieving a best mean intersection over union (IOU) value of 0.993. The performance of various U-Net models is compared and analyzed. Specific to AM porosity segmentation with XCT images, several techniques in data augmentation and model development are introduced. This article demonstrates that U-Net can be effectively applied for automatic segmentation of AM porosity from XCT images with high accuracy. The method can potentially help improve the quality control of AM parts in an industry setting.

References

1.
Gibson
,
I.
,
Rosen
,
D.
, and
Stucker
,
B.
,
2010
,
Additive Manufacturing Technologies
,
Springer
,
New York
.
2.
Ngo
,
T. D.
,
Kashani
,
A.
,
Imbalzano
,
G.
,
Nguyen
,
K. T. Q.
, and
Hui
,
D.
,
2018
, “
Additive Manufacturing 3D Printing: A Review of Materials, Methods, Applications and Challenges
,”
Composites, Part B
,
143
, pp.
172
196
.
3.
Leung
,
Y.
,
Kwok
,
T.
,
Li
,
X.
,
Yang
,
Y.
,
Wang
,
C. C. L.
, and
Chen
,
Y.
,
June 2019
, “
Challenges and Status on Design and Computation for Emerging Additive Manufacturing Technologies
,”
ASME J. Comput. Inf. Sci. Eng.
,
19
(
2
), p.
021013
.
4.
Reese
,
R.
,
Bheda
,
H.
, and
Mondesir
,
W.
,
2016
, “
Method to Monitor Additive Manufacturing Process for Detection and In-Situ Correction of Defects
,” Pub. No.: US 2016/0271610 A1 Patent Application Publication.
5.
Wu
,
H.
,
Wang
,
Y.
, and
Yu
,
Z.
,
2016
, “
In Situ Monitoring of FDM Machine Condition via Acoustic Emission
,”
Int. J. Adv. Manuf. Technol.
,
84
(
5–8
), pp.
1483
1495
.
6.
Faes
,
M.
,
Abbeloos
,
W.
,
Vogeler
,
F.
,
Valkenaers
,
H.
,
Coppens
,
K.
,
Goedemé
,
T.
, and
Ferraris
,
E.
,
2014
, “
Process Monitoring of Extrusion Based 3D Printing via Laser Scanning
,”
International Conference on Polymers and Moulds Innovations (PMI)
,
Guimaraes, Portugal
,
Sept. 10–12
.
7.
Rao
,
P. K.
,
Liu
,
J. P.
,
Roberson
,
D.
, and
Kong
,
Z. J.
,
2015
, “
Sensor-Based Online Process Fault Detection in Additive Manufacturing
,”
ASME 2015 International Manufacturing Science and Engineering Conference
,
Guangzhou, China
,
Nov. 28–29
, p. V002T04A010.
8.
Buffière
,
J.-Y.
,
Savelli
,
S.
,
Jouneau
,
P. H.
,
Maire
,
E.
, and
Fougères
,
R.
,
2001
, “
Experimental Study of Porosity and Its Relation to Fatigue Mechanisms of Model Al–Si7–Mg0.3 Cast Al Alloys
,”
Mater. Sci. Eng. A
,
316
(
1–2
), pp.
115
126
.
9.
Guo
,
Y.
,
Liu
,
Y.
,
Georgiou
,
T.
, and
Lew
,
M. S.
,
2018
, “
A Review of Semantic Segmentation Using Deep Neural Networks
,”
Int. J. Multimed. Inf. Retr.
,
7
(
2
), pp.
87
93
.
10.
He
,
K.
,
Gkioxari
,
G.
,
Dollár
,
P.
, and
Girshick
,
R.
,
2017
, “
Mask R-CNN
,”
2017 IEEE International Conference on Computer Vision (ICCV)
,
Venice, Italy
,
Oct. 22–29
, pp.
2980
2988
.
11.
Pesaresi
,
M.
, and
Benediktsson
,
J. A.
,
2001
, “
A New Approach for the Morphological Segmentation of High-Resolution Satellite Imagery
,”
IEEE Trans. Geosci. Remote Sens.
,
39
(
2
), pp.
309
320
.
12.
Bhatt
,
P. M.
,
Malhan
,
R. K.
,
Rajendran
,
P.
,
Shah
,
B. C.
,
Thakar
,
S.
,
Yoon
,
Y. J.
, and
Gupta
,
S. K.
,
2021
, “
Image-Based Surface Defect Detection Using Deep Learning: A Review
,”
ASME J. Comput. Inf. Sci. Eng.
,
21
(
4
), p.
040801
.
13.
Nie
,
Z.
,
Jiang
,
H.
, and
Kara
,
L. B.
,
February 2020
, “
Stress Field Prediction in Cantilevered Structures Using Convolutional Neural Networks
,”
ASME J. Comput. Inf. Sci. Eng.
,
20
(
1
), p.
011002
.
14.
Ferguson
,
M.
,
Ak
,
R.
,
Lee
,
Y.-T. T.
, and
Law
,
K. H.
,
2018
, “
Detection and Segmentation of Manufacturing Defects With Convolutional Neural Networks and Transfer Learning
,”
Smart Sustain. Manuf. Syst.
,
2
(
1
), pp.
137
164
.
15.
Liu
,
T.
,
Bao
,
J.
,
Wang
,
J.
, and
Zhang
,
Y.
,
2020
, “
A Coarse-Grained Regularization Method of Convolutional Kernel for Molten Pool Defect Identification
,”
ASME J. Comput. Inf. Sci. Eng.
,
20
(
2
), p.
021005
.
16.
Milletari
,
F.
,
Navab
,
N.
, and
Ahmadi
,
S.-A.
,
2016
, “
V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation
,”
IEEE International Conference on 3D Vision
,
Stanford, CA
,
Oct. 25–28
, pp.
565
571
.
17.
Cicek
,
O.
,
Abdulkadir
,
A.
,
Lienkamp
,
S. S.
,
Brox
,
T.
, and
Ronneberger
,
O.
,
2016
, “
3D U-Net: Learning Dense Volumetric Segmentation From Sparse Annotation
,”
International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI)
,
Athens, Greece
,
Oct. 17–21
, pp.
424
432
.
18.
Lee
,
K.
,
Zung
,
J.
,
Li
,
P.
,
Jain
,
V.
, and
Seung
,
H. S.
,
2017
, “
Superhuman Accuracy on the SNEMI3D Connectomics Challenge
,”
arXiv Preprint
19.
Singh
,
S. P.
,
Wang
,
L.
,
Gupta
,
S.
,
Goli
,
H.
,
Padmanabhan
,
P.
, and
Gulyás
,
B.
,
2020
, “
3D Deep Learning on Medical Images: A Review
,”
Sensors
,
20
(
18
), p.
5097
.
20.
Henry
,
T.
,
Carre
,
A.
,
Lerousseau
,
M.
,
Estienne
,
T.
,
Robert
,
C.
,
Paragios
,
N.
, and
Deutsch
,
E.
,
2020
, “
Top 10 BraTS 2020 Challenge Solution: Brain Tumor Segmentation With Self-Ensembled, Deeply-Supervised 3D-Unet Like Neural Networks
,” arXiv Preprint arXiv:2011.01045.
21.
Wang
,
J.
,
Bao
,
Y.
,
Wen
,
Y.
,
Lu
,
H.
,
Luo
,
H.
,
Xiang
,
Y.
,
Li
,
X.
,
Liu
,
C.
, and
Qian
,
D.
,
2020
, “
Prior-Attention Residual Learning for More Discriminative COVID-19 Screening in CT Images
,”
IEEE Trans. Med. Imaging
,
39
(
8
), pp.
2572
2583
.
22.
Siddique
,
N.
,
Paheding
,
S.
,
Elkin
,
C. P.
, and
Devabhaktuni
,
V.
,
2021
, “
U-Net and Its Variants for Medical Image Segmentation: A Review of Theory and Applications
,”
IEEE Access
,
9
, pp.
82031
82057
.
23.
Wong
,
V. W. H.
,
Ferguson
,
M.
,
Law
,
K. H.
,
Lee
,
Y.-T. T.
, and
Witherell
,
P.
,
2020
, “
Automatic Volumetric Segmentation of Additive Manufacturing Defects With 3D U-Net
,”
AAAI 2020 Spring Symposia
,
Stanford, CA
,
Mar. 23–25
, arXiv Preprint arXiv:2101.08993.
24.
Wong
,
V. W. H.
,
Ferguson
,
M.
,
Law
,
K. H.
,
Lee
,
Y.-T. T.
, and
Witherell
,
P.
,
2021
, “
Segmentation of Additive Manufacturing Defects Using U-Net
,”
Proceedings of the ASME 2021 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference IDETC/CIE 2021
,
Virtual
,
Aug. 17–19
.
25.
Caesar
,
H.
,
Uijlings
,
J.
, and
Ferrari
,
V.
,
2016
, “
Region-Based Semantic Segmentation With End-to-End Training
,”
The 14th European Conference on Computer Vision (ECCV)
,
Amsterdam, The Netherlands
,
Oct. 8–16
.
26.
Long
,
J.
,
Shelhamer
,
E.
, and
Darrell
,
T.
,
2015
, “
Fully Convolutional Networks for Semantic Segmentation
,”
IEEE Conference on Computer Vision and Pattern Recognition
,
Boston, MA
,
June 7–12
, pp.
3431
3440
.
27.
Ronneberger
,
O.
,
Fischer
,
P.
, and
Brox
,
T.
,
2015
, “
U-Net: Convolutional Networks for Biomedical Image Segmentation
,”
International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI)
,
Munich, Germany
,
Oct. 5–9
, pp.
234
241
.
28.
Yu
,
Q.
,
Xia
,
Y.
,
Xie
,
L.
,
Fishman
,
E. K.
, and
Yuille
,
A. L.
,
2019
, “
Thickened 2D Networks for 3D Medical Image Segmentation
,” arXiv Preprint arXiv 1904.01150.
29.
Isensee
,
F.
,
Jaeger
,
P. F.
,
Full
,
P. M.
,
Wolf
,
I.
,
Engelhardt
,
S.
, and
Maier-Hein
,
K. H.
,
2017
, “
Automatic Cardiac Disease Assessment on Cine-MRI via Time-Series Segmentation and Domain Specific Features
,”
The 8th International Statistical Atlases and Computational Modeling of the Heart (STACOM) Workshop
,
Quebec City, Canada
,
Sept. 10–14
, pp.
120
129
.
30.
Zhang
,
B.
,
Liu
,
S.
, and
Shin
,
Y. C.
,
2019
, “
In-Process Monitoring of Porosity During Laser Additive Manufacturing Process
,”
Addit. Manuf.
,
28
, pp.
497
505
.
31.
Shevchik
,
S. A.
,
Kenel
,
C.
,
Leinenbach
,
C.
, and
Wasmer
,
K.
,
2018
, “
Acoustic Emission for in Situ Quality Monitoring in Additive Manufacturing Using Spectral Convolutional Neural Networks
,”
Addit. Manuf.
,
21
, pp.
598
604
.
32.
Mutiargo
,
B.
,
Pavlovic
,
M.
,
Malcolm
,
A. A.
,
Goh
,
B.
,
Krishnan
,
M.
,
Shota
,
T.
,
Shaista
,
H.
,
Jhinaoui
,
A.
, and
Putro
,
M. I. S.
,
2019
, “
Evaluation of X-Ray Computed Tomography (CT) Images of Additively Manufactured Components Using Deep Learning
,”
3rd Singapore International Non-Destructive Testing Conference and Exhibition (SINCE2019)
,
Singapore
,
Dec. 4–5
.
33.
Kim
,
F. H.
,
Moylan
,
S. P.
,
Garboczi
,
E. J.
, and
Slotwinski
,
J. A.
,
2017
, “
Investigation of Pore Structure in Cobalt Chrome Additively Manufactured Parts Using X-Ray Computed Tomography and Three-Dimensional Image Analysis
,”
Addit. Manuf.
,
17
, pp.
23
38
.
34.
Kim
,
F. H.
,
Moylan
,
S. P.
,
Garboczi
,
E. J.
, and
Slotwinski
,
J. A.
,
2019
,
High-Resolution X-Ray Computed Tomography (XCT) Image Data Set of Additively Manufactured Cobalt Chrome Samples Produced With Varying Laser Powder Bed Fusion Processing Parameters, CoCr AM XCT Data
,
National Institute of Standards and Technology
,
Gaithersburg, MD
.
35.
Buades
,
A.
,
Coll
,
B.
, and
Morel
,
J.-M.
,
2011
, “
Non-Local Means Denoising
,”
Image Process. Line
,
1
, pp.
208
212
.
36.
Sun
,
W.
,
Brown
,
S. B.
, and
Leach
,
R. K.
,
2012
,
An Overview of Industrial X-Ray Computed Tomography, Technical Report ENG 32
,
National Physical Laboratory
,
Teddington, Middlesex, UK
.
37.
Bernsen
,
J.
,
1986
, “
Dynamic Thresholding of Gray-Level Images
,”
8th International. Conference on Pattern Recognition
,
Paris, France
,
Oct. 27–31
, pp.
1251
1255
.
38.
Ioffe
,
S.
, and
Szegedy
,
C.
,
2015
, “
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
,”
32nd International Conference on Machine Learning
,
Lille, France
,
July 6–11
.
39.
Wu
,
Y.
, and
He
,
K.
,
2020
, “
Group Normalization
,”
Int. J. Comput. Vis.
,
128
, pp.
742
755
.
40.
Ulyanov
,
D.
,
Vedaldi
,
A.
, and
Lempitsky
,
V.
,
2016
, “
Instance Normalization: The Missing Ingredient for Fast Stylization
,” arXiv Preprint arXiv:1607.08022.
41.
Werbos
,
P. J.
,
1990
, “
Backpropagation Through Time: What It Does and How to Do It
,”
Proc. IEEE
,
78
(
10
), pp.
1550
1560
.
42.
Rakhlin
,
A.
,
Davydow
,
A.
, and
Nikolenko
,
S. I.
,
2018
, “
Land Cover Classification From Satellite Imagery With U-Net and Lovasz-Softmax Loss
,”
2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
,
Salt Lake City, UT
,
June 18–22
.
43.
Diakogiannis
,
F. I.
,
Waldner
,
F.
,
Caccetta
,
P.
, and
Wu
,
C.
,
2020
, “
ResUNet-a: A Deep Learning Framework for Semantic Segmentation of Remotely Sensed Data
,”
ISPRS J. Photogramm. Remote Sens.
,
162
, pp.
94
114
.
44.
He
,
K.
,
Zhang
,
X.
,
Ren
,
S.
, and
Sun
,
J.
,
2016
, “
Deep Residual Learning for Image Recognition
,”
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
,
Las Vegas, NV
,
June 26–July 1
, pp.
770
778
.
45.
Hinton
,
G.
,
2012
, “
Neural Networks for Machine Learning—Lecture 6a—Overview of Mini-Batch Gradient Descent
,” https://www.cs.toronto.edu/~tijmen/csc321/slides/lecture_slides_lec6.pdf, Accessed July 2020.
46.
GitHub, Inc.
, “
UNet: Semantic Segmentation With PyTorch
,” https://github.com/milesial/Pytorch-UNet, Accessed November 2020.
47.
Kingma
,
D.
, and
Ba
,
J.
,
2015
, “
Adam: A Method for Stochastic Optimization
,”
3rd International Conference for Learning Representations (ICLR)
,
San Diego, CA
,
May 7–9
.
48.
Wolny
,
A.
,
2019
, “
Pytorch-3DUnet: PyTorch Implementation of 3D U-Net
,”
Zenodo
.
49.
Isensee
,
F.
,
Jäger
,
P. F.
,
Kohl
,
S. A. A.
,
Petersen
,
J.
, and
Maier-Hein
,
K. H.
,
2020
, “
nnU-Net: A Self-Configuring Method For Deep Learning-Based Biomedical Image Segmentation
,”
Nature Methods
,
18
, pp.
203
211
.
50.
Drozdzal
,
M.
,
Vorontsov
,
E.
,
Chartrand
,
G.
,
Kadoury
,
S.
, and
Pal
,
C.
,
2016
, “
The Importance of Skip Connections in Biomedical Image Segmentation
,”
Deep Learn. Data Labeling Med. Appl.
,
10008
, pp.
179
187
.
51.
Khened
,
M.
,
Kollerathu
,
V. A.
, and
Krishnamurthi
,
G.
,
2019
, “
Fully Convolutional Multi-Scale Residual DenseNets for Cardiac Segmentation and Automated Cardiac Diagnosis Using Ensemble of Classifiers
,”
Med. Image Anal.
,
51
, pp.
21
45
.
52.
He
,
K.
,
Zhang
,
X.
,
Ren
,
S.
, and
Sun
,
J.
,
2015
, “
Delving Deep Into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
,”
Proceedings of the IEEE International Conference on Computer Vision
,
Santiago, Chile
,
Dec. 7–13
, pp.
1026
1034
.
53.
Yurii
,
N.
,
2004
,
Introductory Lectures on Convex Optimization: A Basic Course
, 1,
Springer Science & Business Media
,
Boston, MA
.
54.
Oktay
,
O.
,
Schlemper
,
J.
,
Folgoc
,
L. L.
,
Lee
,
M.
,
Heinrich
,
M.
,
Misawa
,
K.
,
Mori
,
K.
,
McDonagh
,
S.
,
Hammerla
,
N. Y.
,
Kainz
,
B.
,
Glocker
,
B.
, and
Rueckert
,
D.
,
2018
, “
Attention U-Net: Learning Where to Look for the Pancreas
,”
Medical Imaging with Deep Learning (MIDL)
,
Amsterdam, The Netherlands
,
July 4–6
.
You do not currently have access to this content.