Abstract

3D modeling accurately depicts the physical world but typically requires substantial data acquisition resources and significant storage space. We introduce a novel three-dimensional slice-reconstruction model (3DSR) to address these challenges. This 3D data super-resolution model leverages low-resolution 3D data as input to generate high-resolution results promptly and accurately, reducing the time and storage required to create detailed 3D models. To enhance the computational efficiency and accuracy of deep learning models, the 3D data are partitioned into multiple slices. The 3DSR processes each slice into a high-resolution 2D image, which is then reassembled into high-resolution 3D data. Our slice-up method and slice-reconstruction technique are specifically designed to preserve the primary characteristics of the 3D data. We employ a pre-trained deep 2D convolutional neural network to expand the resolution of the 2D image, resulting in excellent performance. This approach reduces the time required for training deep learning models and enhances computational efficiency during the resolution improvement process. Importantly, our model can deliver superior performance even when trained on fewer data.

References

1.
Wang
,
W.
,
Huang
,
Q.
,
You
,
S.
,
Yang
,
C.
, and
Neumann
,
U.
,
2017
, “
Shape Inpainting Using 3D Generative Adversarial Network and Recurrent Convolutional Networks
,”
Proceedings of the IEEE International Conference on Computer Vision
,
Venice, Italy
,
Oct. 22–29
, pp.
2298
2306
.
2.
Yang
,
B.
,
Rosa
,
S.
,
Markham
,
A.
,
Trigoni
,
N.
, and
Wen
,
H.
,
2018
, “
Dense 3D Object Reconstruction From a Single Depth View
,”
IEEE Trans. Pattern. Anal. Mach. Intell.
,
41
(
12
), pp.
2820
2834
.
3.
Li
,
Z.
,
Oskarsson
,
M.
, and
Heyden
,
A.
,
2022
, “
Detailed 3D Human Body Reconstruction From Multi-View Images Combining Voxel Super-Resolution and Learned Implicit Representation
,”
Appl. Intell.
,
52
(
6
), pp.
6739
6759
.
4.
Redmon
,
J.
, and
Farhadi
,
A.
,
2018
, “
Yolov3: An Incremental Improvement
,” arXiv preprint arXiv:1804.02767.
5.
Ronneberger
,
O.
,
Fischer
,
P.
, and
Brox
,
T.
,
2015
, “
U-Net: Convolutional Networks for Biomedical Image Segmentation
,”
Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference
,
Munich, Germany
,
Oct. 5–9
, pp.
234
241
.
6.
Cao
,
C.
,
Liu
,
X.
,
Yang
,
Y.
,
Yu
,
Y.
,
Wang
,
J.
,
Wang
,
Z.
,
Huang
,
Y.
, and
Wang
,
L.
,
2015
, “
Look and Think Twice: Capturing Top-Down Visual Attention With Feedback Convolutional Neural Networks
,”
Proceedings of the IEEE International Conference on Computer Vision
,
Santiago, Chile
,
Dec. 13–16
, pp.
2956
2964
.
7.
Li
,
K.
,
Wu
,
Z.
,
Peng
,
K.-C.
,
Ernst
,
J.
, and
Fu
,
Y.
,
2018
, “
Tell Me Where to Look: Guided Attention Inference Network
,”
Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition
,
Salt Lake City, UT
,
June 18–22
, pp.
9215
9223
.
8.
Zhang
,
Z.
,
Wang
,
Z.
,
Lin
,
Z.
, and
Qi
,
H.
,
2019
, “
Image Super-Resolution by Neural Texture Transfer
,”
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
,
Long Beach, CA
,
June 16–20
, pp.
7982
7991
.
9.
Yoon
,
Y.
,
Jeon
,
H.-G.
,
Yoo
,
D.
,
Lee
,
J.-Y.
, and
So Kweon
,
I.
,
2015
, “
Learning a Deep Convolutional Network for Light-Field Image Super-Resolution
,”
Proceedings of the IEEE International Conference on Computer Vision Workshops
,
Santiago, Chile
,
Dec. 13–16
, pp.
24
32
.
10.
Ledig
,
C.
,
Theis
,
L.
,
Huszár
,
F.
,
Caballero
,
J.
,
Cunningham
,
A.
,
Acosta
,
A.
, and
Aitken
,
A.
,
2017
, “
Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network
,”
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
,
Honolulu, HI
,
July 21–26
, pp.
4681
4690
.
11.
Lim
,
B.
,
Son
,
S.
,
Kim
,
H.
,
Nah
,
S.
, and
Mu Lee
,
K.
,
2017
, “
Enhanced Deep Residual Networks for Single Image Super-Resolution
,”
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops
,
Honolulu, HI
,
July 21–26
, pp.
136
144
.
12.
Xu
,
Q.
,
Nie
,
Z.
,
Xu
,
H.
,
Zhou
,
H.
,
Attar
,
H. R.
,
Li
,
N.
,
Xie
,
F.
, and
Liu
,
X.-J.
,
2022
, “
Supermeshing: A New Deep Learning Architecture for Increasing the Mesh Density of Physical Fields in Metal Forming Numerical Simulation
,”
J. Appl. Mech.
,
89
(
1
), pp.
002
011
.
13.
Niu
,
B.
,
Wen
,
W.
,
Ren
,
W.
,
Zhang
,
X.
,
Yang
,
L.
,
Wang
,
S.
,
Zhang
,
K.
,
Cao
,
X.
, and
Shen
,
H.
,
2020
, “
Single Image Super-Resolution Via a Holistic Attention Network
,”
Computer Vision–ECCV 2020: 16th European Conference
,
Glasgow, UK
,
Aug. 23–28
, pp.
191
207
.
14.
Wang
,
F.
,
Jiang
,
M.
,
Qian
,
C.
,
Yang
,
S.
,
Li
,
C.
,
Zhang
,
H.
,
Wang
,
X.
, and
Tang
,
X.
,
2017
, “
Residual Attention Network for Image Classification
,”
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
,
Honolulu, Hawaii
,
July 21–26
, pp.
3156
3164
.
15.
Woo
,
S.
,
Park
,
J.
,
Lee
,
J.-Y.
, and
Kweon
,
I. S.
,
2018
, “
CBAM: Convolutional Block Attention Module
,”
Proceedings of the European Conference on Computer Vision (ECCV)
,
Munich, Germany
,
Sept. 8–14
, pp.
3
19
.
16.
Redmon
,
J.
, and
Farhadi
,
A.
,
2017
, “
Yolo9000: Better, Faster, Stronger
,”
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
,
Honolulu, HI
,
July 21–26
, pp.
7263
7271
.
17.
Simonyan
,
K.
, and
Zisserman
,
A.
,
2014
, “
Very Deep Convolutional Networks for Large-Scale Image Recognition
,” arXiv preprint arXiv:1409.1556.
18.
He
,
K.
,
Zhang
,
X.
,
Ren
,
S.
, and
Sun
,
J.
,
2016
, “
Deep Residual Learning for Image Recognition
,”
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
,
Las Vegas, NV
,
June 26–July 1
, pp.
770
778
.
19.
Hinton
,
G. E.
, and
Salakhutdinov
,
R. R.
,
2006
, “
Reducing the Dimensionality of Data With Neural Networks
,”
Science
,
313
(
5786
), pp.
504
507
.
20.
Zeiler
,
M. D.
,
Krishnan
,
D.
,
Taylor
,
G. W.
, and
Fergus
,
R.
,
2010
, “
Deconvolutional Networks
,”
2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition
,
San Francisco, CA
,
June 13–18
, pp.
2528
2535
.
21.
Hu
,
J.
,
Shen
,
L.
, and
Sun
,
G.
,
2018
, “
Squeeze-and-Excitation Networks
,”
Proceedings of the IEEE conference on Computer Vision and Pattern Recognition
,
Salt Lake City, UT
,
June 18–22
, pp.
7132
7141
.
22.
Bachtiar
,
Y.
, and
Adiono
,
T.
,
2019
, “
Convolutional Neural Network and Maxpooling Architecture on ZYNQ SOC FPGA
,”
International Symposium on Electronics and Smart Devices (ISESD)
,
Yogyakarta, Indonesia
,
Oct. 17–19
, pp.
1
5
.
23.
Chu
,
T.
,
Chen
,
Y.
,
Huang
,
L.
,
Tan
,
H.
,
Cao
,
J.
, and
Xu
,
Z.
,
2020
, “
Street View Image Retrieval With Average Pooling Features
,”
IGARSS 2020-2020 IEEE International Geoscience and Remote Sensing Symposium
,
Waikoloa Village, HI
,
July 16–26
, pp.
1205
1208
.
24.
Maas
,
A. L.
,
Hannun
,
A. Y.
, and
Ng
,
A. Y.
,
2013
, “
Rectifier Nonlinearities Improve Neural Network Acoustic Models
,”
Proc. ICML
,
Atlanta, GA
,
June 16–21
, p.
3
.
25.
Johnson
,
J.
,
Alahi
,
A.
, and
Fei-Fei
,
L.
,
2016
, “
Perceptual Losses for Real-Time Style Transfer and Super-Resolution
,” European Conference on Computer Vision,
Springer
, pp.
694
711
.
26.
Zhao
,
P.
, and
Lai
,
L.
,
2020
, “
Minimax Optimal Estimation of Kl Divergence for Continuous Distributions
,”
IEEE Trans. Inf. Theory
,
66
(
12
), pp.
7787
7811
.
27.
Sun
,
X.
,
Wu
,
J.
,
Zhang
,
X.
,
Zhang
,
Z.
,
Zhang
,
C.
,
Xue
,
T.
,
Tenenbaum
,
J. B.
, and
Freeman
,
W. T.
,
2018
, “
Pix3d: Dataset and Methods for Single-Image 3D Shape Modeling
,”
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
,
Salt Lake City, UT
,
June 18–22
, pp.
2974
2983
.
28.
Ben-Shabat
,
Y.
,
Yu
,
X.
,
Saleh
,
F.
,
Campbell
,
D.
,
Rodriguez-Opazo
,
C.
,
Li
,
H.
, and
Gould
,
S.
,
2021
, “
The IKEA ASM Dataset: Understanding People Assembling Furniture Through Actions, Objects and Pose
,”
Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision
,
Virtually
,
Jan. 5–9
, pp.
847
-
859
.
29.
Donahue
,
J.
,
Anne Hendricks
,
L.
,
Guadarrama
,
S.
,
Rohrbach
,
M.
,
Venugopalan
,
S.
,
Saenko
,
K.
, and
Darrell
,
T.
,
2015
, “
Long-Term Recurrent Convolutional Networks for Visual Recognition and Description
,”
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
,
Boston, MA
,
June 7–12
, pp.
2625
2634
.
30.
Park
,
J. J.
,
Florence
,
P.
,
Straub
,
J.
,
Newcombe
,
R.
, and
Lovegrove
,
S.
,
2019
, “
DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation
,”
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
,
Long Beach, CA
,
June 16–20
, pp.
165
174
.
You do not currently have access to this content.