Abstract

Three-dimensional urban reconstruction requires the combination of data from different sensors, such as cameras, inertial systems, GPS, and laser sensors. In this technical report, a complete system for the generation of textured volumetric global maps (deep vision) is presented. Our acquisition platform is terrestrial and moves through different urban environments digitizing them. The report is focused on describing the three main problems identified in this type of works. (1) The acquisition of three-dimensional data with high precision, (2) the extraction of the texture and its correlation with the 3D data, and (3) the generation of the surfaces that describe the components of the urban environment. It also describes the methods implemented to extrinsically calibrate the acquisition platform, as well as the methods developed to eliminate the radial and tangential image distortion; and the subsequent generation of a panoramic image. Procedures are developed for the sampling of 3D data and its smoothing. Subsequently, the process to generate textured global maps with a negligible uncertainty is developed and the results are presented. Finally, the process of surface generation and the post-process of eliminating certain holes/occlusions in the meshes are reported. In each section, results obtained are shown. Using the methods presented here for geometric and photorealistic reconstruction of urban environments, high-quality 3D models are generated. The results achieved the following objectives: generate global textured models that preserve the geometry of the scanned scenes.

References

1.
Yi
,
C.
,
Zhang
,
Y.
,
Wu
,
Q.
,
Xu
,
Y.
,
Remil
,
O.
,
Wei
,
M.
, and
Wang
,
J.
,
2017
, “
Urban Building Reconstruction From Raw Lidar Point Data
,”
Compu.-Aided Des.
,
93
, pp.
1
14
. 10.1016/j.cad.2017.07.005
2.
Boussaha
,
M.
,
Vallet
,
B.
, and
Rives
,
P.
,
2018
, “
Large Scale Textured Mesh Reconstruction From Mobile Mapping Images and LIDAR Scans
,”
International Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences ISPRS
,
Italy
.
3.
Sünderhauf
,
N.
,
Pham
,
T. T.
,
Latif
,
Y.
,
Milford
,
M.
, and
Reid
,
I.
,
2017
, “
Meaningful Maps with Object-Oriented Semantic Mapping
,”
International Conference on Intelligent Robots and Systems (IROS), 2017 IEEE/RSJ
, IEEE,
New York
, pp.
5079
5085
.
4.
Ocando
,
M. G.
,
Certad
,
N.
,
Alvarado
,
S.
, and
Terrones
,
A.
,
2017
, “
Autonomous 2d SLAM and 3d Mapping of an Environment Using a Single 2d Lidar and Ros
,”
Robotics Symposium (LARS) and 2017 Brazilian Symposium on Robotics (SBR), 2017 Latin American
, IEEE,
New York
, pp.
1
6
.
5.
Yang
,
S.
,
Huang
,
Y.
, and
Scherer
,
S.
,
2017
, “
Semantic 3D Occupancy Mapping Through Efficient High Order CRFs
,”
IEEE/RSJ International Conference on Intelligent Robots and Systems
,
Canada
.
6.
Liu
,
L.
, and
Stamos
,
I.
,
2012
, “
A Systematic Approach for 2d-image to 3d-range Registration in Urban Environments
,”
Comput. Vision Image Underst.
,
116
(
1
), pp.
25
37
. 10.1016/j.cviu.2011.07.009
7.
Liu
,
L.
, and
Stamos
,
I.
,
2005
, “
Automatic 3d to 2d Registration for the Photorealistic Rendering of Urban Scenes
,”
IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005. CVPR 2005
,
San Diego, CA
, Vol.
2
,
IEEE
,
New York
, pp.
137
143
.
8.
Stamos
,
I.
,
Liu
,
L.
,
Chen
,
C.
,
Wolberg
,
G.
,
Yu
,
G.
, and
Zokai
,
S.
,
2008
, “
Integrating Automated Range Registration with Multiview Geometry for the Photorealistic Modeling of Large-scale Scenes
,”
Int. J. Comput. Vision
,
78
(
2–3
), pp.
237
260
. 10.1007/s11263-007-0089-1
9.
Pandey
,
G.
,
McBride
,
J. R.
,
Savarese
,
S.
, and
Eustice
,
R.
,
2012
, “
Automatic Targetless Extrinsic Calibration of a 3d LIDAR and Camera by Maximizing Mutual Information
,”
Proceedings of the Twenty-Sixth AAAI Conference on Artificial Intelligence
.
10.
Pandey
,
G.
,
McBride
,
J. R.
,
Savarese
,
S.
, and
Eustice
,
R. M.
,
2014
, “
Automatic Extrinsic Calibration of Vision and Lidar by Maximizing Mutual Information
,”
J. Field Rob.
,
32
(
5
), pp.
696
722
. 10.1002/rob.21542
11.
Zeng
,
F.
, and
Zhong
,
R.
,
2014
, “
The Algorithm to Generate Color Point-cloud With the Registration Between Panoramic Image and Laser Point-cloud
,”
IOP Conference Series: Earth and Environmental Science
,
Beijing, China
, Vol.
17
,
IOP Publishing
, p.
012160
.
12.
Shi
,
Y.
,
Ji
,
S.
,
Shao
,
X.
,
Yang
,
P.
,
Wu
,
W.
,
Shi
,
Z.
, and
Shibasaki
,
R.
,
2015
, “
Fusion of a Panoramic Camera and 2d Laser Scanner Data for Constrained Bundle Adjustment in Gps-denied Environments
,”
Image Vision Comput.
,
40
, pp.
28
37
. 10.1016/j.imavis.2015.06.002
13.
Zhu
,
L.
,
Hyyppä
,
J.
,
Kukko
,
A.
,
Kaartinen
,
H.
, and
Chen
,
R.
,
2011
, “
Photorealistic Building Reconstruction From Mobile Laser Scanning Data
,”
Remote Sens.
,
3
(
7
), pp.
1406
1426
. 10.3390/rs3071406
14.
Di Angelo
,
L.
,
Di Stefano
,
P.
,
Giaccari
,
L.
,
2011
, “
A New Mesh-growing Algorithm for Fast Surface Reconstruction
,”
Comput.-Aided Design
,
43
(
6
), pp.
639
650
. 10.1016/j.cad.2011.02.012
15.
Carr
,
J. C.
,
Beatson
,
R. K.
,
Cherrie
,
J. B.
,
Mitchell
,
T. J.
,
Fright
,
W. R.
,
McCallum
,
B. C.
, and
Evans
,
T. R.
,
2001
, “
Reconstruction and Representation of 3d Objects With Radial Basis Functions
,”
Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques
,
ACM
,
New York
, pp.
67
76
.
16.
Amenta
,
N.
,
Bern
,
M.
, and
Kamvysselis
,
M.
,
1998
, “
A New Voronoi-Based Surface Reconstruction Algorithm
,”
Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques
, ACM,
New York
, pp.
415
421
.
17.
Cohen-Steiner
,
D.
, and
Da
,
F.
,
2004
, “
A Greedy Delaunay-based Surface Reconstruction Algorithm
,”
Visual Comput.
,
20
(
1
), pp.
4
16
. 10.1007/s00371-003-0217-z
18.
Cazals
,
F.
, and
Giesen
,
J.
,
2006
, “Delaunay Triangulation Based Surface Reconstruction,”
Effective Computational Geometry for Curves and Surfaces
,
Springer
,
Berlin
, pp.
231
276
.
19.
Bernardini
,
F.
,
Mittleman
,
J.
,
Rushmeier
,
H.
,
Silva
,
C.
, and
Taubin
,
G.
,
1999
, “
The Ball-Pivoting Algorithm for Surface Reconstruction
,”
IEEE Trans. Vis. Comput. Graph.
,
5
(
4
), pp.
349
359
. 10.1109/2945.817351
20.
Kazhdan
,
M.
,
Bolitho
,
M.
, and
Hoppe
,
H.
,
2006
, “
Poisson Surface Reconstruction
,”
Proceedings of the Fourth Eurographics Symposium on Geometry Processing
,
Cagliari, Sardinia, Italy
.
21.
Kazhdan
,
M.
, and
Hoppe
,
H.
,
2013
, “
Screened Poisson Surface Reconstruction
,”
ACM Trans. Graph. (TOG)
,
32
(
3
), p.
29
. 10.1145/2487228.2487237
22.
García-Moreno
,
A.-I.
,
González-Barbosa
,
J.-J.
,
Ramírez-Pedraza
,
A.
,
Hurtado-Ramos
,
J. B.
, and
Ornelas-Rodriguez
,
F.-J.
,
2016
, “
Accurate Evaluation of Sensitivity for Calibration Between a Lidar and a Panoramic Camera Used for Remote Sensing
,”
J. Appl. Remote Sens.
,
10
(
2
), p.
024002
. 10.1117/1.JRS.10.024002
23.
García-Moreno
,
A.-I.
,
Gonzalez-Barbosa
,
J.-J.
,
Ornelas-Rodriguez
,
F.-J.
,
Hurtado-Ramos
,
J. B.
, and
Primo-Fuentes
,
M.-N.
,
2013
, “Lidar and Panoramic Camera Extrinsic Calibration Approach Using a Pattern Plane,”
Pattern Recognition
,
Springer
,
Berlin
.
24.
Cristian
,
F.
,
1989
, “
Probabilistic Clock Synchronization
,”
Distrib. Comput.
,
3
(
3
), pp.
146
158
. 10.1007/BF01784024
25.
Garcia
,
D.
,
2010
, “
Robust Smoothing of Gridded Data in One and Higher Dimensions With Missing Values
,”
Comput. Stat. Data Anal.
,
54
(
4
), pp.
1167
1178
. 10.1016/j.csda.2009.09.020
26.
Pandey
,
G.
,
McBride
,
J. R.
, and
Eustice
,
R. M.
,
2011
, “
Ford Campus Vision and Lidar Data Set
,”
Int. J. Rob. Res.
,
30
(
13
), pp.
1543
1552
. 10.1177/0278364911400640
27.
Harris
,
C.
,
1993
, “Geometry From Visual Motion,”
Active Vision
,
MIT Press
,
Cambridge, MA
, pp.
263
284
.
28.
Brown
,
M.
,
Szeliski
,
R.
, and
Winder
,
S.
,
2005
, “
Multi-Image Matching Using Multi-Scale Oriented Patches
,”
IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005. CVPR 2005
, Vol.
1
,
IEEE
,
Silver Spring, MD
, pp.
510
517
.
29.
Kushilevitz
,
E.
,
Ostrovsky
,
R.
, and
Rabani
,
Y.
,
2000
, “
Efficient Search for Approximate Nearest Neighbor in High Dimensional Spaces
,”
SIAM J. Comput.
,
30
(
2
), pp.
457
474
. 10.1137/S0097539798347177
30.
García-Moreno
,
A.-I.
,
Hernandez-García
,
D.-E.
,
Gonzalez-Barbosa
,
J.-J.
,
Ramírez-Pedraza
,
A.
,
Hurtado-Ramos
,
J. B.
, and
Ornelas-Rodriguez
,
F.-J.
,
2014
, “
Error Propagation and Uncertainty Analysis Between 3d Laser Scanner and Camera
,”
Rob. Auton. Syst.
,
62
(
6
), pp.
782
793
. 10.1016/j.robot.2014.02.004
31.
Hartley
,
R.
, and
Zisserman
,
A.
,
2000
,
Multiple View Geometry in Computer Vision
,
Cambridge University Press
,
Cambridge
.
32.
Rusinkiewicz
,
S.
, and
Levoy
,
M.
,
2001
, “
Efficient Variants of the ICP Algorithm
,”
Third International Conference on 3-D Digital Imaging and Modeling, 2001, Proceedings
,
Canada
,
IEEE
,
Silver Spring, MD
, pp.
145
152
.
33.
Wang
,
M.
, and
Tseng
,
Y.-H.
,
2010
, “
Automatic Segmentation of Lidar Data Into Coplanar Point Clusters Using An Octree-based Split-and-merge Algorithm
,”
Photogramm. Eng. Remote Sens.
,
76
(
4
), pp.
407
420
. 10.14358/PERS.76.4.407
You do not currently have access to this content.