Abstract

We introduce universal solution manifold network (USM-Net), a novel surrogate model, based on artificial neural networks (ANNs), which applies to differential problems whose solution depends on physical and geometrical parameters. We employ a mesh-less architecture, thus overcoming the limitations associated with image segmentation and mesh generation required by traditional discretization methods. Our method encodes geometrical variability through scalar landmarks, such as coordinates of points of interest. In biomedical applications, these landmarks can be inexpensively processed from clinical images. We present proof-of-concept results obtained with a data-driven loss function based on simulation data. Nonetheless, our framework is non-intrusive and modular, as we can modify the loss by considering additional constraints, thus leveraging available physical knowledge. Our approach also accommodates a universal coordinate system, which supports the USM-Net in learning the correspondence between points belonging to different geometries, boosting prediction accuracy on unobserved geometries. Finally, we present two numerical test cases in computational fluid dynamics involving variable Reynolds numbers as well as computational domains of variable shape. The results show that our method allows for inexpensive but accurate approximations of velocity and pressure, avoiding computationally expensive image segmentation, mesh generation, or re-training for every new instance of physical parameters and shape of the domain.

References

1.
Alber
,
M.
,
Buganza Tepole
,
A.
,
Cannon
,
W. R.
,
De
,
S.
,
Dura-Bernal
,
S.
,
Garikipati
,
K.
,
Karniadakis
,
G.
,
Lytton
,
W. W.
,
Perdikaris
,
P.
,
Petzold
,
L.
, and
Kuhl
,
E.
,
2019
, “
Integrating Machine Learning and Multiscale Modeling—Perspectives, Challenges, and Opportunities in the Biological, Biomedical, and Behavioral Sciences
,”
NPJ Digital Med.
,
2
(
1
), pp.
1
11
.10.1038/s41746-019-0193-y
2.
Anderson
,
J. D.
, and
Wendt
,
J.
,
1995
,
Computational Fluid Dynamics
, Vol.
206
,
Springer
, Berlin.
3.
Parolini
,
N.
, and
Quarteroni
,
A.
,
2005
, “
Mathematical Models and Numerical Simulations for the America's Cup
,”
Comput. Methods Appl. Mech. Eng.
,
194
(
9-11
), pp.
1001
1026
.10.1016/j.cma.2004.06.020
4.
Formaggia
,
L.
,
Quarteroni
,
A.
, and
Veneziani
,
A.
,
2010
,
Cardiovascular Mathematics: Modeling and Simulation of the Circulatory System
, Vol.
1
,
Springer Science & Business Media
, Berlin.
5.
Brunton
,
S. L.
,
Noack
,
B. R.
, and
Koumoutsakos
,
P.
,
2020
, “
Machine Learning for Fluid Mechanics
,”
Annu. Rev. Fluid Mech.
,
52
, pp.
477
508
.10.1146/annurev-fluid-010719-060214
6.
Quarteroni
,
A.
,
2017
,
Numerical Models for Differential Problems
, 3rd ed.,
Springer
, Berlin.
7.
Hughes
,
T. J.
,
Cottrell
,
J. A.
, and
Bazilevs
,
Y.
,
2005
, “
Isogeometric Analysis: CAD, Finite Elements, NURBS, Exact Geometry and Mesh Refinement
,”
Comput. Methods Appl. Mech. Eng.
,
194
(
39–41
), pp.
4135
4195
.10.1016/j.cma.2004.10.008
8.
Sederberg
,
T. W.
, and
Parry
,
S. R.
,
1986
, “
Free-Form Deformation of Solid Geometric Models
,”
Proceedings of the 13th Annual Conference on Computer Graphics and Interactive Techniques
, Dallas, TX, Aug. 18–22, pp.
151
160
.https://people.eecs.berkeley.edu/~sequin/CS285/PAPERS/Sederberg_Parry.pdf
9.
Lamousin
,
H. J.
, and
Waggenspack
,
N.
,
1994
, “
NURBS-Based Free-Form Deformations
,”
IEEE Comput. Graphics Appl.
,
14
(
6
), pp.
59
65
.10.1109/38.329096
10.
Buhmann
,
M. D.
,
2000
, “
Radial Basis Functions
,”
Acta Numer.
,
9
, pp.
1
38
.10.1017/CBO9780511543241
11.
Heimann
,
T.
, and
Meinzer
,
H.-P.
,
2009
, “
Statistical Shape Models for 3D Medical Image Segmentation: A Review
,”
Med. Image Anal.
,
13
(
4
), pp.
543
563
.10.1016/j.media.2009.05.004
12.
Jolliffe
,
I. T.
, and
Cadima
,
J.
,
2016
, “
Principal Component Analysis: A Review and Recent Developments
,”
Philos. Trans. R. Soc., A
,
374
(
2065
), p.
20150202
.10.1098/rsta.2015.0202
13.
Quarteroni
,
A.
,
Rozza
,
G.
, and
Manzoni
,
A.
,
2011
, “
Certified Reduced Basis Approximation for Parametrized Partial Differential Equations and Applications
,”
J. Math. Ind.
,
1
(
1
), pp.
1
49
.10.1186/2190-5983-1-3
14.
Carlberg
,
K.
,
Bou-Mosleh
,
C.
, and
Farhat
,
C.
,
2011
, “
Efficient Non-Linear Model Reduction Via a Least-Squares Petrov–Galerkin Projection and Compressive Tensor Approximations
,”
Int. J. Numer. Methods Eng.
,
86
(
2
), pp.
155
181
.10.1002/nme.3050
15.
Quarteroni
,
A.
,
Manzoni
,
A.
, and
Negri
,
F.
,
2015
,
Reduced Basis Methods for Partial Differential Equations: An Introduction
, Vol.
92
,
Springer
, Cham, Switzerland.
16.
Hesthaven
,
J. S.
,
Rozza
,
G.
, and
Stamm
,
B.
,
2016
,
Certified Reduced Basis Methods for Parametrized Partial Differential Equations
, Vol.
590
,
Springer
, Cham,Switzerland.
17.
Peherstorfer
,
B.
, and
Willcox
,
K.
,
2016
, “
Data-Driven Operator Inference for Nonintrusive Projection-Based Model Reduction
,”
Compt. Methods Appl. Mech. Eng.
,
306
, pp.
196
215
.10.1016/j.cma.2016.03.025
18.
Taira
,
K.
,
Brunton
,
S. L.
,
Dawson
,
S. T.
,
Rowley
,
C. W.
,
Colonius
,
T.
,
McKeon
,
B. J.
,
Schmidt
,
O. T.
,
Gordeyev
,
S.
,
Theofilis
,
V.
, and
Ukeiley
,
L. S.
,
2017
, “
Modal Analysis of Fluid Flows: An Overview
,”
AIAA J.
,
55
(
12
), pp.
4013
4041
.10.2514/1.J056060
19.
Bonizzoni
,
F.
,
Nobile
,
F.
, and
Perugia
,
I.
,
2018
, “
Convergence Analysis of Padé Approximations for Helmholtz Frequency Response Problems
,”
ESAIM: Math. Modell. Numer. Anal.
,
52
(
4
), pp.
1261
1284
.10.1051/m2an/2017050
20.
Bonizzoni
,
F.
,
Nobile
,
F.
,
Perugia
,
I.
, and
Pradovera
,
D.
,
2020
, “
Fast Least-Squares Padé Approximation of Problems With Normal Operators and Meromorphic Structure
,”
Math. Comput.
,
89
(
323
), pp.
1229
1257
.10.1090/mcom/3511
21.
Samareh
,
J.
,
2004
, “
Aerodynamic Shape Optimization Based on Free-Form Deformation
,”
AIAA
Paper No. 2004-4630.10.2514/6.2004-4630
22.
Lassila
,
T.
, and
Rozza
,
G.
,
2010
, “
Parametric Free-Form Shape Design With PDE Models and Reduced Basis Method
,”
Compt. Methods Appl. Mech. Eng.
,
199
(
23-24
), pp.
1583
1592
.10.1016/j.cma.2010.01.007
23.
Morris
,
A.
,
Allen
,
C.
, and
Rendall
,
T.
,
2008
, “
CFD-Based Optimization of Aerofoils Using Radial Basis Functions for Domain Element Parameterization and Mesh Deformation
,”
Int. J. Numer. Methods Fluids
,
58
(
8
), pp.
827
860
.10.1002/fld.1769
24.
Rendall
,
T. C.
, and
Allen
,
C. B.
,
2008
, “
Unified Fluid–Structure Interpolation and Mesh Motion Using Radial Basis Functions
,”
Int. J. Numer. Methods Eng.
,
74
(
10
), pp.
1519
1559
.10.1002/nme.2219
25.
Manzoni
,
A.
,
Quarteroni
,
A.
, and
Rozza
,
G.
,
2012
, “
Model Reduction Techniques for Fast Blood Flow Simulation in Parametrized Geometries
,”
Int. J. Numer. Methods Eng.
,
28
(
6-7
), pp.
604
625
.10.1002/cnm.1465
26.
Sangalli
,
L. M.
,
Secchi
,
P.
,
Vantini
,
S.
, and
Veneziani
,
A.
,
2009
, “
A Case Study in Exploratory Functional Data Analysis: Geometrical Features of the Internal Carotid Artery
,”
J. Am. Stat. Assoc.
,
104
(
485
), pp.
37
48
.10.1198/jasa.2009.0002
27.
Li
,
S.
, and
Liu
,
W. K.
,
2002
, “
Meshfree and Particle Methods and Their Applications
,”
ASME Appl. Mech. Rev.
,
55
(
1
), pp.
1
34
.10.1115/1.1431547
28.
Sirignano
,
J.
, and
Spiliopoulos
,
K.
,
2018
, “
DGM: A Deep Learning Algorithm for Solving Partial Differential Equations
,”
J. Comput. Phys.
,
375
, pp.
1339
1364
.10.1016/j.jcp.2018.08.029
29.
Raissi
,
M.
,
Perdikaris
,
P.
, and
Karniadakis
,
G. E.
,
2019
, “
Physics-Informed Neural Networks: A Deep Learning Framework for Solving Forward and Inverse Problems Involving Nonlinear Partial Differential Equations
,”
J. Comput. Phys.
,
378
, pp.
686
707
.10.1016/j.jcp.2018.10.045
30.
Raissi
,
M.
,
Yazdani
,
A.
, and
Karniadakis
,
G. E.
,
2020
, “
Hidden Fluid Mechanics: Learning Velocity and Pressure Fields from Flow Visualizations
,”
Science
,
367
(
6481
), pp.
1026
1030
.10.1126/science.aaw4741
31.
Jin
,
X.
,
Cai
,
S.
,
Li
,
H.
, and
Karniadakis
,
G. E.
,
2021
, “
NSFnets (Navier–Stokes flow nets): Physics-Informed Neural Networks for the Incompressible Navier–Stokes Equations
,”
J. Comput. Phys.
,
426
, p.
109951
.10.1016/j.jcp.2020.109951
32.
Regazzoni
,
F.
,
Pagani
,
S.
,
Alessandro
,
C.
,
Alessandro
,
L.
, and
Quarteroni
,
A. M.
,
2021
, “
A Physics-Informed Multi-Fidelity Approach for the Estimation of Differential Equations Parameters in Low-Data or Large-Noise Regimes
,”
Atti Accad. Naz. Lincei Cl. Sci. Fis. Mat. Natur.
,
32
(
3
), pp.
437
470
.10.4171/RLM/943
33.
Cai
,
S.
,
Mao
,
Z.
,
Wang
,
Z.
,
Yin
,
M.
, and
Karniadakis
,
G. E.
,
2021
, “
Physics-Informed Neural Networks (Pinns) for Fluid Mechanics: A Review
,”
Acta Mech. Sin.
, 37, pp.
1727
1738
.10.1007/s10409-021-01148-1
34.
Tripathy
,
R. K.
, and
Bilionis
,
I.
,
2018
, “
Deep UQ: Learning Deep Neural Network Surrogate Models for High Dimensional Uncertainty Quantification
,”
J. Comput. Phys.
,
375
, pp.
565
588
.10.1016/j.jcp.2018.08.036
35.
Berg
,
J.
, and
Nyström
,
K.
,
2018
, “
A Unified Deep Artificial Neural Network Approach to Partial Differential Equations in Complex Geometries
,”
Neurocomputing
,
317
, pp.
28
41
.10.1016/j.neucom.2018.06.056
36.
Sun
,
L.
,
Gao
,
H.
,
Pan
,
S.
, and
Wang
,
J.-X.
,
2020
, “
Surrogate Modeling for Fluid Flows Based on Physics-Constrained Deep Learning Without Simulation Data
,”
Compt. Methods Appl. Mech. Eng.
,
361
, p.
112732
.10.1016/j.cma.2019.112732
37.
Cybenko
,
G.
,
1989
, “
Approximation by Superpositions of a Sigmoidal Function
,”
Math. Control Signals Syst.
,
2
(
4
), pp.
303
314
.10.1007/BF02551274
38.
Mhaskar
,
H. N.
,
1996
, “
Neural Networks for Optimal Approximation of Smooth and Analytic Functions
,”
Neural Comput.
,
8
(
1
), pp.
164
177
.10.1162/neco.1996.8.1.164
39.
Montanelli
,
H.
, and
Du
,
Q.
,
2019
, “
New Error Bounds for Deep RELU Networks Using Sparse Grids
,”
SIAM J. Math. Data Sci.
,
1
(
1
), pp.
78
92
.10.1137/18M1189336
40.
Botella
,
O.
, and
Peyret
,
R.
,
1998
, “
Benchmark Spectral Results on the Lid-Driven Cavity Flow
,”
Comput. Fluids
,
27
(
4
), pp.
421
433
.10.1016/S0045-7930(98)00002-4
41.
Chiastra
,
C.
,
Iannaccone
,
F.
,
Grundeken
,
M. J.
,
Gijsen
,
F. J.
,
Segers
,
P.
,
De Beule
,
M.
,
Serruys
,
P. W.
,
Wykrzykowska
,
J. J.
,
van der Steen
,
A. F.
, and
Wentzel
,
J. J.
,
2016
, “
Coronary Fractional Flow Reserve Measurements of a Stenosed Side Branch: A Computational Study Investigating the Influence of the Bifurcation Angle
,”
Biomed. Eng. Online
,
15
(
1
), pp.
1
16
.10.1186/s12938-016-0211-0
42.
Abadi
,
M.
,
Agarwal
,
A.
,
Barham
,
P.
,
Brevdo
,
E.
,
Chen
,
Z.
,
Citro
,
C.
,
Corrado
,
G. S.
, et al.,
2015
, “TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems,” TensorFlow, accessed Aug. 31, 2022, https://www.tensorflow.org/about/bib
43.
Virtanen
,
P.
,
Gommers
,
R.
,
Oliphant
,
T. E.
,
Haberland
,
M.
,
Reddy
,
T.
,
Cournapeau
,
D.
,
Burovski
,
E.
, et al.,
2020
, “
SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python
,”
Nat. Methods
,
17
, pp.
261
272
.10.1038/s41592-020-0772-5
44.
Kingma
,
D. P.
, and
Ba
,
J.
,
2014
, “
Adam: A Method for Stochastic Optimization
,” arXiv:1412.6980.
45.
Goodfellow
,
I.
,
Bengio
,
Y.
,
Courville
,
A.
, and
Bengio
,
Y.
,
2016
,
Deep Learning
, Vol.
1
.
MIT Press
,
Cambridge, MA
.
46.
Benner
,
P.
,
Mehrmann
,
V.
, and
Sorensen
,
D. C.
,
2005
,
Dimension Reduction of Large-Scale Systems
, Vol.
45
,
Springer-Verlag, Berlin
.
47.
Antoulas
,
A. C.
,
Sorensen
,
D. C.
, and
Gugercin
,
S.
,
2001
, “
A Survey of Model Reduction Methods for Large-Scale Systems
,”
Contem. Math.
, 280, pp.
193
219
.10.1090/conm/280/04630
48.
Lassila
,
T.
,
Manzoni
,
A.
,
Quarteroni
,
A.
, and
Rozza
,
G.
,
2014
, “
Model Order Reduction in Fluid Dynamics: Challenges and Perspectives
,” A. Quarteroni and G. Rozza, eds.,
Reduced Order Methods for Modeling and Computational Reduction
, MS&A - Modeling, Simulation and Applications, Vol. 9, Springer, Cham, pp.
235
273
.10.1007/978-3-319-02090-7_9
49.
Benner
,
P.
,
Gugercin
,
S.
, and
Willcox
,
K.
,
2015
, “
A Survey of Projection-Based Model Reduction Methods for Parametric Dynamical Systems
,”
SIAM Rev.
,
57
(
4
), pp.
483
531
.10.1137/130932715
50.
Peherstorfer
,
B.
, and
Willcox
,
K.
,
2015
, “
Dynamic Data-Driven Reduced-Order Models
,”
Compt. Methods Appl. Mech. Eng.
,
291
, pp.
21
41
.10.1016/j.cma.2015.03.018
51.
Hesthaven
,
J. S.
, and
Ubbiali
,
S.
,
2018
, “
Non-Intrusive Reduced Order Modeling of Nonlinear Problems Using Neural Networks
,”
J. Comput. Phys.
,
363
, pp.
55
78
.10.1016/j.jcp.2018.02.037
52.
Carlberg
,
K. T.
,
Jameson
,
A.
,
Kochenderfer
,
M. J.
,
Morton
,
J.
,
Peng
,
L.
, and
Witherden
,
F. D.
,
2019
, “
Recovering Missing CFD Data for High-Order Discretizations Using Deep Neural Networks and Dynamics Learning
,”
J. Comput. Phys.
,
395
, pp.
105
124
.10.1016/j.jcp.2019.05.041
53.
Dal Santo
,
N.
,
Deparis
,
S.
, and
Pegolotti
,
L.
,
2020
, “
Data Driven Approximation of Parametrized PDES by Reduced Basis and Neural Networks
,”
J. Comput. Phys.
,
416
, p.
109550
.10.1016/j.jcp.2020.109550
54.
O'Leary-Roseberry
,
T.
,
Du
,
X.
,
Chaudhuri
,
A.
,
Martins
,
J. R.
,
Willcox
,
K.
, and
Ghattas
,
O.
,
2021
, “
Adaptive Projected Residual Networks for Learning Parametric Maps from Sparse Data
,” preprint
arXiv:2112.07096
.https://www.researchgate.net/publication/357046501_Adaptive_Projected_Residual_Networks_for_Learning_Parametric_Maps_from_Sparse_Data
55.
Roney
,
C. H.
,
Pashaei
,
A.
,
Meo
,
M.
,
Dubois
,
R.
,
Boyle
,
P. M.
,
Trayanova
,
N. A.
,
Cochet
,
H.
,
Niederer
,
S. A.
, and
Vigmond
,
E. J.
,
2019
, “
Universal Atrial Coordinates Applied to Visualisation, Registration and Construction of Patient Specific Meshes
,”
Med. Image Anal.
,
55
, pp.
65
75
.10.1016/j.media.2019.04.004
You do not currently have access to this content.