PhD Position F/ M PhD position F/ M Building physics-based multilevel surrogate models from neural networks. Application to electromagnetic wave propagation

Contexte et atouts du poste

Numerical simulations of electromagnetic wave propagation problems primarily rely on a space discretization of the system of Maxwell’s equations using methods such as finite differences or finite elements. For complex and realistic three-dimensional situations, such a process can be computationally prohibitive, especially when the end goal consists in many-query analyses (e.g., optimization design and uncertainty quantification). Therefore, developing cost-effective surrogate models is of great practical significance.

There exist different possible ways of building surrogate models for a given system of partial differential equations (PDEs) in a non-intrusive way (i.e., with minimal modifications to an existing discretization-based simulation methodology). In recent years, approaches based on neural networks (NNs) and Deep Learning (DL) have shown much promise, thanks to their capability of handling nonlinear or/and high dimensional problems. Model-based neural networks, as opposed to purely data-driven neural networks, are currently the subject of intense research for devising high-performance surrogate models of parametric PDEs.

The concept of Physics-Informed Neural Networks (PINNs) introduced in [1], and later revisited in [2], is one typical example. PINNs are neural networks trained to solve supervised learning tasks while respecting some given physical laws, described by a (possibly nonlinear) PDE system. PINNs can be seen as a continuous approximation of the solution to the PDE. They seamlessly integrate information from both data and PDEs by embedding the PDEs into the loss function of a neural network. Automatic differentiation is then used to actually differentiate the network and compute the loss function.

Following similar ideas, and relying on the widely known result that NNs are universal approximators of continuous functions, DeepONets [3] are deep neural networks (DNNs) whose goal is to learn continuous operators or complex systems from streams of scattered data. A DeepONet consists of a DNN for encoding the discrete input function space (branch net) and another DNN for encoding the domain of the output functions (trunk net). PINNs and DeepONet are merely two examples of many DNNs that have contributed to making the field of Scientific Machine Learning (SciML) so popular in recent years.

This PhD project is expected to be a follow-up to the Master internship referred as

Mission confiée

The main challenge when devising scalable physics-based DNNs for realistic applications is the computational cost of network training, especially when they are only used for forward modeling. Another important issue lies in their capacity to accurately deal with high frequency and/or multiscale problems. In particular, it has been observed that, when higher frequencies and multiscale features are present in the PDE solution, the accuracy of PINNs usually rapidly decreases, while the cost of training and evaluation drastically increases. There are multiple reasons for this behavior. One is the spectral bias of NNs, which is the well-studied property that NNs have difficulties learning high frequencies. Another reason is that, as high frequencies and multiscale features are added, more collocation points as well as a larger NN with significantly more free parameters, are typically required to accurately approximate the solution. This leads to an increase in the complexity of the optimization problem to be solved when training the NN.

In the present PhD project, we propose to study multilevel distributed strategies for fast training of physics-based DNNs for modeling electromagnetic wave propagation in the frequency domain. We will in particular investigate strategies that can accurately and efficiently deal with the simulation of electromagnetic wave interaction with heterogeneous media, and geometrically complex scattering structures. In this context, the ultimate goal of this project is to develop high-performance parametric NN surrogates that will be used as the forward model in inverse design studies. The first step will be to develop novel methodologies, and assess their performance in a simplified two-dimensional case, on a Helmholtz-type PDE. The extension to the more general three-dimensional Maxwell’s equations will be considered in a second step, informed by the results obtained in the Helmholtz case in two space dimensions.

[1] I.E. Lagaris, A. Likas and D.I. Fotiadis. Artificial neural networks for solving ordinary and partial differential equations. IEEE Trans. Neur. Netw., Vol. 9, No. 5, pp. 987-1000

[2] M. Raissi, P. Perdikaris and G.E. Karniadakis. Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comp. Phys., Vol. 378, pp. 686-707

[3] L. Lu, P. Jin, G. Pang, Z. Zhang and G. E. Karniadakis. Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators. Nat. Mach. Intell., Vol. 3, pp. 218-229

[4] V. Dolean, A. Heinlein, S. Mishra, B. Moseley. Multilevel domain decomposition-based architectures for physics-informed neural networks. arXiv preprint arXiv:2306.054862

Principales activités

Compétences

Technical skills and level required

Software development skills : Python programming, TensorFlow, Pytorch

Relational skills : team worker (verbal communication, active listening, motivation and commitment)

Other valued appreciated : good level of spoken and written english

Rémunération

Gross Salary per month: 2082€ brut per month (year 1 & 2) and 2190€ brut per month (year 3)