An extrapolation-driven network architecture for physics-informed deep learning

Yong Wang; Yanzhong Yao; Zhiming Gao

doi:10.1016/j.neunet.2024.106998

An extrapolation-driven network architecture for physics-informed deep learning

Neural Netw. 2024 Dec 5:183:106998. doi: 10.1016/j.neunet.2024.106998. Online ahead of print.

Authors

Yong Wang¹, Yanzhong Yao², Zhiming Gao³

Affiliations

¹ Institute of Applied Physics and Computational Mathematics, Beijing 100088, China; Graduate School of China Academy of Engineering Physics, Beijing 100088, China; National Key Laboratory of Computational Physics, Beijing 100088, China.
² Institute of Applied Physics and Computational Mathematics, Beijing 100088, China; National Key Laboratory of Computational Physics, Beijing 100088, China. Electronic address: yao_yanzhong@iapcm.ac.cn.
³ Institute of Applied Physics and Computational Mathematics, Beijing 100088, China; National Key Laboratory of Computational Physics, Beijing 100088, China.

PMID: 39657525
DOI: 10.1016/j.neunet.2024.106998

Abstract

Current physics-informed neural network (PINN) implementations with sequential learning strategies often experience some weaknesses, such as the failure to reproduce the previous training results when using a single network, the difficulty to strictly ensure continuity and smoothness at the time interval nodes when using multiple networks, and the increase in complexity and computational overhead. To overcome these shortcomings, we first investigate the extrapolation capability of the PINN method for time-dependent PDEs. Taking advantage of this extrapolation property, we generalize the training result obtained in a specific time subinterval to larger intervals by adding a correction term to the network parameters of the subinterval. The correction term is determined by further training with the sample points in the added subinterval. Secondly, by designing an extrapolation control function with special characteristics and combining it with a correction term, we construct a new neural network architecture whose network parameters are coupled with the time variable, which we call the extrapolation-driven network architecture. Based on this architecture, using a single neural network, we can obtain the overall PINN solution of the whole domain with the following two characteristics: (1) it completely inherits the local solution of the interval obtained from the previous training, (2) at the interval node, it strictly maintains the continuity and smoothness that the true solution has. The extrapolation-driven network architecture allows us to divide a large time domain into multiple subintervals and solve the time-dependent PDEs one by one in a chronological order. This training scheme respects the causality principle and effectively overcomes the difficulties of the conventional PINN method in solving the evolution equation on a large time domain. Numerical experiments verify the performance of our method. The data and code accompanying this paper are available at https://github.com/wangyong1301108/E-DNN.

Keywords: Deep learning; Evolution equation; Extrapolation; Physics-informed neural networks.