Physics Informed Neural Networks : A Deeper Look

The last article discussed the basic formulation of Physics Informed Neural Networks (PINNs), their advantages and difficulties, and the promise they hold in the world of simulations and digital twins to revolutionize applications like product design, health monitoring etc. This article will take a deeper dive into some more recent advances in PINNs and Operator Networks, as well as take a closer look at an applications that demonstrates the potential of PINNs as a surrogate modeling tool that can change the way simulation is done.

As discussed previously, one of the main, most exciting use cases of PINNs is to create robust, accurate surrogate models while using little to no solution data. Traditional surrogate modeling methods all require plenty of data to create good models, and PINNs have the potential to remove the costly step of preparing said data. However, in their current form, there are two primary challenges with doing the same.

First, convergence of PINNs to the correct solution is not trivial, since the loss function that needs to be minimized to satisfy all the constraints is almost always a weighted linear combination of each constraint, and this can lead to convergence to bad minima if the correct weights are not used. Indeed, computing these weights correctly as the training progresses is a massive area of research today, with several strategies being investigated.

Figure 1: The loss of a standard PINN problem is usually a linear combination of multiple terms. Usually these are PDE residuals, and boundary/initial condition losses. Each of these loss terms are themselves highly nonconvex, and the selecting the right values for w is required either statically (same value throughout training) or dynamically (changes during training)

Second, in order to solve PDEs parametrically, the network has to be able to parametrize the quantities of interest, that we require the network be able to deal with. An example of this is if we wish the network solve the energy equation for a given range of conductivities, then the range of conductivities has to be given as an input to the network. However in scientific ML, we very often are also interested in parametrizing fields of inputs. An example of this is parametrizing the heat flux field around a geometry undergoing cooling. In this case, the network now has to learn a mapping between functions to solutions, not just vector valued inputs. To deal with this, operator networks were introduced, which are based on the ability of neural networks to map between infinite dimensional Banach spaces instead of just vector spaces. The two most well-developed types of operator networks today are the Fourier Neural Operator and DeepONets. We have adopted the use of DeepONets to create surrogates, and we will discuss using Physics Informed DeepONets to parametrize fields.

Figure 2: A physics informed DeepONet as depicted in Perdikaris et al.[1].In addition to the standard PINN setup, DeepONets also posses branch nets that parametrize fields.

In order to demonstrate the use of DeepONets, we solve the following problem: we wish to create a surrogate model for solving the energy equation in a grid of simplified cell models, that are undergoing cooling via an airflow. We want to predict the temperature in the cells, given a volumetric source term within them that generates heat and the cooling flux. This problem is one of many particularly amenable to surrogate modeling via DeepONets, since for the grid of cells the only difference between the PDE problem of each cell, is that the flux field on the surface would be different. Hence, if the flux field can be parametrized, we can train a model in a physics informed manner to predict the temperature profile inside a cell given an external cooling flux field.

Figure 3: Depiction of our surrogate modeling experiment. We train (in a physics informed manner) on a 3x3 grid of identical cell geometries with identical B.C except the flux boundary condition on the top and curved surface, for the shown inflow velocities. We then use the network to predict the solution for unseen velocities for any size of grid, in near real time.

We set this up as a 2 axis parametrization problem: we want to parametrize the inflow velocity (since we solve only the energy equation for now, this is manifested in the different fields generated on each cylinder) and the number of cylinders present in the grid. The figure above depicts the experiment training and a test case geometry. Once the model has been trained (in a data-free manner) on the smaller grid (each cell in the grid is a separate training input to the network, which only sees one cylinder), on a few inflow velocities, we then test the ability of the model to predict the temperature in all cells on a larger/smaller grid of identical geometries, for values of inflow velocities the network has not seen before. 

Figure 4: Test grid that we use to deploy the trained physics informed model. The training setup was the same training setup from figure 4. We use a velocity of 0.7(that the network did not see previously) units to obtain the flux fields from AcuSolve, and predict the temperature fields in each cell.

Figure 5: Lengthwise temperature comparisons between AcuSolve (red) and PINN(blue) online predictions for the 4x4 grid of cells. Each prediction for a cell takes only a fraction of a second to make. The generalizations are fairly accurate, with the largest error being around 2% for any cell prediction.

The above experiment really demonstrates the promise of PINNs and DeepONets as surrogate modeling tools, that are economical to create in terms of effort required. A designer can use a trained PINN surrogate to quickly explore many combinations of inflow velocity and number of cells in the grid, and their effect on a bigger simulation of say, a battery pack. One of our immediate goals is to extend this parametrization to shape changes (of the cylinder for example) as well, and to be able to parametrize the flow side of the problem as well.

References:

[1] ] LEARNING THE SOLUTION OPERATOR OF PARAMETRIC PARTIAL DIFFERENTIAL EQUATIONS WITH PHYSICS-INFORMED DEEPONETS: Wang, Teng and Perdikaris 2020