Nikola B. Kovachki

selected publications

1. JMLR

   Neural Operator: Learning Maps Between Function Spaces With Applications to PDEs

   Kovachki, Nikola B, Li, Zongyi, Liu, Burigede, Azizzadenesheli, Kamyar, Bhattacharya, Kaushik, Stuart, Andrew M, and Anandkumar, Anima

   Journal of Machine Learning 2023

   Abs Bib HTML PDF

   The classical development of neural networks has primarily focused on learning mappings between finite dimensional Euclidean spaces or finite sets. We propose a generalization of neural networks to learn operators, termed neural operators, that map between infinite dimensional function spaces. We formulate the neural operator as a composition of linear integral operators and nonlinear activation functions. We prove a universal approximation theorem for our proposed neural operator, showing that it can approximate any given nonlinear continuous operator. The proposed neural operators are also discretization-invariant, i.e., they share the same model parameters among different discretization of the underlying function spaces. Furthermore, we introduce four classes of efficient parameterization, viz., graph neural operators, multi-pole graph neural operators, low-rank neural operators, and Fourier neural operators. An important application for neural operators is learning surrogate maps for the solution operators of partial differential equations (PDEs). We consider standard PDEs such as the Burgers, Darcy subsurface flow, and the Navier-Stokes equations, and show that the proposed neural operators have superior performance compared to existing machine learning based methodologies, while being several orders of magnitude faster than conventional PDE solvers.

   @article{kovachki2023neural,
     abbr = {JMLR},
     bibtex_show = {true},
     title = {Neural Operator: Learning Maps Between Function Spaces With Applications to PDEs},
     author = {Kovachki, Nikola B and Li, Zongyi and Liu, Burigede and Azizzadenesheli, Kamyar and Bhattacharya, Kaushik and Stuart, Andrew M and Anandkumar, Anima},
     journal = {Journal of Machine Learning},
     volume = {24},
     year = {2023},
     number = {89},
     html = {https://www.jmlr.org/papers/v24/21-1524.html},
     pdf = {https://www.jmlr.org/papers/volume24/21-1524/21-1524.pdf},
     selected = {true}
   }

2. arXiv

   An Approximation Theory Framework for Measure-Transport Sampling Algorithms

   Baptista, Ricardo, Hosseini, Bamdad, Kovachki, Nikola B, Marzouk, Youssef M, and Sagiv, Amir

   CoRR 2023

   Abs Bib HTML PDF

   This article presents a general approximation-theoretic framework to analyze measure transport algorithms for probabilistic modeling. A primary motivating application for such algorithms is sampling – a central task in statistical inference and generative modeling. We provide a priori error estimates in the continuum limit, i.e., when the measures (or their densities) are given, but when the transport map is discretized or approximated using a finite-dimensional function space. Our analysis relies on the regularity theory of transport maps and on classical approximation theory for high-dimensional functions. A third element of our analysis, which is of independent interest, is the development of new stability estimates that relate the distance between two maps to the distance (or divergence) between the pushforward measures they define. We present a series of applications of our framework, where quantitative convergence rates are obtained for practical problems using Wasserstein metrics, maximum mean discrepancy, and Kullback–Leibler divergence. Specialized rates for approximations of the popular triangular Knothe-Rosenblatt maps are obtained, followed by numerical experiments that demonstrate and extend our theory.

   @article{baptista2023anapproximation,
     abbr = {arXiv},
     bibtex_show = {true},
     author = {Baptista, Ricardo and Hosseini, Bamdad and Kovachki, Nikola B and Marzouk, Youssef M and Sagiv, Amir},
     title = {An Approximation Theory Framework for Measure-Transport Sampling Algorithms},
     journal = {CoRR},
     volume = {abs/2302.13965},
     year = {2023},
     html = {https://arxiv.org/abs/2302.13965},
     pdf = {https://arxiv.org/pdf/2302.13965.pdf},
     selected = {true}
   }

3. JMPS

   A Learning-based Multiscale Method and its Application to Inelastic Impact Problems

   Liu, Burigede, Kovachki, Nikola B, Li, Zongyi, Azizzadenesheli, Kamyar, Anandkumar, Anima, Stuart, Andrew M, and Bhattacharya, Kaushik

   Journal of the Mechanics and Physics of Solids 2022

   Abs Bib HTML PDF

   The macroscopic properties of materials that we observe and exploit in engineering application result from complex interactions between physics at multiple length and time scales: electronic, atomistic, defects, domains etc. Multiscale modeling seeks to understand these interactions by exploiting the inherent hierarchy where the behavior at a coarser scale regulates and averages the behavior at a finer scale. This requires the repeated solution of computationally expensive finer-scale models, and often a priori knowledge of those aspects of the finer-scale behavior that affect the coarser scale (order parameters, state variables, descriptors, etc.). We address this challenge in a two-scale setting where we learn the fine-scale behavior from off-line calculations and then use the learnt behavior directly in coarse scale calculations. The approach builds on the recent success of deep neural networks by combining their approximation power in high dimensions with ideas from model reduction. It results in a neural network approximation that has high fidelity, is computationally inexpensive, is independent of the need for a priori knowledge, and can be used directly in the coarse scale calculations. We demonstrate the approach on problems involving the impact of magnesium, a promising light-weight structural and protective material.

   @article{liu2022alearningbased,
     abbr = {JMPS},
     bibtex_show = {true},
     title = {A Learning-based Multiscale Method and its Application to Inelastic Impact Problems},
     journal = {Journal of the Mechanics and Physics of Solids},
     volume = {158},
     year = {2022},
     html = {https://www.sciencedirect.com/science/article/pii/S0022509621002982},
     pdf = {https://arxiv.org/pdf/2102.07256.pdf},
     author = {Liu, Burigede and Kovachki, Nikola B and Li, Zongyi and Azizzadenesheli, Kamyar and Anandkumar, Anima and Stuart, Andrew M and Bhattacharya, Kaushik},
     selected = {true}
   }

4. ICLR

   Fourier Neural Operator for Parametric Partial Differential Equations

   Li, Zongyi, Kovachki, Nikola B, Azizzadenesheli, Kamyar, Liu, Burigede, Bhattacharya, Kaushik, Stuart, Andrew M, and Anandkumar, Anima

   In International Conference on Learning Representations (ICLR) 2021

   Abs Bib HTML PDF

   The classical development of neural networks has primarily focused on learning mappings between finite-dimensional Euclidean spaces. Recently, this has been generalized to neural operators that learn mappings between function spaces. For partial differential equations (PDEs), neural operators directly learn the mapping from any functional parametric dependence to the solution. Thus, they learn an entire family of PDEs, in contrast to classical methods which solve one instance of the equation. In this work, we formulate a new neural operator by parameterizing the integral kernel directly in Fourier space, allowing for an expressive and efficient architecture. We perform experiments on Burgers’ equation, Darcy flow, and Navier-Stokes equation. The Fourier neural operator is the first ML-based method to successfully model turbulent flows with zero-shot super-resolution. It is up to three orders of magnitude faster compared to traditional PDE solvers. Additionally, it achieves superior accuracy compared to previous learning-based solvers under fixed resolution.

   @inproceedings{li2021fourier,
     abbr = {ICLR},
     bibtex_show = {true},
     author = {Li, Zongyi and Kovachki, Nikola B and Azizzadenesheli, Kamyar and Liu, Burigede and Bhattacharya, Kaushik and Stuart, Andrew M and Anandkumar, Anima},
     title = {Fourier Neural Operator for Parametric Partial Differential Equations},
     booktitle = {International Conference on Learning Representations (ICLR)},
     volume = {9},
     publisher = {OpenReview.net},
     year = {2021},
     pdf = {https://openreview.net/pdf?id=c8P9NQVtmnO},
     html = {https://iclr.cc/virtual/2021/poster/3281},
     selected = {true}
   }

5. SMAI-JCM

   Model Reduction and Neural Networks for Parametric PDEs

   Bhattacharya, Kaushik, Hosseini, Bamdad, Kovachki, Nikola B, and Stuart, Andrew M

   The SMAI journal of computational mathematics 2021

   Abs Bib HTML PDF

   We develop a general framework for data-driven approximation of input-output maps between infinite-dimensional spaces. The proposed approach is motivated by the recent successes of neural networks and deep learning, in combination with ideas from model reduction. This combination results in a neural network approximation which, in principle, is defined on infinite-dimensional spaces and, in practice, is robust to the dimension of finite-dimensional approximations of these spaces required for computation. For a class of input-output maps, and suitably chosen probability measures on the inputs, we prove convergence of the proposed approximation methodology. We also include numerical experiments which demonstrate the effectiveness of the method, showing convergence and robustness of the approximation scheme with respect to the size of the discretization, and compare it with existing algorithms from the literature; our examples include the mapping from coefficient to solution in a divergence form elliptic partial differential equation (PDE) problem, and the solution operator for viscous Burgers’ equation.

   @article{bhattacharya2021model,
     abbr = {SMAI-JCM},
     bibtex_show = {true},
     author = {Bhattacharya, Kaushik and Hosseini, Bamdad and Kovachki, Nikola B and Stuart, Andrew M},
     title = {Model Reduction and Neural Networks for Parametric PDEs},
     journal = {The SMAI journal of computational mathematics},
     publisher = {Soci\'et\'e de Math\'ematiques Appliqu\'ees et Industrielles},
     volume = {7},
     year = {2021},
     html = {https://smai-jcm.centre-mersenne.org/articles/10.5802/smai-jcm.74/},
     pdf = {https://smai-jcm.centre-mersenne.org/item/10.5802/smai-jcm.74.pdf},
     selected = {true}
   }