Data-Driven Fluid Mechanics: Combining First Principles and Machine Learning

A book based on the von Karman Institute Lecture Series
Machine Learning for Fluid Mechanics: Analysis, Modeling, Control and Closures
Image

About the lecture Series

This is the first lecture series from the von Karman Institute for Fluid Dynamics dedicated to machine learning for fluid mechanics.  The course was hosted by the Université libre de Bruxelles (ULB) from 24 to 28 February 2020, in the classic VKI lecture series format. These are one-week courses on specialized topics, selected by the VKI faculty and typically organized 8-12 times per year. These courses have gained a worldwide recognition and are among the most influential and distinguished European teaching forums, where pioneers in fluid mechanics have been training young talents for many decades.
The video of the lectures that were given in the VKI lecture series are available here.

The lecture series was co-organized by Miguel A. Mendez from the von Karman Institute (Belgium), Alessandro Parente from the Université libre de Bruxelles (Belgium), Andrea Ianiro from Universidad Carlos III de Madrid (Spain), Bernd R. Noack from Harbin Institute of Technology, Shenzhen (China) and TU Berlin (Germany) and Steven L. Brunton from University of Washington (US).

Outline of the Course

The course followed the historical path through which this field has been developed in the latest few decades, opening with the tools which were pioneered by the fluid dynamics community with the aim of identifying coherent structures in turbulent flows, and proceeding towards recent machine learning methods that were initially developed in machine vision, pattern recognition, and artificial intelligence.

 

The first day was dedicated to the coherent patterns (structures) in turbulent flows and aims at providing an overview of the subject and of the applications to fluid mechanics in which machine learning methods are bringing substantial contributions. In lecture 1, this overview was given using a well-known benchmark to build confidence, intuition, and a clear understanding of the motivation of the course. Lecture 2 on turbulent structures further elaborated on the motivation of the field by addressing the problem of coherent structures in turbulent flows, giving a phenomenological picture of their origin, dynamics, and their effects on crucial engineering quantities (e.g., skin friction, noise production, flow separation phenomena, dynamic loads, etc.). The identification and the objective definition of these coherent structures have been the primary motivation for the development of decomposition methods in fluid mechanics. The following two lectures were dedicated to two classical decompositions that were originally developed with this aim: The Proper Orthogonal Decomposition (POD) and the Dynamic Mode Decomposition (DMD).

 

The second day was dedicated to the mathematical analysis. Lectures 5 and 6 on mathematical fundamentals will review continuous and discrete linear time-invariant systems and their properties (impulse response and transfer functions, poles, zeros, etc), the fundamental transforms used in their investigation (Laplace, Fourier and Z transform), methods for time-frequency analysis (Gabor transform, continuous and discrete wavelets) and the related uncertainty principle (Heisenberg principle). Lecture 7 builds on the first two by presenting a generalized framework of data-driven modal analysis and discusses limitations and strengths of various methods. The last Lecture of this day presented a collection of applications of data decompositions to numerical and experimental data typically encountered in fluid mechanics (e.g. DNS simulations or experimental velocity fields from PIV), presenting best practices and potential pitfalls.

 

The third day was dedicated to dynamical systems. Lecture 9 introduced stability analysis of fluid flows, including classical methods of analysis (normal modes, linearization, and Orr-Sommerfeld problem) and modern formulation based on optimization methods. Lecture 10 introduced linear dynamical systems, from their general formulation to their phase portrait and the basis of linear control theory (controllability, observability, Gramians, etc.). Lecture 11 covered nonlinear dynamical systems and their distinctive features, including attractors, bifurcations and the route to chaos, with a connection to the theory of turbulent flows. Finally, Lecture 12 gave an overview of system identification methods, such as Eigensystem realization algorithm (ERA), observer Kalman Filter identification (OKID), autoregressive moving average (ARMA) and more.

 

The fourth day was dedicated to machine-learning methods and their application in data-driven reduced-order modeling. Most problems in fluid mechanics can be formulated as regression problems, i.e., optimizing a function with respect to data or a given system. Examples are feature extraction, model-order reduction (autoencoders), prediction, control, estimation, and modeling. Lecture 13 provided an overview of the powerful regression solvers of machine learning (e.g., supervised versus non-supervised methods, clustering and classification methods, support vector machines, kernel methods, neural networks) while Lecture 14 gave an overview of how these are entering in the fluid dynamics community. Lecture 15 presented how dynamic reduced-order models can be developed with machine-learning methods while Lecture 16 presented their applications for the extremely challenging case of reactive flows, illustrating how machine learning techniques can help reduce the computational burden and decode the complexity of turbulence-chemistry interactions.

 

The fifth day was dedicated to applications and the interplay of machine and human learning in the grand challenge problems of the field: turbulence closures and turbulence control. Lecture 17 is dedicated to the importance of reduced order modeling and machine learning for aerodynamic applications. Lecture 18 highlighted recent successes of automated learning of complex nonlinear MIMO control laws in experiments and simulations with a surprisingly simple and efficient regression solver. Lecture 19 closed the course offering a fascinating perspective on how the rapid developments of direct numerical simulations, together with machine learning methods, can be used for suggesting questions, rather than providing answers. This perspective opens a debate on whether machine learning will change the way we advance scientific knowledge and whether we will one day consider machines as colleagues.

Content of the lecture series

Part I: Coherent Structures

Lecture 1 - Prof. B.R. Noack Analysis, Modeling and Control of the Cylinder Wake

Lecture 2 - Prof. J. Jiménez Coherent Structures in Turbulent Flows

Lecture 3 - Prof. S.T.M. Dawson The Proper Orthogonal Decomposition

Lecture 4 - Prof. P.J. Schmid The Dynamic Mode Decomposition: From Koopman Theory to Applications

Part II: Mathematical Analysis

Lecture 5 - Prof. Mendez Mathematical Tools, Part I: Continuous and Discrete LTI Systems

Lecture 6 - Prof. Discetti Mathematical Tools, Part II: Time-Frequency Analysis

Lecture 7 - Prof. Mendez Generalized Modal Analysis and Multiscale POD

Lecture 8 - Prof. Ianiro Applications and Good Practice

Part III: Dynamical Systems

Lecture 9 - Prof. P.J. Schmid Modern Tools for the Stability Analysis of Fluid Flows

Lecture 10 - Prof. S.T.M. Dawson Linear Dynamical Systems and Control

Lecture 11 - Prof. S.L. Brunton Nonlinear Dynamical Systems

Lecture 12 - Prof. S.L. Brunton Methods for System Identification

Part IV: Reduced-Order Modeling

Lecture 13 - Prof. S.L. Brunton Introduction to Machine Learning Methods

Lecture 14 - Prof. S.L. Brunton Machine Learning in Fluids: Pairing Methods with Problems

Lecture 15 - Prof. B.R. Noack Machine Learning for Reduced-Order Modeling

Lecture 16 - Prof. A. Parente Advancing Reacting Flow Simulations with Data-Driven Models: Chemistry Accelerations and Reduced-Order Modeling

Part V: Control, Closure and Perspectives

Lecture 17 - Prof. S. Görtz Reduced-Order Modeling for Aerodynamic Applications and MDO

Lecture 18 - Prof. B.R. Noack Machine Learning for Turbulence Control

Lecture 19 - Prof. J. Jiménez The Computer as Turbulence Researcher

Outline of the Course

The course follows the historical path through which this field has been developed in thelatest few decades, opening with the tools which were pioneered by the fluid dynamics community with the aim of identifying coherent structures in turbulent flows, and proceeding towards recent machine learning methods that were initially developed in machine vision, pattern recognition, and artificial intelligence.
The first day is dedicated to the coherent patterns (structures) in turbulent flows, and aims at providing an overview of the subject and of the applications to fluid mechanics in which machine learning methods are bringing substantial contributions. In lecture 1, this overview is given using a well-known benchmark to build confidence, intuition and a clear understanding of the motivation of the course. Lecture 2 on turbulent structures further elaborates on the motivation of the field by addressing the problem of coherent structures in turbulent flows, giving a phenomenological picture of their origin, dynamics, and their effects on crucial engineering quantities (e.g., skin friction, noise production, flow separation phenomena, dynamic loads, etc.). The identification and the objective definition of these coherent structures have been the primary motivation for the development of decomposition methods in fluid mechanics. The following two lectures are dedicated to two classical decompositions that were originally developed with this aim: the Proper Orthogonal Decomposition (POD) and the Dynamic Mode Decomposition (DMD).
The second day is dedicated to the mathematical analysis. Lectures 5 and 6 on mathematical fundamentals will review continuous and discrete linear time-invariant systems and their properties (impulse response and transfer functions, poles, zeros, etc), the fundamental transforms used in their investigation (Laplace, Fourier and Z transform), methods for time-frequency analysis (Gabor transform, continuous and discrete wavelets) and the related uncertainty principle (Heisenberg principle). Lecture 7 builds on the first two by presenting a generalized framework of data-driven modal analysis and discusses limitations and strengths of various methods. The last Lecture of this day presents a collection of applications of data decompositions to numerical and experimental data typically encountered in fluid mechanics (e.g. DNS simulations or experimental velocity fields from PIV), presenting best practices and potential pitfalls.
The third day is dedicated to dynamical systems. Lecture 9 introduces stability analysis of fluid flows, including classical methods of analysis (normal modes, linearization, and Orr-Sommerfeld problem) an extensions to multiscale and multiphysics flows. Lecture 10 introduces linear dynamical systems, from their general formulation to their phase portrait and the basis of linear control theory (controllability, observability, Gramians, etc.). Lecture 11 introduces nonlinear dynamical systems and their distinctive features, including attractors, bifurcations and the route to chaos, with a connection to the theory of turbulent flows. Finally, Lecture 12 gives an overview of system identification methods, such as Eigensystem realization algorithm (ERA), observer Kalman Filter identification (OKID), autoregressive moving average (ARMA) and more.
The fourth part is dedicated to machine-learning methods and their application in data-driven reduced-order modeling. Most problems in fluid mechanics can be formulated as regression problems, i.e., optimizing a function with respect to data or a given system. Examples are feature extraction, model-order reduction (autoencoders), prediction, control, estimation, and modeling. Lecture 13 will provide an overview of the powerful regression solvers of machine learning (e.g., supervised versus non-supervised methods, clustering and classification methods, support vector machines, kernel methods, neural networks) while Lecture 14 gives an overview of how these are entering in the fluid dynamics community. Lecture 15 presents how dynamic reduced-order models can be developed with machine-learning methods while Lecture 16 presents their applications for the extremely challenging case of reactive flows.
The fifth day is dedicated to applications and the interplay of machine and human learning in the grand challenge problems of the field: turbulence closures and turbulence control. Lecture 17 is dedicated to the importance of reduced order modeling and machine learning for aerodynamic applications. Lecture 18 highlights recent successes of automated learning of complex nonlinear MIMO control laws in experiments and simulations with a surprisingly simple and efficient regression solver. Lecture 19 closes the course offering a fascinating perspective on how the rapid developments of direct numerical simulations, together with machine learning methods, can be used for suggesting questions, rather than providing answers. This perspective will also open a debate on whether machine learning will change the way we advance scientific knowledge and whether we will one day consider machines as colleagues.