This post is part of a series of posts on optimal control theory. We take a detalied look on how classical LQR control is derived. A simple implementation is provided for clarity.
LQR is an extremely popular optomal control framework. This blog closely follows (Duriez, Brunton, and Noack 2017).
Small
Let’s consider the linear system
\[\begin{align} \dot{\mathbf{x}} &= \mathbf{A}\mathbf{x} + \mathbf{B}\mathbf{u}\\ \mathbf{y} &= \mathbf{C}\mathbf{x} + \mathbf{D}\mathbf{u}\tag{1} \end{align}\]
If the system in (1) is controllable then a proportional controller can be designed as
\[\begin{equation} \mathbf{u} = -\mathbf{K}_r \mathbf{x}\tag{2} \end{equation}\]
Hence the closed loop system becomes
\[\begin{equation} \dot{\mathbf{x}} = (\mathbf{A}-\mathbf{B}\mathbf{K}_r)\mathbf{x}\tag{3} \end{equation}\]
We can construct a quadratic cost \(J\) that balances the regulation of \(\mathbf{x}\) with the cost of control input \(\mathbf{u}\),
\[\begin{equation} J(t) = \int_0^t [\mathbf{x}^T(\tau)\mathbf{Q}\mathbf{x}(\tau)] + \mathbf{u}^T(\tau)\mathbf{R}\mathbf{u}(\tau)]\tag{4} \end{equation}\]
By solving Algebraic Riccati Equation (ARE) we get the optimal control law,
\[\begin{equation} \mathbf{K}_r = \mathbf{R}^{-1}\mathbf{B}^T\mathbf{P}\tag{5} \end{equation}\]
where the ARE is expressed as
\[\begin{equation} \mathbf{A}^T\mathbf{P} + \mathbf{P}\mathbf{A} - \mathbf{P}\mathbf{B}\mathbf{R}^{-1}\mathbf{B}^T\mathbf{P} + \mathbf{Q} = 0\tag{6} \end{equation}\]
Duriez, Thomas, Steven L Brunton, and Bernd R Noack. 2017. Machine Learning Control-Taming Nonlinear Dynamics and Turbulence. Springer.