Adaptive Tuning Structures for Optimal Process Control and Graphical Games
Speaker:Prof. Frank L. Lewis
Fellow IEEE, Fellow IFAC, Fellow U.K. Inst. MC
Moncrief-O'Donnell Endowed Chair Professor
Head, Advanced Controls & Sensors Group
The University of Texas at Arlington
Date & Time:11 Nov 2013 (Monday) 10:30 - 11:30
Organized by:Department of Electromechanical Engineering


This talk will discuss some new adaptive control structures for learning online the solutions to optimal control problems and multi-player differential games. Applications in Process control and vehicle control are shown. Techniques from reinforcement learning are used to design a new family of adaptive controllers based on actor-critic mechanisms that converge in real time to optimal control and game theoretic solutions. A new sort of distributed game is defined- namely multi-agent graphical games, where the interactions between players are restricted by a distributed communication topology.

Optimal Control. Optimal feedback control design has been responsible for much of the successful performance of engineered systems in aerospace, industrial processes, vehicles, ships, robotics, and elsewhere since the 1960s. Optimal design allows for savings in fuel consumption, reduced resource usage, increased actuator life, and minimum processing times. However, optimal control design is performed offline by solving optimal design equations including the algebraic Riccati equation (ARE) and the Game ARE. It is difficult to perform optimal designs for nonlinear systems since they rely on solutions to complicated Hamilton-Jacobi-Bellman (HJB). Finally, optimal design generally requires that the full system dynamics models be known. System identification to determine the process dynamics can be expensive and may lead to inaccurate dynamics models. Also, dynamics models may change over time.

Optimal Adaptive Control. Adaptive control has provided powerful techniques for online learning of effective controllers for unknown nonlinear systems. It has especially been used in industrial process control for decades. Adaptive control does not generally produce optimal control solutions. Therefore, in this talk we discuss online adaptive algorithms for learning optimal control solutions for continuous-time linear and nonlinear systems. This is a novel class of adaptive control algorithms that converge to optimal control solutions by online learning in real time. It is a data-based approach that uses measurements of process inputs and outputs in real time. System dynamics models are not needed. In the linear quadratic (LQ) case, the algorithms learn the solution to the ARE by adaptation along the system motion trajectories. In the case of nonlinear systems with general performance measures, the algorithms learn the (approximate smooth local) solutions of HJB equations. The algorithms are based on actor-critic reinforcement learning techniques.

Multi-Player Differential Games. Multi-player control systems are important in disturbance rejection and control of multiple coordinated processes or vehicles. New algorithms will be presented for solving online the non zero-sum multi-player games for continuous-time systems. Applications to disturbance compensation will be shown. We use an adaptive control structure motivated by reinforcement learning policy iteration. The result is an adaptive control system with multiple tuned control loops that learns based on the interplay of agents in a game, to deliver true online gaming behavior.

Graphical Games. A new formulation for control of multi-agent cooperative systems is given. A novel form of game among agents in a communication graph is formulated where each agent is allowed to interact only with its neighbors. A new notion of Nash equilibrium is defined that is suitable for graphical games.


F.L. Lewis, Fellow IEEE, Fellow IFAC, Fellow U.K. Institute of Measurement & Control, PE Texas, U.K. Chartered Engineer, is Distinguished Scholar Professor, Distinguished Teaching Professor, and Moncrief-O’Donnell Chair at The University of Texas at Arlington Research Institute. IEEE Control Systems Society Distinguished Lecturer. He obtained the Bachelor's Degree in Physics/EE and the MSEE at Rice University, the MS in Aeronautical Engineering from Univ. W. Florida, and the Ph.D. at Ga. Tech. He works in feedback control, reinforcement learning, intelligent systems, and distributed control systems. He is author of 6 U.S. patents, 273 journal papers, 375 conference papers, 15 books, 44 chapters, and 11 journal special issues. He received the Fulbright Research Award, NSF Research Initiation Grant, ASEE Terman Award, Int. Neural Network Soc. Gabor Award 2009, U.K. Inst Measurement & Control Honeywell Field Engineering Medal 2009. Received IEEE Computational Intelligence Society Neural Networks Pioneer Award 2012. Distinguished Foreign Scholar, Nanjing Univ. Science & Technology. Project 111 Professor at Northeastern University, China. Received Outstanding Service Award from Dallas IEEE Section, selected as Engineer of the Year by Ft. Worth IEEE Section. Listed in Ft. Worth Business Press Top 200 Leaders in Manufacturing. Received the 2010 IEEE Region 5 Outstanding Engineering Educator Award and the 2010 UTA Graduate Dean’s Excellence in Doctoral Mentoring Award. Elected to UTA Academy of Distinguished Teachers 2012. He served on the NAE Committee on Space Station in 1995. Founding Member of the Board of Governors of the Mediterranean Control Association. Helped win the IEEE Control Systems Society Best Chapter Award (as Founding Chairman of DFW Chapter), the National Sigma Xi Award for Outstanding Chapter (as President of UTA Chapter), and the US SBA Tibbets Award in 1996 (as Director of ARRI’s SBIR Program).