Dynamic Programming and Stochastic Control

The course covers the basic models and solution techniques for stochastic optimal control problems and sequential decision-making under uncertainty. The course starts with a class of stochastic processes known as Markov chains. We then extend the model to include control variables (Markov Decision Processes, or in short MDPs) and study optimal control of a dynamical system over a finite number of stages. We also discuss an important class of problems known as optimal stopping problems. Different applications from engineering and finance to operations research and management science will be covered.