Yunus Emre Demirci

Reading List

Recommended Papers

.

Note: .

Foundations of Stochastic Control

  1. Bertsekas, D. P., and Shreve, S. E. Stochastic Optimal Control: The Discrete-Time Case. Academic Press, 1978. A canonical reference for measurability subtleties in dynamic programming.
  2. Hernández-Lerma, O., and Lasserre, J. B. Discrete-Time Markov Control Processes. Springer, 1996. Clean treatment of average-cost criteria and existence results.
  3. Yüksel, S., and Başar, T. Stochastic Networked Control Systems. Birkhäuser, 2013. Information-theoretic perspective on control under communication constraints.

Nonlinear Filtering & Ergodicity

  1. Van Handel, R. The stability of conditional Markov processes and Markov chains in random environments. Annals of Probability, 37(5):1876–1925, 2009. Foundational result on filter stability under mild assumptions.
  2. Chigansky, P., Liptser, R., and van Handel, R. Intrinsic methods in filter stability. Handbook of Nonlinear Filtering, Oxford Univ. Press, 2011. Pedagogical survey — good entry point into the area.
  3. McDonald, C., and Yüksel, S. Exponential filter stability via Dobrushin's coefficient. Electronic Communications in Probability, 25, 2020. A short, sharp account of the Dobrushin contraction technique.

POMDPs & Reinforcement Learning

  1. Kara, A. D., and Yüksel, S. Near optimality of finite memory feedback policies in partially observed Markov decision processes. Journal of Machine Learning Research, 23, 2022. Finite-window approximations with explicit error bounds.
  2. Saldi, N., Linder, T., and Yüksel, S. Finite Approximations in Discrete-Time Stochastic Control. Birkhäuser, 2018. Quantization-based approximations with convergence guarantees.
  3. Singh, S., Jaakkola, T., and Jordan, M. Learning without state-estimation in partially observable Markovian decision processes. ICML, 1994. Classic motivation for finite-memory policies in RL.

Probability & Graph Theory

  1. Aldous, D., and Fill, J. Reversible Markov Chains and Random Walks on Graphs. Unfinished monograph, 2002. The standard reference for mixing, cover, and hitting times.
  2. Levin, D., Peres, Y., and Wilmer, E. Markov Chains and Mixing Times. AMS, 2nd ed., 2017. A friendly modern textbook on mixing-time techniques.