Distinguished Lecture - Reinforcement Learning as Software Engineering for Agents

Location: 2150 Torgersen Hall
Date: Friday December 5, 2014
Time: 11:15-12:30pm
This talk is open to the general public.

Dr. Michael Littman
Brown University


Reinforcement learning (RL) is a subarea of machine learning and artificial intelligence concerned with agents improving their behavior over time via experience. Introduced as a concept in psychology in the 1940s, a central tenant of RL is that behavior is driven by a desire to maximize rewarding stimuli. I argue that RL, in a computer-science context, can be seen as a software engineering methodology for agents in complex, uncertain environments. In this analogy, reward functions are programs and learning algorithms are compilers. The field has focused almost exclusively on the design of algorithms for finding reward maximizing behavior (compilers), but not much attention has been paid to the role of the rewards themselves (the programming language). I will describe efforts by my colleagues and me to probe the nature of the rewards-as-programs idea with the goal of moving toward higher-level specifications with a clear semantics.


Michael L. Littman's research in machine learning examines algorithms for decision making under uncertainty.  He has earned multiple awards for teaching and his research has been recognized with three best-paper awards on the topics of meta-learning for computer crossword solving, complexity analysis of planning under uncertainty, and algorithms for efficient reinforcement learning.  Littman has served on the editorial boards for the Journal of Machine Learning Research and the Journal of Artificial Intelligence Research.  He was general chair of International Conference on Machine Learning 2013 and program chair of the Association for the Advancement of Artificial Intelligence Conference 2013. He is one of the co-organizers of Brown's interdisciplinary "Humanity Centered Robotics Initiative" (hcri.brown.edu <http://hcri.brown.edu/> ).