Toronto Networking Seminar

Since the first multi-armed bandit (MAB) problem posed by Thompson in 1933 for the application of clinical trials, MAB has developed into an important branch in stochastic optimization and machine learning and has found a wide range of applications in economics and finance, medicine, and industrial engineering. It has recently received increasing attention from the communications and networking research community for formulating and tackling the optimization of learning and activation in dynamic systems with unknown models. A mathematical abstraction of the MAB problems involves a player who can operate one of N arms at each time, with each yielding a random reward drawn from an unknown distribution when operated. The objective is an arm selection policy that minimizes the regret defined as the performance loss with respect to a genie who knows the reward model of each arm. In this talk, we present our recent results that extend the classic MAB theory in three directions: from exponential family of reward distributions to heavy tail reward distributions, from a single player to multiple distributed players, and from i.i.d. reward
models to restless Markov reward models.