Real-world applications pose distinct challenges for decision-making algorithms, especially during the deployment phase. These include high-dimensional observation and action spaces, tasks that may be partially observable or non-stationary, and feedback that is often unspecified, delayed, or corrupted. Furthermore, feedback rarely comes in the form of a scalar reward function, exploration is often prohibitively costly, and safety considerations are a prerequisite for trained models to be deployed.

Reinforcement learning (RL) and contextual bandit (CB) algorithms have been studied in many settings, including healthcare, recommender and advertising systems, resource allocation and operations management, hardware and engineering design, and more recently, large foundation models. Despite many studies focusing on applying RL to such domains, the majority of solutions are not eventually deployed. An important question for the RL community to ask is:

This workshop aims to place a spotlight on this topic, with the goal of advancing RL and CB algorithms towards becoming a widespread industry standard. To this end, we invite contributions on theory and practice of RL aimed at facilitating deployment to real-world problems, examples of which include but are not limited to:

• Methods to enable deployability in RL: offline RL, offline policy evaluation (OPE), offline to online RL, safe learning and exploration, preference learning, learning in partially observable settings, interpretable policies in high-stakes settings, and methods for improving deployment efficiency.
• Applications in personalization and recommendation systems: applications of RL and CB methods in technology platforms that interact with humans (e.g., audio and video recommendations, online advertising, LLMs).
• Applications in industrial automation: applications of RL and CB methods in the control of physical systems or allocation of resources (e.g., inventory management, ride-sharing, commercial cooling systems, data center congestion control).
• Applications in decision systems for healthcare: applications of RL and CB methods in healthcare (e.g., managing chronic conditions, digital treatment recommendations, human-in-the-loop RL).

The above are only a handful of suitable topics. We welcome submissions on any topic that focuses on the RL deployment process. The submissions can be 4-8 pages in length and be of either a theoretical/methodological or empirical nature.

Keynote Speakers

Hamsa Bastani
Wharton, University of Pennsylvania

Daniel Russo
Columbia Business School

Aviral Kumar
Google DeepMind
CMU

John Langford
Microsoft Research NYC

Hongseok Namkoong
Columbia Business School

Panel

Omer Gottsman
Amazon

Daniel Russo
Columbia Business School

Kaushik Subramanian
Sony AI

Cathy Wu
MIT

Zheqing (Bill) Zhu
Meta
Panel moderator

Schedule

The schedule below is tentative and subject to change.

Organizers

Jalaj Bhandari

Meta

Yonathan Efroni
Meta

Mohammad Ghavamzadeh
Amazon

Daniel R. Jiang
Meta

Aldo Pacchiano
Boston University

Yi Wan
Meta

Kelly W. Zhang
Columbia Business School

Angela Zhou
Marshall School of Business, USC