Siddharth (Sid) Mysore

Siddharth (Sid) Mysore

AI Scientist at Electronic Arts

Publications

Multi-Critic Actor Learning: Teaching RL Policies to Act with Style

Published in International Conference on Learning Representations, 2022

Using a single value function (critic) shared over multiple tasks in Actor-Critic multi-task reinforcement learning (MTRL) can result in negative interference between tasks, which can compromise learning performance. Multi-Critic Actor Learning (MultiCriticAL) proposes instead maintaining separate critics for each task being trained while training a single multi-task actor.

Recommended citation: Mysore, S., Cheng, G., Zhao, Y., Saenko, K, & Wu, M. "Multi-Critic Actor Learning: Teaching RL Policies to Act with Style". International Conference on Learning Representations https://openreview.net/forum?id=rJvY_5OzoI

Designing Composites with Target Effective Young’s Modulus using Reinforcement Learning

Published in ACM Symposium on Computational Fabrication, 2021

In this work, we develop and utilize a Reinforcement learning (RL)-based framework for the design of composite structures which avoids the need for user-selected training data. Models can be trained using just 2.78% of the total design space and are capable of finding designs at a success rate exceeding 90%.

Recommended citation: Gongora, A.E., Mysore, S., Li, B., Shou, W., Matusik, W., Morgan, E.F., Brown, K.A., & Whiting, E. "Designing Composites with Target Effective Youngs Modulus using Reinforcement Learning". ACM Symposium on Computational Fabrication https://arxiv.org/abs/2110.05260

Honey, I Shrunk The Actor: A Case Study on Preserving Performance with Smaller Actors in Actor-Critic RL

Published in IEEE Conference on Games, 2021

By relaxing the assumption of architectural symmetry, it is often possible for smaller actors to achieve comparable policy performance to their symmetric counterparts. Our experiments show up to 97% reduction in the number of network weights with an average reduction of 77% over multiple algorithms on multiple tasks. Project Website: http://ai.bu.edu/littleActor/

Recommended citation: Mysore, S., Mabsout, B., Mancuso, R., & Saenko, K. (2021). "Honey, I Shrunk The Actor: A Case Study on Preserving Performance with Smaller Actors in Actor-Critic RL"IEEE Conference on Games 2021 https://arxiv.org/abs/2102.11893

Regularizing Action Policies for Smooth Control with Reinforcement Learning

Published in IEEE International Conference on Robotics and Automation (ICRA), 2021

We introduce Conditioning for Action Policy Smoothness (CAPS), an effective yet intuitive regularization on action policies, which offers consistent improvement in the smoothness of the learned state-to-action mappings of neural network controllers, reflected in the elimination of high-frequency components in the control signal. Project Website: http://ai.bu.edu/caps/

Recommended citation: Mysore, S., Mabsout, B., Mancuso, R., & Saenko, K. (2021). "Regularizing Action Policies for Smooth Control with Reinforcement Learning", IEEE International Conference on Robotics and Automation 2021, Xian, China. https://arxiv.org/abs/2012.06644

How to Train your Quadrotor: A Framework for Consistently Smooth and Responsive Flight Control via Reinforcement Learning

Published in ACM Transactions on Cyber-Physical Systems, 2021

To combat issues of instability in RL agents, we propose a systematic framework, REinforcement-based transferable Agents through Learning (RE+AL), for designing simulated training environments which preserve the quality of trained agents when transferred to real platforms.

Recommended citation: Mysore, S., Mabsout, B., Saenko, K., & Mancuso, R. (2021). "How to Train your Quadrotor: A Framework for Consistently Smooth and Responsive Flight Control via Reinforcement Learning". ACM Trans. Cyber-Phys. Syst. 5, 4, Article 36 (October 2021), 24 pages. DOI:https://doi.org/10.1145/3466618 https://arxiv.org/abs/2012.06656

[Workshop paper] Reward-guided Curriculum for Learning Robust Action Policies

Published in Workshop on Multi-task and Lifelong Reinforcement Learning at ICML 2019, 2019

We propose a novel method to develop robust actor policies, by automatically developing a sampling curriculum over environment settings to use in training

Recommended citation: Mysore, S., Platt, R., Saenko, K. (2019). "Reward-guided Curriculum for Learning Robust Action Policies.", Workshop on Multi-task and Lifelong Reinforcement Learning at ICML 2019, Long beach CA http://sidmysore.github.io/files/RgC_MTLRL_Workshop_ICML_2019.pdf