Reinforcement Learning: A Declarative Paradigm for Game AI

A Retrospective on my work at yuri.ai

How I started working on Yuri

When I left Microsoft, I had a rough idea that I wanted to work in the gaming space but not exactly what. I had about 2 years to figure something out which I thought was ample time.

What was the point of Yuri

I thought I’d program some simple AI to play against. But, every time I would make a design change, I would need to go update my AI and over time this was making me furious.

  1. Dropping in bots that behave like players that just got disconnected
  2. A guided trainer that’s always slightly better than you to become pro
  • Medium companies: Couldn’t justify spending $10K on server costs for RL training
  • Small companies: Couldn’t pay me for what amounted to time consuming consulting work

Reinforcement Learning vs Decision Tree

TL;DR:

  • Decision Tree: Procedural, need to spend money on programmers and domain experts
  1. It’s too expensive
  2. It’s too slow
  3. It’s unreliable
  4. It’s too hard to interpret and debug, how am I supposed to make updates
  5. It removes the craftsmanship, I need to make my AI fun
  1. Tools to make an agent slightly worse in case it’s too good
  2. Tools to bootstrap training

Next Steps

I hope this article helps you better understand what I was trying to do with my Reinforcement Learning research at Yuri.ai and why I still believe that Reinforcement Learning will be a game changer for Strategy Games.

Acknowledgements

Thank you to Stephane Mourani for carefully proofreading this article.

Robots will save us

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store