The Robot Overlord Manual

8 min readNov 13, 2019

After reading this book you should be a lot better prepared to start designing robots at home. I’m mostly focused on math and software because that’s what I know best.

This is a roughly 250 page textbook covering what are in my mind the most powerful ideas in Robotics and Machine Learning. My chapters have regularly made it to the front page of Hacker News and Google Search so subscribe and join the hundreds of thousands of readers that are looking to become Robot Overlords.

Prologue

How to Build Robots at Home — will give you a broad overview of the robotics at home problem including manufacturing, electronics and software.

The Robot Overlord Manual is primarily about simulating robots in software.

Deep Learning

Deep Learning is a programming technique where instead of describing how to go from inputs to outputs, you have labels from inputs to outputs and can generate the program automatically. Most applications so far have focused on image classification or text classification or generation but more recently Deep Learning has been used for robotic control problems.

Can I use Deep Learning for that? A Type Theoretic Heuristic — This is a useful heuristic I’ve developed to figure out what’s possible and what isn’t with Deep Learning.
Automatic Differentiation Step by Step — The reason we can even think of Neural Networks as types or functions from one problem space to another is because Automatic Differentiation abstracts away all the complexity. It’s the secret sauce of all the popular Deep Learning libraries out there today.
Probabilistic Programming — coming soon

Reinforcement Learning

Reinforcement Learning is a technique to describe goals to robots and then have them execute them without telling them how to do it.

How to Implement a Reinforcement Learning Library from Scratch — Reinforce.jl deep dive — How does Reinforcement Learning work and how does a real Reinforcement Learning library work?
Controlling a 2D Robotic Arm with Deep Reinforcement Learning from scratch — Getting Reinforcement Learning to work on a continuous robotics control problem requires a few tweaks and someone still needs to build a robotic simulator.
Build your own Reinforcement Learning environments with Unity ML agents — Programming your own Reinforcement Learning environment without an engine is challenging, Game Development is by far the most complex field in Computer Science. Unity is an easy to use game engine which will help you design arbitrarily complex robot simulations.
How to Solve Board Games — Board games are useful to test cognitive skills without involving physics. We can test strategic thinking or bluffing depending on whether we’re looking to play Chess or Poker. If the robots ever rise up, they will likely get rich playing and gambling on online games first.
Multi Agent Reinforcement Learning — coming soon

Physics

Robots are physical objects moving in space and it’s challenging to describe them using just Newton’s laws. Different representations have different advantages so this section is about how we can improve physics engines to support more scalable Reinforcement Learning

How to Turn physics into an Optimization Problem — Physics Engines are hard to build and slow but we can simplify them with a unique insight that’s very familiar to the Machine Learning community.
How to Move? Lie Group Robotics — calculations for robotic joints involve all sorts of complicated expressions, we can simplify these expressions drastically to come up with almost trivial solutions to the Inverse and Forward Kinematics Problems
Representation Theory for Robotics — draft, still needs illustrations
Dynamical Systems — coming soon

Software Design Patterns

Building software is hard so it’s important to constantly cultivate better tools and design patterns. Object Oriented Programming teaches lots of bad habits and often results in poor performance unless you’re an expert user.

Design patterns that I find a lot more exciting

Writing Compositional Software with Multiple Dispatch — most design pattern books don’t talk about design patterns that are particularly well suited for scientific code. This interactive article in a Pluto.jl notebook will show you how multiple dispatch lets you write scientific code the way it should have always been written.
Entity Component System is all you need — ECS is a design pattern that finds it roots in game development to write more maintainable data oriented code and also take better advantage of improvements in CPUs while making better use of cache locality to help out RAM which has barely improved over the years

Epilogue

If there’s anything else you’d really like me to cover, please don’t hesitate to reach out to me either in the comments section below or on Twitter.

But if you’re hungry for more content.

Machine Learning: The Great Stagnation — Risk free rewards, feedback loops, fake rigor and graduate student descent have created an environment where risk free rewards are now celebrated Machine Learning is now facing a great stagnation of ideas, the bureaucrats are running the asylum
Deschooling Society — A summary of my thoughts on the modern education system. Explains why I wrote this book and why I believe that self learning is superior to universities. If you’re interested in more homeschooling articles I’d recommend “One World Schoolhouse”
How to get a great job — Learning tons of stuff and sharing what you’re learning can help you land a dream job in a more fulfilling and reliable way than hacking a CV
Technical Books I Love — My work involved reading tons of books so naturally I found a few amazing ones. This is the full list, 8 years in the making.
Exterior Product — Geometric Algebra is a language for geometry problems that can act as a replacement for Linear Algebra. Given how often we deal with geometry and spaces in robotics and deep learning, it’s only natural we look for better formalisms.
How I Read Technical Books — Because I had to read a lot of books, I’ve developed some tricks to make the reading process more efficient and less painful. Hopefully they help you too
A Very Brief Visual History of Mathematics — People often tell you should only read math when you really need it but my experience tells me that knowing more about the history of math makes the motivation of each mathematical technique crystal clear. My goal with this post was that anyone should be able to read it.
Reinforcement Learning: A Declarative Paradigm for Game AI — reflections on my work at yuri.ai — The question that started it all. Working on Yuri was how I got interested in robotics, graphics and reinforcement learning in the first place. This book is a summary of everything I learned working on Yuri.
Under-explored Game Genres for Reinforcement Learning Research — different video games test vastly different skills so you’re guaranteed to learn something new bench-marking on a new kind of game instead of getting +0.001% on Atari.
A beginner’s guide to DnD — DnD is not a tactics game, it’s a collaborative storytelling exercise you play with your friends. It’s a great way to open up and be creative and will help you craft better stories.

Why should you read this?

The state of the textbook

When I was working on Yuri.ai, I had to self teach myself a lot of unfamiliar concepts from math to physics to machine learning.

I learn best from textbooks but they have a few problems that I’ve always hated.

Problems with no solutions: From homework to exercises left to the reader — you can’t validate your knowledge.
Long and comprehensive: Many textbooks are written with the intent to survive hundreds of years, they try to synthesize all advances in a field without focusing on the few key ideas. Textbooks were invented before the internet and have failed to adapt to it.
No motivation: Ideas are always declared to be important without any explanations of its pros and cons. Textbooks are great at details but often fail to communicate the point of learning some technique.
Written by professors for professors: Textbooks are generally read by PhD students and Professors within a given community. The “curse of knowledge” means notation and explanations will favor experts not beginners.
Once you’re stuck you’re stuck: Ideas become progressively more complicated in a textbook so once you’re lost, you’re lost.