ITP-NYU :: 4/28/2016

Game AI and deep reinforcement learning

Class notes

News / admin

• ML amplifies privilege
  • sentiment analysis + automated content
• OpenAI reinforcement learning gym
• incredible style transfer video with temporal loss term
• alt-AI exhibition

Review: the whole class "in 10 minutes"

• Esepcailly review convnets and applications, and RNNs
• repetition = mental backpropagation

Generative Models

• Autoencoders
  • MNIST variational autoencer in Tensorflow
  • Training autoencoders on subsets of celebrity faces
• Generative adversarial networks
  • Deep convolutional generative adversarial networks and code
  • A Book from the Sky
  • DCGAN manga (@mattya)
  • DCGAN flowers (@vintermann)
  • VAE + GAN making numbers (@hardmaru)

Video game AI + reinforcement learning

• Reinforcement learning
  • Markov decision process + {States, Actions, Rewards}
  • Q-Learning in video games
    • goal: function which advises an action given a state
    • Convnets playing Atari games (DeepMind)
• challenges
  • time matters
  • reward can be delayed
  • exploration vs. exploitation
    • multi-armed bandits
    • discount factor
• Applications
  • maneuver a body (robotics)
  • manage something (investment portfolio, power stations, industrial applications)
  • play games :)
    • arms race? AI vs impossible super mario levels?

How AlphaGo works

• Tic-Tac-Toe
  • parsing game trees
• Chess
  • Monte Carlo Tree Search
  • handcrafted value function
  • DeepBlue vs. Gary Kasparov
• Go
  • AlphaGo Nature paper
  • policy + value convnets
  • self-play to train value network
  • MCTS + policy + value
  • AlphaGo vs. Lee Se-dol
• It gets waaaay harder (like Doom)