ITP-NYU :: 4/14/2016

Convnet applications

Class notes

News / admin

• making up materials from last week
• office hours
  • unofficial open office hours at DBRSLabs
• theory vs. code

Critical reading

• bias, ownership, privacy (1) (2)
• Stranger Visions (Heather Dewey-Hagborg)
  • data reconstruction from neural nets

Review convolutional neural networks

• Feature abstractions in neural nets
  • CIFAR-10 weights
• Understanding volumes
• Convolution + pooling layers
• What do the activations mean?
  • n-1 layer
• CIFAR-10 confusion matrix (live demo)

Applications of convnets

• Visualizing and interpreting convnets
  • Visualizing convnets (Zeiler, Fergus)
    • image patches which maximally activate neurons
    • occlusion + classification accuracy
  • applications to localization + segmentation + attention
  • reconstructing images from activations
    • applications to compression, security
  • deconvolution + guided backpropagation
  • Deep visualization toolbox (Yosinski, Clune)
• Image synthesis
  • Synthesizing images which excite neurons
  • Deepdream
    • Reddit leak
    • amplifying activations (set gradient = activations)
    • Regularizing with bilateral filters (Kyle McDonald + Mike Tyka)
  • Style transfer
    • A neural algorithm of artistic style
    • How style transfer works
      • balancing content and style loss
      • Gram matrices and "style" computation
    • Applications
      • Style transfer experiments
        • Alice in Wonderland stylenet
      • Style transfer studies
      • Deep forger
      • Neural doodle (semantic style transfer)
    • what if content loss is discarded?
      • Deep Texture
• Transfer learning
  • Low-level vs. high-level features
  • What if you want to train a complex model but have few training examples
  • ConvnetOSC -> Wekinator
• t-SNE
  • t-SNE (Laurens van der Maaten)
  • use t-SNE on n-1 activations
  • t-SNE in other contexts
    • wiki-TSNE (text)
  • implementations
    • ofxTSNE
    • sklearn
  • Misc
    • Visualizing MNIST (@colah)
    • Indico visualizations

Deep learning landscape (not in the video)

• Main platforms
  • Caffe, Theano, Torch, TensorFlow
  • Wrappers :)
    • Keras, tflearn
    • Lasagne, nolearn
  • openFrameworks support
    • real-time style transfer (@_mayfa)
    • ofxTensorFlow
• Set up
  • local machine?
    • Pros: free (more or less) and you own it
    • Cons: lots of dependencies, install is hard/inconsistent
  • Amazon EC2, Google Cloud Compute
    • Pros: no install, scales well
    • Cons: expensive
  • Terminal.com
    • Pros: same as Amazon/GCC but even nicer and more features
    • Cons: still expensive, and somewhat unreliable GPU support
  • ml-notebook
    • Pros: best of both worlds (local + easyish to install)
    • Cons: no GPU support