Date Topic covered Assignments
22-Aug Class syllabus
Introduction to Artificial Intelligence (AI)
Introduction to Deep Learning
Introduction to Keras/TensorFlow
29-Aug Applied Mathematics (all) Deep Learning,
Introduction to Linear Algebra (all) Unstructured Data,
Introduction to Probability (all) Deep Learning, Chapters 1-4
Introduction to Information Theory (optional) Artificial Neural Networks,
HW0 (all) Big O,
5-Sep Information Theory & Machine Learning Fundamentals (all) Deep Learning, Chapters 4-5
Introduction to Machine Learning (Part 1)
Assignment 1
HW0 due
12-Sep Introduction to Machine Learning (Part 2) (all) Deep Learning, Chapter 5
19-Sep The Multilayer Perceptron & Feedforward Networks (all) Deep Learning, Chapter 6
Architecture & Gradient-Based Learning (optional) "Learning Representations by Back-Propagating Errors" (Nature),
Team 1 Learning to learn by gradient descent by gradient descent(
Team 2 Distilling the knowledge in a neural network(
26-Sep Combating Overparametrization:  Regularization (all) Deep Learning, Chapter 7
Team 3 Dropout: A simple way to prevent neural networks from overfitting(
Team 4 Regularizing Deep Neural Networks by Noise: Its Interpretation and Optimization(
Assignment 1 Due
3-Oct Team Presentations: Project Proposal
Team Order 1 to 12
10-Oct Team Presentations continued; TensorFlow Examples
Team 5 Modified quasi-Newton methods for training neural networks(
Team 6 Incorporating Nesterov Momentum into Adam.(
17-Oct Optimization (all) Deep Learning, Chapter 8
Team 7 Layer Normalization(
Team 8 A Framework for Self-Tuning Optimization Algorithm(
Assignment 2
24-Oct Convolutional Networks (all) Deep Learning, Chapter 9
Team 9 Inception-v4, inception-resnet and the impact of residual connections on learning(
Team 10 Rethinking the inception architecture for computer vision(
31-Oct Recurrent Neural Networks (all) Deep Learning, Chapter 10
Team 11 Exploring the limits of language modeling(
Team 12 Effective approaches to attention-based neural machine translation(
7-Nov Practical Methodology, Autoencoders (all) Deep Learning, Chapter 11; Autoencoders, Chapter 14
Assignment 2 Due (optional) Neural Networks: Tricks of the Trade, 2nd Edition -- "Practical Recommendations for Gradient-Based Training of Deep Architectures",
Team 13 Algorithms for Hyper-Parameter Optimization(
Team 14 Gradient-based Hyperparameter Optimization through Reversible Learning(
14-Nov Neural Networks with Memory (all)
Local representation Alignment
CNNs plus RNNs
GANs for Distractor Generation
21-Nov Thanksgiving break, no classes
28-Nov Team Presentations
Teams 1-7
5-Dec Team Presentations
Teams 8-14
10-Dec Project Reports Due 2 hard copies - ACM paper format 9pt single space double sided.