Date | Topic covered | Assignments | ||||||||

22-Aug | Class syllabus | |||||||||

Introduction to Artificial Intelligence (AI) | https://en.wikipedia.org/wiki/Artificial_intelligence | |||||||||

Introduction to Deep Learning | https://en.wikipedia.org/wiki/Deep_learning | |||||||||

Introduction to Keras/TensorFlow | ||||||||||

29-Aug | Applied Mathematics | (all) Deep Learning, https://en.wikipedia.org/wiki/Deep_learning | ||||||||

Introduction to Linear Algebra | (all) Unstructured Data, http://en.wikipedia.org/wiki/Unstructured_data | |||||||||

Introduction to Probability | (all) Deep Learning, Chapters 1-4 | |||||||||

Introduction to Information Theory | (optional) Artificial Neural Networks, https://en.wikipedia.org/wiki/Artificial_neural_network | |||||||||

HW0 | (all) Big O, http://en.wikipedia.org/wiki/Big_O_notation | |||||||||

5-Sep | Information Theory & Machine Learning Fundamentals | (all) Deep Learning, Chapters 4-5 | ||||||||

Introduction to Machine Learning (Part 1) | ||||||||||

Assignment 1 | ||||||||||

HW0 due | ||||||||||

12-Sep | Introduction to Machine Learning (Part 2) | (all) Deep Learning, Chapter 5 | ||||||||

19-Sep | The Multilayer Perceptron & Feedforward Networks | (all) Deep Learning, Chapter 6 | ||||||||

Architecture & Gradient-Based Learning | (optional) "Learning Representations by Back-Propagating Errors" (Nature), https://www.nature.com/nature/journal/v323/n6088/abs/323533a0.html | |||||||||

Team 1 | Learning to learn by gradient descent by gradient descent(https://arxiv.org/abs/1606.04474) | |||||||||

Team 2 | Distilling the knowledge in a neural network(https://arxiv.org/abs/1503.02531) | |||||||||

26-Sep | Combating Overparametrization: Regularization | (all) Deep Learning, Chapter 7 | ||||||||

Team 3 | Dropout: A simple way to prevent neural networks from overfitting(https://www.cs.toronto.edu/~hinton/absps/JMLRdropout.pdf) | |||||||||

Team 4 | Regularizing Deep Neural Networks by Noise: Its Interpretation and Optimization(https://arxiv.org/abs/1710.05179) | |||||||||

Assignment 1 Due | ||||||||||

3-Oct | Team Presentations: Project Proposal | |||||||||

Team Order 1 to 12 | ||||||||||

10-Oct | Team Presentations continued; TensorFlow Examples | |||||||||

Team 5 | Modified quasi-Newton methods for training neural networks(https://www.sciencedirect.com/science/article/pii/0098135495002286) | |||||||||

Team 6 | Incorporating Nesterov Momentum into Adam.(https://openreview.net/pdf?id=OM0jvwB8jIp57ZJjtNEZ) | |||||||||

17-Oct | Optimization | (all) Deep Learning, Chapter 8 | ||||||||

Team 7 | Layer Normalization(https://arxiv.org/abs/1607.06450) | |||||||||

Team 8 | A Framework for Self-Tuning Optimization Algorithm(https://arxiv.org/abs/1312.5667) | |||||||||

Assignment 2 | ||||||||||

24-Oct | Convolutional Networks | (all) Deep Learning, Chapter 9 | ||||||||

Team 9 | Inception-v4, inception-resnet and the impact of residual connections on learning(https://arxiv.org/abs/1602.07261) | |||||||||

Team 10 | Rethinking the inception architecture for computer vision(https://arxiv.org/abs/1512.00567) | |||||||||

31-Oct | Recurrent Neural Networks | (all) Deep Learning, Chapter 10 | ||||||||

Team 11 | Exploring the limits of language modeling(https://arxiv.org/abs/1602.02410) | |||||||||

Team 12 | Effective approaches to attention-based neural machine translation(https://arxiv.org/abs/1508.04025) | |||||||||

7-Nov | Practical Methodology | (all) Deep Learning, Chapter 11 | ||||||||

Assignment 2 Due | (optional) Neural Networks: Tricks of the Trade, 2nd Edition -- "Practical Recommendations for Gradient-Based Training of Deep Architectures", https://link.springer.com/chapter/10.1007/978-3-642-35289-8_26 | |||||||||

Team 13 | Algorithms for Hyper-Parameter Optimization(https://papers.nips.cc/paper/4443-algorithms-for-hyper-parameter-optimization.pdf) | |||||||||

Team 14 | Gradient-based Hyperparameter Optimization through Reversible Learning(https://arxiv.org/abs/1502.03492) | |||||||||

14-Nov | Neural Networks with Memory | (all) Assigned papers | ||||||||

Representation Learning | (all) Deep Learning, Chapter 14 | |||||||||

Neural Discrete Representation Learning(https://arxiv.org/abs/1711.00937) | ||||||||||

CM-GANs: Cross-modal Generative Adversarial Networks for Common Representation Learning(https://arxiv.org/abs/1710.05106) | ||||||||||

21-Nov | Thanksgiving break, no classes | |||||||||

28-Nov | Team Presentations | |||||||||

5-Dec | Team Presentations | |||||||||

10-Dec | Project Reports Due | 2 hard copies - ACM paper format 9pt single space double sided. | ||||||||