Date Topic covered Assignments
     
22-Aug Class syllabus
Introduction to Artificial Intelligence (AI) https://en.wikipedia.org/wiki/Artificial_intelligence
Introduction to Deep Learning https://en.wikipedia.org/wiki/Deep_learning
Introduction to Keras/TensorFlow
29-Aug Applied Mathematics (all) Deep Learning, https://en.wikipedia.org/wiki/Deep_learning
Introduction to Linear Algebra (all) Unstructured Data, http://en.wikipedia.org/wiki/Unstructured_data
Introduction to Probability (all) Deep Learning, Chapters 1-4
Introduction to Information Theory (optional) Artificial Neural Networks, https://en.wikipedia.org/wiki/Artificial_neural_network
HW0 (all) Big O, http://en.wikipedia.org/wiki/Big_O_notation
5-Sep Information Theory & Machine Learning Fundamentals (all) Deep Learning, Chapters 4-5
Introduction to Machine Learning (Part 1)
Assignment 1
HW0 due
12-Sep Introduction to Machine Learning (Part 2) (all) Deep Learning, Chapter 5
19-Sep The Multilayer Perceptron & Feedforward Networks (all) Deep Learning, Chapter 6
Architecture & Gradient-Based Learning (optional) "Learning Representations by Back-Propagating Errors" (Nature), https://www.nature.com/nature/journal/v323/n6088/abs/323533a0.html
Team 1 Learning to learn by gradient descent by gradient descent(https://arxiv.org/abs/1606.04474)
Team 2 Distilling the knowledge in a neural network(https://arxiv.org/abs/1503.02531)
26-Sep Combating Overparametrization:  Regularization (all) Deep Learning, Chapter 7
Team 3 Dropout: A simple way to prevent neural networks from overfitting(https://www.cs.toronto.edu/~hinton/absps/JMLRdropout.pdf)
Team 4 Regularizing Deep Neural Networks by Noise: Its Interpretation and Optimization(https://arxiv.org/abs/1710.05179)
Assignment 1 Due
3-Oct Team Presentations: Project Proposal
Team Order 1 to 12
10-Oct Team Presentations continued; TensorFlow Examples
Team 5 Modified quasi-Newton methods for training neural networks(https://www.sciencedirect.com/science/article/pii/0098135495002286)
Team 6 Incorporating Nesterov Momentum into Adam.(https://openreview.net/pdf?id=OM0jvwB8jIp57ZJjtNEZ)
17-Oct Optimization (all) Deep Learning, Chapter 8
Team 7 Layer Normalization(https://arxiv.org/abs/1607.06450)
Team 8 A Framework for Self-Tuning Optimization Algorithm(https://arxiv.org/abs/1312.5667)
Assignment 2
24-Oct Convolutional Networks (all) Deep Learning, Chapter 9
Team 9 Inception-v4, inception-resnet and the impact of residual connections on learning(https://arxiv.org/abs/1602.07261)
Team 10 Rethinking the inception architecture for computer vision(https://arxiv.org/abs/1512.00567)
31-Oct Recurrent Neural Networks (all) Deep Learning, Chapter 10
Team 11 Exploring the limits of language modeling(https://arxiv.org/abs/1602.02410)
Team 12 Effective approaches to attention-based neural machine translation(https://arxiv.org/abs/1508.04025)
7-Nov Practical Methodology, Autoencoders (all) Deep Learning, Chapter 11; Autoencoders, Chapter 14
Assignment 2 Due (optional) Neural Networks: Tricks of the Trade, 2nd Edition -- "Practical Recommendations for Gradient-Based Training of Deep Architectures", https://link.springer.com/chapter/10.1007/978-3-642-35289-8_26
Team 13 Algorithms for Hyper-Parameter Optimization(https://papers.nips.cc/paper/4443-algorithms-for-hyper-parameter-optimization.pdf)
Team 14 Gradient-based Hyperparameter Optimization through Reversible Learning(https://arxiv.org/abs/1502.03492)
14-Nov Neural Networks with Memory (all) http://clgiles.ist.psu.edu/IST597/materials/papers-computing/papers-lect11/
Local representation Alignment https://arxiv.org/abs/1803.01834
CNNs plus RNNs https://arxiv.org/abs/1804.08588
GANs for Distractor Generation http://clgiles.ist.psu.edu/pubs/naacl18_bea.pdf
21-Nov Thanksgiving break, no classes
28-Nov Team Presentations
Teams 1-7
5-Dec Team Presentations
Teams 8-14
10-Dec Project Reports Due 2 hard copies - ACM paper format 9pt single space double sided.