Date Topic covered Assignments
22-Aug Class syllabus
Introduction to IST 597
Introduction to Artificial Intelligence (AI)
Introduction to Deep Learning
Computational Complexity Big O,
Computational Resources
Assignment 01
Introduction in linear algebra for deep learing - Numpy
29-Aug Review chapters 1-4. This course will assume you know this. (all) Deep Learning,
Machine Learning Fundamentals (all) Unstructured Data,
(all) Deep Learning, Chapters 1-4
Introduction to PyTorch (optional) Artificial Neural Networks,
Assigment 01 due
Assigment 10 given out (all) Deep Learning, Chapter 5
5-Sep The Multilayer Perceptron & Feedforward Networks (all) Deep Learning, Chapter 6
Architecture & Gradient-Based Learning (optional) "Learning Representations by Back-Propagating Errors" (Nature),
Readings and Presentations Assigned
3Blue1Brown series on neural networks
Tinker with Neural Networks
PyTorch Examples Building your first neural network - MLP on mnist/fmnist
Stochastic Back Propagation
12-Sep Practical Methodology, Combating Overparametrization:  Regularization (all) Deep Learning, Chapter 7, Chapter 11
(optional) Neural Networks: Tricks of the Trade, 2nd Edition -- "Practical Recommendations for Gradient-Based Training of Deep Architectures",
PyTorch Examples  Norms, Regularizaion, Early Stopping
Avoiding overfitting of multilayer perceptrons by training derivatives
Overfitting in Neural Nets: Backpropagation, Conjugate Gradient, and Early Stopping
Project discussions by team
Assignment 10 due
Assignment 11 given out
19-Sep Optimization (all) Deep Learning, Chapter 8
Team 1 Improving Generalization Performance by Switching from Adam to SGD
Adaptive Gradient Methods with Dynamic Bound of Learning Rate
26-Sep Team Presentations: Project Proposal - 15 mins
Assignment 11 is due
Teams : 4, 6, 2, 8, 7, 5, 3, 1
3-Oct Convolutional Networks (all) Deep Learning, Chapter 9
Team 2 Mask RCNN
Zero Knowledge proof for CNN
Convolutional Neural Generative Coding
CNN Explainer
10-Oct Recurrent Neural Networks (all) Deep Learning, Chapter 10
Team 8 A Formal Hierarchy of RNN architectures
Higher Order Recurrent Networks and Grammatical Inference
2 page project update reports from all teams due LSTM: A search Space Odyssey
Fundamental of RNN and LSTM
On Multiplicative Integration with RNNs
17-Oct Neural Networks with Memory Discrete recurrent neural networks for grammatical inference
Team 3 Turing Completeness of Bounded Precision RNN
Neural Random-Access Machines
Neural Turing Machines
Theoretical Limitations of Self-Attention in Neural Sequence Models
5 Min. Project Update 
Presentation from each team with quad chart presentations Quad Chart Template
24-Oct Autoencoders and GANs (all) Deep Learning, Chapter 14
Team 7 Robust Vector Quantized-Variational Autoencoder 
Stacked Denoising Autoencoders
Variational Recurrent Auto-Encoders
Stacked Convolutional Auto-Encoders for Hierarchical Feature Extraction 
31-Oct Transformers (all) Deep Learning, Chapter 20
Team 4 Thinking Like Transformers
Solving Reasoning Task with Slot Transformer
Transformers in Vision: A Survey
Attention Is All You Need
7-Nov Large Language Models A Survey of Large Language Models
Team 6 Progressive Learning GPT-4
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT
14-Nov Science and Enginnerng Inspired Neural Networks Deep Learning In Science, Pierre Baldi
Team 5 Wavelets based PINN
Combinatorial Optimization with Physics Inspired GNN
Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next
21-Nov Break
28-Nov Final Team Presentations - Day 1
5-Dec Final Team Presentations - Day 2
11-Dec Project Reports Due PDF and 2 hard copies - ACM paper format 9pt single space double sided.