Date	Topic covered	Assignments
22-Aug	Class syllabus
	Introduction to IST 597	http://clgiles.ist.psu.edu/IST597/
	Introduction to Artificial Intelligence (AI)	https://en.wikipedia.org/wiki/Artificial_intelligence
	Introduction to Deep Learning	https://en.wikipedia.org/wiki/Deep_learning
	Computational Complexity	Big O, http://en.wikipedia.org/wiki/Big_O_notation
	Computational Resources
	Assignment 01
	Introduction in linear algebra for deep learing - Numpy
29-Aug	Review chapters 1-4. This course will assume you know this.	(all) Deep Learning, https://en.wikipedia.org/wiki/Deep_learning
	Machine Learning Fundamentals	(all) Unstructured Data, http://en.wikipedia.org/wiki/Unstructured_data
		(all) Deep Learning, Chapters 1-4
	Introduction to PyTorch	(optional) Artificial Neural Networks, https://en.wikipedia.org/wiki/Artificial_neural_network
	Assigment 01 due	https://pytorch.org/tutorials/index.html
	Assigment 10 given out	(all) Deep Learning, Chapter 5
5-Sep	The Multilayer Perceptron & Feedforward Networks	(all) Deep Learning, Chapter 6
	Architecture & Gradient-Based Learning	(optional) "Learning Representations by Back-Propagating Errors" (Nature), https://www.nature.com/nature/journal/v323/n6088/abs/323533a0.html
	Readings and Presentations Assigned	https://www.nature.com/nature/journal/v323/n6088/abs/323533a0.html
	3Blue1Brown series on neural networks	https://www.3blue1brown.com/lessons/neural-networks
	Tinker with Neural Networks	https://playground.tensorflow.org/
	PyTorch Examples	Building your first neural network - MLP on mnist/fmnist
		Stochastic Back Propagation
12-Sep	Practical Methodology, Combating Overparametrization:  Regularization	(all) Deep Learning, Chapter 7, Chapter 11
		(optional) Neural Networks: Tricks of the Trade, 2nd Edition -- "Practical Recommendations for Gradient-Based Training of Deep Architectures", https://link.springer.com/chapter/10.1007/978-3-642-35289-8_26
	PyTorch Examples	Norms, Regularizaion, Early Stopping
		Avoiding overfitting of multilayer perceptrons by training derivatives
		Overfitting in Neural Nets: Backpropagation, Conjugate Gradient, and Early Stopping
	Project discussions by team
	Assignment 10 due
	Assignment 11 given out
19-Sep	Optimization	(all) Deep Learning, Chapter 8
	Team 1	Improving Generalization Performance by Switching from Adam to SGD
		Adaptive Gradient Methods with Dynamic Bound of Learning Rate
26-Sep	Team Presentations: Project Proposal - 15 mins
	Assignment 11 is due
	Teams : 4, 6, 2, 8, 7, 5, 3, 1
3-Oct	Convolutional Networks	(all) Deep Learning, Chapter 9
	Team 2	Mask RCNN
		Zero Knowledge proof for CNN
		Convolutional Neural Generative Coding
	DeepLizard	https://deeplizard.com/learn/video/YRhxdVk_sIs
	CNN Explainer	https://poloclub.github.io/cnn-explainer/
10-Oct	Recurrent Neural Networks	(all) Deep Learning, Chapter 10
	Team 8	A Formal Hierarchy of RNN architectures
		Higher Order Recurrent Networks and Grammatical Inference
	2 page project update reports from all teams due	LSTM: A search Space Odyssey
		Fundamental of RNN and LSTM
		On Multiplicative Integration with RNNs
17-Oct	Neural Networks with Memory	Discrete recurrent neural networks for grammatical inference
	Team 3	Turing Completeness of Bounded Precision RNN
		Neural Random-Access Machines
		Neural Turing Machines
		Theoretical Limitations of Self-Attention in Neural Sequence Models
	5 Min. Project Update
	Presentation from each team with quad chart presentations	Quad Chart Template
24-Oct	Autoencoders and GANs	(all) Deep Learning, Chapter 14
	Team 7	Robust Vector Quantized-Variational Autoencoder
		Stacked Denoising Autoencoders
		Variational Recurrent Auto-Encoders
		Stacked Convolutional Auto-Encoders for Hierarchical Feature Extraction
		WGAN
31-Oct	Transformers	(all) Deep Learning, Chapter 20
	Team 4	Thinking Like Transformers
		Solving Reasoning Task with Slot Transformer
		Transformers in Vision: A Survey
		Attention Is All You Need
7-Nov	Large Language Models	A Survey of Large Language Models
	Team 6	Progressive Learning GPT-4
		BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
		RLHF
		A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT
14-Nov	Science and Enginnerng Inspired Neural Networks	Deep Learning In Science, Pierre Baldi
	Team 5	Wavelets based PINN
		Combinatorial Optimization with Physics Inspired GNN
		Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next
21-Nov	Break
28-Nov	Final Team Presentations - Day 1
5-Dec	Final Team Presentations - Day 2
11-Dec	Project Reports Due	PDF and 2 hard copies - ACM paper format 9pt single space double sided.
		https://www.acm.org/publications/proceedings-template