IST 597:  Foundations of Deep Learning
 
  Fall 2023

Instructor: Dr. C. Lee Giles

TA: Neisarg Dave



Time and Place:  2:30-5:30, Tuesday, Westgate W219


Office Hours:  Giles, Westgate E350, 2:30-3:30 PM Wednesday
                         Dave, TBA.
                
                        

Course Overview:

This is a three hour course hands on course for graduate students that meets once a week. Students should be familiar with the mathematical foundations of deep learning: linear algebra, numerical computation, probability and information theory. There will be a brief introduction to machine learning basics and computation complexity. We will cover modern practical deep networks and their applications. We will investigate deep feedforward networks, convolutional networks, and recurrent networks as well as practical methodologies, including model regularization and parameter optimization. We will also cover transformers, adversarial, science inspired, and large language models. Practical applications will be utilized for course projects. Students will learn deep learning software from the projects and will give presentations on chosen relevant papers.



Course Mission Statement:

This course is intended to prepare students to understand, design, develop and use deep learning methods.



Course Prerequisites:

Students should be familiar with the basic mathematics of deep learning fundamentals.


Schedule (syllabus):  This schedule is subject to change. Please check it on a regular basis for assignments.



Course Materials and References: Course materials can be found here. There will also be links on the schedule.


Grading:       

                        Research Project: 50% total.
                            Project proposal and presentations: 10%
                            Final project presentation and paper: 40%
 
                       Exercises: 30%

                       Paper reviews and presentations: 10%

                       Class participation: 10%

Exercises:

There will be 3 exercises, each varying in points, all in Python or PyTorch.

All exercises are due by midnight of the due date and must be submitted as a zip or gzip which includes PDF and PythonScript[.py].
Late exercises will receive 50% credit for the first 24 hours and no credit after that time. All exercise solutions must be turned in, even if late.
Failure to do so can result in a deferred grade.


Texts and Readings: 

The reading list is on the schedule above.  We will use chapters and sections from the online text Deep Learning and related papers listed in the syllabus.


   




Research Project:

Student teams will have a research project, either proposed or assigned by Giles and TA.

All proposed projects will have to be approved.

There will be a topic presentation, two updates, a final presentation, and a project paper in PDF and hard copy in the ACM paper format.


Presentations:

There will be at two detailed presentations for the project and brief updates.

For the assigned papers, each team will make a review presentation on their assigned paper(s).

Each eam will write two questions for each presentation and ask one (not required by the presenting team).


Email: All email to the instructor and TA about this class should contain "IST597" in the subject line.  For example, the subject line might read "IST597: Question about ....".  Email without this information might be deleted by spam filters or placed in a folder to be read at a later date.  Email with the appropriate identifier will usually be read within 24 hours of being received.


Acknowledgements!


Reuse: Materials from this course can be publicly reused in other courses. This course supports Open Educational Resources (OER).