Course Syllabus
Welcome to ECE408 / CS483 / CSE408
The aim of this course is to provide knowledge and hands-on experience in developing software for processors with massively parallel computing resources, such as Graphics Processing Units, or GPUs. Effectively programming these processors requires in-depth knowledge about parallel programming principles, as well as the parallelism models, communication models, and resource limitations of these processors. In addition, this course also prepares the students to use the parallelism models of modern programming environments for high-performance computing and AI. The target audiences of the course are students who want to develop applications for these processors for fun, for work, and/or for research, as well as those who want to develop programming tools and future processors.
The course involves 8 programming lab assignments of steadily growing complexity. All assignments will involve programming a massively parallel GPU system using CUDA, which is a popular commercial language extension of C/C++ for GPU programming. Assignments involve tasks such as matrix multiplication, convolution, reduction, histogram calculation, and sparse matrix-vector multiplication. During the final third of the semester, students work on a larger, more complex project/competition.
Course Staff
Instructors
Prof. Volodymyr Kindratenko.
UIUC TAs
Howie Liu (hanwenl4@illinois.edu),
Yifei Song (yifeis7@illinois.edu),
Shengjie Ma (sm138@illinois.edu),
Michael Wang (mjwang6@illinois.edu),
Yue Yuan (yuey10@illinois.edu),
Xiyue Zhu (xiyuez2@illinois.edu).
UIUC Undergraduate Course Assistants
Daksh Kalley (dkalley2@illinois.edu),
Hrishi Shah (hrishis2@illinois.edu),
Colin Zeng (kelinz2@illinois.edu),
Lincoln Lin (ll32@illinois.edu)
ZJUI Undergraduate Course Assistants
Qiqian Fu (qiqianf2@illinois.edu)
Zhixiang Liang (zliang18@illinois.edu)
Chiming Ni (chiming2@illinois.edu)
Class Time and Location
AL1: 1002 ECEB, Tuesdays and Thursdays, 9:30-10:50am US Central Time by Prof. Volodymyr Kindratenko. Lectures will be recorded and posted on mediaspace.
ALL other sections: Recorded lectures posted on mediaspace.
Reading Materials
Textbook
D. Kirk and W. Hwu, “Programming Massively Parallel Processors – A Hands-on Approach,” Morgan Kaufman Publisher, 3rd edition, 2016, ISBN 978-0123814722. Relevant to this course chapters can be found here.
NVIDIA documentation
NVIDIA, CUDA C Programming Guide (CUDA PG)
https://docs.nvidia.com/cuda/cuda-c-programming-guide
https://docs.nvidia.com/cuda/pdf/CUDA_C_Programming_Guide.pdf
NVIDIA, CUDA C++ Best Practices Guide (CUDA BPG)
https://docs.nvidia.com/cuda/cuda-c-best-practices-guide
https://docs.nvidia.com/cuda/pdf/CUDA_C_Best_Practices_Guide.pdf
NVIDIA Developer Blog (NVIDIA DB)
https://developer.nvidia.com/blog
Computer System Setup
We will use NCSA's Delta supercomputer for programming assignments (labs and project) in this course. We will distribute course assignments via a github repository and will collect students' solutions via github as well. Below are instructions on how to get started with Delta and Github.
Delta Access
To get started with Delta, follow these steps:
- Go to https://identity.access-ci.org/new-user and register yourself as a new user. If you already have an ACCESS account (some of you might already have it due to a research project, etc.) do not register as new user, just use the same username. When registering for new account, use "University of Illinois Urbana-Champaign" as your Identity Provider. This way, you can just use your UofI credentials.
- Once your account is created, note your ACCESS user ID and submit this user ID to the course staff via this form: https://forms.illinois.edu/sec/794309499.
- Wait for an email from NCSA Delta team confirming completion of your account registration. This may take several days, so please be patient. After receiving the confirmation email, you should be able to login into Delta via ssh and clone the GitHub repository to get started with the labs.
- While you are waiting for a confirmation that your account has been setup on Delta, go ahead and familiarize yourself with Delta's documentation and also setup your class GitHub repository.
GitHub setup
To get started with GitHub:
- First, follow this link to establish a class repository: https://edu.cs.illinois.edu/create-gh-repo/fa24_ece408. This needs to be done only once. After this step is complete, you should be able to visit https://github.com/illinois-cs-coursework/fa24_ece408_NetID page (where NetID is your NetID) that shows the contents of your repository. Initially, this repository will be empty.
- After completing step 1, you should be able to access the class release repository at https://github.com/illinois-cs-coursework/fa24_ece408_.release. Please read
README.release.md file for further instructions about what to do next. You can proceed with these steps once your Delta account is setup.
Getting Help
UIUC Office hours
Course instructor and TAs will have office hours offered in-person and on-line starting on the second week of the semester (week of 9/2). Watch this space for updates.
| US Central Time | Monday | Tuesday | Wednesday | Thursday | Friday |
| 8 - 9 am | Lincoln (3003 ECEB) |
||||
| 9 - 10am | |||||
| 10 - 11am | Colin (3003 ECEB) |
||||
| 11am - noon | Yue (3003 ECEB) |
||||
| noon - 1pm | |||||
| 1 - 2pm | Daksh (Zoom) | ||||
| 2 - 3pm | Prof. Kindratenko (3050E NCSA) |
Shengjie (3003 ECEB) |
Hrishi (3003 ECEB) |
||
| 3 - 4pm | Xiyue (Zoom) | ||||
| 4 - 5pm | Howie (3034 ECEB) |
||||
| 5 - 6pm |
Michael |
||||
| 6 - 7pm | Colin (2013 ECEB) |
||||
| 7 - 8pm | Yifei (Zoom) | Howie (3034 ECEB) |
|||
| 8 - 9pm |
ZJUI Office hours
All ZJUI office hours are in-person at the library cafe.
Qiqian Fu: Thursday 1030-1130am
Zhixiang Liang: Monday 3-4pm
Chiming Ni: Friday 6-7pm
Discussion board
We will use Campuswire for Q&A. Sign up here if you are not yet enrolled. (You will be asked for a verification code, which is 4870.)
Discussion TA: Michael Wang
Course chatbot
Yes, we are providing a GPT-4 powered AI chatbot trained on ECE 408 course materials! You are welcome to use it to learn, not to cheat! You can start by asking the chatbot how to succeed in this class. :)
Please read this disclaimer before using the chatbot, this a very experimental technology and you must understand its limitations and agree to the provisions outlined in the disclaimer.
Grading
Grads and undergrads in the same grading pool. All grades will be posted in Canvas.
- Exam 1: 25%
- Exam 2: 25%
- Lab Assignments: 25%
- Correctness (determined by the number of data sets passed and the functionality) 90%
- Responses to the quiz 10%
- Lab with the lowest score will be dropped from the final grade, therefore no late submissions will be allowed.
- Lab 0's grade will not be counted towards the final grade.
- Final Project: 25%
- Detailed Rubric is posted with the project description
Lectures Schedule
Lecture slides can be found here.
Reading materials can be found here and here.
Lecture recordings can be found here.
| Week | Date | Activity | Topic | Reading |
| 1 | 8/27/24 | Lecture 1 | Introduction to the Course |
Kirk & Hwu: Chapter 1 CUDA PG: Introduction |
| 8/29/24 | Lecture 2 | Introduction to Parallel Computing and CUDA |
Kirk & Hwu: Chapter 2 CUDA PG: Programming Model |
|
| 8/30/24 | ACCESS and Github setup due |
|
||
| 2 | 9/3/24 | Lecture 3 | CUDA Parallel Execution Model | Kirk & Hwu: Chapter 3 |
| 9/5/24 | Lecture 4 | CUDA Memory Model | Kirk & Hwu: Chapter 4 | |
| 9/6/24 | Lab 0 due | Device Query | ||
| 3 | 9/10/24 | Lecture 5 | Locality and Tiled Matrix Multiply |
Kirk & Hwu: Chapter 4 CUDA PG: Shared Memory NVIDIA DB: Shared Memory |
| 9/12/24 | Lecture 6 | Generalized Tiling and DRAM Bandwidth |
Kirk & Hwu: Chapter 5 CUDA BPG: Device Memory Spaces NVIDIA BD: Global Memory |
|
| 9/13/24 | Lab 1 due | Vector Addition | ||
| 4 | 9/17/24 | Lecture 7 | Convolution concept (1D and 2D); 1D Basic Convolution Kernel, and constant cache |
Kirk & Hwu: Chapter 7 CUDA BPG: Device Memory Spaces |
| 9/19/24 | Lecture 8 | 2D Tiled Convolution Kernel |
Kirk & Hwu: Chapter 7 NVIDIA DB: Shared Memory |
|
| 9/20/24 | Lab 2 due | Simple Matrix Multiplication | ||
| 5 | 9/24/24 | Lecture 9 | Shared Memory Reuse Analysis | Kirk & Hwu: Chapter 7 |
| 9/26/24 | Lecture 10 | Introduction to Machine Learning |
|
|
| 9/27/24 | Lab 3 due | Tiled Matrix Multiplication | ||
| 6 | 10/1/24 | Lecture 11 | Deep Networks: Inference and Training | Kirk & Hwu: Chapter 16 |
| 10/3/24 | Lecture 12 | Computation in CNNs and Transformers | Kirk & Hwu: Chapter 16 | |
| 10/4/24 | Lab 4 due | 3D Convolution | ||
| 7 | 10/08/24 |
(no in-class lecture, please watch pre-recorded lecture) |
Application Profiling with Nsight |
|
| 10/10/24 |
Lecture 13 |
Reductions |
Kirk & Hwu: Chapter 5 |
|
| 10/11/24 | PM1 due | Project Milestone 1 Code & Report | ||
| 8 | 10/15/24 | Midterm 1 | Lectures 1-12 and "Application Profiling with Nsight" lecture, Labs 1-4, and PM1 | |
| 10/17/24 | Lecture 14 | Parallel Scan: Kogge-Stone | Kirk & Hwu: Chapter 8 | |
| 10/18/24 10/20/24 | Lab 5 due | List Reduction | ||
| 9 | 10/22/24 | Lecture 15 | Parallel Scan: Brent-Kung | Kirk & Hwu: Chapter 8 |
| 10/24/24 | Lecture 16 | Histogramming and Atomic Operations |
Kirk & Hwu: Chapter 9 CUDA PG: Atomic Functions |
|
|
10/25/24 |
Lab 6 due | Scan |
|
|
| 10 | 10/29/24 | Lecture 17 | Sparse Matrix I | Kirk & Hwu: Chapter 10 |
| 10/31/24 | Lecture 18 | Sparse Matrix II | Kirk & Hwu: Chapter 10 | |
| 11/1/24 | Lab 7 due | Histogram | ||
| 11 | 11/5/24 | Lecture 19 | GPU Systems Architecture | |
| 11/7/24 | Lecture 20 | Tasks Parallelism and Asynchronous Data Transfer | Kirk & Hwu: Chapter 18 | |
| 11/8/24 | PM2 due | Project Milestone 2 Code & Report | ||
| 12 | 11/12/24 | Lecture 21 | Other Accelerations APIs | |
| 11/14/24 |
in-class |
Guest lecture by Dr. Vikram Mailthody, NVIDIA |
||
| 11/15/24 | Lab 8 due | Sparse Matrix Multiplication | ||
| 13 | 11/19/24 | in-class Lecture 22 |
Guest lecture by Prof. Sanjay Patel, NVIDIA Tensor Processing APIs |
CUDA PG: Warp Matrix Functions |
| 11/21/24 | Guest Lecture by Jeff Larkin, NVIDIA | No More Porting! Programming GPUs with C++, Fortran, and Python. (recording on MediaSpace) | ||
| Fall Break | ||||
| 14 | 12/3/24 | in-class Guest Lecture |
Guest lecture by Alex Settle, NVIDIA | |
| 12/5/24 | Lecture 23 | Course Retrospective | ||
|
12/6/24 |
PM3 due | Project Milestone 3 Code & Report | ||
| 15 | 12/10/24 | Midterm 2 | ||
| 12/13/24 | Project Competition due | Project Competition | ||
Labs (aka MPs)
Lab (aka MP) assignments will be done using NCSA's Delta system for coding and Canvas for quiz submission. Programming part of each lab will be distributed via GitHub repository. Q&A part will be done as a quiz via Canvas. Do not start working on the quiz part before completing the programming part as the quiz will require providing info from the actual working lab. Labs are individual assignments; they must be carried out and submitted individually.
Lab assignments are due on Fridays at 8:00pm US Central Time, unless otherwise noted or explicitly extended by the teaching staff.
| Date released | Due Date | Notes | |
| Lab 0: Device Query | 8/30/24 | 9/6/24 | not counted toward the final grade |
| Lab 1: Vector Addition | 9/6/24 | 9/13/24 | |
| Lab 2: Simple Matrix Multiplication | 9/13/24 | 9/20/24 | |
| Lab 3: Tiled Matrix Multiplication | 9/20/24 | 9/27/24 | |
| Lab 4: 3D Convolution | 9/27/24 | 10/4/24 | |
| Lab 5: List Reduction | 10/11/24 | 10/18/24 10/20/24 | deadline moved to 10/20/24 |
| Lab 6: Scan | 10/18/24 |
10/25/24 |
deadline moved to 10/27/24 |
| Lab 7: Histogram | 10/25/24 | 11/1/24 | |
| Lab 8: Sparse Matrix Multiplication | 11/8/24 | 11/15/24 |
Lab TA: Yue Yuan
Project
We release the project in the same GitHub repository as the labs. Use "git pull" if you have checked out the project before. We will update instructions one milestone at a time. Projects are individual assignments, they must be carried out and submitted individually. We will use Delta for project as well.
Milestones are due on Fridays at 8:00pm US Central Time, unless otherwise noted or explicitly extended by the teaching staff. Project reports are submitted via Canvas.
| Date Released | Due Date | Notes | |
| Milestone 1 | 9/27/24 | 10/11/24 | |
| Milestone 2 | 10/16/24 | 11/8/24 | |
| Milestone 3 | 11/12/24 | 12/6/24 | |
| Final Competition | 12/13/24 | optional |
Project TAs: Yifei Song, Shengjie Ma
Late Submission for Labs and Project - updated (again)
All assignments are due by Friday at 8pm. However, recognizing potential issues such as Delta system maintenance, interviews, or illness, from now on we provide a three-day grace period. You can submit an assignment by Monday at 8pm and still receive full credit without penalty. You do not need to notify us if you intend to use this default 3-day grace period. However, to extend your grace period beyond 3 days, you must notify us by Friday 8pm via email to the TA(s) responsible for the assignment you are asking about. Extensions to the grace period are given out for extenuating circumstances at our discretion. Requests after this time will not be considered. Please also note that the code in your GitHub repository at 8pm on Monday will be considered the final version, even if you had a correct version earlier. We will retrieve and grade the code at that time.
Exams
There are two midterm exams, there is no final exam.
| Midterm 1 | Midterm 2 |
|
When: UIUC: 10/15/24, time 7-9pm. Where: UIUC:
ZJUI: RC1 Multifunction Hall. What: Lectures 1-12 and "Application Profiling with Nsight" lecture, Labs 1-4, PM1 How: paper-based Alternative Exam Time: as arranged Study materials: see Files HKN review session: 3-5:30pm on Sunday in 1002 ECEB Last-minute Office Hours: 2:30-4:30pm on the exam day in 3081 ECEB The cover page and the reference sheet [link] will be provided. |
When: 12/10/24, time 7-10pm. Where: 1002, 1013, 1015 ECEB. What: All lectures, including all guest lectures, all Labs, all project milestones. With a focus on everything after Exam 1. How: paper-based Alternative Exam Time: as arranged Study materials: see Files The cover page and the reference sheet [link] will be provided. |
Cheating and Academic Integrity
We take cheating very seriously.
The Student Code defines cheating as:
Using or attempting to use in any academic exercise materials, information, study aids, or electronic data that the student knows or should know is unauthorized (§1-402(a))
and notes that:
Substantial portions of the same academic work may not be submitted for credit more than once or by more than one student without authorization. (§1-402(a)(4))
The student code also disallows facilitation of cheating, prohibiting
Helping or attempting to help another to commit an infraction of academic integrity, where one knows or should know that through one’s acts or omissions such an infraction may be facilitated. (§1-402(c))
Cheating is an offense to the entire academic ecosystem: you, your classmates, and the course staff. It creates extra work for the course staff, it creates grading imbalances for your classmates, and it prevents you from learning the material that the rest of your academic career is built on.
We will take cheating very seriously. If we catch you cheating, or helping someone cheat, the penalty will not be less than a zero on the applicable assignment and the incident will be reported to the College.