Welcome To Spring 2026-CS 588: Autonomous Vehicle System Engineering

Instructor: Prof. Shenlong Wang (shenlong@illinois.edu), TA: Henry Che (hungdc2@illinois.edu)

Time: Mon/Wed 9:30am-10:45am Spring 2026, Location: 101 Transportation Building

Office Hour:

Wed 11am - 12pm (Siebel CS Room 4124, weekly, for course material/HWs)
Fri 9:00am - 10:00am (Highbay Test Facility, sometimes, for labs/final projects)

Lab: Fri 9:00am - 10:00am (Highbay Test Facility)

1. Introduction

Overview: This course introduces the computational principles underlying autonomous vehicles, with a strong emphasis on hands-on experience through practical lab work on a real self-driving vehicle, and is intended for graduate students with research interests in autonomous driving and robotics who are motivated to engage with real physical systems. The course covers the full autonomy stack, including the history of self-driving, vehicle kinematics and dynamics, hardware, camera- and LiDAR-based perception (object detection and tracking, localization and mapping, BEV representations), planning and control (path planning, sampling-based kinodynamic planning, optimal control, and trajectory optimization), and selected advanced topics such as learning-based methods, hybrid model- and learning-based systems, safety assurance, foundation models for driving, and neural and generative simulation. Evaluation consists of several hands-on machine problems and a final project implemented and tested on a physical vehicle.

Prerequisites: algorithms (CS 374 / ECE 374 or equivalent), machine learning (CS 444 / CS 441 / CS 446 or equivalent), and basic robotics (ECE 484 / ECE 470 or equivalent).

Course Objective: Through this course, you will gain a deep understanding of the core modules of autonomous vehicles, hands-on experience implementing the underlying algorithms, and practical experience deploying and testing them on a real vehicle.

Participation: We expect you to attend all lectures and lab sessions in-person unless pressing and unforeseen conflicts arise. Conflicts that are persistent (e.g. registering for two classes with half an hour overlap) are not excused.

2. Grading & Deliverables

75%: Machine Problems
1. Five individual MPs in total;
2. Code and quiz; PDF and code need to be submitted
3. Total grade: 15% each;
4. 1-2% Bonus question might be included for each MP;
20%: Final Project
1. 5% Mid-term progress report (2 pages)
2. 5% Video & demo presentation
3. 10% Final report (6 pages)
5%: Participation in class and lab

Grading policy: I'm a big believer in mastery learning. My goal is to help you really understand the material so that everyone has the opportunity to learn and succeed. I am most successful if everyone in class earns an A. This class will not be curved.

Late submission policy: Free late days (machine problem only): Each student has a total of five free late days for homework assignments across the entire semester. As long as you remain within your total late-day budget, no extension request is needed and no late penalty will be applied. Late penalty (after free days are used): Once you have used all of your free late days, your assignment score will be multiplied by 0.8 for each additional late day. Assignments that are more than five days late beyond any free late days will not be accepted. No partial submissions: You may not submit different parts of an assignment at different times in order to apply late penalties to only part of the work. The assignment is treated as a single unit for lateness. No extension request will be accepted for final projects deliverables, with exception below!

Extensions: Except for the free late days, no extensions will be granted, with the sole exception of documented medical emergencies with doctoral approval.

Letter grade. Letter grades will be assigned based on the following thresholds:

A+ 100, A [95,100) A- [90,95), B+ [86.7,90), B [83.3,86.7), B- [80,83.3), C+ [76.7,80), C [73.3,76.7), C- [70,73.3), D [60,70), F [0,60).
Range notation [90,93) means 90 is included and 93 is not

3. Final Project

The final project will be completed in small groups of 3-5 students. From the beginning of the semester during the first lab session, you can start forming a group and deciding on one of the suggested project topics. Your group will be asked to follow the mid-term report template to update progress, preliminary results in simulation, and challenges you face, and complete it by the mid-term deadline. Your group will need to perform the described proof-of-concept experiment, demonstrate the working demo during lab sessions, write the final report and perform a conference style spotlight video presentation.

Project topics and scope: Your project should fall within one or more topics in self-driving / autonomous driving systems. The approach must include components that are integrated and run onboard the GEMe4 platform; it cannot be purely offboard (e.g., an algorithm that only runs on a separate workstation). The project does not need to be limited to algorithms covered in this class; however, it must originate in this class and cannot be the same as, or heavily dependent on, your or others’ ongoing research (for example, you may not use your ongoing CVPR submission as your final project). It is strongly recommended that you implement your own version of the core components and integrate them into the GEM platform. That said, you are allowed to reuse and build upon open-source implementations, provided that you clearly document which parts are borrowed from existing code and what you have implemented yourself, and that your main effort is on meaningful integration, system building, and/or algorithmic contributions that enable new capabilities on the platform. If you are unsure about the intellectual merit, feasibility, or risk level of your idea, you are encouraged to discuss it with the instructor or TA.

Example Project Directions

Perception:

A LiDAR–SLAM / visual–SLAM based online mapping system
A particle-filter-based online 6-DoF localization system
An end-to-end online camera-based 3D reconstruction system
A BEV perception system with LiDAR and camera fusion

Calibration

Target-less online sensor calibration

Simulation

A real-time hardware-in-the-loop simulation system
Real-time onboard neural simulation system

End-to-end:

An imitation-learning pipeline for lane following
A sim-to-real reinforcement-learning-based lane-following agent
A vision-based end-to-end neural motion planner
A LiDAR-based neural motion planner
A VLM based dual-system autonomy

Modular System:

A pedestrian emergency braking system involving object detection, waypoint following, and a braking controller
An open-set object obstacle avoidance system using 3D occupancy grid perception
An online human gesture recognition and reaction system
An automatic, FSD-style parking system

4. Participation

In-class & lab Participation (5%): We actively promote a friendly, engaging, and interactive classroom environment. Students are strongly encouraged to ask questions at any time during class, labs and office hours. Up to a 1% bonus for each will be awarded to the most active students. During the semester, attendance may be checked in three randomly selected lectures via a short attendance form. Lab attendance will be recorded at every lab session.

Canvas Rules:

We use Canvas for notifications, discussions on conceptual and technical questions among classmates. There is no anonymity. Please be respectful to your classmates.
Please use Forum for all communications as much as possible. Others will benefit from answers and discussions on public questions.
Canvas is moderated by the teaching staff.

Email Rules:

Please email Henry or Shenlong only if Slack does not work for you. Otherwise, emails to individual teaching staff members will be ignored.

5. Machine Problems (MP)

MP0 — Sensors, Visualization, and Calibration

Visualize multi-sensor autonomous driving data.
Perform camera and camera–LiDAR calibration.
Project LiDAR points into images and verify alignment.

MP1 — Mapping and Sensor Fusion

Build a 3D map by aligning LiDAR scans.
Fuse camera color or semantics into the map.
Visualize and assess map quality.
(Optional) Mesh or neural map reconstruction.

MP2 — Object Detection and Tracking

Train an object detector on driving data.
Evaluate and visualize detection results.
Run perception on real vehicle logs.
(Optional) Add multi-object tracking.

MP3 — Planning and Control

Implement a rule-based and a learned planner.
Train planners from data and compare behavior.
Evaluate planning performance in simulation.
(Optional) Generative or diffusion-based planning.

MP4 — BEV Perception and Forecasting

Train a BEV perception model.
Predict future BEV states from history.
Visualize and evaluate predictions.
(Optional) Joint perception–prediction–planning.

6. Misc

DRES Accommodations: To obtain disability-related academic adjustments and/or auxiliary aids, students with disabilities must contact the course instructor and the Disability Resources and Educational Services (DRES) as soon as possible. To contact DRES you may visit 1207 S. Oak St., Champaign, call 333-4603 (V/TDD), or e-mail a message to disability@uiuc.edu.

Academic Integrity: All work that you submit should be written solely by you and your group, and you should cite any significant sources of ideas. If any part of your project builds on efforts prior to the semester (e.g. your ongoing research project), be sure to discuss with instructor in advance. Plagiarism and other integrity violations will go on record at the university, and the minimum penalty will be a zero for the entire assignment.

Anti-racism and Inclusivity Statement: The intent of this section is to raise student and instructor awareness of the ongoing threat of bias and racism and of the need to take personal responsibility in creating an inclusive learning environment. The Grainger College of Engineering is committed to the creation of an anti-racist, inclusive community that welcomes diversity along a number of dimensions, including, but not limited to, race, ethnicity and national origins, gender and gender identity, sexuality, disability status, class, age, or religious beliefs. The College recognizes that we are learning together in the midst of the Black Lives Matter movement, that Black, Hispanic, and Indigenous voices and contributions have largely either been excluded from, or not recognized in, science and engineering, and that both overt racism and micro-aggressions threaten the well-being of our students and our university community. The effectiveness of this course is dependent upon each of us to create a safe and encouraging learning environment that allows for the open exchange of ideas while also ensuring equitable opportunities and respect for all of us. Everyone is expected to help establish and maintain an environment where students, staff, and faculty can contribute without fear of personal ridicule, or intolerant or offensive language. If you witness or experience racism, discrimination, micro-aggressions, or other offensive behavior, you are encouraged to bring this to the attention of the course director if you feel comfortable. You can also report these behaviors to the Bias Assessment and Response Team (BART) (https://bart.illinois.edu/). Based on your report, BART members will follow up and reach out to students to make sure they have the support they need to be healthy and safe. If the reported behavior also violates university policy, staff in the Office for Student Conflict Resolution may respond as well and will take appropriate action.