CS 489/698: Introduction to Natural Language Processing

Instructors: Freda Shi and Victor Zhong

Grading Policy

There will be no exam.

CS 489: Grades will be composed of two assignments (25% each) and a course project (the Interstellar Autocomplete Challenge, 50%).
The Interstellar Autocomplete Challenge will have a project initial proposal (due on February 2), a midterm check-in (due on February 23), and a final submission (due during the final exam period).

CS 698: Grades will be composed of two assignments (25% each), the Interstellar Autocomplete Challenge (requirements are the same as CS 489, but it will make up 25% of the grade for CS 698), and an NLP-related research project (25%).
The independent research project will have a project proposal/check-in that describes your project idea and what you have done for the project (5%, due on February 23), and a final submission (20%, due during the final exam period).

All projects must be done individually.

Location

There are two parallel sections: The content of the two sections is identical. You may attend either section, even if you are registered in the other.

Course Schedule

Week Monday Wednesday Instructor
1 January 5 Introduction and Fundamentals
[slides] 		January 7 Words: Definition, Tokenization, Morphology [slides]
[Reading: Sennrich et al. (2016)]
[Reading: Kudo (2018)] 		Freda
2 January 12 Lexical Semantics and Word Embeddings [slides]
[Reading: SLP 3, Chapter 6] 		January 14 Building a Text Classifier
[slides] [Reading: SLP 3, Chapter 6] 		Freda
3 January 19 Common Neural Architectures [slides] January 21 Common Neural Architectures (cont.) Assignment 1 Out [A1 Description] [A1 Data] A1 Kaggle Competitions (please login with UW email): [Task 1] [Task 2.1] [Task 2.2] [Task 3] Freda
4 January 26 No class: campus closed January 28 Language Modeling [slides]
[Reading: SLP 3, Chapter 3] 		Freda
5 February 2 Language Modeling (cont.)
[Reading: Holtzman et al., 2020]
[Reading: HuggingFace Language Model Tutorial Chapter 7.6: Training GPT-2] Project Initial Proposal Due 		February 4 Neural Language Models and Language Model Analysis
[slides]
[Reading: Devlin et al., 2019]
[Reading: HuggingFace Tutorial on Masked Language Modeling] 		Freda
6 February 9 Syntax and Context-Free Grammars
[slides]
[Reading: SLP 3, Chapter 18]
[Reading: SLP 3, Chapter 19] Assignment 1 Due 		February 11 Language Grounding and Multimodal Language Models
[slides]
Assignment 2 Out (Friday Feb 13): Decipher PCFG from Transformers [A2 Description] [A2 Material] A2 Kaggle Competitions (please login with UW email): [Task 1] [Task 2] [Task 3] 		Freda
7 No class (reading week)
8 February 23 Pretraining
[slides]
[Reading: Radford et al., 2018]
Midterm Project(s) Check-In Due: 1 for CS 489, 2 for CS 698 (extended to March 1) 		February 25 Instruction Fine-tuning
[slides]
[Reading: Raffel et al., 2019]
[Reading: Brown et al., 2020]
[Reading: Zhou et al., 2023] 		Victor
9 March 2 Reinforcement Learning and Alignment
[slides]
[Reading: Schulman et al., 2017]
[Reading: Ouyang et al., 2022]
[Reading: Rafailov et al., 2023]
[Reading: Shao et al., 2024]
 March 4 Test-time Scaling and Inference Compute Assignment 2 Due (Friday Mar 6) [slides]
[Reading: Wei et al., 2022]
[Reading: Yao et al., 2022]
[Reading: Shinn et al., 2023]
[Reading: Chen et al., 2026]
 Victor
10 March 9 Retrieval Augmented Generation [slides]
[Reading: Lewis et al., 2020]
[Reading: Karpukhin et al., 2020]
[Reading: Shi et al., 2023]
 March 11 Advanced Topic: Mixture of Experts [slides]
[Reading: Shazeer et al., 2017]
[Reading: Fedus et al., 2021]
[Reading: Zoph et al., 2022]
[Reading: Dai et al., 2024]
 Victor
11 March 16 Advanced Topic: Physical Devices and Compute [slides]
[Reading: Cunha, 2024]
[Reading: Sekar and Subbu, 2026]
[Reading: Armbruster, 2024]
[Reading: Dao et al., 2022]
 March 18 Advanced Topic: Agents [slides]
[Reading: Green, 2017]
[Reading: Jimenez al., 2024]
[Reading: Xie et al., 2024]
[Reading: Pascanu al., 2017]
[Reading: Wang et al., 2025]
 Victor
12 March 23 History of NLP [slides]
March 25 Open QA & High Level Project Discussions Victor
14 Final Projects (1 project for undergrads and 2 for grads) Submission Due