CS 489/698: Introduction to Natural Language Processing

Instructors: Freda Shi and Victor Zhong

Grading Policy

There will be no exam.

CS 489: Grades will be composed of two assignments (25% each) and a course project (the Interstellar Autocomplete Challenge, 50%).
The Interstellar Autocomplete Challenge will have a project initial proposal (due on February 2), a midterm check-in (due on February 23), and a final submission (due during the final exam period).

CS 698: Grades will be composed of two assignments (25% each), the Interstellar Autocomplete Challenge (requirements are the same as CS 489, but it will make up 25% of the grade for CS 698), and an NLP-related research project (25%).
The independent research project will have a project proposal/check-in that describes your project idea and what you have done for the project (5%, due on February 23), and a final submission (20%, due during the final exam period).

All projects must be done individually.

Location

There are two parallel sections:

• MW 11:30am-12:50pm (MC 2035)
• MW 4:00pm-5:20pm (RCH 305)

The content of the two sections is identical. You may attend either section, even if you are registered in the other.

Course Schedule

Week	Monday	Wednesday	Instructor
1	January 5 Introduction and Fundamentals [slides]	January 7 Words: Definition, Tokenization, Morphology [slides] [Reading: Sennrich et al. (2016)] [Reading: Kudo (2018)]	Freda
2	January 12 Lexical Semantics and Word Embeddings [slides] [Reading: SLP 3, Chapter 6]	January 14 Building a Text Classifier [slides] [Reading: SLP 3, Chapter 6]	Freda
3	January 19 Common Neural Architectures [slides]	January 21 Common Neural Architectures (cont.) Assignment 1 Out [A1 Description] [A1 Data] A1 Kaggle Competitions (please login with UW email): [Task 1] [Task 2.1] [Task 2.2] [Task 3]	Freda
4	January 26 No class: campus closed	January 28 Language Modeling [slides] [Reading: SLP 3, Chapter 3]	Freda
5	February 2 Language Modeling (cont.) [Reading: Holtzman et al., 2020] [Reading: HuggingFace Language Model Tutorial Chapter 7.6: Training GPT-2] Project Initial Proposal Due	February 4 Neural Language Models and Language Model Analysis [slides] [Reading: Devlin et al., 2019] [Reading: HuggingFace Tutorial on Masked Language Modeling]	Freda
6	February 9 Syntax and Context-Free Grammars [slides] [Reading: SLP 3, Chapter 18] [Reading: SLP 3, Chapter 19] Assignment 1 Due	February 11 Language Grounding and Multimodal Language Models [slides] Assignment 2 Out (Friday Feb 13): Decipher PCFG from Transformers [A2 Description] [A2 Material]	Freda
7	No class (reading week)
8	February 23 Pretraining Midterm Project(s) Check-In Due: 1 for CS 489, 2 for CS 698	February 25 Instruction Fine-tuning	Victor
9	March 2 Inference Scaling	March 4 Reinforcement Learning and Alignment Assignment 2 Due (Friday Mar 6)	Victor
10	March 9 Retrieval Augmented Generation	March 11 Advanced Topic: Mixture of Experts	Victor
11	March 16 Advanced Topic: Physical Devices and Compute	March 18 Advanced Topic: Agents	Victor
12	March 23 History of NLP	March 25 Final Review & Project Help	Victor
14	Final Projects (1 project for undergrads and 2 for grads) Submission Due