Modern NLP Systems
Undergraduate Course, CuCEng, 2027
Course Objectives
The primary objective of this course is to equip students with the practical engineering skills required to design, develop, and deploy modern Natural Language Processing systems based on Large Language Models (LLMs). The course aims to transition students from theoretical NLP concepts to building advanced, production-ready AI applications by utilizing state-of-the-art frameworks, external knowledge retrieval systems, and autonomous agent architectures.
Course Content
This project-centric course covers the end-to-end integration of LLMs into modern software systems. Key topics include the architecture and implementation of Retrieval-Augmented Generation (RAG) pipelines, vector database management, and chunking strategies for semantic search. The curriculum also focuses on adapting open-source models to domain-specific tasks using Parameter-Efficient Fine-Tuning (PEFT) and LoRA methodologies. Furthermore, students will explore agentic workflows, function calling, and tool integration using frameworks like LangChain and LlamaIndex. Through hands-on labs and team projects, students will develop and deploy functional AI prototypes capable of solving complex, real-world scenarios.
Course Materials
- Sebastian Raschka, Build a Large Language Model (From Scratch), Manning Publications, 2024. A reference for understanding how LLMs are coded from scratch, hardware optimizations, and fine-tuning processes.
- Lewis Tunstall, Leandro von Werra, and Thomas Wolf, Natural Language Processing with Transformers: Building Language Applications with Hugging Face, O’Reilly Media, 2022. A practical guide to the open-source ecosystem.
- Official Documentations: LangChain and LlamaIndex.
Assessment
- Three team projects with presentations: RAG Pipeline, Fine-Tuning, and Agentic Workflows
Prerequisites
None. Taking Natural Language Processing in the fall semester is recommended.
Weekly Schedule
| Week | Subjects | Note |
|---|---|---|
| 1 | Introduction to Modern NLP Systems and API Ecosystem | |
| 2 | Vector Spaces, Embedding Models, and Semantic Search | |
| 3 | Vector Database Management and Text Chunking Strategies | |
| 4 | Implementation of RAG (Retrieval-Augmented Generation) Architecture | |
| 5 | RAG Systems Project Delivery and Application Development (UI/UX) | |
| 6 | Open Source Language Models and the Hugging Face Ecosystem | |
| 7 | Dataset Preparation and Formatting for Language Models | RAG Pipeline due |
| 8 | Midterm Week — no exam for this course | |
| 9 | Parameter-Efficient Fine-Tuning (PEFT) and the Math of LoRA | |
| 10 | Open Source Model Training (Fine-Tuning) on Local Hardware | |
| 11 | Delivery of Fine-Tuned Domain-Specific Models | Fine-Tuning due |
| 12 | Function Calling and Tool Use in Large Language Models | |
| 13 | Autonomous Agent Architectures and LangChain-LlamaIndex Frameworks | |
| 14 | Agentic Workflows Project Delivery and Live Demos | Agentic Workflows due |
| 15 | Delivery of Projects |