Fine-Tuning and Evaluating Local Models

What you'll learn

Fine-tuning has a reputation for being expensive, mysterious and GPU-hungry. Most of that reputation is about the *scale*, not the *ideas*. The ideas, gradient descent on a loss, low-rank adapters, quantization, preference optimization, evaluation, fit on one slide and can be understood completely.

This course teaches those ideas by building them. Students implement, from scratch in NumPy, a reverse-mode autograd engine, a small GPT, LoRA and DoRA, NF4 quantization and QLoRA, an AdamW trainer, an evaluation harness, and Direct Preference Optimization, on a model tiny enough to fine-tune on a laptop CPU in seconds. Then each module shows how the production tools, Hugging Face PEFT, TRL and bitsandbytes, do exactly the same thing at the scale of a 7-billion-parameter model on a single GPU.

The result is an engineer who can read a fine-tuning recipe and know what every line does, choose between full fine-tuning, LoRA, QLoRA and preference tuning from evidence, and, above all, *measure* whether a fine-tune helped, because an unmeasured fine-tune is a guess.

Everything in the lab kit runs offline and deterministically: no GPU, no downloads, and every number in the course is reproducible. The "at scale" sections point to the real tools and the hardware they need.

Learning outcomes

By the end of the course, you can:

1. Decide between prompting, retrieval, fine-tuning and preference tuning for a task, with a justification and a cost estimate.
2. Explain how a model learns, in terms of autograd and gradient descent, and implement both.
3. Implement and apply LoRA and DoRA, and explain the low-rank hypothesis and its limits.
4. Build, clean and format a supervised fine-tuning dataset with the right loss mask.
5. Run a training loop well: optimizer, learning-rate schedule, gradient clipping, and the hyperparameters that matter.
6. Quantize weights (int8, NF4) and fine-tune a 4-bit base with QLoRA, trading precision for memory knowingly.
7. Evaluate a fine-tuned model with held-out loss, task metrics, the train/test gap, and catastrophic forgetting.
8. Apply preference tuning (DPO) to align a model to ranked preferences without a reward model or RL.
9. Merge and serve adapters, and reason about multi-adapter deployment.
10. Map every technique in the lab kit onto the production tools (PEFT, TRL, bitsandbytes) and the hardware a real fine-tune needs.

Who it's for

• Engineers who use local models and want to specialize them for a domain, a format or a task
• ML and platform engineers moving from "call an API" to training and serving their own adapters
• Technical leads who must decide when fine-tuning is worth it, and how to evaluate the result

Prerequisites

Comfortable Python and basic NumPy, the command line, and secondary-school calculus (the chain rule). No deep-learning framework experience is required; the course builds the autograd engine it uses. RNK-CORP-301 (datasets) and RNK-LLM-301 (local serving) are helpful but not required.

Equipment and software

See the setup guide. Python 3.12+, NumPy, Git and a text editor. The optional at-scale tools (PyTorch, PEFT, TRL, bitsandbytes) and a GPU are needed only for the production exercises.

The full setup guide, with verification steps, is in your learning materials when you enroll.

Format and study time

42 guided hours (10 modules × ~4 h + capstone clinic) plus ~15 h capstone build. Instructor-led cohort, 5-day intensive, or self-paced with office hours.

Each scheduled class below states its own format, every meeting date and time, and its time zone.

What a seat includes

Curriculum

1. 01
   Why fine-tune (and when not to)

   4 h · Lab: Measure the gap a base model has on the target task
2. 02
   How models learn: autograd and a tiny GPT

   4 h · Lab: A gradient-checked autograd engine; a pretrained tiny GPT
3. 03
   LoRA and DoRA

   4 h · Lab: Full fine-tuning vs LoRA, with parameter counts and merging
4. 04
   Data for fine-tuning

   4 h · Lab: A dataset audit; mask, template, label-noise and split experiments
5. 05
   The training loop

   4 h · Lab: Optimiser and learning-rate experiments
6. 06
   Generation and decoding

   4 h · Lab: Sampling, temperature, top-k, and exposure bias
7. 07
   Quantization and QLoRA

   4 h · Lab: NF4/int8 error and compression; a QLoRA fine-tune
8. 08
   Evaluating a fine-tune

   4 h · Lab: Intervals, a paired comparison, forgetting, LoRA and replay
9. 09
   Preference tuning (DPO)

   4 h · Lab: SFT vs DPO vs DPO+SFT, measured on ranking and on generation
10. 10
    Serving fine-tuned models

    4 h · Lab: Two customer adapters on one base; a loader that refuses the wrong base
11. ★
    Capstone clinic & demo day

    2 h · Lab: Capstone project

Assessment and completion

• Pass: ≥ 70% overall, with the capstone ≥ 70% and at least 8 of 10 labs submitted
• Merit: ≥ 80% overall
• Distinction: ≥ 90% overall, and the capstone's evaluation report shows a measured improvement over its baseline with held-out metrics

Simply opening a lesson or a video never counts as completing an assessment. Completion is recorded only when the published requirements are met and your instructor has confirmed the result.

Course policies

• Late work: −10% per day, up to 3 days; after that, by arrangement. One free 48-hour extension per student.
• Collaboration: discuss freely; write your own code and reports. Capstone teams of two document who did what.
• AI assistants: allowed for coding, with disclosure in each report. You must be able to explain everything you submit, including the math. Fabricating results (loss curves, metrics) is academic misconduct.
• Data: the Halcyon scenario is fictional. If you bring your own data for the capstone, you are responsible for its licence and for not including personal or confidential information.
• Compute: the course runs on a CPU. The optional "at scale" exercises need a GPU; the instructor will say what is available. No student is disadvantaged for lacking a GPU.
• Accessibility: materials are searchable PDFs; their Markdown sources are available on request. Contact the instructor for accommodations.
• Attendance (A/B): ≥ 80% required for certification; missed sessions can be made up by submitting the lab with a 1-page reflection.

Upcoming classes