The Distillation Lab is a web app on your-gpu-host (/distillation/) that runs the full teacher → dataset → student pipeline from a browser. This documents the tooling; for the underlying concepts see Knowledge Distillation, End to End and Does Distillation Add Knowledge or Just Style?.
The pipeline
- Teacher — a large model generates high-quality completions for your prompts. The Lab's default teacher is
glm-5.2:cloud. - Dataset — those completions are captured as an instruction-tuning dataset (prompt + teacher response).
- Student — a small local model is fine-tuned on that dataset to imitate the teacher's behavior on the target task.
Mind the meter
glm-5.2:cloudburns Ollama Cloud Pro limits. Teacher generation is the expensive step — every dataset row is a cloud call. Check theusage.phpledger before and after a run so a large dataset doesn't blow the monthly quota.
Practical levers to control cost:
- Generate a small, diverse seed set rather than a huge redundant one — distillation transfers style and task behavior efficiently from relatively few good examples.
- Cache teacher outputs; never regenerate rows you already have.
- Consider a local teacher on your-llm-host or the dual-GPU box when the task doesn't need frontier quality — see Ollama Cloud vs Local: Model Routing on the Fleet.
After the run
- Fine-tune the student (e.g. with Unsloth — see Fine-Tuning with Unsloth).
- Evaluate honestly before shipping — see Evaluating a Fine-Tuned Model and Picking a Teacher Model.
- Export the student to GGUF and import to Ollama — see Converting HF Models to GGUF with llama.cpp.
The usage.php ledger is your record of what each distillation run cost; review it as part of every session.