Fine-tuning

Prerequisites

• Large language model
• Overfitting

How it works

Fine-tuning continues training from existing parameters for a new task or distribution. Full fine-tuning updates many or all weights; parameter-efficient methods train a smaller set. LoRA, for example, learns low-rank weight updates while freezing the base weights. Prompting changes input context and ordinarily leaves weights unchanged.

Worked example

For an illustrative report-formatting task, collect approved input-output pairs and hold out whole document families. Compare the base model with a prompt-only approach and an adapted model. Evaluate whether formatting improves without increasing invented facts or degrading unrelated tasks.

A useful parameterization

LoRA represents an update as Delta_W = B A, where the intermediate rank is small relative to the original matrix dimensions. This reduces trainable parameters; it does not remove every other training-memory cost.

What to test

Check unseen documents, long inputs, malformed requests and general-language regressions. Record the base model, tokenizer, adapter, dataset split and training settings so the result can be reproduced.

Common pitfall

Fine-tuning on examples is not a dependable way to keep changing facts current. Separate behavior adaptation from document retrieval and factual verification.

Sources for the expanded explanation

• LoRA: Low-Rank Adaptation of Large Language Models ↗

Editorial synthesis checked against the linked primary material; no independent replication is claimed.

Original example

Adapt a pretrained model to support conversations.

Context and aliases

Level: Foundational. Stages: Training · Adaptation · Evaluation.

Tags: DL · SLM · LLM · NLP.

Aliases: Finetuning.

Related concepts

• Transfer learning
• Full fine-tuning

Concepts linking here

• In-context learning
• Pretraining
• Task-adaptive pretraining
• Transfer learning

Technology connections

• Axolotl — Configuration-driven model fine-tuning toolkit.
• bitsandbytes — Low-bit quantization and memory-saving optimizer primitives.
• Hugging Face PEFT — Parameter-efficient model adaptation methods.
• Hugging Face TRL — Post-training trainers for supervised and preference/RL-style methods.
• LLaMA-Factory — Unified training and fine-tuning toolkit for supported models.
• NVIDIA Model Optimizer — Model optimization tools including supported quantization, pruning and distillation paths.
• OpenRLHF — Distributed RLHF and post-training framework.
• TorchAO — PyTorch architecture optimization and quantization tooling.
• torchtune — PyTorch-native post-training recipes; development has wound down.
• Unsloth — Optimized fine-tuning tooling for supported model families.

Original references

• Transformers glossary ↗ · Consulted
• PEFT: LoRA conceptual guide ↗ · Consulted
• TRL: supervised fine-tuning ↗ · Reference link
• TRL documentation ↗ · Consulted
• Original reference URL ↗