Home › Hugging Face › LLM fine-tuning (TRL + PEFT) AGENTS.md

LLM fine-tuning (TRL + PEFT) AGENTS.md

AGENTS.md for fine-tuning LLMs with Hugging Face TRL and PEFT (LoRA/QLoRA): data format, configs, evals, and safety.

AGENTS.mdHugging Face

Copy AGENTS.md

How to use

Save this as AGENTS.md in your repository root, commit it, and your agent will pick it up automatically. Edit the commands to match your project.

Note: Review and test before use. Provided as is, without warranty. See Terms.

# AGENTS.md: Fine-tuning with TRL + PEFT

## Commands
- Debug run (tiny model, 20 steps, CPU/GPU): `uv run python train.py --config configs/debug.yaml`
- Real run (ask first, costs GPU time): `uv run accelerate launch train.py --config configs/<name>.yaml`
- Eval: `uv run python eval.py --adapter outputs/<run>/ --suite core`

## Data
- Training data is JSONL in `data/` using the conversational format (`{"messages": [{"role": "user", ...}, {"role": "assistant", ...}]}`) or prompt/completion pairs. Validate schema and dedupe before training.
- Hold out an eval split; never train on eval or test examples. Remove secrets and personal data.

## Training
- Use `SFTTrainer` with an `SFTConfig` (or `DPOTrainer`/`GRPOTrainer` for preference/RL tuning), all hyperparameters in YAML config: learning rate, epochs, batch size, gradient accumulation, max sequence length, packing.
- LoRA via `peft.LoraConfig` (rank, alpha, dropout, target modules) by default; QLoRA (4-bit with bitsandbytes) when memory is tight. Full fine-tuning only with explicit approval.
- Use the model's chat template; check a few tokenized samples (loss is computed on assistant tokens only when configured).
- bf16 on supported GPUs, gradient checkpointing for long sequences, fixed seeds, logging to the configured tracker.
- Save adapters and the resolved config per run in `outputs/<run-name>/`.

## Evaluation
- Compare against the base model on the same eval suite; report metric deltas and a few qualitative samples. Check for regressions in general ability and safety prompts.

## Don't
Don't push models or datasets to the Hub, or merge adapters into base weights, without approval.