# AGENTS.md: Fine-tuning with TRL + PEFT

## Commands
- Debug run (tiny model, 20 steps, CPU/GPU): `uv run python train.py --config configs/debug.yaml`
- Real run (ask first, costs GPU time): `uv run accelerate launch train.py --config configs/<name>.yaml`
- Eval: `uv run python eval.py --adapter outputs/<run>/ --suite core`

## Data
- Training data is JSONL in `data/` using the conversational format (`{"messages": [{"role": "user", ...}, {"role": "assistant", ...}]}`) or prompt/completion pairs. Validate schema and dedupe before training.
- Hold out an eval split; never train on eval or test examples. Remove secrets and personal data.

## Training
- Use `SFTTrainer` with an `SFTConfig` (or `DPOTrainer`/`GRPOTrainer` for preference/RL tuning), all hyperparameters in YAML config: learning rate, epochs, batch size, gradient accumulation, max sequence length, packing.
- LoRA via `peft.LoraConfig` (rank, alpha, dropout, target modules) by default; QLoRA (4-bit with bitsandbytes) when memory is tight. Full fine-tuning only with explicit approval.
- Use the model's chat template; check a few tokenized samples (loss is computed on assistant tokens only when configured).
- bf16 on supported GPUs, gradient checkpointing for long sequences, fixed seeds, logging to the configured tracker.
- Save adapters and the resolved config per run in `outputs/<run-name>/`.

## Evaluation
- Compare against the base model on the same eval suite; report metric deltas and a few qualitative samples. Check for regressions in general ability and safety prompts.

## Don't
Don't push models or datasets to the Hub, or merge adapters into base weights, without approval.