Blog
Notes on adapters, fine-tuning, and running models on your own hardware.
Learning Rate Schedules for Adapter Training
How learning rate and schedule choice affect LoRA stability, convergence, and overfitting risk.
2024-12-02QLoRA: 4-bit Training on Modest GPUs
Training LoRA on top of a 4-bit quantized base model to cut VRAM without hurting quality much.
2024-11-05Merging a LoRA into the Base Model
When to bake an adapter into the base weights and when to keep them separate.
2024-10-11
Choosing Target Modules for LoRA Training
How to decide which layers receive LoRA updates and why the choice changes training behavior and artifact size.
2024-09-23Preparing Image Datasets for Style Adapters
How to curate image datasets for style LoRA training without polluting the signal with bad crops, bad captions, or bad balance.
2024-09-08RAG or a Fine-Tuned Adapter: Choosing the Path
When retrieval augmentation is enough and when a small LoRA is the better tool.
2024-08-15Captioning Image Datasets for Better LoRA Results
How to write image captions that help a LoRA learn the right visual concept without overfitting to generic tags.
2024-07-15Train and Validation Splits That Catch Real Regressions
How to split datasets so validation actually measures generalization instead of leakage or duplicate memorization.
2024-07-01Dataset Cleaning Checklist for Fine-Tuning
A practical checklist for cleaning instruction and chat datasets before a fine-tune job starts.
2024-06-10