Unsloth: 2x Faster LLM Fine-Tuning at Half the Memory

Unsloth is an open-source Python library that dramatically accelerates LLM fine-tuning — delivering 2x faster training speed and 60% less memory usage compared to standard Hugging Face + PEFT approaches. It's become the default fine-tuning tool for developers who want to run LoRA or QLoRA training on consumer hardware without hitting VRAM limits.

What Unsloth actually does

Unsloth optimizes the core computational bottlenecks in transformer fine-tuning through three main techniques:

• Custom CUDA kernels — rewrites key operations (attention, RoPE embeddings, cross-entropy loss) in optimized CUDA for 2x faster forward and backward passes
• Memory-efficient backpropagation — computes gradients in chunks rather than materializing the full gradient matrix, significantly reducing peak VRAM usage
• Intelligent weight offloading — moves unused weights to CPU during forward pass, loading them back for the backward pass. Enables training models much larger than your GPU's VRAM

Unsloth vs standard LoRA training

Getting started

pip install unsloth

from unsloth import FastLanguageModel
import torch

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="unsloth/llama-3.2-8b-bnb-4bit",
    max_seq_length=2048,
    dtype=None,
    load_in_4bit=True,
)

# Add LoRA adapters
model = FastLanguageModel.get_peft_model(
    model,
    r=16,
    target_modules=["q_proj", "v_proj", "k_proj", "o_proj"],
    lora_alpha=16,
    lora_dropout=0,
    bias="none",
    use_gradient_checkpointing="unsloth",
)

# Then train normally with SFTTrainer or your own loop

What you can fine-tune on consumer hardware with Unsloth

Unsloth for inference

Unsloth also accelerates inference via its GGUF export pipeline. After fine-tuning, you can export directly to quantized GGUF for use with llama.cpp or Ollama:

model.save_pretrained_gguf("my-model", tokenizer, quantization_method="q4_k_m")
# Outputs my-model-unsloth.Q4_K_M.gguf — ready for Ollama or llama.cpp

The token cost angle

Faster fine-tuning means lower cloud GPU costs. A fine-tuning run that takes 4 hours on standard LoRA takes ~2 hours with Unsloth — halving your GPU rental bill. For iterative fine-tuning workflows where you run many experiments, this compounds significantly.

And a well fine-tuned model can replace verbose system prompts — baking behavior into weights rather than injecting it as context tokens on every call. See our guide on LLM token waste for how this reduces per-call costs.