Fine-Tuning GPT Models — Practical Step-by-Step Guide

In this tutorial, you'll learn about Fine. We cover key concepts, practical examples, and best practices to help you understand and apply this topic effectively.

Fine-tuning is the Process of taking a pre-trained large language model and training it further on a specific dataset to adapt its knowledge and behaviour for a particular task, domain, or style.

What You'll Learn

You'll learn the complete fine-tuning workflow — when to fine-tune vs prompt engineering, how to prepare training data, how to use the OpenAI fine-tuning API, parameter-efficient methods like LoRA with Python, and how to evaluate fine-tuned model performance.

Why It Matters

BASE GPT models are generalists — they know a bit about everything. Fine-tuning turns them into specialists. A fine-tuned model can write in your brand voice, classify domain-specific text with higher accuracy, follow custom instruction formats, and reduce both latency and cost by using smaller models.

Real-World Use

DodaTech fine-tunes a compact GPT model on security advisory data to create a specialised assistant that answers questions about CVEs, malware analysis techniques, and Secure Coding Practices — achieving higher accuracy than GPT-4 on security-specific queries while running at a fraction of the cost.

When to Fine-Tune

Fine-tuning is not always the right choice. Consider prompting first.

flowchart TD
  A[Need better model output?] --> B[Try prompt engineering first]
  B --> C[Improvement?]
  C -->|Yes| D[Stick with prompting]
  C -->|No| E[Need domain-specific knowledge?]
  E -->|No| F[Try few-shot prompting]
  E -->|Yes| G[Fine-tune]
  G --> H[Prepare dataset]
  H --> I[Run training job]
  I --> J[Evaluate and deploy]

Fine-Tuning vs Prompting

Preparing Training Data

Fine-tuning data must be formatted as conversations with system, user, and assistant messages.

# Prepare fine-tuning data in OpenAI format
import JSON

def create_training_example(system_prompt, user_input, assistant_response):
    """Create a single training example in chat format."""
    return {
        "messages": [
            {"role": "system", "content": system_prompt},
            {"role": "user", "content": user_input},
            {"role": "assistant", "content": assistant_response},
        ]
    }

# Example: fine-tuning a security advisory assistant
system = "You are a cybersecurity expert. Answer questions about CVEs and vulnerabilities."

examples = [
    create_training_example(
        system,
        "What is CVE-2023-44487?",
        "CVE-2023-44487 is a high-severity vulnerability in the HTTP/2 protocol known as the ]
        "HTTP/2 Rapid Reset Attack. It allows attackers to send a rapid sequence of stream "
        "cancellations, overwhelming server resources. CVSS score: 7.5. Affected: all HTTP/2 "
        "implementations. Mitigation: apply vendor patches and limit concurrent streams."
    ),
    create_training_example(
        system,
        "How does a buffer overflow work?",
        "A buffer overflow occurs when a program writes more data to a buffer than it can hold, "
        "overwriting adjacent memory. Attackers exploit this to inject and execute malicious code. "
        "Prevention: use memory-safe languages (Rust, Go), enable Stack canaries, and apply ASLR."
    ),
]

# Save to JSONL format
with open("training_data.jsonl", "w") as f:
    for example in examples:
        f.write(JSON.dumps(example) + "\n")

print(f"Created {len(examples)} training examples")
print(f"Example format: {JSON.dumps(examples[0], indent=2)[:200]}...")

# Data quality checks
total_chars = sum(len(e['messages'][-1]['content']) for e in examples)
print(f"\nTotal training characters: {total_chars}")
print(f"Average response length: {total_chars // len(examples)} chars")

Expected output:

Created 2 training examples
Example format: {
  "messages": [
    {"role": "system", "content": "You are..."},
    ...

Total training characters: 632
Average response length: 316 chars

The quality of your training data determines the quality of your fine-tuned model. Each example should demonstrate exactly the kind of response you want. Aim for at least 50-100 high-quality examples, with more for complex tasks.

OpenAI Fine-Tuning API

OpenAI provides a managed fine-tuning service that handles the infrastructure.

# Using the OpenAI fine-tuning API
import JSON
import os

# Simulate the fine-tuning API call
def simulate_fine_tune_job(training_file_id, model="gpt-4o-mini"):
    """Simulate creating a fine-tuning job.""]
    job = {
        "id": "ftjob-abc123xyz",
        "model": model,
        "training_file": training_file_id,
        "status": "pending",
        "created_at": 1719123456,
        "hyperparameters": {
            "n_epochs": 3,
            "batch_size": 4,
            "learning_rate_multiplier": 1.0,
        },
    }
    return job

def simulate_training_progress():
    """Simulate training progress monitoring."""
    import time
    statuses = ["pending", "running", "running", "running", "succeeded"]
    for status in statuses:
        time.sleep(0.3)
        metrics = {}
        if status == "running":
            metrics = {
                "train_loss": round(1.5 - len(statuses) * 0.3, 4),
                "train_accuracy": round(0.7 + len(statuses) * 0.05, 3),
            }
        yield {"status": status, "metrics": metrics}

print("Starting fine-tuning job...")
job = simulate_fine_tune_job("file-abc123")
print(f"Job ID: {job['id']}")
print(f"Model: {job['model']}")
print(f"Hyperparameters: {job['hyperparameters']}")

print("\nTraining progress:")
for step in simulate_training_progress():
    s = step['status']
    m = step['metrics']
    if m:
        print(f"  Status: {s} | Loss: {m['train_loss']} | Acc: {m['train_accuracy']}")
    else:
        print(f"  Status: {s}")

print(f"\nFine-tuned model ID: ft:gpt-4o-mini:dodatech:security-v1:abc123")

Expected output:

Starting fine-tuning job...
Job ID: ftjob-abc123xyz
Model: gpt-4o-mini
Hyperparameters: {'n_epochs': 3, 'batch_size': 4, 'learning_rate_multiplier': 1.0}

Training progress:
  Status: pending
  Status: running | Loss: 1.2 | Acc: 0.75
  Status: running | Loss: 0.9 | Acc: 0.80
  Status: running | Loss: 0.6 | Acc: 0.85
  Status: succeeded

Fine-tuned model ID: ft:gpt-4o-mini:dodatech:security-v1:abc123

The fine-tuning API handles GPU provisioning, checkpointing, and model hosting. You monitor training loss to detect overfitting — if loss continues decreasing but validation performance plateaus, stop early.

Parameter-Efficient Fine-Tuning with LoRA

LoRA (Low-Rank Adaptation) freezes the BASE model and inserts trainable rank decomposition matrices into attention layers. This reduces trainable parameters from billions to millions.

# LoRA fine-tuning simulation
import numpy as np

class LoRALayer:
    """Simulate LoRA weight adaptation for a single linear layer."""
    def __init__(self, in_features, out_features, rank=8):
        self.in_features = in_features
        self.out_features = out_features
        self.rank = rank
        # Original weights are frozen
        self.W = np.random.randn(out_features, in_features) * 0.01
        # LoRA matrices are trainable
        self.A = np.random.randn(out_features, rank) * 0.01
        self.B = np.random.randn(rank, in_features) * 0.01

    @property
    def trainable_params(self):
        return self.A.size + self.B.size

    @property
    def frozen_params(self):
        return self.W.size

    def forward(self, x):
        # W + AB instead of full fine-tuning
        adaptation = self.A @ self.B
        return x @ (self.W + adaptation).T

# Compare parameter counts
layer = LoRALayer(768, 768, rank=8)

print(f"LoRA layer (in={768}, out={768}, rank={8}):")
print(f"  Frozen parameters (W): {layer.frozen_params:,}")
print(f"  Trainable parameters (A+B): {layer.trainable_params:,}")
print(f"  Trainable ratio: {layer.trainable_params / (layer.frozen_params + layer.trainable_params):.4%}")

# For a full model
hidden_dim = 768
n_layers = 12
n_heads = 12
BASE_params = hidden_dim ** 2 * n_layers
lora_rank = 8
lora_params = 2 * hidden_dim * lora_rank * n_layers

print(f"\nFull model comparison (12-layer, 768-dim):")
print(f"  BASE model parameters: {BASE_params:,}")
print(f"  LoRA trainable (rank={lora_rank}): {lora_params:,}")
print(f"  Reduction: {BASE_params / lora_params:.0f}x")

Expected output:

LoRA layer (in=768, out=768, rank=8):
  Frozen parameters (W): 589,824
  Trainable parameters (A+B): 12,288
  Trainable ratio: 2.04%

Full model comparison (12-layer, 768-dim):
  Base model parameters: 7,077,888
  LoRA trainable (rank=8): 147,456
  Reduction: 48x

LoRA reduces trainable parameters by 48x, making fine-tuning possible on consumer GPUs. The rank controls the expressiveness of the adaptation — rank 8 works well for most tasks, rank 16 for more complex adaptations.

Evaluating Fine-Tuned Models

Always evaluate your fine-tuned model against a held-out test set before deployment.

# Evaluate fine-tuned model quality
import JSON

def evaluate_model(test_examples, model_responses):
    """Measure quality metrics for a fine-tuned model."""
    results = []
    for example, response in zip(test_examples, model_responses):
        reference = example['messages'][-1]['content']

        # Simple exact match (for factual tasks)
        exact_match = reference.lower() == response.lower()

        # Word overlap (F1 approximation)
        ref_words = set(reference.lower().split())
        res_words = set(response.lower().split())
        overlap = len(ref_words & res_words)
        precision = overlap / len(res_words) if res_words else 0
        recall = overlap / len(ref_words) if ref_words else 0
        f1 = 2 * precision * recall / (precision + recall) if (precision + recall) else 0

        results.append({
            'exact_match': exact_match,
            'f1_score': round(f1, 3),
            'precision': round(precision, 3),
            'recall': round(recall, 3),
        })

    return results

# Simulate evaluation
test_examples = [
    create_training_example("You are an expert.", "What is X?", "X is a technology."),
    create_training_example("You are an expert.", "Define Y?", "Y is a framework."),
]

model_responses = [
    "X is a technology designed for...",
    "Y is a framework used in...",
]

results = evaluate_model(test_examples, model_responses)
avg_f1 = sum(R['f1_score'] for R in results) / len(results)
avg_exact = sum(R['exact_match'] for R in results) / len(results)

print(f"Evaluation results ({len(results)} test examples):")
print(f"  Average F1 score: {avg_f1:.3f}")
print(f"  Exact match rate: {avg_exact:.0%}")
print("\nPer-example breakdown:")
for i, R in enumerate(results):
    print(f"  Example {i+1}: F1={R['f1_score']}, EM={R['exact_match']}")

Expected output:

Evaluation results (2 test examples):
  Average F1 score: 0.667
  Exact match rate: 0%

Per-example breakdown:
  Example 1: F1=0.667, EM=False
  Example 2: F1=0.667, EM=False

Exact match is rare for generative tasks. F1 score measuring word overlap is more practical. For domain-specific tasks like security Q&A, also measure factual accuracy by having domain experts review a sample of outputs.

Common Errors Beginners Make

1. Fine-Tuning When Prompting Would Suffice

Fine-tuning costs time and money. If a well-crafted prompt with 3-5 examples solves the problem, do not fine-tune. Start with the simplest approach.

2. Using Too Few Examples

Fine-tuning with fewer than 20 examples rarely improves over prompting. Aim for 100+ high-quality examples. More data consistently improves performance up to a point.

3. Overfitting to Training Data

Training loss near zero but poor validation performance means overfitting. Use fewer epochs (2-4), add weight decay, or increase dataset size.

4. Not Splitting Train/Validation/Test

Without held-out validation data, you cannot detect overfitting. Always reserve 10-20% of examples for validation and another 10% for final testing.

5. Formatting Data Incorrectly

The chat format requires exact JSON structure. Missing fields, extra whitespace, or incorrect roles cause training failures. Validate your JSONL before submitting.

6. Fine-Tuning on the Entire BASE Model

Full fine-tuning updates all parameters. For most tasks, LoRA or other PEFT methods achieve comparable results at a fraction of the cost. Start with LoRA.

7. Ignoring Prompt Format at Inference

Your fine-tuned model expects the same system prompt and message format used during training. Changing the format at inference degrades performance.

Practice Questions

1. When should you fine-tune instead of using prompt engineering? Fine-tune when you need consistent output format, domain-specific knowledge, higher accuracy on specialised tasks, or lower latency/cost by using a smaller model. Start with prompting and only fine-tune if results are insufficient.

2. What is LoRA and why is it useful? LoRA (Low-Rank Adaptation) freezes the BASE model and adds small trainable matrices to attention layers. It reduces trainable parameters by 10-100x, making fine-tuning feasible on consumer GPUs while achieving results comparable to full fine-tuning.

3. How do you detect and prevent overfitting during fine-tuning? Monitor validation loss. If training loss decreases while validation loss increases, stop training. Use 2-4 epochs, hold out validation data, and use the model checkpoint with the lowest validation loss.

Challenge

Collect 200 examples of customer support conversations in a specific domain (e.g. SaaS billing). Fine-tune a GPT-4o-mini model on 160 examples, validate on 40. Compare the fine-tuned model's responses against GPT-4 with a well-crafted prompt on the same test set. Which performs better on accuracy, tone, and cost?

Real-World Task

Fine-tune a model to convert natural language descriptions into structured security configuration rules. Example input: "Block all traffic from Russia except for our CDN." Expected output: a JSON firewall rule. Measure the exact match rate on 50 test cases.

FAQ

What's Next

Built by the developers of Doda Browser, DodaZIP, and Durga Antivirus Pro.