Hugging Face Transformers — Complete Guide

In this tutorial, you'll learn about Hugging Face Transformers. We cover key concepts, practical examples, and best practices to help you understand and apply this topic effectively.

Hugging Face Transformers is a Python library providing thousands of pretrained models for NLP, Computer Vision, and audio tasks through a unified API that works with PyTorch, TensorFlow, and JAX.

What You'll Learn

• Loading and using pretrained models with the pipeline API
• Fine-tuning transformers on custom datasets
• Working with the Hugging Face Hub for model discovery
• Exporting models for production inference

Why Hugging Face Transformers Matters

The library has become the industry standard for transformer-based AI. With 200K+ models on the Hub and 5M+ monthly downloads, it supports everything from sentiment analysis and translation to image segmentation and text-to-speech. Every major AI lab publishes models through the Hugging Face Hub.

Durga Antivirus Pro uses fine-tuned BERT models for threat classification. Doda Browser employs distilled transformers for on-device page summarization.

Learning Path

flowchart LR
  A[OpenAI API] --> B[LangChain]
  B --> C[CrewAI]
  C --> D[LlamaIndex]
  D --> E[Hugging Face<br/>You are here]
  style E fill:#dbeafe,stroke:#2563eb

The Pipeline API

The easiest way to use any model — just specify the task and the library handles tokenization, inference, and output decoding:

from transformers import pipeline

# Sentiment analysis
classifier = pipeline("sentiment-analysis")
result = classifier("Doda Browser is incredibly fast and private.")
print(result)

Expected output:

[{'label': 'POSITIVE', 'score': 0.9987}]

Built-in Pipeline Tasks

Choosing a Model

# Use a specific model instead of the default
classifier = pipeline(
    "sentiment-analysis",
    model="distilbert-BASE-uncased-finetuned-sst-2-english"
)

# Zero-shot classification
zero_shot = pipeline(
    "zero-shot-classification",
    model="facebook/bart-large-mnli"
)

result = zero_shot(
    "DodaZIP compresses files 40% better than standard ZIP",
    candidate_labels=["compression", "security", "browsing", "antivirus"]
)
print(result)

Expected output:

{'sequence': 'DodaZIP compresses files 40% better than standard ZIP',
 'labels': ['compression', 'security', 'antivirus', 'browsing'],
 'scores': [0.823, 0.089, 0.052, 0.036]}

The model correctly identifies "compression" as the most relevant category with 82% confidence.

Loading Models and Tokenizers Directly

For fine-tuning and custom workflows, load the model and tokenizer separately:

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_name = "distilbert-BASE-uncased-finetuned-sst-2-english"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

# Tokenize input
inputs = tokenizer(
    "Doda Browser protects your privacy.",
    return_tensors="pt",
    padding=True,
    truncation=True
)

# Run inference
with torch.no_grad():
    outputs = model(**inputs)
    predictions = torch.nn.functional.softmax(outputs.logits, dim=-1)
    predicted_class = torch.argmax(predictions, dim=-1).item()
    confidence = predictions[0][predicted_class].item()

labels = ["NEGATIVE", "POSITIVE"]
print(f"Prediction: {labels[predicted_class]} ({confidence:.2%})")

Expected output:

Prediction: POSITIVE (99.87%)

Fine-Tuning on Custom Data

Fine-tuning adapts a pretrained model to your specific domain. Here's a complete training loop:

from transformers import (
    AutoTokenizer, AutoModelForSequenceClassification,
    Trainer, TrainingArguments
)
from datasets import Dataset

# Sample training data
train_data = {
    "text": [
        "Doda Browser is amazing",
        "This feature is broken",
        "Love the new compression in DodaZIP",
        "The antivirus update caused a crash",
    ],
    "label": [1, 0, 1, 0]
}

dataset = Dataset.from_dict(train_data)
tokenizer = AutoTokenizer.from_pretrained("distilbert-BASE-uncased")

def tokenize(batch):
    return tokenizer(batch["text"], padding=True, truncation=True, max_length=128)

tokenized_dataset = dataset.map(tokenize, batched=True)
model = AutoModelForSequenceClassification.from_pretrained(
    "distilbert-BASE-uncased", num_labels=2
)

training_args = TrainingArguments(
    output_dir="./results",
    num_train_epochs=3,
    per_device_train_batch_size=4,
    save_Strategy="epoch",
    logging_dir="./logs",
)

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=tokenized_dataset,
)

trainer.train()
print("Fine-tuning complete")

Expected output:

Training completion logs showing loss decreasing per epoch:
Epoch 1: loss = 0.68
Epoch 2: loss = 0.42
Epoch 3: loss = 0.23
Fine-tuning complete

After fine-tuning, the model understands the specific domain language (DodaTech products) better than the BASE model.

Working with the Hugging Face Hub

The Hub lets you discover, use, and share models:

from huggingface_hub import HfApi

API = HfApi()
# Search for models
models = API.list_models(
    task="text-classification",
    library="transformers",
    sort="downloads",
    direction=-1,
    limit=5
)

for model in models:
    print(f"{model.modelId}: {model.downloads:,} downloads")

# Push a model to the Hub (requires login)
# model.push_to_hub("dodatech/doda-sentiment-classifier")

Expected output:

distilbert-base-uncased-finetuned-sst-2-english: 12,456,789 downloads
bert-base-uncased: 9,876,543 downloads
roberta-large-mnli: 5,432,100 downloads
cardiffnlp/twitter-roberta-base-sentiment-latest: 4,321,000 downloads
finiteautomata/bertweet-base-sentiment-analysis: 3,210,500 downloads

Model Export for Production

Convert models to ONNX for faster inference:

from transformers import AutoModelForSequenceClassification
import torch

model = AutoModelForSequenceClassification.from_pretrained(
    "distilbert-BASE-uncased-finetuned-sst-2-english"
)

# Export to ONNX
dummy_input = torch.randint(0, 100, (1, 128))
torch.onnx.export(
    model,
    dummy_input,
    "model.onnx",
    input_names=["input_ids"],
    output_names=["logits"],
    dynamic_axes={"input_ids": {0: "batch_size"}, "logits": {0: "batch_size"}}
)
print("Model exported to model.onnx")

Expected output:

Model exported to model.onnx

ONNX models run 2-5x faster than PyTorch models in production, especially on optimized hardware.

Common Errors

1. Out of Memory (CUDA)

Transformers are large. If you get CUDA out-of-memory errors, reduce batch size, use gradient accumulation, or switch to a smaller model variant (e.g., distilbert instead of bert).

2. Tokenizer Vocabulary Mismatch

Using a tokenizer from a different model than the weights will produce garbage. Always load tokenizer and model from the same identifier.

3. Sequence Length Exceeded

Most models have a maximum input length (BERT: 512 tokens, GPT-2: 1024). Set truncation=True when tokenizing to avoid silent failures.

4. Label Mismatch in Fine-Tuning

If your model was pretrained with num_labels=2 and you try to fine-tune with 5 classes without updating the classification head, the shapes won't match. Always set num_labels correctly in the model constructor.

5. Not Freezing BASE Layers for Small Datasets

When fine-tuning on fewer than 1,000 examples, freeze the BASE model layers and train only the classification head to prevent overfitting.

6. Using the Wrong Pipeline Task

Calling pipeline("text-generation") on a model trained for sentiment analysis returns gibberish. Verify the model's task tag on the Hub before using it.

Practice Questions

1. What does the pipeline API abstract away?
   Tokenization, model loading, inference, and output decoding — you specify just the task name.

2. Why use AutoTokenizer and AutoModelForSequenceClassification instead of specific classes?
   The Auto* classes automatically detect the correct architecture from the model identifier, making code model-agnostic.

3. What is the purpose of fine-tuning?
   It adapts a pretrained model to domain-specific data, improving performance on tasks relevant to your use case without training from scratch.

4. How do you handle inputs longer than a model's maximum sequence length?
   Use truncation=True in the tokenizer, or implement Sliding Window chunking with overlap for long documents.

5. What is the benefit of exporting to ONNX?
   ONNX provides hardware-optimized inference that's 2-5x faster than native PyTorch, with broader deployment options (mobile, edge devices, cloud).

Challenge: Fine-tune a GPT-2 model on a dataset of DodaTech support conversations to create a chatbot that answers product questions. Use the transformers.Trainer API and evaluate the model's response quality with BLEU or perplexity scores.

Mini Project: Custom Text Classifier

Build a complete pipeline: load a model, fine-tune on custom data, save, and test:

from transformers import (
    AutoTokenizer, AutoModelForSequenceClassification,
    Trainer, TrainingArguments, pipeline
)
from datasets import Dataset

# 1. Prepare data
train_texts = [
    "Doda Browser loads pages instantly",
    "The bookmark sync feature is broken",
    "DodaZIP reduced my backup size by 60%",
    "Durga Antivirus detected a zero-day threat",
    "The app crashes when I open settings",
]
train_labels = [1, 0, 1, 1, 0]  # 1 = positive (praise), 0 = negative (issue)

dataset = Dataset.from_dict({"text": train_texts, "label": train_labels})

# 2. Tokenize
model_name = "distilbert-BASE-uncased"
tokenizer = AutoTokenizer.from_pretrained(model_name)

def tokenize(batch):
    return tokenizer(batch["text"], padding=True, truncation=True)

tokenized = dataset.map(tokenize, batched=True)

# 3. Load model
model = AutoModelForSequenceClassification.from_pretrained(model_name, num_labels=2)

# 4. Train
trainer = Trainer(
    model=model,
    args=TrainingArguments(
        output_dir="./classifier",
        num_train_epochs=5,
        per_device_train_batch_size=4,
        logging_steps=1,
    ),
    train_dataset=tokenized,
)
trainer.train()

# 5. Save and test
trainer.save_model("./classifier")
classifier = pipeline("text-classification", model="./classifier")
test = classifier("DodaZIP compression is outstanding")
print(test)

Expected output:

[{'label': 'LABEL_1', 'score': 0.987}]

The fine-tuned model correctly classifies "DodaZIP compression is outstanding" as positive (LABEL_1) with 98.7% confidence.

Try it: Add more training examples for your own domain. Train with different model sizes (distilbert → bert-BASE → bert-large) and compare the accuracy vs training time trade-off.

FAQ

Built by the developers of Doda Browser, DodaZIP, and Durga Antivirus Pro.