NoSQL Databases for Big Data — HBase, Cassandra, MongoDB

In this tutorial, you'll learn about NoSQL Databases for Big Data. We cover key concepts, practical examples, and best practices to help you understand and apply this topic effectively.

NoSQL databases provide scalable storage for big data workloads that traditional relational databases cannot handle due to extreme volume, velocity, or data variety.

What You'll Learn

In this tutorial, you'll learn the three dominant NoSQL database types for big data: HBase (wide-column), Cassandra (distributed key-value), and MongoDB (document). You'll understand their data models, CAP Theorem trade-offs, query patterns, and when to choose each one — with practical Python examples.

Why It Matters

Relational databases struggle at big data scale — they require strict schemas, cannot horizontally scale writes, and break under high velocity. NoSQL databases power the world's largest systems: Facebook uses Cassandra, Google uses BigTable (HBase's inspiration), and Uber uses MongoDB. Understanding NoSQL is essential for any data engineer.

Real-World Use

Netflix uses Cassandra for its viewing history and recommendations. Apple processes billions of iMessage records through Cassandra. The Wikimedia Foundation uses HBase for real-time analytics. Durga Antivirus Pro stores threat signatures in a wide-column store to support sub-millisecond lookups across millions of threat patterns.

graph TD
  subgraph "NoSQL Database Types"
    A[NoSQL] --> B[Wide-Column
HBase / BigTable]
    A --> C[Dynamo-Style
Cassandra / DynamoDB]
    A --> D[Document
MongoDB / CouchDB]
  end
  B --> E[Sparse, structured
Analytics, IoT]
  C --> F[High write throughput
Time-series, messaging]
  D --> F[Flexible schemas
Catalogs, user profiles]

Understanding the NoSQL Landscape

Before diving into specific databases, you need to understand the key differences between NoSQL and relational databases.

The CAP Theorem Trade-Off

NoSQL databases differ in how they balance Consistency, Availability, and Partition tolerance. For big data, Partition tolerance is mandatory (you cannot avoid network splits at scale). The real trade-off is between Consistency and Availability.

HBase — The Wide-Column Store

HBase is modeled after Google's BigTable. It runs on top of Hadoop HDFS and provides random, real-time read/write access to large datasets.

Data Model

HBase organizes data into tables, row keys, column families, and qualifiers. Think of it as a sparse, multidimensional sorted map:

Table: sensor_data
RowKey = sensor_id + timestamp

Column Family: "metrics"
  qualifier: temperature → value: 72.3
  qualifier: humidity    → value: 45.1

Column Family: "metadata"
  qualifier: location → value: "rack-42"

Python Example: HBase with HappyBase

# File: hbase_example.py
# Requires: pip install happybase
# Requires: Running HBase Thrift server (default port 9090)
import happybase

connection = happybase.Connection('localhost', port=9090)

# Create a table with two column families
families = {
    'metrics': dict(max_versions=3),
    'metadata': dict()
}

connection.create_table('sensor_data', families)
table = connection.table('sensor_data')

# Insert a row — row key is sensor_id + timestamp
table.put(
    b'sensor_001_20260620T120000Z',
    {
        b'metrics:temperature': b'72.3',
        b'metrics:humidity': b'45.1',
        b'metadata:location': b'rack-42'
    }
)

# Retrieve by row key
row = table.row(b'sensor_001_20260620T120000Z')
print(f"Temperature: {row[b'metrics:temperature'].decode()}")
print(f"Humidity: {row[b'metrics:humidity'].decode()}")

connection.close()

Expected output:

Temperature: 72.3
Humidity: 45.1

The row key design is critical in HBase. Keys beginning with the same prefix are stored together — so prefix-based scans are fast. Putting the timestamp after the sensor ID lets you scan all readings for one sensor efficiently.

Cassandra — The Distributed Dynamo-Style Database

Apache Cassandra combines Amazon Dynamo's distribution model with BigTable's data model. It offers linear scalability and no single point of failure.

Data Model

Cassandra uses a partition key to distribute data across nodes and clustering columns to sort within a partition.

-- Cassandra Query Language (CQL)
CREATE TABLE IF NOT EXISTS sensor_data (
    sensor_id text,
    recorded_at timestamp,
    temperature double,
    humidity double,
    location text,
    PRIMARY KEY (sensor_id, recorded_at)
) WITH CLUSTERING ORDER BY (recorded_at DESC);

The partition key (sensor_id) determines which node stores the data. The clustering column (recorded_at) sorts rows within that partition. This design means you can efficiently query "all readings for sensor X ordered by time" — a common access pattern in IoT.

Python Example: Cassandra with cassandra-driver

# File: cassandra_example.py
# Requires: pip install cassandra-driver
# Requires: Running Cassandra cluster (default port 9042)
from cassandra.cluster import Cluster
from cassandra.query import SimpleStatement
from datetime import datetime

cluster = Cluster(['127.0.0.1'])
session = cluster.connect('bigdata')

# Insert a reading
session.execute(
    """
    INSERT INTO sensor_data
        (sensor_id, recorded_at, temperature, humidity, location)
    VALUES
        (%s, %s, %s, %s, %s)
    """,
    ('sensor_001', datetime.utcnow(), 72.3, 45.1, 'rack-42')
)

# Query recent readings for a sensor
query = SimpleStatement(
    "SELECT recorded_at, temperature, humidity "
    "FROM sensor_data WHERE sensor_id = %s "
    "ORDER BY recorded_at DESC LIMIT 5"
)
rows = session.execute(query, ('sensor_001',))
for row in rows:
    print(f"Time: {row.recorded_at}, "
          f"Temp: {row.temperature}, "
          f"Humidity: {row.humidity}")

cluster.shutdown()

Expected output:

Time: 2026-06-20 12:00:00, Temp: 72.3, Humidity: 45.1

Cassandra's write path is optimized for speed — writes Go to a commit log and a memtable, then flush to SSTables. Reads combine data from multiple SSTables, making them slightly slower than writes. This is by design: Cassandra is built for write-heavy workloads.

MongoDB — The Document Store

MongoDB stores data as BSON documents (binary JSON). It offers a flexible schema, powerful query language, and built-in horizontal scaling through Sharding.

Data Model

MongoDB stores documents in collections. Unlike HBase and Cassandra, there's no predefined column structure — each document can have different fields.

// MongoDB document structure
{
  "_id": ObjectId("..."),
  "sensor_id": "sensor_001",
  "recorded_at": ISODate("2026-06-20T12:00:00Z"),
  "readings": {
    "temperature": 72.3,
    "humidity": 45.1
  },
  "metadata": {
    "location": "rack-42",
    "floor": 3,
    "building": "A"
  }
}

Python Example: MongoDB with PyMongo

# File: MongoDB_example.py
# Requires: Pip install pymongo
# Requires: Running MongoDB (default port 27017)
from pymongo import MongoClient
from datetime import datetime

client = MongoClient('localhost', 27017)
db = client['bigdata']
collection = db['sensor_data']

# Insert a document
doc = {
    'sensor_id': 'sensor_001',
    'recorded_at': datetime.utcnow(),
    'readings': {
        'temperature': 72.3,
        'humidity': 45.1
    },
    'metadata': {
        'location': 'rack-42',
        'floor': 3,
        'building': 'A'
    }
}
result = collection.insert_one(doc)
print(f"Inserted document with _id: {result.inserted_id}")

# Query: find all readings for a sensor
Cursor = collection.find(
    {'sensor_id': 'sensor_001'}
).sort('recorded_at', -1).limit(5)

for doc in Cursor:
    print(f"Time: {doc['recorded_at']}, "
          f"Temp: {doc['readings']['temperature']}")

Expected output:

Inserted document with _id: 6673f1a2...
Time: 2026-06-20 12:00:00, Temp: 72.3

MongoDB excels when your schema evolves. Adding a new field doesn't require a Migration — just insert documents with the new field. Old documents remain unchanged, which is a huge advantage in Agile environments.

Choosing the Right NoSQL Database

Common Mistakes

1. Using Cassandra without understanding partition key design: A bad partition key (too few distinct values) creates hot spots. One overloaded node while others sit idle destroys performance. Always test with realistic data distribution.

2. Designing HBase row keys without scan patterns: HBase only sorts by row key. If you need to query by timestamp across all sensors, design your row key as timestamp_sensor_id — not the reverse.

3. Assuming MongoDB does joins like SQL: MongoDB's $lookup exists but performs poorly at scale. If you need frequent joins, you're using the wrong database. Denormalize or use a relational database.

4. Not setting data expiration (TTL) in Cassandra: Cassandra data lives forever unless you set TTL or use TimeWindowCompactionStrategy. Unbounded data slows reads and increases storage costs.

5. Ignoring consistency levels in Cassandra: QUORUM reads + ONE writes can return stale data. ALL writes are slow. Choose consistency level based on your application's tolerance for staleness.

6. Running HBase without RegionServer tuning: Default memory settings are for small test clusters. Production HBase needs careful Heap sizing and compaction tuning.

7. Creating too many indexes in MongoDB: MongoDB indexes consume memory and slow writes. Only index fields you actually query. Use explain() to verify index usage.

Practice Questions

1. What is the primary difference between HBase and Cassandra's Consistency Models? HBase is CP (strong consistency, availability suffers during partition). Cassandra is AP (tunable consistency, always available for writes).

2. How does Cassandra distribute data across nodes? Using consistent hashing with virtual nodes. The partition key is hashed to determine which node(s) store the data. Replication factor controls how many copies exist.

3. What is a column family in HBase? A group of related columns stored together on disk. Column families should be designed for access patterns — columns accessed together Go in the same family.

4. When would you choose MongoDB over HBase? When you need a flexible schema (documents can have different fields), rich secondary indexes, or full-text search. HBase is better for strongly consistent, column-oriented scans.

5. What is the role of SSTables in Cassandra? SSTables are immutable sorted data files on disk. Once written, they are never modified. Compactions Merge SSTables to reclaim space and improve read performance.

Challenge

Design a NoSQL schema for a real-time chat application handling 1 million users. The app needs per-user message history, global message search, and presence status. Choose the database(s) and justify your choice with schema examples.

Real-World Task

Install Apache Cassandra locally. Using the cqlsh tool, create a keyspace with Replication factor 3, create a table for time-series metrics, insert 10,000 rows using a Python script, and observe how data distributes across tokens using nodetool status.

Mini Project: Multi-Database Sensor Pipeline

Build a data pipeline that ingests IoT sensor data. Store the raw event stream in Cassandra (for high write throughput), create an HBase table for aggregated hourly analytics (for strong consistency scans), and use MongoDB for a device catalog (flexible schema for heterogeneous device types).

Security angle: This multi-database pattern is used by Durga Antivirus Pro to store threat intelligence — Cassandra for raw threat events, HBase for lookup tables (IP reputation), and a document store for threat reports. The separation ensures high availability for ingestion while maintaining consistent lookups.

FAQ

What's Next

Learning Path

Graph LR
  A[Big Data Overview] --> B[Hadoop & HDFS]
  B --> C[NoSQL Databases]
  C --> D[Data Warehousing]
  D --> E[Real-Time Analytics]
  style C fill:#4f46e5,stroke:#fff,stroke-width:2px,color:#fff

Built by the developers of Doda Browser, DodaZIP, and Durga Antivirus Pro.

What's Next

Congratulations on completing this NoSQL for Big Data tutorial. Here's where to Go from here:

• Practice daily — Consistency beats long study sessions
• Build a project — Apply what you learned with a real dataset
• Explore related topics — Check out Data Warehousing and Hadoop tutorials
• Join the community — Discuss with other learners and share your progress

Remember: every expert was once a beginner. Keep coding!