Cloud Parts

Wide-column stores

Huge tables split into partitions by key, with rows kept sorted inside each one, built for heavy writes.

3 technologies on this page ↓
Live model
What you’re watchingIn Cassandra and ScyllaDB, any node can coordinate a write and sends it to every replica of the partition, here three. At consistency level QUORUM it can answer the client once two have acknowledged, so when replica C goes down, writes keep succeeding and C catches up after it returns.

How it works

A wide-column store keeps tables of rows, but it isn’t relational. Each row belongs to a partition, chosen by hashing its partition key (Cassandra, ScyllaDB) or by ranges of sorted row keys (Bigtable, HBase), and rows are stored in sorted order, by a clustering column such as a timestamp in Cassandra or by row key in Bigtable. That makes two things cheap: writing, and reading one partition or a slice of it, such as “the last hour of readings from sensor 7”. Rows don’t need the same columns, and a row can have a huge number of them, which is where “wide” comes from. Writes go to a commit log and an in-memory table, which is later flushed to immutable sorted files on disk, so even a delete is a write, called a tombstone. Filtering on other columns or joining tables is slow or unsupported, so you design one table per query and often write the same data to several. The name is loose: Google describes Bigtable as a sorted key-value map, and DB-Engines calls these databases two-dimensional key-value stores.

Think of it as

A filing drawer per customer, with pages kept in date order. Adding a page at the back is instant, and so is pulling last week’s pages; finding every page that mentions a refund means emptying every drawer.

Reach for it when

  • Time series, event logs, messages and IoT readings: lots of writes, read back by key and time range
  • Very large datasets that must keep accepting writes across racks or regions
  • Access patterns you know well enough to design tables around

Think twice when

  • Ad-hoc queries, filters on arbitrary columns, or joins
  • Small datasets, where the operational work doesn’t pay off
  • Updates that need transactions across many rows or partitions

A real example

A fleet-tracking company writes each truck’s GPS position every few seconds to Cassandra, in a table partitioned by truck and day and sorted by time, newest first. Drawing a truck’s route for today reads one partition in order. Finding every truck that passed through a given town, though, needs a second table written with that question in mind.

Technologies

Specific products, and what each does differently.

What each provider calls it

AWS
Amazon Keyspaces (Cassandra-compatible); HBase on Amazon EMR
Google Cloud
Bigtable, which also offers an HBase-compatible API
Azure
Azure Managed Instance for Apache Cassandra; Azure Cosmos DB for Apache Cassandra
Open source & others
Apache Cassandra, ScyllaDB (source-available since 2025), Apache HBase

In code · sql

-- CQL: Cassandra, ScyllaDB, Amazon Keyspaces
CREATE TABLE positions (
  truck_id text,
  day      date,
  ts       timestamp,
  lat      double,
  lon      double,
  PRIMARY KEY ((truck_id, day), ts)   -- partition key, then clustering column
) WITH CLUSTERING ORDER BY (ts DESC);

INSERT INTO positions (truck_id, day, ts, lat, lon)
VALUES ('tr-118', '2026-10-06', '2026-10-06 09:30:05', 59.91, 10.75);

-- one partition, already sorted newest first
SELECT ts, lat, lon FROM positions
WHERE truck_id = 'tr-118' AND day = '2026-10-06'
LIMIT 100;

Worth exploring

WebsiteDB-EnginesDB-Engines, provided by Redgate SoftwareA directory of hundreds of database systems with their data models, licences and features, which you can compare side by side. Its monthly ranking measures popularity (search results, Google Trends, Stack Overflow questions, job ads, LinkedIn profiles, posts on X), not quality or speed.
All resources →

Often confused with

Key-value storesDocument databasesTime-series databasesRelational databasesWide tablesKey-value, document or wide-column?
Where it came from2006–2024

In 2006 Google described Bigtable, the storage system behind web indexing, Google Earth and Google Finance. Its data model was “a sparse, distributed, persistent multi-dimensional sorted map”, indexed by row key, column key and timestamp. Rows were kept in key order and split into ranges called tablets, each served by one machine, with the data files stored in Google’s distributed file system.

Open-source versions followed quickly. HBase, begun at Powerset in 2007, rebuilt Bigtable on Hadoop’s HDFS. Cassandra, built at Facebook for inbox search and open-sourced in 2008, took Bigtable’s column families and log-structured storage but spread data the way Amazon’s Dynamo did, with no master, keys hashed around a ring and every replica accepting writes. One of its two authors, Avinash Lakshman, had co-written the Dynamo paper. Both projects became top-level Apache projects in 2010.

Cassandra’s interface then moved closer to SQL. CQL, a SQL-like language, gave tables declared columns, a partition key and clustering columns, so a partition holds many rows sorted by the clustering columns. ScyllaDB rewrote Cassandra in C++ from 2015, and the clouds now sell managed versions: Bigtable itself, Amazon Keyspaces since 2020, and Azure’s managed Cassandra.

Why it’s built this way

Keep rows sorted by key
Bigtable stored rows in key order so related data sat together and could be read as a range. Web pages were stored under reversed host names, such as com.google.maps/index.html, so one domain’s pages were neighbours.
Append writes, sort them later
Writing to a commit log and memory, then flushing immutable sorted files, turns random writes into sequential ones, so heavy write loads stay cheap. Reads and background compaction pay for it later.
Every replica accepts writes (Cassandra)
Facebook’s inbox search had to take billions of writes a day and replicate across data centers where, at that scale, components fail continuously. So Cassandra borrowed Dynamo’s masterless replication instead of Bigtable’s single server per range.

Timeline

  1. 2006

    Google publishes the Bigtable paper

  2. 2007

    HBase code is posted and ships with Hadoop 0.15.0

  3. 2008

    Facebook open-sources Cassandra

  4. 2010

    Cassandra and HBase become top-level Apache projects

  5. 2015

    ScyllaDB’s first release, a C++ rewrite of Cassandra

  6. 2020

    Amazon Keyspaces (for Apache Cassandra) becomes generally available

  7. 2024

    Cassandra 5.0 adds Storage Attached Indexes for non-key columns

Sources14

Improve this page on GitHub