Cloud Parts

Analytics

Turning piles of files into answers.

11 concepts · 82 technologies · 2 comparisons · 2 real stacks

Concepts

Start with File formats
  1. File formats17 technologiesHow rows and columns are laid out inside a data file, which decides how much a query has to read.
  2. Table formats12 technologiesA metadata layer that turns a folder of data files into a real table, with transactions, schema changes and time travel.
  3. Lakehouse catalogs7 technologiesThe service every engine asks “where is this table right now?”: it maps table names to their current metadata, makes commits atomic, and decides who may read which files.
  4. Query engines32 technologiesCompute that runs SQL over data it doesn’t own, and scales separately from the storage.
  5. Data warehouses9 technologiesA columnar database for big analytical questions over all your data.
  6. Real-time analytics databases11 technologiesColumnar databases that make streaming events queryable within seconds and answer many small analytical queries at once, fast enough to sit behind an app.
  7. Data lakes3 technologiesRaw files of any kind in cheap storage, given a structure only when someone reads them.
  8. Lakehouses21 technologiesWarehouse-style tables on open files in object storage, shared by every engine.
  9. ETL & ELT pipelines13 technologiesCopy data from app databases, SaaS APIs, files and streams into a warehouse or lake, then reshape it into tables people can query.
  10. Semantic layers2 technologiesOne shared, code-defined model of metrics, dimensions and joins that BI tools, apps and AI agents query by name, which the layer compiles to SQL against the warehouse and often caches.
  11. Data notebooks2 technologiesA document of code cells, their outputs and notes, backed by a live process that keeps variables in memory, so you can explore data a step at a time and keep the results next to the code that made them.

Technologies

Grouped by the concept each one mainly implements

Commonly confused

Data lake, lakehouse or warehouse?Data lakes · Lakehouses · Data warehousesFile format or table format?File formats · Table formats

See it in a real stack

Architectures that use analytics concepts
Small lakehouse on DuckLakeDuckDB writes and queries tables whose catalog lives in PostgreSQL and whose data sits as Parquet on S3.DuckDBDuckLakePostgreSQLAmazon S3Open lakehouse on IcebergKafka streams events into Spark, which writes Iceberg tables on S3 that Trino queries for dashboards.Apache KafkaApache SparkApache IcebergAmazon S3Trino

Worth exploring

All resources →
Article· September 2025Understanding Apache FlussJack VanlightlyA long, detailed walk through Fluss internals, reverse-engineered from the code: log tables, primary-key tables that emit changelogs, and tiering to lakehouse tables. The author works at Confluent, which runs Flink but not Fluss, and says so up front.
Article· August 2021What Is a Data Lakehouse and Answers to Other Frequently Asked QuestionsMichael Armbrust, Bharath Gowda, Reynold Xin, Matei Zaharia and Ali Ghodsi, DatabricksDatabricks answers ten questions: how a lakehouse differs from a data lake and a warehouse, how it compares on cost and speed, what governance it supports, and how it relates to a data mesh. Databricks sells a lakehouse, so read it as the case for one.
WebsiteAwesome DuckDBDavid Gasquez and contributors, on GitHubA community-curated list of more than 400 DuckDB links: client APIs, core and community extensions, BI and ETL tools built on DuckDB (grouped by commercial, open core and open source), WebAssembly and SQL clients, plus talks, podcasts, blog posts and books. Use it to find what already exists around DuckDB; entries arrive by pull request with one-line descriptions, so judge each project yourself.
WebsiteThe Umbra Database SystemThomas Neumann’s group, Technical University of MunichThe research site for Umbra, the successor to HyPer and the basis of CedarDB: a feature overview and the full publication list with PDFs, from the CIDR 2020 paper “Umbra: A Disk-Based System with In-Memory Performance” onwards. You can try queries in the browser or pull a Docker image, but its terms allow only evaluation, not production use.

Benchmarks

How to read a benchmark →
Benchmark · ClickHouse Inc.ClickBenchHow fast analytical databases run 43 SQL queries over one 100-million-row table of web analytics events, on the same cloud machines.32 systems in this catalog · 7 caveats to read firstBenchmark · Tiger Data (formerly Timescale)Time Series Benchmark Suite (TSBS)A Go toolkit that generates synthetic server-monitoring or IoT time series, bulk-loads it into a database and times a fixed set of time-series queries.7 systems in this catalog · 6 caveats to read first