Cloud Parts

Apache Impala

An MPP SQL engine in C++ for interactive queries over Hive tables on HDFS, Kudu, Iceberg and object storage; built and mainly sold by Cloudera.

What it is
Query engines
Type
Open source
Made by
Apache Software Foundation
License
Apache 2.0
Facts checked
6 October 2026
Website Source code

What it is

Impala runs long-lived daemons instead of launching jobs. An impalad process runs on every worker host, often next to an HDFS DataNode so it can read blocks locally; any of them can accept a query from impala-shell, JDBC or ODBC, plan it, and act as coordinator while the others execute fragments and stream results back. Since Impala 2.9 you can dedicate some hosts to coordinating and others to executing. Table definitions come from the Hive Metastore, which Impala shares with Hive. A catalogd daemon caches that metadata and file locations, and a single statestored daemon tracks which impalads are alive and broadcasts catalog changes. The planner is Java; the executor is C++ and uses LLVM to generate machine code for each query’s inner loops, and operators spill to disk under memory pressure.

Impala reads Parquet, ORC, Avro and text files on HDFS, S3, ADLS and Ozone, queries Apache Kudu tables (which allow in-place UPDATE, DELETE and UPSERT) and HBase tables, and since 4.0 (2021) reads and writes Iceberg tables, including DELETE, UPDATE and MERGE on Iceberg v2. The Apache project ships source releases under Apache 2.0; the current line is 4.5. Cloudera created Impala and sells it as part of its platform: in Cloudera Data Warehouse, an Impala Virtual Warehouse is one of two engine types, next to Hive, that query the same Database Catalog.

Why teams pick it

  • Low-latency SQL for BI tools on data already in a Hadoop or Cloudera lake, with no per-query job startup
  • Shares tables and the Hive Metastore with Hive and Spark, so one copy of the data serves batch and interactive SQL
  • C++ executor with LLVM code generation; Cloudera’s CIDR paper measured up to 5.7x speedups from codegen alone
  • Writes Iceberg v2 tables, and Kudu tables give it fast in-place UPDATE, DELETE and UPSERT next to full scans
  • Active community: the project reports high activity in 2026, with work on Iceberg v3, a Calcite planner and a Kubernetes operator

Watch out for

  • Metadata is cached: after Hive, Spark or any other tool adds files or partitions, you must run REFRESH or INVALIDATE METADATA before Impala sees them
  • No mid-query recovery: if a node crashes, the query fails; transparent retries rerun the whole query only when cluster membership changed
  • Partial format support: ORC is read-only, and Iceberg writes produce only Parquet data files and position deletes (no equality deletes)
  • In practice a Cloudera product: the Apache project publishes source tarballs, and the managed offering is Cloudera Data Warehouse
  • A single statestored and catalogd: if you run DDL while the statestore is down, queries that touch the new objects fail

Alternatives

What all query engines have in common →
Amazon AthenaAWSAWS’s serverless SQL engine for data in S3: no cluster to run, and on-demand queries cost $5 per TB scanned.Apache Arrow AceroExperimentalOpen sourceThe streaming execution engine inside Arrow’s C++ library: you hand it an exact plan of nodes (scan, filter, project, aggregate, join, sort) and it streams Arrow record batches through them, with no SQL parser or optimizer of its own.Apache CalciteOpen sourceA Java framework with everything a database needs to understand a query (SQL parser, validator, relational algebra and a rule-based, cost-based optimizer) but no storage of its own, which Hive, Flink, Druid, Dremio, Beam and many other systems embed as their SQL front end and planner.Apache DataFusionOpen sourceA query engine library in Rust, built on Arrow, that other projects embed and extend to build their own databases and data tools.+1 Apache DataFusion CometApache HadoopOpen sourceThe project that made big-data clusters normal: a distributed file system (HDFS), a cluster scheduler (YARN) and the MapReduce engine.Apache HiveOpen sourceThe first SQL engine for Hadoop, and the source of the Hive Metastore, the table catalog that Trino, Spark, Impala and Flink still read.Apache PigDormantOpen sourceA step-by-step scripting language, Pig Latin, that compiles to MapReduce, Tez or Spark jobs on Hadoop. Still released, but the project calls itself dormant.Apache SparkOpen sourceThe general-purpose distributed engine for large batch jobs, streaming and machine learning, in Python, SQL, Scala, Java or R.Apache ZeppelinOpen sourceA self-hosted web notebook where each paragraph runs through an interpreter: Spark, Flink, JDBC, Python and more. Maintained, but behind Jupyter and Databricks in reach.chDBOpen sourceThe ClickHouse engine packaged as a library: run ClickHouse SQL inside a Python, Go, Rust or Node.js program, with no server.Cloudflare Basin SQLRenamedCommercialCloudflare’s serverless, distributed SQL engine for Apache Iceberg tables in Basin Catalog on R2, called R2 SQL until it became generally available on 1 October 2026, billed by the data it scans.DuckDBOpen sourceAn in-process analytical SQL database: a library with no server that queries Parquet, CSV, Iceberg and S3 directly.GizmoSQLOpen sourceA database server that wraps DuckDB (or SQLite) and serves it to many clients over Arrow Flight SQL, with TLS and authentication.GlareDBDormantOpen sourceA small, embeddable SQL engine in Rust for querying Parquet and CSV files locally, over HTTP or in S3 and GCS, from a CLI, Python or the browser. It began as a DataFusion-based federated query service and was rewritten in 2025 on its own engine, and it has had no release since June 2025.HotdataCommercialA managed SQL engine for AI agents, built in Rust on Apache DataFusion and Arrow: each agent gets short-lived “instant databases” to load data into, then runs read-only SQL, vector, full-text and geospatial queries over an HTTP API or CLI.IbisOpen sourceA Python dataframe API that compiles to SQL, so the same code runs on DuckDB on a laptop and on BigQuery, Snowflake, PostgreSQL or more than 20 other engines.PolarsOpen sourceA DataFrame library with a Rust query engine underneath: you write Python (or Rust, SQL, Node.js, R) and it plans, optimises and runs the query on every core, streaming data bigger than memory.Presto (PrestoDB)Open sourceFacebook’s distributed SQL engine, now governed by the Linux Foundation, moving its workers from Java to C++ on Velox. Trino is its 2019 fork.RayforceOpen sourceA small embeddable engine in pure C that runs columnar analytics, graph traversals and recursive Datalog queries in one optimised plan.Spice.ai OSSOpen sourceA single-binary SQL engine in Rust, built on DataFusion and Arrow, that federates queries across databases, warehouses and lakes, keeps hot datasets in a local accelerated copy, and adds search and LLM APIs on top.TrinoOpen sourceA distributed SQL engine that queries data where it lives, from Iceberg tables on S3 to a live PostgreSQL database, in one query.VeloxOpen sourceMeta’s open-source C++ execution engine library: it takes an already-optimized query plan and runs it with vectorized operators, and it is the engine inside Presto’s C++ workers (Prestissimo) and Apache Gluten’s native Spark.

How it fits with other technologies

An Iceberg table in impala-shell, plus the REFRESH that outside writes need · sql

-- impala-shell -i impala-coordinator:21050
CREATE TABLE sales.orders (
  order_id BIGINT, customer_id BIGINT, amount DECIMAL(10,2), ts TIMESTAMP)
PARTITIONED BY SPEC (day(ts))
STORED AS ICEBERG
TBLPROPERTIES ('format-version' = '2');

INSERT INTO sales.orders VALUES (1, 4711, 19.90, now());
DELETE FROM sales.orders WHERE customer_id = 4711;

-- a Hive table that a Spark job just added files to
REFRESH web.page_views;
COMPUTE STATS web.page_views;

SELECT url, count(*) AS views, appx_median(ms) AS p50_ms
FROM web.page_views
WHERE ds = '2026-10-05'
GROUP BY url ORDER BY views DESC LIMIT 10;

Ways to run it#

Cloudera Data Warehouse

managed

Impala Virtual Warehouses are compute environments that scale independently and query a shared Database Catalog, next to Hive Virtual Warehouses on the same data. They expose HiveServer2-compatible endpoints for Beeline, impala-shell and Impyla.

Apache source releases

build it yourself

The Apache project publishes signed source tarballs (4.5.x, with older 4.x and 3.4 lines still listed). You build and deploy the impalad, catalogd and statestored daemons yourself, alongside a Hive Metastore.

Kubernetes

in progress

The August 2026 board report lists a Kubernetes operator and Helm chart support among current work, together with Iceberg v3 support and OpenTelemetry integration.

See it in a real stack

No architecture uses Apache Impala yet. Drop it into the playground and wire it up yourself.Open playground →

Benchmarks

Benchmark · ClickHouse Inc.ClickBenchHow fast analytical databases run 43 SQL queries over one 100-million-row table of web analytics events, on the same cloud machines.Includes Apache Impala · 7 caveats to read first
How to read a benchmark →
Where it came from2012–2025

By 2012 Hadoop could store and process petabytes cheaply, but SQL on it meant Hive, which compiled each query into MapReduce jobs. Batch frameworks like that could not give BI and analytic queries the low latency and high concurrency of a traditional analytic database.

A Cloudera team, with Marcel Kornacker as lead author of its 2015 paper, built Impala from scratch in C++ and Java rather than forking Postgres, as many SQL-on-Hadoop systems did. It reused Hadoop’s own pieces, HDFS, HBase, the Hive Metastore and YARN, and replaced MapReduce with daemons on every node that run queries directly. Impala’s beta came out in October 2012 and it reached general availability in May 2013, with nearly one million downloads by early 2015.

Impala showed that an MPP engine could run directly on the Hadoop stack and share its tables. Cloudera donated it to the Apache Incubator in December 2015, and it became a top-level project in November 2017.

Why it’s built this way

Always-running daemons instead of batch jobs
Starting MapReduce jobs added latency to every query. Daemons on every DataNode host could start work immediately and read blocks from local disks without crossing the network.
A new engine in C++, not a Postgres fork
Impala was architected from the ground up for Hadoop: its storage, file formats and distributed execution, rather than retrofitting a database built for a single machine.
Runtime code generation with LLVM
Generic interpreted operators pay for virtual function calls on every row. Compiling each query’s inner loops into machine code at runtime removed that overhead.
Reuse the Hive Metastore
Users already defined their tables in Hive. Sharing the Metastore meant Impala could query those tables without a migration, and both tools stayed in sync.

Timeline

  1. 2012

    Cloudera releases the Impala beta in October

  2. 2013

    General availability in May

  3. 2014

    Impala 2.0 released in October

  4. 2015

    Enters the Apache Incubator in December

  5. 2017

    Becomes a top-level Apache project in November

  6. 2021

    Impala 4.0 adds Iceberg table support

  7. 2025

    Impala 4.5.0 released in February

Sources14 · facts checked 6 October 2026

Improve this page on GitHub