Cloud Parts

Apache Calcite

A Java framework with everything a database needs to understand a query (SQL parser, validator, relational algebra and a rule-based, cost-based optimizer) but no storage of its own, which Hive, Flink, Druid, Dremio, Beam and many other systems embed as their SQL front end and planner.

What it is
Query engines
Type
Open source
License
Apache 2.0
Facts checked
11 October 2026

What it is

Calcite takes SQL text, parses it, checks it against a schema, and turns it into a tree of relational operators: scan, filter, project, join, aggregate and so on. Its optimizer then rewrites that tree with rules. Calcite ships hundreds of them, plus two planner engines to fire them: a cost-based one modeled on Volcano, which keeps sets of equivalent plans and picks the cheapest, and a cheaper one that fires rules, without costing them, until the plan stops changing. Physical properties such as sort order and partitioning are “traits” of an operator, and so is the calling convention, the system that will run it. That lets one plan span engines: a filter can be pushed into Splunk while the join with a MySQL table runs elsewhere.

On purpose, Calcite leaves out data storage, the algorithms that process data and a metadata repository. You bring those through adapters. An adapter tells Calcite which collections count as tables and can add rules that push work into its backend; the project includes adapters for JDBC databases, Cassandra, MongoDB, Elasticsearch, Druid, Kafka, Redis, Arrow, Spark, CSV and plain Java objects. What can’t be pushed down runs in Calcite’s own Enumerable operators, which iterate over rows in generated Java code. Most users never see Calcite directly: Hive uses it for cost-based optimization, Flink to parse and optimize batch and streaming SQL, Druid to parse and plan Druid SQL, and Dremio, Beam, Drill, Kylin, Feldera and Alibaba’s MaxCompute for parsing or planning. It also has a JDBC driver, built on its Avatica subproject, so an application can query any mix of adapters as one database. The latest release is 1.42.0 (31 May 2026).

Why teams pick it

  • A complete, mature SQL front end you don’t have to write: parser, validator with dialect and conformance settings (MySQL, Oracle, BigQuery, SQL Server lexical rules), and conversion to relational algebra
  • An optimizer built for extension: pluggable rules, cost models, statistics and planner engines, with hundreds of rules included
  • Federation through adapters: one query can join tables from different systems, with rules pushing filters, joins and aggregations down to each source
  • Proven by adoption: Hive, Flink, Druid, Dremio, Beam, Drill and many more rely on it, so its parser and planner are exercised by a lot of production workloads

Watch out for

  • Java only: an engine written in C++ or Rust has to run a JVM beside it, or pass plans across a language boundary
  • Not a database: no storage or catalog, and its built-in Enumerable execution runs row by row over iterators, so heavy work has to be pushed into an engine
  • The library API moves: release notes open with a “Breaking Changes” list (1.42.0 renamed methods and changed how DDL nodes report their operands), so embedders must budget for upgrades
  • Planning costs time: the cost-based planner can search exhaustively, so systems stop early when the cost stops improving or split rules into phases and run some with the cheaper planner that doesn’t cost plans

Alternatives

What all query engines have in common →
Amazon AthenaAWSAWS’s serverless SQL engine for data in S3: no cluster to run, and on-demand queries cost $5 per TB scanned.Apache Arrow AceroExperimentalOpen sourceThe streaming execution engine inside Arrow’s C++ library: you hand it an exact plan of nodes (scan, filter, project, aggregate, join, sort) and it streams Arrow record batches through them, with no SQL parser or optimizer of its own.Apache DataFusionOpen sourceA query engine library in Rust, built on Arrow, that other projects embed and extend to build their own databases and data tools.+1 Apache DataFusion CometApache HadoopOpen sourceThe project that made big-data clusters normal: a distributed file system (HDFS), a cluster scheduler (YARN) and the MapReduce engine.Apache HiveOpen sourceThe first SQL engine for Hadoop, and the source of the Hive Metastore, the table catalog that Trino, Spark, Impala and Flink still read.Apache ImpalaOpen sourceAn MPP SQL engine in C++ for interactive queries over Hive tables on HDFS, Kudu, Iceberg and object storage; built and mainly sold by Cloudera.Apache PigDormantOpen sourceA step-by-step scripting language, Pig Latin, that compiles to MapReduce, Tez or Spark jobs on Hadoop. Still released, but the project calls itself dormant.Apache SparkOpen sourceThe general-purpose distributed engine for large batch jobs, streaming and machine learning, in Python, SQL, Scala, Java or R.Apache ZeppelinOpen sourceA self-hosted web notebook where each paragraph runs through an interpreter: Spark, Flink, JDBC, Python and more. Maintained, but behind Jupyter and Databricks in reach.chDBOpen sourceThe ClickHouse engine packaged as a library: run ClickHouse SQL inside a Python, Go, Rust or Node.js program, with no server.Cloudflare Basin SQLRenamedCommercialCloudflare’s serverless, distributed SQL engine for Apache Iceberg tables in Basin Catalog on R2, called R2 SQL until it became generally available on 1 October 2026, billed by the data it scans.DuckDBOpen sourceAn in-process analytical SQL database: a library with no server that queries Parquet, CSV, Iceberg and S3 directly.GizmoSQLOpen sourceA database server that wraps DuckDB (or SQLite) and serves it to many clients over Arrow Flight SQL, with TLS and authentication.GlareDBDormantOpen sourceA small, embeddable SQL engine in Rust for querying Parquet and CSV files locally, over HTTP or in S3 and GCS, from a CLI, Python or the browser. It began as a DataFusion-based federated query service and was rewritten in 2025 on its own engine, and it has had no release since June 2025.HotdataCommercialA managed SQL engine for AI agents, built in Rust on Apache DataFusion and Arrow: each agent gets short-lived “instant databases” to load data into, then runs read-only SQL, vector, full-text and geospatial queries over an HTTP API or CLI.IbisOpen sourceA Python dataframe API that compiles to SQL, so the same code runs on DuckDB on a laptop and on BigQuery, Snowflake, PostgreSQL or more than 20 other engines.PolarsOpen sourceA DataFrame library with a Rust query engine underneath: you write Python (or Rust, SQL, Node.js, R) and it plans, optimises and runs the query on every core, streaming data bigger than memory.Presto (PrestoDB)Open sourceFacebook’s distributed SQL engine, now governed by the Linux Foundation, moving its workers from Java to C++ on Velox. Trino is its 2019 fork.RayforceOpen sourceA small embeddable engine in pure C that runs columnar analytics, graph traversals and recursive Datalog queries in one optimised plan.Spice.ai OSSOpen sourceA single-binary SQL engine in Rust, built on DataFusion and Arrow, that federates queries across databases, warehouses and lakes, keeps hot datasets in a local accelerated copy, and adds search and LLM APIs on top.TrinoOpen sourceA distributed SQL engine that queries data where it lives, from Iceberg tables on S3 to a live PostgreSQL database, in one query.VeloxOpen sourceMeta’s open-source C++ execution engine library: it takes an already-optimized query plan and runs it with vectorized operators, and it is the engine inside Presto’s C++ workers (Prestissimo) and Apache Gluten’s native Spark.

How it fits with other technologies

Parse SQL over plain Java objects, push a filter below the join, and print the plan as PostgreSQL · java

// calcite-core 1.42.0 on JDK 21
import org.apache.calcite.adapter.java.ReflectiveSchema;
import org.apache.calcite.config.Lex;
import org.apache.calcite.plan.RelOptUtil;
import org.apache.calcite.plan.hep.*;
import org.apache.calcite.rel.RelNode;
import org.apache.calcite.rel.rel2sql.RelToSqlConverter;
import org.apache.calcite.rel.rules.CoreRules;
import org.apache.calcite.schema.SchemaPlus;
import org.apache.calcite.sql.dialect.PostgresqlSqlDialect;
import org.apache.calcite.sql.parser.SqlParser;
import org.apache.calcite.tools.*;

public class Demo {
  public static class Emp {  // public fields become columns
    public final int deptno; public final String name; public final int salary;
    Emp(int deptno, String name, int salary) { this.deptno = deptno; this.name = name; this.salary = salary; }
  }
  public static class Dept {
    public final int deptno; public final String name;
    Dept(int deptno, String name) { this.deptno = deptno; this.name = name; }
  }
  public static class Hr {
    public final Emp[] emps = { new Emp(10, "Bill", 10000), new Emp(10, "Theodore", 11500) };
    public final Dept[] depts = { new Dept(10, "Sales") };
  }

  public static void main(String[] args) throws Exception {
    SchemaPlus root = Frameworks.createRootSchema(true);
    root.add("hr", new ReflectiveSchema(new Hr()));  // any Java object can be a schema
    Planner planner = Frameworks.getPlanner(Frameworks.newConfigBuilder()
        .defaultSchema(root).parserConfig(SqlParser.config().withLex(Lex.JAVA)).build());

    // 1. Parse, validate against the schema, convert to relational algebra
    RelNode plan = planner.rel(planner.validate(planner.parse(
        "SELECT d.name, COUNT(*) AS n FROM hr.emps AS e JOIN hr.depts AS d "
      + "ON e.deptno = d.deptno WHERE e.salary > 5000 GROUP BY d.name"))).project();

    // 2. Rewrite it with a rule: push the filter below the join
    HepPlanner hep = new HepPlanner(HepProgram.builder()
        .addRuleInstance(CoreRules.FILTER_INTO_JOIN).build());
    hep.setRoot(plan);
    RelNode better = hep.findBestExp();
    System.out.println(RelOptUtil.toString(better));
    // LogicalAggregate(group=[{0}], n=[COUNT()])
    //   LogicalProject(name=[$4])
    //     LogicalJoin(condition=[=($0, $3)], joinType=[inner])
    //       LogicalFilter(condition=[>(CAST($2):INTEGER NOT NULL, 5000)])
    //         LogicalTableScan(table=[[hr, emps]])
    //       LogicalTableScan(table=[[hr, depts]])

    // 3. Turn the plan back into SQL for another engine
    var pg = PostgresqlSqlDialect.DEFAULT;
    System.out.println(new RelToSqlConverter(pg).visitRoot(better).asStatement().toSqlString(pg));
    // SELECT "depts"."name", COUNT(*) AS "n"
    // FROM (SELECT *
    // FROM "hr"."emps"
    // WHERE CAST("salary" AS INTEGER) > 5000) AS "t"
    // INNER JOIN "hr"."depts" ON "t"."deptno" = "depts"."deptno"
    // GROUP BY "depts"."name"
  }
}

What Calcite gives you, and what it leaves out#

SQL parser and validator

included

Standard SQL with sub-queries, window aggregates, GROUPING SETS and streaming extensions, checked against your schema. Lexical rules for Oracle (the default), MySQL, BigQuery, SQL Server and Java.

Relational algebra

included

The plan as a tree of operators, which you can also build directly in code. The 2014 proposal noted that Hive did just that, skipping Calcite’s SQL parser.

Optimizer

included

Rules, metadata providers (row counts, selectivity, cost) and two planner engines: cost-based (Volcano-style) and a rule-firing one that skips costing. Materialized-view rewriting and lattices on top.

Adapters

included

Make a source look like tables and push work into it: JDBC, Cassandra, MongoDB, Elasticsearch, Druid, Kafka, Redis, Arrow, files and Java objects.

JDBC driver

included

Built on Avatica, local or remote over JSON or Protobuf on HTTP, so tools can connect to a Calcite schema like any database.

Storage, execution at scale, catalog

left out

You bring them: an engine such as Hive, Flink or Druid runs the optimized plan, and its own metastore supplies the tables.

See it in a real stack

No architecture uses Apache Calcite yet. Drop it into the playground and wire it up yourself.Open playground →
Where it came from2012–2026

Traditional databases bundled storage, execution, a parser, a planner and a catalog into one product. The Hadoop and NoSQL systems of the early 2010s split that stack apart, and most of them had fast storage and runtimes but no SQL and no real optimizer. As the 2014 incubator proposal put it, building a query planner is hard, so “many systems muddle along without a planner” until customers demand one. Julian Hyde, lead developer of the Mondrian OLAP engine, started Optiq as a reusable one. The first commits, in May 2012, imported source files from the Eigenbase project and added Optiq code that could parse and validate SQL but not yet plan.

Before it reached Apache, Optiq was already inside Apache Drill, Apache Hive and Cascading Lingual. It entered the Apache Incubator on 19 May 2014 with committers from Concurrent, Hortonworks, MapR and Salesforce, was renamed Calcite that September, and became a top-level Apache project on 22 October 2015. Hive was a key early user: as it moved from batch jobs toward interactive SQL, it needed a strong optimizer at its core and adopted Calcite for that. Flink, Storm, Druid and MapD were using it by 2018, and the project’s powered-by list now also names Dremio, Beam, Drill, Kylin and Feldera. A 2018 SIGMOD paper by Edmon Begoli, Jesús Camacho-Rodríguez, Julian Hyde, Michael Mior and Daniel Lemire described the architecture and why it spread.

Why it’s built this way

Leave out storage, execution algorithms and the metadata store
Those are what specialized systems do best and want to keep. Omitting them made Calcite a fit for mediating between applications and many data stores, and a base on which to build a new database: “just add data”.
The engine that runs an operator is just another trait
Treating the calling convention like sort order or partitioning let one cost-based search decide where each part of a query runs, so a single plan can span MySQL, Splunk and Spark.
Two planner engines instead of one
A Volcano-style cost-based planner finds good plans but can be slow; a second engine fires rules quickly without costing them. Offering both lets systems split optimization into phases and control planning time.
Java
Most of the data systems it aimed at, especially in the Hadoop ecosystem, were written in Java or Scala, so a Java library could be embedded in them directly.

Timeline

  1. 2012

    Julian Hyde commits the first Optiq code, which can parse and validate SQL

  2. 2014

    Optiq enters the Apache Incubator in May and is renamed Calcite in September

  3. 2015

    1.0.0 in January; graduates to a top-level Apache project on 22 October

  4. 2018

    SIGMOD paper describes the framework, by then used by Hive, Flink, Storm, Druid and MapD

  5. 2026

    1.42.0 on 31 May, from 39 contributors, resolving 249 issues

Sources8 · facts checked 11 October 2026

Improve this page on GitHub