Cloud Parts

Elasticsearch

The distributed search and analytics engine built on Apache Lucene: JSON documents in, full-text, vector and log queries out, over a REST API.

What it is
Search indexes
Used for
Logs,Observability
Type
Commercial product
Made by
Elastic N.V.
License
Source code: AGPLv3, SSPL 1.0 or Elastic License 2.0, your choice, except paid features under x-pack, which are Elastic License 2.0 only. Default downloads: Elastic License 2.0. Apache 2.0 up to 7.10.2
Facts checked
6 October 2026
Website Source code

What it is

Elasticsearch stores JSON documents in indices and splits each index into shards. Every shard is a complete Lucene index holding a subset of the documents, and each document lands in one primary shard, chosen by hashing its routing value (its _id by default). Each primary has replica shards on other nodes (one by default), which take over when a node dies and also serve searches. The number of primary shards is fixed when the index is created; the number of replicas can change at any time. Every node knows the others and can forward a request to the nodes that hold the shards it needs.

Inside a shard, Lucene keeps an inverted index in immutable segments. New documents go into an in-memory buffer, and a refresh writes them into a new segment that searches can open without a full commit to disk. That happens every second by default (only on indices searched in the last 30 seconds), which is why Elasticsearch calls itself near real-time: a document you just indexed is searchable about a second later, not instantly. On top of text search it does aggregations, a piped query language (ES|QL), and vector search: dense_vector fields indexed as HNSW graphs, quantized automatically to save memory. Together with Logstash and Kibana it formed the ELK stack, still one of the most common ways to search logs.

The licence has changed twice. Elasticsearch was Apache 2.0 until 7.10.2. From 7.11 (February 2021) the source was under SSPL or the Elastic License, which forbids offering it as a managed service; AWS answered by forking 7.10.2 as OpenSearch. In August 2024 Elastic added AGPLv3, an OSI-approved open-source licence, as a third option for the free parts of the source. The downloads you usually run are still under the Elastic License 2.0, and some features, such as calling machine learning models for inference, need a paid subscription.

Why teams pick it

  • Full-text search, aggregations and vector search in one engine, with BM25 ranking and configurable text analyzers
  • Scales out by adding nodes: shards spread and rebalance automatically, and replicas add both failover and read capacity
  • The ELK ecosystem: Kibana for exploring and dashboards, Logstash and Elastic Agent for ingest, and preinstalled support in Grafana
  • Run it yourself, on Kubernetes with the ECK operator, or as Elastic Cloud on AWS, Google Cloud or Azure

Watch out for

  • The primary shard count is set at index creation; choosing badly for a growing index means reindexing it
  • Near real-time: a write becomes searchable only after the next refresh (every second by default, and not until searched on an idle index)
  • Deep paging is capped: from + size may not exceed 10,000 hits by default, so you page with search_after instead
  • Licence and feature split: the default distribution isn’t open source, AGPL covers only the free code, and clouds may not sell it as a service without complying with one of the three licences

Alternatives

What all search indexes have in common →
OpenSearchOpen sourceThe Apache 2.0 fork of Elasticsearch and Kibana 7.10.2 that AWS started in 2021, now run by a Linux Foundation project.pg_trgmOpen sourceA PostgreSQL extension that splits text into three-letter pieces, so `LIKE '%foo%'`, regex and typo-tolerant searches can use an index.UrsaDBOpen sourceCERT Polska’s byte-trigram index for malware collections: it narrows a YARA hunt over terabytes of files down to a few candidates.

How it fits with other technologies

Create an index with three shards, add a document and search it · bash

ES=https://localhost:9200; AUTH="elastic:$ELASTIC_PASSWORD"   # security is on by default since 8.0

curl -u $AUTH --cacert http_ca.crt -X PUT $ES/products -H 'Content-Type: application/json' -d '{
  "settings": { "number_of_shards": 3, "number_of_replicas": 1 },
  "mappings": { "properties": { "name": { "type": "text", "analyzer": "english" } } }
}'

# refresh=wait_for: return only once the document is searchable
curl -u $AUTH --cacert http_ca.crt -X POST "$ES/products/_doc/1?refresh=wait_for" \
  -H 'Content-Type: application/json' -d '{"name": "Noise-cancelling headphones", "price": 199}'

# the english analyzer stems both sides, so “headphone” matches “headphones”
curl -u $AUTH --cacert http_ca.crt "$ES/products/_search" -H 'Content-Type: application/json' \
  -d '{"query": {"match": {"name": "headphone"}}}'

See it in a real stack

No architecture uses Elasticsearch yet. Drop it into the playground and wire it up yourself.Open playground →
Where it came from2005–2024

In 2005 Shay Banon open-sourced Compass, a Java framework on top of Apache Lucene, while building a recipe app for his wife. Lucene was a library: it could index and search on one machine, but spreading an index over many machines, replicating it and talking to it from other languages was left to you. Banon wrote that Elasticsearch “was born out of my frustration with the fact that there isn’t really a good, open source, solution for distributed search engine out there”.

He started it in 2009 and announced it on February 8, 2010, under the line “You Know, for Search”, as “an open source, distributed, RESTful, search engine” built on Lucene; the first release was 0.4.0. Banon and others founded a company around it in 2012. In 2013 Elasticsearch joined forces with Logstash and Kibana as the ELK stack, which made it the default place to send and search logs, and version 1.0 followed in 2014.

Cloud providers began selling Elasticsearch as a service, and in January 2021 Elastic moved new versions from Apache 2.0 to SSPL and its own licence to stop that without “contributing back”. AWS forked the last Apache version as OpenSearch. Version 8.0 (2022) brought approximate vector search with HNSW and turned security on by default, and in August 2024 Elastic added AGPLv3 so it could call Elasticsearch open source again.

Why it’s built this way

Build on Lucene instead of writing a new index
Banon had spent five years building Compass on Lucene. Reusing it gave Elasticsearch a mature inverted index and scoring from day one, so the new work could go into distribution, replication and the API.
Distributed from the start: shards that are whole Lucene indices
Splitting an index into self-contained shards, each with replicas on other nodes, lets a cluster grow by adding machines and survive losing one, which a single Lucene index on one server couldn’t.
JSON over HTTP
Lucene was a Java library. A RESTful API with JSON documents meant any language with an HTTP client could index and search.
Near real-time refresh instead of commit-per-write
Opening a new Lucene segment is much cheaper than committing to disk. Doing it every second makes new documents searchable quickly while keeping indexing throughput high.

Timeline

  1. 2005

    Shay Banon open-sources Compass, a Java framework on top of Lucene

  2. 2010

    Elasticsearch announced and released as 0.4.0 in February

  3. 2012

    Elasticsearch, the company, is founded

  4. 2013

    Joins Logstash and Kibana as the ELK stack

  5. 2014

    Version 1.0: snapshot and restore, aggregations, circuit breakers

  6. 2021

    7.11 moves from Apache 2.0 to SSPL or the Elastic License; AWS forks 7.10.2 as OpenSearch

  7. 2022

    Version 8.0: HNSW vector search and security on by default

  8. 2024

    AGPLv3 added as a third licence option in August

Sources17 · facts checked 6 October 2026

Improve this page on GitHub