Cloud Parts

Apache Ozone

A Hadoop-born object store that serves the same buckets over S3 and as a Hadoop file system.

What it is
Object storage,File storage
Type
Open source
Made by
Apache Software Foundation
License
Apache 2.0
Facts checked
5 October 2026
Website Source code

What it is

Apache Ozone is a distributed object store built by Hadoop engineers to sit beside or replace HDFS. It splits metadata in two. The Ozone Manager (OM) keeps the namespace of volumes, buckets and keys. The Storage Container Manager (SCM) tracks the Datanodes and storage containers: 5 GB units that each hold many blocks. Datanodes report and replicate whole containers rather than individual blocks, so billions of small files don’t swamp the metadata services. OM and SCM replicate their own state with Apache Ratis, a Raft implementation. Open containers are copied three ways through Raft; a bucket can use erasure coding instead, such as 6 data plus 3 parity pieces, which cuts the storage overhead from 200% to 50%.

Clients reach the data through several front ends. The S3 Gateway is a separate, stateless service, so you run several behind a load balancer; each S3 bucket is a bucket in the /s3v volume. Hadoop tools such as Spark, Hive and Trino use ofs://, a Hadoop-compatible file system that sees every volume and bucket. A bucket’s layout decides what works: file-system-optimized (FSO) buckets, the default for buckets made with Ozone’s own tools, store real directories, so renaming or deleting a whole directory is one metadata change. Object-store (OBS) buckets, the default for buckets made through S3, store flat keys and can’t be used through ofs. Security builds on Kerberos, with Ozone’s own ACLs or Apache Ranger for authorization.

Why teams pick it

  • One copy of the data over S3 and as a Hadoop file system, so Spark jobs and S3 applications can share buckets
  • Designed for billions of keys: separate namespace and block services, with containers keeping small files cheap to track
  • Per-bucket choice between three-way Raft replication and erasure coding for colder data
  • Governed by the Apache Software Foundation; Tencent, Cloudera, Shopee, DiDi and others list production use
  • Fits existing Hadoop security: Kerberos, Ranger policies and transparent data encryption

Watch out for

  • Many moving parts (OM, SCM, Datanodes, S3 Gateway, Recon), each with its own high-availability setup: far heavier to run than a single MinIO binary
  • Partial S3 coverage: no bucket versioning, object lock, S3 server-side encryption, bucket policies or S3 Select, and lifecycle rules, replication and event notifications are still on the roadmap
  • S3 access uses Ozone’s own permission model, and on a secured cluster users need Kerberos just to get S3 keys
  • Buckets created through S3 use the flat layout and can’t be read through ofs, so create FSO buckets up front when both sides need the data

Alternatives

What all object storage have in common →
Amazon S3AWSAWS’s object storage, and the de facto API most object stores copy.Apache OpenDALOpen sourceA Rust library, with bindings for Python, Java, Node.js, Go and more, that gives one API (read, write, stat, list, delete) over 50-plus storage backends: S3 and other object stores, local and HDFS file systems, Google Drive and Dropbox, and key-value stores.Azure Blob StorageAzureAzure’s object storage, with three blob types and an optional real directory tree for analytics.Backblaze B2 Cloud StorageCommercialLow-cost object storage on Backblaze’s own hardware, with its own API and an S3-compatible one.CephOpen sourceA distributed storage cluster that serves S3 buckets, block devices and a file system from the same pool of disks.Cloudflare R2CommercialS3-compatible object storage from Cloudflare that charges nothing for data going out.GarageOpen sourceA lightweight S3 object store that keeps copies of your data in several sites linked by ordinary Internet connections.Google Cloud StorageGoogle CloudGoogle Cloud’s object storage, where even the archive class answers in milliseconds.MinIOArchivedOpen sourceA self-hosted object store that speaks the S3 API. The open-source edition is archived and its images are gone from Docker Hub; MinIO now sells AIStor.PGSTY SiloOpen sourceA community-maintained fork of the open-source MinIO server, with builds, images, security fixes and the full web console.RustFSOpen sourceA young Rust reimplementation of MinIO’s design: S3-compatible, erasure-coded, Apache 2.0, and generally available since September 2026.SeaweedFSOpen sourceA distributed blob store with an S3 gateway and a file system layer, built to store and serve billions of small files.TigrisCommercialGlobally distributed S3-compatible object storage that moves data to where it’s read, with no egress fees.Versity S3 GatewayOpen sourceA stateless S3 server that exposes an existing file system, or another object store, through the S3 API.Wasabi Hot Cloud StorageCommercialS3-compatible object storage sold at one flat price per TB, with no egress or request fees.Zenko CloudServerOpen sourceScality’s open-source Node.js S3 server: a light S3 endpoint for development and tests, and a front end that can store objects in other clouds.

How it fits with other technologies

Try Ozone with Docker, then write over S3 and read through ofs · bash

# evaluation cluster: OM, SCM, 3 Datanodes, S3 Gateway, Recon (replication factor 1)
curl -O https://raw.githubusercontent.com/apache/ozone-docker/refs/heads/latest/docker-compose.yaml
docker compose up -d --scale datanode=3

# a file-system-optimized bucket in the S3 volume, usable from S3 and ofs
docker compose exec om ozone sh bucket create /s3v/lake --layout FILE_SYSTEM_OPTIMIZED

# without Kerberos the S3 Gateway accepts any key pair
export AWS_ACCESS_KEY_ID=testuser AWS_SECRET_ACCESS_KEY=anything AWS_DEFAULT_REGION=us-east-1
aws s3 cp ./events.json s3://lake/events/2026/events.json --endpoint-url http://localhost:9878

# the same object through the Hadoop-compatible file system
docker compose exec om ozone fs -ls -R ofs://om/s3v/lake/

Moving from MinIO#

Ozone fits best when your MinIO cluster mainly fed Spark, Hive or Trino: the same buckets can then be read over S3 and through ofs://. Check the gaps first. Ozone’s S3 Gateway doesn’t implement bucket versioning, object lock, S3 ACLs, bucket policies or S3 server-side encryption, and lifecycle rules, replication and bucket notifications are still on its roadmap. If you relied on MinIO’s object locking for backups or its bucket notifications, Ozone can’t replace them yet. Conditional writes (If-None-Match, If-Match), which tools use to update manifests and metadata safely, arrived in Ozone 2.2.

Create the target buckets before copying, and choose the layout. Buckets made through S3 get the flat object-store layout, so create file-system-optimized buckets with ozone sh bucket create /s3v/<name> --layout FILE_SYSTEM_OPTIMIZED when Hadoop jobs will use them too. On a Kerberos-secured cluster, each user gets S3 credentials with ozone s3 getsecret.

Then define the MinIO endpoint and the Ozone S3 Gateway as two S3 remotes in rclone, run rclone sync minio:bucket ozone:bucket --checksum, and compare both sides with rclone check. rclone copies objects only: users, policies and lifecycle rules have to be set up again, and on Ozone access is managed with Ozone ACLs or Apache Ranger.

See it in a real stack

No architecture uses Apache Ozone yet. Drop it into the playground and wire it up yourself.Open playground →
Where it came from2014–2026

HDFS, Hadoop’s file system, keeps every file and block in the memory of one NameNode. It scales to huge clusters of large files but not to huge numbers of small ones: a 2018 post introducing Ozone said HDFS “struggles with over 400 Million files”. In October 2014 Jitendra Nath Pandey opened HDFS-7240, proposing an object store built on HDFS’s Datanodes but independent of the NameNode’s namespace.

The work grew inside the Apache Hadoop project as Hadoop Ozone, on a new block layer, the Hadoop Distributed Data Store, that groups blocks into containers. The first alpha, 0.2.1, came out on 1 October 2018 with a Hadoop-compatible file system API; an S3-compatible REST API and high availability were next. Version 1.0.0 was tagged in September 2020, and in October 2020 the ASF board made Ozone a top-level project of its own.

Since then Ozone has added erasure coding, file-system-optimized buckets with atomic directory renames and steady S3 Gateway work. Ozone 2.2.0, in July 2026, added S3 conditional requests, a compare-and-set primitive for systems that coordinate through objects, such as table catalogs.

Why it’s built this way

Separate the namespace from block management
HDFS’s NameNode holds both and runs out of memory first. Splitting them into the Ozone Manager and the Storage Container Manager lets each scale on its own.
Group blocks into large containers
Datanodes report and replicate containers, not blocks, so billions of small files mean billions of local block entries but only a manageable number of containers for SCM to track.
Strong consistency through Raft
The designers wanted strict serializability because it makes applications simpler. Ratis replicates metadata and open containers so every replica agrees on each write.
A Hadoop file system API from the first release
Hive, Spark and MapReduce could use Ozone without changes, and Ozone could run beside HDFS on the same cluster, which made trying it cheap for Hadoop users.

Timeline

  1. 2014

    HDFS-7240 proposes an object store on HDFS Datanodes

  2. 2018

    First alpha, Hadoop Ozone 0.2.1, released in October

  3. 2020

    Version 1.0.0 in September; top-level Apache project in October

  4. 2026

    Ozone 2.2.0 adds S3 conditional requests (July)

Sources18 · facts checked 5 October 2026

Improve this page on GitHub