Storage
Where files and disks live.
4 concepts · 22 technologies · 1 comparison · 7 real stacks
Concepts
Start with Object storage- Object storage17 technologiesPractically unlimited storage for files of any size, addressed by key.
- Block storage2 technologiesA virtual hard drive you attach to one machine.
- File storage5 technologiesA shared network drive that many servers mount at once.
- Backup & restore3 technologiesCopies of your data kept apart from the original, so you can get it back after a deletion, corruption, ransomware or a lost region.
Technologies
Grouped by the concept each one mainly implements
Commonly confused
See it in a real stack
Architectures that use storage conceptsSmall lakehouse on DuckLakeDuckDB writes and queries tables whose catalog lives in PostgreSQL and whose data sits as Parquet on S3.DuckDBDuckLakePostgreSQLAmazon S3Open lakehouse on IcebergKafka streams events into Spark, which writes Iceberg tables on S3 that Trino queries for dashboards.Apache KafkaApache SparkApache IcebergAmazon S3TrinoServerless web app on AWSCloudFront serves the site from S3 and sends API calls through API Gateway to Lambda functions backed by DynamoDB.Amazon CloudFrontAmazon S3Amazon API GatewayAWS LambdaAmazon DynamoDBCQRS with a static read sideCommands update PostgreSQL; a Lambda projector rewrites the affected JSON views in S3, and CloudFront serves every read without touching a database.Amazon API GatewayAWS LambdaPostgreSQLAmazon SQSAmazon S3Amazon CloudFrontBackground jobs with a queueThe API drops slow work onto SQS and returns at once; a pool of Fargate workers drains the queue at a steady pace.AWS LambdaAmazon SQSAWS FargateAmazon S3Code sandboxes for AI agentsAn agent’s code runs in a fresh Firecracker microVM per task, isolated by its own kernel, with outputs saved to S3.FirecrackerAmazon S3IoT telemetry pipelineDevices stream readings into Kinesis; Lambda writes them to TimescaleDB for Grafana, and raw data is archived to S3.Amazon Kinesis Data StreamsAWS LambdaTimescaleDBAmazon S3Grafana
Worth exploring
All resources →Slides· September 2025How and Why Datadog moved from cloud-managed to self-managed PostgresNick Canzoneri, Datadog, at PGConf NYC 2025Slides from a talk on why a company running hundreds of Postgres clusters across several clouds left managed services for its own Postgres platform on Kubernetes, how it moved a whole region, and what got better and worse. The stack is Patroni (with ZooKeeper for leader election), WAL-G backups, a Helm chart rather than an operator, and Temporal workflows for automation. Useful before you weigh RDS-style convenience against running Postgres yourself.
Article· August 2023Using Kubernetes with AWS Lambda: Scaling Up Your Serverless ApplicationsRohit Ghumare, LumigoA vendor blog post whose concrete part is a walkthrough of creating a Lambda function from an EKS cluster with the AWS Controllers for Kubernetes (ACK) Lambda controller, closely following AWS’s own 2022 tutorial. The benefits section is vague and blurs what Kubernetes and Lambda each do, and the post ends with a pitch for Lumigo’s observability product.
Article· August 2022Deploying AWS Lambda functions using AWS Controllers for Kubernetes (ACK)Rajdeep Saha, AWS Compute BlogAWS’s step-by-step tutorial for managing Lambda functions as Kubernetes custom resources: install the ACK Lambda controller in EKS, give it IAM permissions through IRSA, then
kubectl apply a Function manifest that points at code in S3. It explains why you might (one GitOps and RBAC model for pods and functions) but, being AWS’s own post, not the drawbacks.Article· February 2021Alternative to AWS lambda serverless concept in Kubernetes?Stack Overflow question by ajsgA team running everything on Kubernetes asks how to get Lambda-style, per-request scaling for loading AI models from S3. The 2021 answers name Knative, KFServing (now KServe) and Kubeless (since archived), which makes it a quick snapshot of the options rather than an up-to-date guide.