YataiArchived

A Kubernetes operator and web console for deploying BentoML services, with custom resources for building Bento images and running them as autoscaled deployments. Archived in June 2026; it never supported BentoML 1.2’s service API.

What it is: Model serving and MLOps
Part of: Tool for BentoML
Type: Open source
Made by: BentoML (now part of Modular)
License: Elastic License 2.0 (source-available, not OSI open source)
Status: Archived since 11 June 2026
Facts checked: 23 October 2026

What it is

Yatai (屋台, a Japanese food cart) took Bentos, the packaged services that BentoML builds, and ran them on any Kubernetes cluster. It came in three parts. Yatai itself was a web UI and API with a central registry for Bentos and models, user accounts and API tokens; its quick install also set up PostgreSQL for metadata and MinIO for storage. yatai-image-builder added the Bento and BentoRequest custom resources and built a container image from a Bento inside the cluster. yatai-deployment added the BentoDeployment resource, which turned a Bento into Kubernetes deployments, services and ingress.

Developers pushed a Bento with bentoml push, and DevOps teams deployed it either from the UI or by applying a BentoDeployment manifest, which fit GitOps tools and CI pipelines. In BentoML 1.0 and 1.1 a service was split into an API server and Runners that did the model inference, and Yatai gave each Runner its own pods, resources and autoscaling range, so a GPU model could scale separately from CPU preprocessing. Version 1.0 also wired in Prometheus and Grafana for metrics, Loki for logs, and tracing through Zipkin, Jaeger or OpenTelemetry.

BentoML 1.2 (February 2024) introduced service classes in place of Runners, and Yatai never caught up: its README says “Yatai for BentoML 1.2 is currently under construction”. The repository was archived on 11 June 2026. The separate yatai-deployment repository still receives CI and Go version updates, but the project as a whole is no longer developed.

Why teams pick it

Kubernetes-native: a BentoDeployment is a custom resource you apply with kubectl, review in Git and roll out with existing CI/CD
Separate autoscaling and hardware for each Runner, so preprocessing stays on cheap CPU nodes while the model scales on GPU nodes
Builds images inside the cluster from a pushed Bento, so data scientists never write a Dockerfile
One place to see Bentos, models, deployments, metrics and logs, with a web UI for teams who don’t use kubectl

Watch out for

Archived since June 2026, with no release since October 2023 and no support for BentoML 1.2 and later
Heavy to install: its own users’ top complaint was setup, with stateful components such as a database and several namespaces, and a user system that didn’t fit Kubernetes RBAC
Elastic License 2.0, not an open-source license; the planned move to Apache 2.0 came with the 2.0 rewrite that never shipped
In-cluster image building was hard to optimize and debug, and teams asked to deploy their own prebuilt images instead

Alternatives, recommended first

What all model serving and mlops have in common →Compare side by side →

KubernetesOpen sourceThe standard open-source orchestrator: declare the containers you want, and controllers keep the cluster matching it.+14 Amazon EKS, Azure Kubernetes Service, Google Kubernetes Engine, K3s…

Ray ServeOpen sourceRay’s library for online inference: Python classes become deployments whose replicas run as Ray actors across a cluster, with request batching, model composition, autoscaling to zero and an OpenAI-compatible LLM server built on vLLM.

How it fits with other technologies

What Yatai uses

From the README: build an image for a Bento, then deploy it with autoscaling · yaml

apiVersion: resources.yatai.ai/v1alpha1
kind: BentoRequest
metadata:
    name: iris-classifier
    namespace: yatai
spec:
    bentoTag: iris_classifier:3oevmqfvnkvwvuqj  # check the tag by `bentoml list iris_classifier`
---
apiVersion: serving.yatai.ai/v2alpha1
kind: BentoDeployment
metadata:
    name: my-bento-deployment
    namespace: yatai
spec:
    bento: iris-classifier
    ingress:
        enabled: true
    autoscaling:
        maxReplicas: 10
        minReplicas: 2
    runners:
        - name: iris_clf
          autoscaling:
              maxReplicas: 4
              minReplicas: 1

See it in a real stack

No architecture uses Yatai yet. Drop it into the playground and wire it up yourself.Open playground →

Where it came from2021–2026

BentoML packaged a model and its code into a Bento, but teams then had to run it at scale themselves. They asked for multi-model deployments, scalable inference graphs and a central model store for collaboration, noting that storing models in S3 “doesn’t scale forever”. BentoML started Yatai in mid-2021 as an open-source system for running those deployments on Kubernetes, and released Yatai 1.0 in November 2022, with separate autoscaling groups per Runner, Prometheus and Grafana dashboards, log shipping with Loki and GitOps deployments through a custom resource. Mission Lane and CJ Express wrote about running it, and Kubeflow documented an integration.

In February 2024, the same month BentoML 1.2 changed its programming model, the team published a Yatai 2.0 proposal. It listed what users had learned: setup was too intrusive, the in-cluster image builder was hard to optimize, stateful components were either expensive or unreliable, and the Elastic License limited partners. The plan was a single BentoDeployment controller installed with Helm, no database, support for custom registries and an S3-based model store, a move to Apache 2.0, and a beta in September or October 2024.

The beta never appeared. The proposal issue stayed open, and a user asked in July 2025 whether it was still in development. After Modular bought BentoML in February 2026, the Yatai repository was archived on 11 June 2026.

Why it’s built this way

Custom resources instead of a deployment script: DevOps teams already managed everything through Kubernetes manifests and GitOps. A BentoDeployment resource let a model service go through the same review, CI and rollback process as any other workload.
One autoscaling group per Runner: In a typical service, preprocessing and each model have different bottlenecks and hardware needs. Splitting a Bento into separately scaled pods let GPU nodes run only the models and cheaper CPU nodes run the rest.
Build images in the cluster: Data scientists push a Bento, not a container, so the platform built the image for them. In practice this became the part users found hardest to debug, and the 2.0 plan was to accept prebuilt images instead.

Timeline

2021
Yatai repository created (July)
2022
Yatai 1.0 released (November)
2023
Last release, 1.1.13 (October)
2024
Yatai 2.0 proposal (February); last commit to the main repository (May)
2026
Repository archived (June 11)

Sources7 · facts checked 23 October 2026

Improve this page on GitHub