LLMs or JEPA?
| Dimension | Large language models | JEPA (Joint-Embedding Predictive Architecture) |
|---|---|---|
| In one line | Predict the next token of text, one at a time, and keep going | Predict the representation of a hidden part of the input, never its pixels or tokens |
| Training signal | Self-supervised on raw text: the next token is the label, so trillions of tokens need no annotation | Self-supervised on images or video: mask regions and predict their embeddings, with no labels, text or negatives |
| What it outputs | Text (or code, or JSON), sampled token by token from a probability over the vocabulary | Embeddings. Text or actions come from a probe, a decoder or a planner built on top |
| Handling the unpredictable | Spreads probability over a finite vocabulary, which works for discrete text | Encoders drop details that can’t be predicted; a latent variable can stand for the remaining choices |
| Main data | Text and code from the web and books | Images and video: V-JEPA 2 used over 1 million hours of video and 1 million images |
| Typical size | Billions to hundreds of billions of weights: Qwen3-8B has 8.2 billion, DeepSeek-R1 671 billion | Around a billion: I-JEPA’s ViT-H is 632 million parameters, V-JEPA 2 is 1.2 billion |
| Planning | In text: the model writes out steps, or an agent loop calls tools and feeds results back | In the world model: V-JEPA 2-AC searches action sequences whose predicted state lands closest to a goal image |
| Maturity in October 2026 | Products: hosted APIs from many providers and open-weight models you can run locally | Research: papers, code and checkpoints from Meta FAIR; AMI Labs founded to build on it, with no product planned for years |
| Licences of the best-known models | Provider terms for APIs; Apache 2.0 (Qwen3) or the Llama Community License for open weights | MIT for V-JEPA 2; CC BY-NC 4.0, non-commercial, for I-JEPA and V-JEPA |
| Weak spots | Fluent but sometimes wrong; knows only what is in its training data or context; LeCun argues its common sense is shallow without experience of the physical world | Can’t say or draw anything on its own; robot success of 65% to 80% on pick-and-place in lab tests; most published results are on images and video |
Rule of thumb
They answer different questions. Use an LLM when the input or output is text and you need a working product today. Look at JEPA when the input is video or sensor data and you want a frozen encoder or a world model to plan with, and accept that it is research code. The two are starting to meet: LLM-JEPA adds a JEPA objective to LLM training, and VL-JEPA answers questions by predicting text embeddings.
Sources
- Claude docs: Glossary (tokens, pretraining, next-token prediction)
- Hugging Face: Qwen/Qwen3-8B model page (8.2B parameters, Apache 2.0)
- Wikipedia: Large language model (DeepSeek-R1, open-weight models)
- Hugging Face: meta-llama/Llama-3.1-8B-Instruct model card (Llama 3.1 Community License)
- Yann LeCun: A Path Towards Autonomous Machine Intelligence (JEPA; LLMs, tokens and common sense) · archived copy, original
- Meta AI: I-JEPA announcement (632M parameters) · archived copy, original
- arXiv: V-JEPA, Revisiting Feature Prediction for Learning Visual Representations from Video (no labels, text or negatives)
- Meta AI: V-JEPA 2 world model and new benchmarks (1.2B parameters, data, 65 to 80% success) · archived copy, original
- arXiv: V-JEPA 2 paper, HTML version (planning toward goal images)
- GitHub: facebookresearch/vjepa2 (MIT)
- GitHub: facebookresearch/ijepa (CC BY-NC 4.0)
- arXiv: LLM-JEPA (2025)
- arXiv: VL-JEPA (2025)
- Wikipedia: Advanced Machine Intelligence Labs (research organization, no product for around five years)