Own Your Embeddings Before You Own Your Model
Goku Mohandas lays out a five-stage climb toward a full RL loop (embeddings → private evals → SFT on traces → a minimal RL environment → a production loop) and is explicit that you do not build the whole thing at once. The first rung is the one almost any team can reach: own your embeddings.
His argument: “not every team is ready to train a full foundation model, but almost any team can own its embeddings.” You train or post-train an embedding model on your own data, then reuse those embeddings across every downstream product, search, recommenders, content matching, fraud, and retrieval for AI agents, and “watch every downstream consumer improve from these contextual embeddings.” The value is leverage from a single asset: build the embedding once, reuse it everywhere, and the improvement compounds across teams that used to ship their own ad-hoc features.
He grounds it in real cases: Tripadvisor produces image, review, and geo embeddings reused from location search to agent RAG; Nubank’s nuFormer is a single transaction-sequence backbone whose embeddings lift credit, fraud, churn, and recsys at once; Adobe Firefly computes embeddings inside the training loop itself. The lesson is sequencing: ownership of your intelligence is a staircase, and embeddings are the cheap, high-leverage first step that earns the right to climb to evals, fine-tuning, and RL.
Related
- Hosting Is Table Stakes Extending Is the Moat — this note is the concrete entry point onto the “extending” staircase that note describes
- The Moat Is the Learning Loop Not the Model — owning embeddings is the first piece of data you stop renting on the way to owning the full loop
- An RL Environment Is a Programmable Simulator of Your Business — the top of the staircase this note starts; embeddings are stage one, the RL environment is stage four
- Skills Are the Unit of Reusable Expertise — same compounding logic in the agent world: build a reusable asset once and every downstream use benefits