Seeing a lot of mixed takes on what LLMs do to vertical software moats. Most are framed as if the biggest threat to software is "an LLM in a chat window." But the real threat is the north star: "AI Agents working with 100% reliability, enterprise context, and at institutional scale" that is slowly becoming reality. Some quick thoughts on the biggest misses I'm seeing:
Business logic and institutional context are THE most important value add. Firms want expert vertical AI Agents not because they want another chatbot, but because they want something that can deliver outcomes with deep, firm-specific context. Similarly, software companies that are deeply embedded in business context have a meaningful moat. But AI's advantage is that it can be built to adapt to different types of business logic while using a shared platform.
Thoughtful UI matters. UI alone is not a moat. But assuming that sophisticated AI Agents can be reduced to a chat box is a huge oversight. Users need to understand how to build Agents that best represent their workflows, how to collaborate and provide feedback, how to get them to reliably interface with other tools. Overloading all of that into a chat box just doesn't work — our lived experience at Samaya — and the need for guided, structured interaction is real.
Strong engineering is still not "trivially accessible." While AI coding tools offer a real increase in productivity, building reliable, fast, scalable, and secure systems — the foundation for enterprise-grade AI Agents — is still a substantial lift. And that's not even mentioning model training work that requires deep technical understanding. AI and software teams that maintain a bar for technical excellence continue to have an important edge. With the coding tools, it's technical excellence coupled with focus and velocity. (See e.g.
x.com/ibab/status/1983356398… from
@ibab )
Nailing the "long heavy tail." When I was at Google, I spent a lot of time training models to be accurate on the "long heavy tail" — a large set of less common but important use cases. Fast forward to now: what's in the long heavy tail has changed, but not its existence. We consistently find small errors in our AI systems (e.g., company tickers, mixing up metrics) that have to be corrected with urgency so they don't compound as the Agent executes. I expect we will always have a changing "long heavy tail" that needs custom development.
Proprietary data is not necessarily a moat. On the surface, proprietary data seems like a strong moat, especially for software incumbents. But in practice, a lot of proprietary data is not truly proprietary. It may be a data product with revenues and pressures tied to licensing it out, aggregated across different primary sources — now easier with AI — or tied to other third parties. I expect we'll see a trend towards data becoming more broadly accessible as the value accrues to what you do with the data.
A common mistake that AI companies make nowadays is to not give their engineers enough time and mental calm to do their best work. Constant deadlines, pressure and distractions from daily AI news are poison for writing good code and systems that scale well. That’s why most AI APIs and products have reliability issues.
A good company culture that mixes excellence with focus and enough rest leads to faster and better results. The best example of how to do it well is the early Google culture from 1998 which resulted in one of the largest scale and most reliable services on the web in just a few short years. Founders should copy some of the strategies that Larry and Sergey used. They are still underrated IMO despite their huge reputation.
