Working With vs. Working On: The Coming Schism in LLM Development
The Prediction
At some point in the near future, AI companies will confront a structural truth they’ve so far managed to avoid: building large language models and building usable LLM-driven systems are two fundamentally different disciplines. And yet today, the same people—Machine Learning PhDs—are tasked with both.
This arrangement is not just inefficient. It’s epistemologically incoherent. And it will not last.
The future of LLM development will be defined by a crucial division of labor: working on LLMs (training the underlying models) versus working with LLMs (designing systems around them). These are not merely different roles. They are different ways of thinking, requiring different backgrounds, heuristics, and forms of expertise.
We are living through a moment akin to the early days of computing, where hardware engineers built machines and were then asked to design user interfaces, operating systems, and enterprise applications. It worked, sort of. But it was never sustainable.
The Current Misalignment
The LLM space is still dominated by the prestige and inertia of its origins.
- ML researchers built the early breakthroughs.
- Investors learned to associate those breakthroughs with this class of technologist.
- ML specialists now run the orgs, set the roadmaps, and control the hiring pipelines.
The result is a monoculture: highly skilled, deeply quantitative thinkers trying to solve applied cognitive design problems by throwing more tokens, more parameters, and more fine-tuning at the issue.
But usability is not an emergent property of scale.
- It is an architectural problem.
- A linguistic problem.
- A human problem.
And most ML researchers are not trained for that kind of work.
What Working With LLMs Requires
To build usable, interpretable, trustworthy LLM systems, the talent stack must shift. What’s needed is not just more model engineers, but a new breed of multi-disciplinary teams:
- Linguists, who understand not just syntax, but pragmatics — the way meaning emerges from context, tone, and social interaction.
- Philosophers, who can articulate and formalize concepts like truth, reference, intention, and the ethics of dialogue.
- Cognitive psychologists, who understand how humans frame questions, anchor on answers, and fall prey to systematic bias.
- Enterprise software architects, who know how to build systems that scale, interoperate, and recover gracefully when things go wrong.
Today, those voices are marginal. Soon, they will be central.
LLMs Are Not the Product
Right now, many companies still treat the LLM as the product. It isn’t.
It is the engine.
The product is the LLM-augmented system: the co-pilot, the tutor, the assistant, the dialogic interface, the complex scaffold of constraints and affordances that turn stochastic token streams into usable tools.
And those systems do not emerge automatically from better fine-tuning. They must be engineered. Not optimized. Designed.
This is where current org charts are misaligned. The skills that train the model are not the skills that can harness it. And the longer we pretend otherwise, the more brittle and incoherent our AI products will remain.
Organizational Inertia and Domain Capture
The deeper problem is sociological. The AI field has undergone classic domain capture:
- A new field emerges.
- A narrow slice of experts achieves early dominance.
- That group defines the boundaries of legitimacy.
- They hire in their own image and shape incentives to match their worldview.
This is not a conspiracy. It’s an inevitability. And it happens in every emerging domain.
But at some point, someone from outside the priesthood builds a better system—simpler, more usable, more elegant. And suddenly, the illusion of total technical control begins to crack.
That crack is coming for the LLM space.
A Copernican Shift
The future is not more ML-shaped solutions. It is system-shaped solutions, where ML is a single component.
The dominant metaphor will shift from “building a better brain” to “designing a better conversation.” And that shift will require a wholesale realignment of who gets hired, who gets funded, and who gets to decide what counts as progress.
We will stop asking ML researchers to write the user manual for the system they barely interact with. We will stop pretending that all AI problems are just engineering problems in disguise. We will remember that intelligence is not prediction alone. It is judgment, responsiveness, narrative, context.
And that means a new kind of architecture—epistemic, dialogic, interpretive.
It’s worth noting that Machine Learning as a term is not intrinsically narrow. One could imagine a more expansive vision in which the fields of philosophy, linguistics, psychology, and software architecture are seen as vital components of an enriched ML ecosystem. But in practice, what is currently taught in graduate programs under the ML banner is overwhelmingly geared toward model optimization, not systemic integration. The argument here is not against Machine Learning, but against the limitations of its institutional definition and application.
The question is not if this transition will happen. It’s how long it will take for the institutions to catch up.
If you’re building for the next five years, don’t just hire people who know how to build models. Hire people who know how to build around them.