#PatternsinScalingAI
In the past couple of years, I've built and scaled AI apps using a number of different approaches. This was largely because there were no best practices for building and scaling GenAI-based apps back then. The situation is a lot better now. However, one crucial part that most people still miss, especially in #EnterpriseAI, is that #Trust in AI needs to be built in the first few iterations. If this doesn't happen, it takes longer to deploy at scale. When and how much to use LLMs has a big role to play here.
More broadly, I've tried 2 ways of building AI apps.
In one set of projects, we started with a deterministic flow and later added LLM layers. We were forcing deterministic logic too early. The workflow had its own nuances and kept breaking at edge cases. The problem with this approach was that during UAT, users felt the system lacked depth. Building initial trust in the application took time — and so did the deployment. Worst case, since these UAT users were supposed to be champions of the system, adoption suffered.
In another set, I went LLM-heavy from day one and used LLMs for multiple steps. The speed was great. I could ship quickly, learn faster, and get UAT testers to trust the product early. But later, latency became a real issue, and for some steps the model didn't consistently follow guidelines. Economically (and even logically), it didn't make sense to keep using LLMs where we were already dead sure a deterministic flow would work better.
What I've learned: both roads can lead to the same destination, but starting LLM-heavy and then reducing LLM usage has been easier in practice. You get early momentum and trust first, then replace stable parts with deterministic logic to improve reliability, cost, and accuracy.
For me, the long-term pattern looks like this: LLM-first for discovery, deterministic-first for scale.