Week 4: Why I Don't Use One AI Model for Everything

Build in Public — Week 4 (May 11–May 17)

Every time a user sends a message to Lumi, the app needs to figure out what they want. Sometimes it’s simple: “remind me at 3 PM.” Sometimes it’s complex: “remind me every other Sunday starting next month, but skip holidays.” The question I faced in Week 4 was whether every request deserved the same level of processing.

The obvious answer is no. But obvious answers are often the hardest to implement because they require you to build more than one path.

If every request goes through the same heavy AI processing, the app becomes slower and more expensive to run than it needs to be. But if you try to handle everything with a lightweight approach, complex requests will fail. The trick is knowing the difference and building a system that can tell them apart.

The Router Concept

The solution I settled on was a two-stage approach. The first stage is a fast, lightweight process that doesn’t try to solve the user’s request — it just figures out what kind of request it is. Is the user creating a reminder, editing one, deleting one, or just asking a question? This classification step is intentionally narrow and focused.

Once the type is determined, the request gets passed to the right handler. Common actions like creating a simple one-time reminder take a quick, direct path. More complex requests — recurring schedules, ambiguous times, edits that need to find the right task first — go through a more thorough process that can handle the nuance.

The analogy I used in my notes was “router and specialist.” The router’s job isn’t to be smart — it’s to be fast and accurate enough to send the request to the right place. The specialist handles the detailed work. Neither one needs to do everything. Together, they cover the full range of what users ask for.

A two-stage approach: the router classifies the request, then sends it to the right handler

Why This Matters for a Solo Developer

This approach matters less for technical reasons and more for practical ones. When you’re alone, you don’t have infinite resources. Every AI call costs something — in time, in money, in complexity. If you can handle 60-70% of requests with a lighter process and reserve the heavy processing for the harder cases, you’re not just saving on costs. You’re making the product faster for most users, which is a better experience for everyone.

I also learned something about my own enterprise background during this week. In corporate software, we used workflow engines and routing rules for everything. The terminology was different — “business process management,” “service orchestration” — but the principle was the same: not every task needs the same level of handling. Bringing that thinking to a small AI app felt surprisingly natural.

The Accuracy Tradeoff

Of course, there’s a risk in routing: what if the router misclassifies a request? A complex request sent to a simple handler will produce a bad result. A simple request sent to the heavy handler will be slow and wasteful. Getting the classification right is critical.

For Lumi, I approached this by keeping the router’s job narrow. It doesn’t need to understand everything about the request — just enough to know where to send it. If there’s uncertainty, the system can ask the user for clarification rather than guessing. This “ask instead of guess” principle became one of the design pillars of the product.

What I Learned

Week 4 was when I stopped thinking about AI as a single black box and started thinking about it as part of a system. The model matters, but the flow around the model matters just as much. For a solo developer, that’s good news: you don’t need the most advanced AI to build a good product. You need clear thinking about what each part of your product should do and when.

Next week: why I needed a proxy between Lumi and the AI, and what could go wrong.