What If the LLM Gets It Wrong? How Lumi Protects Your Tasks

Engineering - AI reliability

When building an AI-powered reminder app, one uncomfortable question appears very quickly: what happens if the LLM understands the user incorrectly?

This is not a theoretical problem. A reminder app is not just answering a question. It is creating something that may affect the user’s real day. If the app misunderstands a date, a recurrence rule, or a pause instruction, the mistake may not be visible immediately. The user may only discover it later, when the reminder fires at the wrong time, or does not fire at all.

At first, I treated the LLM response as the main output. The user said something, Lumi parsed it, created the task, and replied with a short confirmation. That flow felt natural because it matched the usual AI chat pattern: input, model understanding, response. But reminders are different from chat. The important output is not the sentence the assistant sends back. The important output is the task that has been created.

That distinction changed the design.

Prompt Quality Is Necessary, but Not Enough

The first instinct is to improve the prompt. If the model misunderstands “next week” or confuses a temporary schedule with a repeating one, it is tempting to add more examples, more rules, and more edge cases to the prompt.

Prompt improvement is useful. Lumi still needs clear instructions for extracting intent, dates, times, recurrence, and task operations. But the prompt cannot enumerate every way a human may describe a reminder. Real language is messy. A user may say “next week” and mean only the seven days from Monday to Sunday. Another user may mean “starting next week and continuing every week.” Both interpretations can sound reasonable depending on context.

For a product, the problem is not simply “how do we make the model smarter?” The better question is:

How do we design the product so that mistakes are visible and correctable?

That question moved the solution away from prompt-only thinking and toward workflow design.

The First Flow Was Too Invisible

The early flow was simple:

User says something -> LLM understands -> Task is created -> Assistant replies

This is fast. It also feels magical when it works. The problem is that the created task can remain hidden behind the assistant’s reply. If the assistant says “Done, I will remind you tomorrow morning,” the user may assume everything is correct. But what exactly was saved? What time is “morning”? Is it a one-time reminder or a recurring one? Did the app understand the date range?

The reply alone was not enough evidence.

In traditional software, users often review a form before saving. In conversational software, that review step can disappear. The user does not see fields; they see a friendly sentence. That creates a trust problem. The system may look confident even when the interpretation deserves inspection.

The Key Shift: Show the Result, Not Just the Reply

The important idea was to show the created task directly inside the conversation. After Lumi created a reminder and replied to the user, the conversation could also show a compact task result: title, time, recurrence, and an entry point to edit or view details.

The flow became:

User says something -> LLM understands -> Task is created -> Assistant replies -> Task result is shown -> User can correct it

This is different from a strict confirmation-before-save workflow. I considered that option too: the model parses the request, the app shows a draft, and the user confirms before the task is created. That is safer in theory, but it also adds friction to every reminder. For a daily reminder tool, speed matters. If every simple reminder requires an approval step, the assistant starts to feel slower than a normal form.

So Lumi uses a lighter pattern: create the task, show the result, and make correction easy. If the result is correct, the user does nothing. If something is wrong, the user has an immediate path to fix it.

This is closer to how a practical assistant works. A human assistant may say, “I’ve set it for tomorrow at 9 AM.” If that is wrong, you correct it right away. The key is that the assistant repeats back the actionable result, not just “Done.”

Why a Visual Result Helps

A task has structure. A sentence hides that structure. A visual result exposes it.

For a reminder, the user usually needs to verify a few things:

• What is the reminder about?
• When will it happen?
• Is it one-time or recurring?
• Does it have an end date?
• Can I open it and change it?

When these details are visible, the user does not need to mentally parse the assistant’s reply. They can scan the result. This is especially useful for recurring reminders, multi-time reminders, pauses, and updates to existing tasks.

This also changes the role of the LLM. The model still does the hard language work, but the product does not ask the user to blindly trust the model. The app turns the model’s interpretation into product data, then lets the user inspect that data.

That is the core reliability pattern:

Do not only make the AI explain what it did. Show the actual product state it created.

Why I Did Not Make Every Task Wait for Confirmation

There is a tradeoff between safety and speed. A full confirmation workflow can reduce mistakes before they enter the database, but it also creates more steps. For reminders, many requests are simple: “Remind me at 8 PM,” “Call John tomorrow morning,” or “Pay the bill on Friday.” Asking for confirmation every time would make the product feel heavy.

The post-action review pattern keeps the main interaction fast. The task is created immediately, and the user receives a normal assistant reply. The visual result then acts as a safety layer. It is available when needed, but it does not block the happy path.

This decision also reduced implementation complexity. A pre-save confirmation system would require draft tasks, pending states, timeout behavior, voice confirmation, and more complicated conversation management. Those may become useful later, but they were not necessary to solve the first reliability problem.

The first version needed a pragmatic balance:

• Keep the conversational flow fast.
• Make the created task visible.
• Let the user correct mistakes immediately.
• Reuse the existing task editing flow instead of inventing a new one.

That balance fit the product better than a perfect but heavy confirmation system.

The Card Became Too Large

The first version of the task result card solved one problem but introduced another. It made the result visible, but it also consumed too much space in the conversation.

This matters because chat is a narrow interface. If every assistant message includes a large task card with all fields, the conversation becomes hard to read. The card competes with the dialogue instead of supporting it. On smaller phones, this problem becomes more obvious.

That led to the second design change: the card should not display everything. It should display only the information needed for a quick check, and provide a link or action to open the full task detail page.

In other words, the card changed from a full task detail view into a compact verification surface.

The compact version keeps the conversation readable. It gives the user enough information to notice obvious mistakes, while the full task detail page remains available for deeper review or editing.

Important Information First, Details One Tap Away

The final pattern became more focused:

• The assistant reply stays short.
• The task result shows the most important fields.
• A detail link opens the task page.
• The task page handles full editing and correction.

This avoids duplicating the full task management UI inside the chat window. The chat should remain a conversation. The task detail page should remain the place for structured inspection and editing.

This separation is important. AI chat is good for capture and intent. Task screens are good for review, comparison, filtering, editing, and control. Trying to make the chat window do everything would make it crowded and fragile.

The better design is to connect them:

Conversation creates the task.
Compact result helps the user verify it.
Task detail page handles full control.

That small link between conversation and task detail became a reliability feature, not just a navigation feature.

Reliability Comes from Layers

The lesson I took from this feature is that AI reliability should not depend on one layer.

The prompt matters. The parser matters. The business logic matters. The database state matters. The UI matters. The editing flow matters. The user also matters, because the user is the only person who truly knows what they meant.

For Lumi, protecting tasks from LLM mistakes became a layered design:

1. Use the LLM to understand natural language.
2. Convert the result into structured task data.
3. Create or update the task through product logic.
4. Show the created result back to the user.
5. Keep the result compact enough for chat.
6. Provide a direct path to details and editing.

None of these layers is perfect alone. Together, they make the product safer.

What I Learned

The biggest lesson was that AI products should not hide their actions behind fluent replies. A confident sentence can be misleading. A visible result is easier to trust.

For a reminder app, the user does not only need the assistant to say “Done.” The user needs to know what “done” means. Was the reminder created for the right day? Does it repeat correctly? Can it be changed if the model got it wrong?

The answer was not to make the prompt infinitely complex. The answer was to make the workflow inspectable.

This is now one of my product principles for Lumi:

AI should reduce effort, but it should not remove control.

The best AI interaction is not always the one with the fewest visible steps. Sometimes the right design is to complete the task quickly, then show just enough structure for the user to trust, verify, and correct it.

That is how Lumi protects tasks when the LLM gets it wrong.