Week 5: Setting Up a Global LLM Proxy - What Could Go Wrong?

Build in Public — Week 5 (May 18–24)

Here’s a rule I learned the hard way: never put your API key inside a mobile app. It doesn’t matter how well you obfuscate it, how many layers of encryption you wrap around it, or how confident you are that no one will look. If the key is in the app, it will eventually be extracted. And once someone has your API key, they can use your AI services at your expense.

This was Week 5’s problem. Lumi needed to communicate with AI services to process user requests, but those services required authentication. The standard approach in a mobile app is to call the AI provider directly with an embedded key — quick, simple, and dangerously insecure for anything beyond a prototype.

The solution was a proxy: a lightweight intermediary that sits between the app and the AI provider. The app talks to the proxy, the proxy authenticates with the AI provider, and the response comes back through the same channel. The API key never leaves the server.

Why Not Just Use a Direct Call?

Direct API calls from a mobile app are simple. You add the API key to your request headers, send the request, and get a response. For a demo or a proof of concept, this works fine. But for a real product with real users, the risks add up.

An embedded key can be extracted through APK decompilation, network traffic inspection, or various other techniques that are well-documented and widely used. Once extracted, there’s nothing stopping someone from using that key however they want — making requests on your account, consuming your quota, and leaving you with the bill.

A proxy eliminates this entire category of risk. The key lives on a server you control, behind additional authentication that can be rotated, revoked, and monitored independently.

The Cloudflare Moment

I chose Cloudflare Workers as the proxy layer for a few reasons. First, they run on a global edge network, which means the proxy is physically close to users regardless of where they are — important for a product that aims to serve people in different regions. Second, the free tier was generous enough to cover the early-stage traffic without additional cost. Third, the deployment model was simple: write the code, deploy with a single command, and it works.

The first time I sent a request from the app through my own proxy and got a response back, it felt like a real milestone. The infrastructure wasn’t just an idea anymore — it was actually handling requests.

How requests flow from the app through the proxy to the AI provider

What Could Go Wrong

Plenty, as it turned out. The proxy introduced a new point of failure in the system. If the proxy goes down, the app can’t process requests. If the proxy is slow, every user feels it. If authentication on the proxy side fails, the app gets an error instead of a reminder.

Debugging proxy issues is also harder than debugging direct calls. When something goes wrong, the failure could be in the app, the proxy, the network between them, or the AI provider at the other end. Tracing the problem requires logging at each layer, which adds complexity to what was supposed to be a simple intermediary.

I also had to think about abuse prevention. Without user accounts, there’s no traditional way to limit how many requests a single user can make. The proxy needed basic safeguards to prevent excessive usage while keeping the experience smooth for legitimate users. Striking that balance took several iterations.

What I Learned

Week 5 taught me that even simple infrastructure requires more care than it seems. A proxy is a straightforward concept — route request A to destination B — but making it reliable, secure, and performant is a project in itself. For a solo developer, the lesson is to start simple, test thoroughly, and be prepared to iterate on infrastructure just as much as on product features.

Next week: the UI went from embarrassing to functional.