Your LLM app runs on vibes. Traces end the guessing.
Date Published
Someone pings you: "the model gave a weird answer." You have two options. Reproduce it by hand and argue about what probably happened, or open a trace and read every prompt, retrieval step, and tool call that led there. Most teams pick the guessing, because observability sounds like more infra than they want to own.
We chose traces. Specifically, we self-hosted Langfuse, the open-source LLM observability platform, for our internal AI stack. No SaaS account, no prompt data leaving our network, no pricing that grows the more you look. Here is what the setup is like in general terms, and what it changed.
What self-hosted Langfuse actually is
Strip the branding and it's a small, boring, well-chosen set of parts, which is exactly what you want from infrastructure:
A web app that serves the UI and ingests trace events from your LLM applications.
A worker that processes those events asynchronously, so ingestion never blocks your app.
ClickHouse as the analytics store. This is the architectural decision that matters. Traces are analytics data, millions of rows you want to slice by user, session, model, cost, or latency. When Langfuse moved to ClickHouse, filtering across thousands of sessions stopped crawling.
Postgres for the relational bits: users, projects, API keys, prompt versions.
Redis as the queue between the web tier and the worker.
Object storage for the bulky attachments that live inside traces.
Boring parts are a feature. When something hurts, you debug it with the same skills you already have, and you upgrade on your own schedule because the image versions are pinned in your own chart. A single Helm deployment brought the whole thing up in an afternoon, next to the auth server and the chat front-end it observes.
Wiring it in was the easiest part
The trace source is our internal chat interface (Open WebUI, if you use it too). The integration is three environment variables: a public key, a secret key, and the Langfuse host. That's all. Every conversation becomes a trace with the session and the user attached.
Once that's in, the questions you can answer change character:
Which sessions burned the most money this week, and why?
Which prompt version made users retry their question?
That weird answer: what context did the model actually receive?
How many tool calls does our agent waste per task?
One honest data point from our own review: when we ranked what our chat deployment still lacked for production hardening, the top gap was backups, not tracing. Tracing was already a solved, unglamorous utility. That is what success looks like for this kind of tool. It stops being a project and becomes a place you look.
What self-hosting buys you
Prompt data stays home. Traces contain full prompts, full outputs, and whatever someone pasted into the chat. For a lot of companies that ends the SaaS conversation on its own. On your own network, "data residency" isn't a policy slide, it's a property of the wiring.
No per-seat, per-event meter. Observability vendors that charge by ingestion teach teams to look less. A stack that costs CPU and disk teaches teams to look more, which is the entire point of having it.
You own the upgrade cycle. Pinned versions, your maintenance window, your rollback plan. The vendor ships when they ship; you adopt when you're ready.
It composes. SSO with the same login you use for everything else. Backups on the same nightly job pattern as the rest of your estate. The tool sits inside your platform instead of beside it.
What it costs you, in fairness
Two stateful databases plus object storage is more to babysit than handing your traces to a vendor. Say that out loud before you commit, then do the two things that make self-hosting survivable:
Back it up from day one. App state in Postgres and traces in ClickHouse are only disposable if you consciously decide they are. Do the backup job before your first incident, not after. Our first enterprise-readiness gap was exactly this.
Put real TLS and SSO on it once it matters. Plain HTTP inside a trusted network is fine for week one and sloppy by month three.
A single web replica and single worker is honest for a small team. It's a sizing decision, not a defect; scale the worker when your trace volume says so.
The part people skip
Running the tool is the easy half. The return shows up in behavior: someone reports a bad answer, you pull the trace by user and session, read the actual prompt and the actual retrieved context, and fix the real cause. No reproduction theater, no "it works on my model." The dashboard is decoration. The trace is evidence.
If your AI apps currently run on screenshots and vibes, the fix is smaller than you think. Langfuse ships the images, the chart, and the docs; the only thing you have to bring is somewhere to run it and an afternoon. Your future debugging stories get to end with "I looked at the trace" instead of "I think maybe."
Build and train your local AI stack with Worlber
Worlber helps organizations design, train, fine-tune, deploy, and observe local AI models on infrastructure they control. From private data pipelines and GPU sizing to model serving, Langfuse tracing, security, and ongoing operations, we turn experimental models into reliable internal platforms.
Planning a private AI or local-model initiative? Talk to Worlber Database Services.