Category
LLM observability tools: what they are and who makes them
open-source first, evaluation-centered platforms, gateway-based logging, instrumentation libraries: Langfuse, Arize Phoenix, Opik (Comet) through Confident AI.
Types of LLM observability tools
Open-source first
Langfuse, Arize Phoenix and Opik publish their code and can run on your own servers. You trade a managed service for control over where traces live, and each also sells a hosted version for teams that would rather not operate a database.
Evaluation-centered platforms
Braintrust and HoneyHive put scoring, datasets and annotation next to tracing. They suit teams that gate releases on test results. LangSmith belongs here too, with the strongest ties to LangChain and LangGraph, so it fits teams already on that framework.
Gateway-based logging
Helicone and Portkey sit between your code and the model vendor, so logging happens as requests pass through. You also get caching, fallbacks and key management, at the cost of adding a network hop that every request now depends on, which matters when latency budgets are tight.
Instrumentation libraries
Traceloop's OpenLLMetry extends OpenTelemetry to model calls and sends spans to the backend you already use. Lunary sits between the groups, with a hosted product focused on chat replays. Both reduce lock-in on the collection side, and both pair with a separate dashboard if you prefer another backend.
Facts at a glance
| Tool | Publisher | Based in | Access |
|---|---|---|---|
| Langfuse | Langfuse GmbH (part of ClickHouse) | Germany | Free tier + API |
| Arize Phoenix | Arize AI, Inc. | United States | Free tier + API |
| Opik (Comet) | Comet ML, Inc. | United States | Free tier + API |
| Braintrust | Braintrust Data, Inc. | United States | Free tier + API |
| Helicone | Helicone, Inc. | United States | Free tier + API |
| LangSmith | LangChain, Inc. | United States | Free tier + API |
| Portkey | Portkey, Inc. | United States | Free tier + API |
| HoneyHive | HoneyHive Inc. | United States | Free tier + API |
| Lunary | Lunary LLC | United States | Paid + API |
| Traceloop | Vecinity (Traceloop) | Israel | Free tier + API |
| Datadog LLM Observability | Datadog, Inc. | United States | Trial + API |
| Confident AI | Confident AI | Not stated on the pages read | Free tier + API |
