LangSmith

An online tool that helps developers get their Large Language Model app from prototype to production. LangSmith is a unified platform for building, testing, evaluating, and monitoring large language model (LLM) applications. It offers complete visibility into agent behavior through tracing, real-time dashboards, and insights, helping developers debug issues and improve performance without adding latency to apps. Key features include step-by-step tracing for debugging, live monitoring of metrics like costs and latency, automatic insights into usage patterns, prompt experimentation, dataset management for evaluations, and collaboration tools for teams. It integrates seamlessly with frameworks like LangChain via a single environment variable. It's ideal for developers and teams building production-grade AI agents, especially those using LangChain or LangGraph. Beginners prototyping apps might find it useful too, but it's geared toward anyone needing observability in non-deterministic LLM workflows, from startups to enterprises. LangSmith works hand-in-hand with LangChain for easy tracing and evaluation. Just set one environment variable, and it captures every step of your chains or agents, making it simple to monitor and debug without extra code. Plans start with a free Developer tier (5k traces/month, 1 seat), move to Plus at $39/user/month (10k traces/month, up to 10 seats), and include a Startup discount for early-stage teams. Enterprise offers custom pricing with self-hosting; traces cost extra based on retention (basic: $0.50/1k for 14 days, extended: $4.50/1k for 400 days). On the Enterprise plan, you can deploy it on your Kubernetes cluster in AWS, GCP, or Azure to keep data in your environment. It's not available on lower tiers, though, so teams prioritizing data privacy often go this route.