> AI agents: this is one page from PostHog's docs. Full index of Markdown docs for LLMs: https://posthog.com/llms.txt # Cost and performance – AI Observability pocket guide Canonical URL: https://posthog.com/learn/ai-observability/pocket-guide/cost-and-performance > **Cloud region:** This page uses US Cloud hosts. For an EU Cloud project, use `https://eu.posthog.com` instead of `https://app.posthog.com` (app host for links). ## On this page - [What goes into a model call](#what-goes-into-a-model-call) - [Separate an expensive request from an expensive conversation](#separate-an-expensive-request-from-an-expensive-conversation) - [Find latency](#find-latency) - [Increase efficiency](#increase-efficiency) - [Investigate cost and performance in your project](#investigate-cost-and-performance-in-your-project) Chapter 05 # Cost and performance – AI Observability pocket guide “Does it have a full kitchen?” is a short question. But answering it involves more than sending those six words to the model. The trip planner also receives instructions, listing information, and the earlier forest and beach exchanges. Twig's engineers want to see what the model processes, what the conversation costs, and where the visitor spends time waiting. ## What goes into a model call Twig's engineers open the kitchen generation and inspect its input. They can see the visitor's question alongside the information the trip planner received about Flamingo's Envy. Models process text in units called [tokens](https://posthog.com/docs/ai-observability/generations.md). The question, instructions, listing information, and conversation history all contribute [input tokens](https://posthog.com/docs/ai-observability/generations.md). The model's answer contributes [output tokens](https://posthog.com/docs/ai-observability/generations.md). Select the forest, beach, and kitchen requests to compare their inputs and recorded usage: The kitchen question depends on the earlier conversation: “it” refers to the beach stay. But that doesn't mean the model needs every fact about every stay in Twig's catalog. The engineers compare the recorded input before deciding what to remove. They need to preserve the information that makes the answer correct, rather than shortening the prompt and hoping for the best. They also verify that the generations contain model and usage data. PostHog calculates cost using [supported model pricing or costs supplied by the application](https://posthog.com/docs/ai-observability/calculating-costs.md). Missing cost information shouldn't make a request look free. ## Separate an expensive request from an expensive conversation Twig's engineers look at the forest request on its own, then add the beach and kitchen requests. [Cost per trace](https://posthog.com/docs/ai-observability/calculating-costs.md) measures the recorded cost of one request. [Cost per AI session](https://posthog.com/docs/ai-observability/calculating-costs.md) measures the conversation containing those related requests. A visitor who asks three questions can generate more model work than someone who accepts the first recommendation. The engineers add the underlying generation costs once. They don't add a trace's total cost to the cost of the generations already included in that total. They also compare how many requests Twig receives. A bigger total bill could mean more visitors are using the trip planner, rather than each answer becoming more expensive. To investigate, the engineers put request counts beside cost per request and conversation cost. They separate the trip planner from the support agent, and keep LLM judge usage identifiable too. Evaluating an answer can involve another model call, even though that call doesn't produce a new reply for the visitor. ## Find latency Cost doesn't explain how long the visitor waits for a recommendation. Twig's engineers inspect [latency](https://posthog.com/docs/ai-observability/traces.md), the time spent on a recorded operation. For the city request, they compare the `search_stays` span with the generation in the trace. This separates the time spent looking up stays from the time spent producing the answer. Compare the operation durations in the trace: This gives the engineers somewhere specific to investigate. A slow catalog lookup needs investigation in Twig's search code – choosing a faster model wouldn't speed up that lookup. ## Increase efficiency Twig's engineers test a smaller input for the kitchen question. Instead of supplying the whole catalog, they supply Flamingo's Envy's listing facts and the conversation context needed to identify the stay. Then they replay the conversation. They inspect two things: whether the input uses fewer tokens, and whether the trip planner still gives the right answers. The beach recommendation must still fit four guests. The kitchen answer must still refer to Flamingo's Envy and its full kitchen. Removing the group size or selected stay would make the input shorter while taking away information the model needs. They keep request volume, cost per request, conversation cost, and latency on a dashboard alongside the relevant evaluation results. After releasing a tested change, they can see whether the production requests use less model work while still producing suitable recommendations. ## Investigate cost and performance in your project Copy this prompt into [PostHog AI](https://app.posthog.com/#panel=max:Compare%20my%20project's%20AI%20cost%20and%20latency%20over%20the%20last%20seven%20days%20with%20the%20previous%20seven.%20First%20identify%20missing%20model%2C%20usage%2C%20cost%2C%20or%20timing%20data.%0A%0ASeparate%20changes%20in%20request%20volume%20from%20changes%20in%20cost%20per%20request%20and%2C%20where%20session%20IDs%20exist%2C%20per%20conversation.%20Break%20down%20the%20difference%20by%20available%20feature%20and%20model%20fields.%0A%0AShow%20representative%20traces%20behind%20the%20change.%20Inspect%20their%20inputs%20and%20compare%20catalog%20lookup%20time%20with%20generation%20time.%20Link%20to%20the%20evidence%20behind%20your%20findings.%0A%0ARecommend%20one%20change%20to%20test.%20Identify%20the%20evaluation%20results%20needed%20to%20verify%20that%20the%20change%20preserves%20answer%20quality%2C%20and%20explain%20what%20evidence%20is%20still%20missing.): ```text Compare my project's AI cost and latency over the last seven days with the previous seven. First identify missing model, usage, cost, or timing data. Separate changes in request volume from changes in cost per request and, where session IDs exist, per conversation. Break down the difference by available feature and model fields. Show representative traces behind the change. Inspect their inputs and compare catalog lookup time with generation time. Link to the evidence behind your findings. Recommend one change to test. Identify the evaluation results needed to verify that the change preserves answer quality, and explain what evidence is still missing. ``` [Product outcomes →](https://posthog.com/learn/ai-observability/pocket-guide/product-outcomes.md) `npx @posthog/wizard ai-observability` **Explore further** [Calculating costs](https://posthog.com/docs/ai-observability/calculating-costs.md) · [Traces](https://posthog.com/docs/ai-observability/traces.md) · [Custom properties](https://posthog.com/docs/ai-observability/custom-properties.md)