GitHub has added enterprise-managed OpenTelemetry configuration to the Copilot app. Administrators can export data about agent sessions, requests to AI models and tool use into compatible monitoring systems. The stated purpose is to follow a session’s flow and investigate unexpected behaviour step by step rather than infer it from a final code change.[1]
This is an important shift. An ordinary chatbot produces an answer; an agent may search, call tools, modify files, invoke another service and retry before it finishes. Without a trace, those intermediate decisions disappear. When the outcome is wrong, the organisation cannot reliably tell whether the failure came from the model, the tool, the input, a permission, a stale dependency or a later human action.
What each observability signal can answer
| Signal | What it records | Useful agent question |
|---|---|---|
| Trace | The end-to-end path of a request through connected operations | Which model and tools were used, in what order and with which latency?Source 2 |
| Span | One unit of work with timing, attributes, events and status | Which individual tool call failed or consumed most of the time?Source 2 |
| Metric | A numerical measurement captured over time | Are retries, cost, latency or failure rates rising?Source 3 |
| Log | A record of an event | What specific error or policy decision occurred?Source 3 |
| Baggage or linked context | Context carried across related operations | Which user, task, repository or approval chain does this work belong to?Source 3Source 2 |
A trace maps execution across connected operations
OpenTelemetry defines a trace as the path of a request through an application and a span as one unit of work. Spans can carry start and end times, parent-child relationships, attributes, events, links and status. That structure is well suited to an agent because one user task can branch into model calls, search, file operations and asynchronous jobs that need to be correlated.[2]
GitHub excludes prompt and response content by default. That is a sensible privacy boundary, especially when code, secrets or personal information may appear in context. It also means a trace may show that a model call occurred without revealing why its answer was wrong. Organisations should decide deliberately whether any content is captured, redacted, sampled or retained rather than turning on maximum logging as a reflex.[1]
A minimum audit record for consequential agent work
| Measure | Value | Context |
|---|---|---|
| Task identity | Stable task, user and repository identifiers | Connect the trace to the business request and authorised scope. |
| Configuration | Model, version, reasoning setting and agent instructions | Make results comparable and explain routing changes. |
| Tool sequence | Name, parameters, target and outcome | Show what the agent actually touched without exposing unnecessary content. |
| Approvals | Who approved which action and when | Distinguish delegated work from unauthorised execution. |
| Artifacts | Diffs, files, tests, receipts and external records | Connect agent activity to the result that must be reviewed. |
| Outcome | Accepted, rejected, rolled back or incomplete | A technically successful span is not the same as a successful business task. |
Completion is not correctness
OpenTelemetry status can say that an operation completed without a technical error. A model request can return normally, a tool can accept an instruction and a file can be written while the overall task is still wrong. Agent evaluation must therefore join the execution trace with acceptance evidence: tests passed, source citations verified, totals reconciled, a human approved the change or the external system returned a valid receipt.
A useful dashboard should keep system health and task quality separate. Latency, token use, queue time and HTTP failures describe the machinery. Accepted-task rate, rollback frequency, human correction time and policy exceptions describe the outcome. Combining them lets a team find the expensive failure: a fast, reliable workflow that consistently produces work nobody can accept.
The practical rollout
Begin with a small set of high-value workflows and define the questions an incident reviewer must be able to answer. Instrument model calls and tool operations, attach task and approval identifiers, and record the final artifact. Then rehearse a failure: a wrong file, a denied permission, a stale dependency and an interrupted asynchronous job. If the trace cannot explain the failure quickly, add the missing evidence rather than simply collecting more volume.
Agent observability is not a substitute for safe design, but it is a prerequisite for operating agents responsibly at scale. An organisation cannot improve what it cannot reconstruct. The final answer tells you what the agent claims it did; the trace, approvals and artifacts tell you what actually happened.
Sources & further reading
- OpenTelemetry in the GitHub Copilot app GitHub Changelog · 22 Sep 2026 · Accessed 9 Oct 2026
- Traces OpenTelemetry · Accessed 9 Oct 2026
- Signals OpenTelemetry · Accessed 9 Oct 2026
- GitHub Copilot for JetBrains adds improved OpenTelemetry configuration and model management GitHub Changelog · 27 Jul 2026 · Accessed 9 Oct 2026