最新消息显示，From Black Box to Transparent: Alibaba Cloud Agent Observability and Audit Data
In 2025, AI agents are moving from the lab to large-scale production. From code assistants used by developers daily to intelligent customer service in enterprise service scenarios, to multi-agent collaboration systems of ever-increasing complexity, AI agents are reshaping software development and business operations at an unprecedented pace. However, once agents are actually running, a critical problem emerges: the actual runtime behavior of AI agents is difficult to observe, trace, and govern. A coding agent autonomously and without authorization modifies core configuration files overnight, with no way to know what changed or why. An intelligent customer service agent autonomously issues a “cancel order” instruction, yet the decision logic, tool calling chain, and token resource consumption cannot be reviewed. A multi-agent collaborative job fails midway, and the failure node and root cause are difficult to pinpoint. These issues point to a common requirement: AI agents need comprehensive observability. Moreover, this observability cannot remain at the shallow statistical dimension of “request success/failure” — it must deeply cover AI agent-specific runtime aspects such as LLM invocation, tool execution, multi-round inference, and memory retrieval. Based on the OpenTelemetry (OTel) community standard and its in-depth practices in observability fields, Alibaba Cloud has developed a complete data collection solution that covers three types of agent forms. Building on the OTel GenAI semantic conventions, Alibaba Cloud has released the LoongSuite GenAI semantic conventions for observability. This paper will systematically introduce the design concept, technical implementation and use of this scheme. The AI agent market is thriving and highly diverse. The runtime models, deployment environments, and use cases of different agent types vary significantly, and their observability and audit needs differ accordingly. We classify mainstream AI agents on the market into three categories: No matter what form is adopted, AI agents will encounter three common problems after large-scale use: Core design principle: Adapt the data collection capability to the native running mode of the AI Agent instead of forcing the Agent to adapt to the data collection tools. Coding agents run on the developer’s local machine, where all core behaviors — code edits, file creation, terminal command execution — happen in the local environment, completely invisible to traditional server-side agents. To address this, we built LoongSuite Pilot, a client-side data collection platform purpose-built for coding agents. 3.2 Personal General-Purpose Assistant: One-Line Command for Full Observability and Audit Personal general-purpose assistants usually run as standalone services, providing end users with dialogue and task-execution capabilities. For this type of agent, we provide a dedicated plugin that enables full tracing with a single command. Design philosophy Take OpenClaw as an example. Although its built-in diagnostics-otel extension can output Metrics and some Trace, it adopts an event-driven architecture. Span is created independently for each event, and there is no parent-child relationship between each other and Trace Context propagation. In essence, it is a group of “standalone data points”. The openclaw plug-in of LoongSuite is a complete distributed tracing by design-all Span share the same traceId and are connected together into a call tree through an explicit parent-child relationship. Span Semantic Model Each type of span is connected to a complete trace tree by using parent-child relationships. O&M personnel can view the number of large model calls, token consumption, tool call list, time-consuming nodes, and fault information of a single request. Essential differences from built-in observability Compared with the built-in observability capabilities of OpenClaw, LoongSuite plug-ins are different in two aspects: Link integrity. Built-in observability is usually flat and independent, and there is no correlation between events. However, our plug-in is based on the OTel Context propagation mechanism to ensure that ENTRY → AGENT → STEP → LLM / TOOL forms a complete call tree, which can restore the complete picture of a request. Data richness. Built-in observability often only records basic metrics such as model usage, while our plug-ins fully record fields such as gen_ai.input.messages, gen_ai.output.messages, gen_ai.system.instructions, gen_ai.tool.call.arguments, and gen_ai.tool.call.result to meet the needs of in-depth audit and troubleshooting. The same plug-in mechanism already covers personal general-purpose assistants such as Hermes Agent and QwenPaw. For agent applications built on frameworks such as LangChain, AgentScope, and Dify, the runtime behaves like a traditional Python application. We provide the LoongSuite Python Agent (deeply customized from OpenTelemetry Python Contrib), which achieves zero-code automatic instrumentation with a single command. # 1. Install the LoongSuite Python Agent pip install loongsuite-distro # 2. Auto-detect and install the required instrumentation libraries loongsuite-bootstrap # 3. Start with one command; probes are injected automatically loongsuite-instrument \ –traces_exporter otlp \ –service_name my-agent-app \ python my_agent_app.py loongsuite-bootstrap automatically scans for installed frameworks (such as langchain, dashscope, and mcp) in the current environment and installs the corresponding instrumentation packages-developers do not need to manually select and install them. Framework Coverage At present, 16 instrumentation libraries have been covered in the LoongSuite Python Agent, covering the mainstream AI agent development framework: Automatically Recognized Span Types The probe automatically detects and generates multiple GenAI span types, covering the entire agent lifecycle: After accessing the preceding collection capabilities, users can obtain observability views in the following dimensions. Take Claude Code as an example. If you want to enable Agent Observability, you only need to log in to CloudMonitor 2.0 Console, click the corresponding card in the access center and follow the steps to complete the installation and access with one line of command. The complete execution process of the agent is presented in the form of a trace tree, from the user request entry (ENTRY) to the agent decision (AGENT), inference step (STEP), LLM call (LLM), and tool execution (TOOL). The hierarchical relationship is clear at a glance. For complex tasks with multiple rounds of ReAct, you can use Step Span to quickly locate which iteration has a problem, and then go to the LLM or Tool Span in the round to analyze the root cause. Troubleshooting pattern: When an agent executes a 10-round ReAct process, you can first use Step Span to identify which round of the problem occurred, and then analyze the specific step in the round. This top-down troubleshooting method greatly improves the fault locating efficiency of complex agents. Based on gen_ai.usage.input_tokens, gen_ai.usage.output_tokens, and gen_ai.usage.total_tokens , as well as cost fields extended by Alibaba Cloud (input_cost, output_cost, and total_cost), you can: Through gen_ai.session.id, gen_ai.turn.id and gen_ai.step.id to build a three-level identification system to achieve: Full conversation traceability across multiple rounds of conversation Step-level fine-grained analysis in a single-round dialogue Session path analysis and user behavior insights You can record the tools that are called by the agent, the parameters that are specified, the results that are returned, and the duration. For the Coding Agent, this means that every file read or write and every command execution is documented. For MCP protocol calls, complete request-response auditing is also provided. Behavior Analysis Dashboard The top count card divides tool calls into dimensions such as command execution, file reading and writing, search, web browsing, and MCP calls by behavior type, and marks the categories with abnormally high call volume with striking red or orange colors to provide a quick snapshot of the overall behavior composition. The right side displays the number of active sessions and the number of users at the same time, which is convenient for correlating the behavior popularity with the usage scale. The session statistics table below is expanded by session and records the number of calls in each session in each dimension of behavior. This allows you to locate the sessions and users in which high-frequency operations are concentrated. Tool Call Distribution The tool invocation distribution page presents the tool usage structure from two perspectives. The pie chart on the left shows the type proportion of all tool calls (such as Read, Write, Bash, TodoWrite, etc.) to help the team understand which tool capabilities the agent relies on most. The pie chart on the right shows the distribution of MCP tool calls independently, revealing which external capabilities are frequently called in cross-system integration. The trend comparison chart below shows the changes in the number of calls for each tool type in a timeline, making it easy to identify phased changes in call patterns-for example, a surge in Bash calls on a certain day may indicate batch script tasks or abnormal behavior. Security Audit Overview The Overview page compresses the security situation of AI agents into a screen-readable risk snapshot based on the multi-dimensional high-risk operation count within a specified time window. The funnel on the left side gradually converges from full sessions to sessions with security risks. This visually shows the proportion of risk surfaces. On the right side, metrics such as high-risk command execution, outbound web requests, outbound command-line requests, sensitive file access, and prompt injection are displayed side by side. With the comparison data, the secur
业内分析认为，AI算力需求与绿色数据中心将成为行业主旋律
如果您正在寻找优质的日本VPS，欢迎访问 www.isclouder.com 了解更多
