DSH Skill Trace: Agent Skill Loading Visibility Plugin

What changed

The DSH Skill Trace plugin, developed by PolinniZhong, aims to provide users with a clear view of the skills an agent loads during a conversation. It transforms the agent's operational process into a reviewable and learnable local record, referred to as a "skill receipt." This addresses the common issue where users only see the final result of an agent's action, without understanding which skills were invoked, the steps taken, or the dependencies involved.

The plugin offers three core interfaces: "Skill Receipt," "Flow Map," and "My Skills." The "Skill Receipt" presents a chronological log of turns, steps, skill loading requests, results, and their status, distinguishing between successful loads, failures, and unobserved states. The "Flow Map" organizes this information into a node-and-relationship structure, illustrating which turn requested which skill, and the success or failure outcomes, without introducing AI-driven process inference. "My Skills" acts as a searchable index and workbench for discovered skills within the current agent preset scope. It displays skill declarations, user-added understanding, verification plans, and manual results, differentiating between actual receipts, session understanding, and historical data.

Key functionalities include observing skill(name) calls and results, differentiating between requests, success, failure, and unknown states. The plugin supports a learning loop where users can review skill declarations, compare them with actual receipts, record their understanding and verification plans, and document manual verification results. It emphasizes a local-first approach to privacy, ensuring that complete prompts, skill bodies, tokens, absolute paths, or project content are not saved. The interface language dynamically adjusts to the DeepSeek Harness settings.

Recent updates to the plugin have focused on robustness and expanded observational capabilities. Version 0.4.0-beta.7 resolved an issue with DeepSeek Harness session format migration (V3 to V4) that caused silent evidence loss by ensuring compatibility with both formats. Version 0.4.0-beta.8 expanded the observation surface to include skills explicitly loaded by users via commands (e.g., /character-asset-kit), which were previously not tracked. It also began recording the DeepSeek Harness persistent skill directory as a declaration baseline and captured limited runtime evidence for tool calls, including invocation, CLI, MCP, and subagent calls.

Further development in 0.4.0-beta.9 normalized runtime events into a unified RuntimeEvent model, aggregated by invocationId. This version also shifted runtime evidence persistence to the end of a turn, as it can be reconstructed from session logs, preventing excessive writes. 0.4.0-beta.10 introduced an association engine and graph reconstruction, prioritizing accuracy over completeness, with a droppedEdgeCount for unresolvable links and unlinked calls for unassigned invocations. It uses a closed vocabulary for edges, avoiding causal language. Version 0.4.0-beta.11 implemented "Declaration ↔ Runtime Alignment," focusing on evidence reporting without scoring, acknowledging that insufficient evidence does not equate to non-execution, and distinguishing direct from generalized evidence. It also improved skill declaration step extraction, now using a dual-channel approach (heading hierarchy + ordered lists) to capture actual process steps more accurately.

Most recently, 0.4.0-beta.13 introduced the "Runtime Graph Canvas," the third view in the plugin. This canvas implements grouping rules to manage graph scale, folding calls within a turn, collapsing turns into intervals, and converting rows to columns to keep the canvas within a manageable size. The layout is deterministic, ensuring the same receipt always renders the same graph. A "checker" provides explanations for each node and edge, clarifying their meaning and what they do not represent, with a significant reduction in response size for large sessions.

Why it matters for builders

This plugin provides builders with unprecedented transparency into how their agents select and utilize skills. By offering detailed logs and visual representations of skill invocations, developers can move beyond simply observing outcomes to understanding the underlying decision-making process. This granular insight is invaluable for debugging complex agent behaviors, identifying inefficiencies in skill chaining, and validating that skills are being used as intended.

Furthermore, the plugin's emphasis on user-recorded understanding and verification plans fosters a more iterative and informed development cycle. Builders can use the recorded evidence to systematically test hypotheses about skill performance, document their findings, and build a personal knowledge base around agent capabilities. This empowers them to refine agent logic, improve skill design, and ultimately build more reliable and predictable AI agents.

Practical impact

Builders can install the DSH Skill Trace plugin via GitHub using the command dsh plugin --profile web add "github:PolinniZhong/dsh-skill-trace#v0.4.0-beta.13&path:/". After installation and restarting DeepSeek Harness Desktop, users can activate the plugin within a session. By running a task and observing the "Skill Receipt" and "Flow Map," developers can analyze which skills were invoked, their sequence, and their outcomes. They can then utilize the "My Skills" interface to record their personal understanding, identify potential improvements, and formulate verification plans for specific skills.

For local development and fallback installation, the project can be cloned using git clone https://github.com/PolinniZhong/dsh-skill-trace.git, followed by cd dsh-skill-trace, npm test, npm run verify, and then installing via a local link: dsh plugin --profile web add "link:$(pwd)". To remove the plugin, the command dsh plugin --profile web remove dsh-skill-trace can be used.

Caveats and source limits

The plugin is currently in a pre-release state (0.4.0-beta.13). While extensive automated tests cover various aspects of its functionality, some aspects are still pending verification, including real-user feedback after closing materials, 24-hour re-testing, navigation of extremely long process maps, and system-level accessibility audits. The plugin explicitly states that manual entries of "meets expectations" only represent the user's record for that specific verification and should not be interpreted as proof of user understanding, universal skill effectiveness, or guaranteed valid output from loading. The plugin does not score skills or provide performance ratings, focusing solely on reporting evidence status counts. It also clarifies that it does not generate or publish modified skills, serving only to aid understanding, record improvement intentions, and prepare verification checklists.