dsh-analytics

English | 中文

Agent FinOps / token analytics for DeepSeek Harness: collect usage from session
events into a local SQLite ledger, price it with a time-aware pricing table
(never hardcoded in logic), and query it through a service, an agent tool, and
JSON API routes.

This is the v1 scope from the design: Overview + Session drill-down + Cost
Engine + Cache analytics, packaged the way the dsh-plugin ecosystem ships
bundles (npm package with dsh.bundle → cordis.patch.yml).

Install

dsh plugin --profile web add dsh-analytics

From git (requires the prepare build allowance):

dsh plugin --profile web add github:you/dsh-analytics#<sha>

From a local checkout:

dsh plugin --profile web add /path/to/dsh-analytics

For local development, mount the source overlay:

dsh --profile web --patch ./cordis.yml

Configuration

The plugin row's config (see cordis.yml for the shape):

Pricing: the table is data, not code

Costs are computed by matching each request against pricing rows:

interface PricingRow {
  provider: string          // request/header config provider
  model: string             // request/header config model
  region?: string           // optional discriminator
  priceType: 'peak' | 'off-peak' | 'flat'
  inputType: 'cache_hit' | 'cache_miss' | 'cache_write' | 'output'
  pricePerMillion: number   // per 1M tokens, in `currency`
  currency: string
  effectiveFrom: string     // ISO 8601, inclusive
  effectiveTo?: string      // ISO 8601, exclusive
}

The match uses model + timestamp + cache hit/miss + input/output, so a
request at time T is always billed with the rows in force at T — later
repricing never rewrites history. cache_write falls back to cache_miss
when no dedicated row exists. Among matching rows the one with the latest
effectiveFrom wins.

The shipped defaults follow DeepSeek's V4 announcement: flat rates until
2026-08-16T16:00:00Z, then peak/off-peak rates (off-peak = 50% of peak).
They are seed data only — supply pricing or pricingFile to own the table.
Without configuration, defaults are seeded only when the table is empty, so a
restart without a pricing config never overwrites recorded prices.

What is collected

The plugin listens to session/event and replays persisted sessions through
ctx.sessionQuery on startup (live-only when the seam is absent):

• assistant/message usage → one request row per (session, seq)
  (deduplicated, replay-safe)
• request/header → provider / model / reasoning effort for pricing
• tool/call + tool/result → tool call rows with error flags
• session headers → created-at, cwd, parent for grouping

Storage is a local SQLite database (dsh_analytics_requests,
dsh_analytics_tool_calls, dsh_analytics_sessions,
dsh_analytics_pricing). Nothing leaves the machine; the plugin never calls
the provider API.

Service: ctx.analytics

All reads are detached snapshots; the service never touches the session store
or the agent loop:

await ctx.analytics.overview({ start, end })   // totals, cost, cache, reasoning, trend, byModel, bySession, budget
await ctx.analytics.session(sessionId)          // request rows, turn waterfall, tools, cache
await ctx.analytics.sessions({ start, end })    // per-session summaries
await ctx.analytics.models({ start, end })      // per-provider/model summaries
await ctx.analytics.tools({ start, end })       // per-tool calls/errors + step-attributed cost
await ctx.analytics.pricing()                   // the pricing table in force
await ctx.analytics.budget()                    // spend vs configured limits + projection

Tool cost is step-level attribution: a model call's cost is distributed
evenly across the tools called in that step, so a step with several tools
shares its cost (never double counted).

Agent tool: analytics_query

The model can ask for the same numbers:

analytics_query(query="overview", range_hours=24)
analytics_query(query="session", session_id="session-1")
analytics_query(query="models" | "sessions" | "tools" | "reasoning" | "agents" | "insights" | "inflation" | "pricing" | "budget", range_hours=24)

range_hours bounds the window ending now (0 = all time, default 24).

Web API

When ctx.webServer is mounted, read-only JSON routes register at
/api/analytics/overview|sessions|models|tools|pricing|budget (with an
optional ?hours= query) and /api/analytics/session/<sessionId>. These are
the data source for a future dashboard page.

Browser dashboard

The same routes power a self-contained dashboard served by the plugin at
/analytics (open the harness web server URL in any browser, or
http://127.0.0.1:<port>/analytics in a headless profile that mounts
ctx.webServer). It is a zero-build vanilla JS app shipped inside the
package (web/):

• Overview — KPI cards (cost, tokens, cache hit rate, reasoning share),
  token/cost trend, composition donut, cache hit-rate trend, cost by model, sessions, budget
• Sessions — list with drill-down: turn waterfall with duration/outcome,
  cumulative context chart, per-turn cost bars, tools, cache
• Reasoning — low/high/max efficiency: calls, success rate, avg duration,
  cost, and cost per success
• Agents — parent/child subagent cost tree with per-node cost bars and a
  top-level cost donut (click any node to drill down)
• Insights — rule-generated optimization suggestions (reasoning effort,
  cache, tool concentration, budget, subagent share, context growth)
• Context Inflation — tool results re-sent in later requests: duplicated
  tokens, avoidable cost, and per-tool share
• Token Flow / Models / Cost / Pricing — per-bucket trend, per-model
  aggregates, per-tool step attribution, latest-rate price comparison, and the pricing table

The range selector (6h/24h/7d/30d/all) applies to every page; the shell-nav
integration inside the harness client is a follow-up (the harness client has
no free plugin page slot today). The top bar also has an EN / 中文 toggle
(persisted across visits; ?lang=zh forces Chinese).

In-shell entry (web profile)

The package also ships a browser client bundle (dsh.client →
exports["./client"], built to the harness module-loader contract). In a web
profile it registers three entries:

• a Token Analytics action beside Settings at the sidebar foot, opening a
  full-screen in-app panel with the same six pages as /analytics
  (Overview / Sessions / Token Flow / Models / Cost / Pricing, with a range
  selector and refresh);
• a per-session Analytics action in the conversation header, opening the
  task-cost overlay (cost, tokens, cache hit rate, turn waterfall, tool
  attribution, and a link to the full dashboard);
• a per-turn cost line under each closed turn's final chat message
  ("tokens · cost · duration"), rendered by the conversation.chat.turnTail entry.

Both read the same read-only JSON routes, so they need the plugin's host half
(and its web routes) mounted. No harness client shell change is required:
the sidebar entry uses the shell's existing sidebar.footer.action hole.

Development

pnpm install
pnpm test
pnpm lint
pnpm build
pnpm doctor

Visual QA: run pnpm preview in one terminal, then
node scripts/visual-qa.mjs — it drives headless Edge over the dashboard and
reports element overlaps, SVG chart-label collisions, text overflow, and the
font-size scale.

Roadmap

• Analytics entry inside the harness client shell (left-nav integration; the
  client has no free plugin page slot today, so the dashboard currently lives
  at /analytics)
• Skill / sub-agent ROI and reasoning-effort efficiency
• Cost anomaly alerts from the budget signals

License

MIT