dsh-headless-json

Structured, machine-readable CI output for DeepSeek Harness (dsh).

中文文档：README.zh.md

dsh's built-in headless mode prints the final assistant text and exits 0/1 —
perfect for a quick smoke test, but thin for CI. dsh-headless-json is a
profile bundle that turns any dsh session into a first-class CI artifact:

• a transaction-level JSON report (session metadata, every structured
  event, outcome, statistics),
• an append-only NDJSON event stream you can tail while the run is live,
• a JUnit XML report for GitLab CI / Jenkins / Azure DevOps / etc.,
• semantic exit codes that distinguish success, failure, timeout,
  blocking, abort and interruption,
• an artifact manifest of file paths referenced in the conversation,
• a privacy layer (text truncation, argument hiding, secret masking,
  path relativization),
• and a dual entry point: dsh tools inside the runtime
  (output_status, output_events, set_options) plus a standalone CLI
  for offline rendering and exit-code plumbing.

Everything is deterministic: given the same session log and the same options,
the produced bytes are identical.

Contents

• Features
• How it works
• Bundle layout
• Installation & integration
• What a run produces
• dsh tools
• CLI reference
• Configuration
• Output formats
  • JSON report
  • Events
  • NDJSON stream
  • JUnit XML
• Exit code semantics
• Privacy & redaction
• Artifacts
• Determinism
• Development
• Limitations & compatibility

Features

How it works

The bundle is a dsh profile bundle: an npm package whose manifest declares

"dsh": { "bundle": { "patch": "./cordis.patch.yml" } }

The patch inserts one plugin (headless-json) into the loader tree. The
plugin subscribes to the session service events, keeps one capture per
session, streams NDJSON lines as events arrive, and writes the JSON/JUnit
reports when the session is disposed (which happens during dsh's normal
graceful teardown).

dsh session event stream
        │  session/created · session/event · session/flush · session/disposed
        ▼
┌─────────────────── CaptureManager ───────────────────┐
│  per-session Capture: derive structured events       │
│  (turn/step/tool latency, tokens, model, errors)     │
│  + artifact scan + type distribution                │
└────────────┬──────────────────────────┬─────────────┘
             │ live NDJSON lines        │ on session end
             ▼                          ▼
       events.ndjson            report.json · junit.xml

• Capture vs. output: events are derived once, in memory; redaction is
  applied at output time, so changing options with set_options affects
  every subsequent write.
• No stdout interference: the plugin never writes to stdout, so it
  composes cleanly with the official headless runner (which owns stdout).
• Error containment: every listener is wrapped; a malformed event can
  never crash the tree or corrupt a report.

Bundle layout

headless-json/
├── package.json              # dsh.bundle.patch manifest + bin
├── cordis.patch.yml          # loader patch (inserts the plugin row)
├── lib/                      # compiled ESM (tsc output)
├── bin/dsh-headless-json.js  # CLI entry
├── src/                      # TypeScript sources (repo only)
├── test/                     # node:test suite (repo only)
├── examples/                 # sample session log + generated outputs
├── README.md / README.zh.md
└── LICENSE                   # MIT

The repository tree is shown above; the published npm package ships lib,
bin, cordis.patch.yml, examples, both READMEs and LICENSE.

Installation & integration

Requirements: Node.js ≥ 18, a dsh installation that composes the sessions
service (any base/headless profile does).

1. Build (only when developing from source)

npm install
npm run build        # or: npm test  (builds + runs the suite)

2. Add the bundle to a dsh profile

Using the CLI (forwards to pnpm):

dsh plugin --profile headless add /path/to/headless-json

or straight from this repository:

dsh plugin --profile headless add github:JohnXu22786/headless-json

or manually: add dsh-headless-json to the profile's package.json
dependencies and to the ordered dsh.profile.bundles list, then
pnpm install in the profile directory. dsh reconciles installed bundles
against dsh.profile.bundles on the next run.

3. Run

dsh --profile headless "run the test suite"

After the run completes you get, in dsh-output/ (relative to the working
directory):

dsh-output/report.json    # transaction-level JSON report
dsh-output/junit.xml      # JUnit XML report
dsh-output/events.ndjson  # only when output.ndjson = true

4. Wire CI

The semantic exit code lives in the report; the CLI turns it into a process
exit:

dsh --profile headless "run the test suite"
code=$(dsh-headless-json exit dsh-output/report.json)
exit $code

or pipe a JUnit report straight into your CI collector:

dsh-headless-json render dsh-output/report.json --format junit --out junit.xml

What a run produces

Per session, the plugin writes:

File names honor a {session} placeholder (e.g.
"json_file": "report-{session}.json"); without it, a second session in the
same process gets a -<id> suffix before the extension. A session that
recorded zero events writes nothing when output.write_empty is false.

dsh tools

The bundle registers three tools (via ctx.tools.register) visible to every
agent. They are resolved against the calling agent's session.

Example conversation:

set_options({ "redact": { "args": "hide" } })
→ { "applied": true, "redact": { "args": "hide", ... } }

CLI reference

dsh-headless-json render <input> [options]   Render a report/event file
dsh-headless-json exit <input> [options]     Print the semantic exit code
dsh-headless-json --version | --help

<input> may be:

• a report.json produced by the plugin,
• an events.ndjson stream produced by the plugin,
• a raw dsh session .jsonl log (header line + session/* event lines),
• a JSON array of raw session events.

Notes:

• exit prints the code and exits the process with it — the CI glue.
• Redaction --set overrides re-render raw session logs only. An
  events.ndjson stream and a report.json are already redacted (redaction
  was applied when each line/file was written at capture time) and are
  re-rendered verbatim.

Configuration

All options are snake_case, configurable through the bundle's patch config
block, the set_options tool, or CLI --set. Unknown keys are tolerated in
configuration files (forward-compatible deployments); set_options and CLI
--set reject unknown keys, including unknown nested keys.

# cordis.patch.yml (profile overlay example)
- id: headless-json
  config:
    output:
      dir: dsh-output
      json: true
      junit: true
      ndjson: true
    redact:
      text_length: 4000
      args: truncate
      paths: relative
      secrets: true

Output formats

JSON report

{
  "schema_version": 1,
  "plugin": { "name": "dsh-headless-json", "version": "0.1.0" },
  "generated_at": 1753000000000,          // session end time (derived, deterministic)
  "session": {
    "id": "...", "cwd": ".", "cwd_name": "repo",
    "created_at": ..., "started_at": ..., "ended_at": ...,
    "event_count": 42,
    "parent_session": null, "agent_preset": "minimal", "delegation_depth": null
  },
  "outcome": {
    "status": "success",                  // stable category
    "exit_code": 0,                       // semantic exit code
    "reason": "completed",                // raw turn/end reason kind
    "complete": true,
    "undelivered_tool_calls": [],
    "error": null                         // {code, message, status?} on error
  },
  "stats": {
    "duration_ms": 4123.5,
    "turns": 1, "steps": 2,
    "assistant_messages": 3, "user_messages": 1,
    "tool_calls": 2, "tool_errors": 0, "undelivered_tool_calls": 0,
    "chunk_count": 14,
    "tokens": { "input": 900, "output": 300, "cache_read": 0, "cache_write": 0, "reasoning": 0 },
    "events_by_type": { "assistant/message": 3, "tool/call": 2, ... },  // raw dsh types
    "by_tool": { "bash": { "calls": 2, "errors": 0, "latency_ms_total": 800, "latency_ms_max": 500 } }
  },
  "events": [ /* see below */ ],
  "events_truncated": null,               // {kept,total,dropped} when trimmed
  "artifacts": [ { "path": "src/options.ts", "kind": "file", "size": 4281, "references": 3 } ]
}

Events

Every event carries seq (session-log sequence, the stable sort key),
time (unix ms) and kind:

Tool calls and results are correlated by call_id; latencies are computed
from the session log timestamps. A tool_call whose tool/result never
arrived by session end is flagged undelivered: true and listed in
outcome.undelivered_tool_calls.

NDJSON stream

One JSON object per line, in arrival order, written immediately:
the same structured events plus a final summary line:

{"kind":"session_end","session_id":"...","generated_at":...,
 "outcome":{"status":"success","exit_code":0,"reason":"completed"},
 "stats":{"duration_ms":4123,"turns":1,"steps":2,"tool_calls":2,"tool_errors":0}}

The file is fsynced on every session/flush checkpoint, so consumers that
read storage after whenIdle() see a durable stream. A tool_call line is
streamed before its result arrives, so the latency_ms / undelivered
fields on those lines are provisional; the correlating tool_result line (or
the final report) carries the authoritative values.

JUnit XML

Timing is in decimal seconds, timestamps are ISO-8601 UTC, and all text is
XML-1.0-escaped (control characters replaced).

Exit code semantics

The final turn/end reason maps to a stable category and a documented exit
code; every value is overridable via exit.*:

Sessions with events but no closing turn/end are reported as
error/incomplete. The 130 code intentionally follows the SIGINT
convention for user-initiated stops.

Privacy & redaction

Defaults are conservative but usable. Everything applies at output time.

• Text truncation — every text/reasoning value is capped at
  redact.text_length with a deterministic …[+N more chars] marker.
• Argument handling — redact.args:
  • full: the parsed argument object, string values masked/truncated;
  • truncate (default): a masked summary capped at redact.arg_length;
  • hide: the literal string [hidden].
• Secret masking (redact.secrets) — built-ins cover sk-… keys,
  Bearer … headers, PEM private keys, GitHub/Google tokens, JWTs, long hex
  values (keeps a short prefix for e.g. commit SHAs) and token-like strings.
  Turn/end error messages are treated as free text and masked too. Add your
  own with redact.secret_patterns (regex source strings, validated at
  configuration time).
• Path relativization — redact.paths: relative (default) strips the
  session cwd from artifact paths and renders session.cwd as .;
  absolute keeps full paths.

What this means in practice: the default output contains no absolute
workspace paths, no raw secret strings, and no unbounded tool output.

Artifacts

When artifacts.collect is on, every user/assistant/tool text is scanned
for path-like candidates (POSIX/Windows absolute paths and
separator-containing relative paths; URLs and emails are excluded). Each
unique candidate becomes an artifact entry with a reference count; when
artifacts.check_exists is on (the default) and the run happens on the same
machine, kind (file/dir/missing) and size are filled in from the
filesystem. The manifest is bounded by artifacts.max_entries.

Determinism

Given the same session events and the same effective options:

• the JSON report is byte-identical across runs — object key order is
  fixed by construction, events are sorted by seq, map-like fields are
  serialized with sorted keys, and numbers are rounded to 3 decimals;
• JUnit XML and NDJSON lines are produced from the same serialization core,
  so all three formats always agree.

generated_at (and the JUnit timestamp) are derived from the session's own
timeline — the last event time — so they never depend on a wall clock.

Development

npm install
npm run build        # tsc -> lib/
npm test             # build + run the full suite (node:test)
npm run test:only    # run tests against the current build
npm run typecheck    # tsc --noEmit

The test suite covers event subscription wiring (against a minimal fake
context), capture derivation, serialization determinism, redaction, exit-code
mapping, JUnit structure/escaping, NDJSON round-tripping and the CLI
end-to-end.

Limitations & compatibility

• Preview APIs. dsh is a developer preview; session event shapes and
  service semantics may evolve. The capture pipeline reads every event
  defensively (malformed input is counted, never fatal) and the session
  event vocabulary used here is documented in the published session typings.
• Tool schema shape. The three dsh tools are registered as plain
  definitions matching the shape ctx.tools.register expects; if a future
  dsh version changes the definition DSL, only src/tools.ts needs updating.
  When ctx.tools is absent from a composition, the plugin still captures
  and reports — only the tools are skipped (with a log warning).
• Exit codes vs. dsh headless. The official headless runner exits 0/1 by
  its own contract; this plugin does not touch headlessIo. Use the
  exit command (or read report.json) for the fine-grained codes.
• Re-rendering redaction. Only raw session logs can be re-rendered with a
  different --set redaction (redaction is a capture-time property of
  report.json and NDJSON lines).

License

MIT — see LICENSE.