dsh-local-perf
Durable DeepSeek Harness plugin bundle carrying the local-model performance
tuning process as a re-installable layer — so it survives dsh updates
instead of living in hand-edited patch files that a version bump can wipe or
whose rationale dies with the author.
Install once, re-apply forever:
# from the GitHub repository
git clone https://github.com/flowingboy/dsh-local-perf.git /tmp/dsh-local-perf
dsh plugin --profile web add file:/tmp/dsh-local-perf
# or from a checkout: dsh plugin --profile web add file:./dsh-local-perfThe bundle joins the profile's layer stack after dsh-base / dsh-web-app,
re-applies all tuning rows on every boot, and vendors its own copy of the
text-toolcall-guard plugin (self-contained, no dsh checkout required).
The complete performance process (why every knob exists)
Everything below was learned the hard way on an M5 Max running four local
OpenAI-compatible servers (Ollama / MLX / Rapid-MLX / MLX-DSpark) against the
DSH web GUI. The bundle encodes the conclusions; this README preserves the
reasoning.
Incident log
| Date | Symptom | Root cause | Fix |
|---|---|---|---|
| 2026-08-18 | "fan spin / no response" | A 114,650-token prefill on Qwen3.5-122B — session context had grown unbounded because the web bundle disables auto-compaction | Re-enable compaction-basic at thresholdRatio: 0.6 |
| 2026-08-19 | session-title starves the interactive step | The title LLM request fires in the same second as a turn's first step; on the single-slot mlx-dspark server it queues behind the interactive generation and one of them starves past the idle budget | Route session-title-llm to the cloud model (deepseek-official / deepseek-v4-flash) |
| 2026-08-19 | mlxdspark timeouts with tools present | The mlx-dspark server buffers the whole generation and emits no data events until it finishes; its 15s SSE keepalive comments are discarded by the OpenAI SDK parser and never reset the idle watchdog | Raise timeoutMs / streamIdleTimeoutMs to 600000 |
| 2026-08-20 | tool calls appear as literal text | Local Qwen3.8-27B-8bit fell out of the structured tool_calls protocol under long tool-heavy steps and wrote <tool_call> prose the harness never executes |
Ship the text-toolcall-guard plugin (vendored here) |
| recurring | hallucinated "today" | No clock context in the prompt | Enable time-context (Asia/Shanghai, 10 min refresh) |
Layer 1 — model configs (settings.example.yaml → ~/.dsh/settings.yaml)
Machine-specific (paths, ports, model ids), so the bundle carries them as a
template, not a runtime patch. Copy the llm-pi-ai section into
~/.dsh/settings.yaml on a fresh machine.
The recurring principles:
- Timeout ≠ prefill tolerance. A slow local server needs
timeoutMs+
streamIdleTimeoutMs≥ worst-case prefill + reasoning + decode. Gemma 4 31B
prefills at ~180 tok/s (system prompt + tool schemas ≈ 13k tokens → ~70s),
mlx-dspark buffers whole generations, so both budgets sit at 300–600s. - Retry only TRANSPORT. Connection-level failures happen before prefill
and are cheap; a TIMEOUT must never re-prefill a long prompt. - Context window ≤ practical prefill budget. 262144 tokens at ~180 tok/s
is minutes of prefill. Lower to 32K–64K; compaction at 0.6× keeps sessions
safely under the server limit. - maxTokens ≤ decode budget. At ~27 tok/s decode, 16K output is ~10 min.
Cap at 8192–16384 so one step's worst case fits the timeout budgets. - Reasoning effort default "off" (or the server's lowest level) for
quick, low-latency local loops; the local model's thinking stream still
renders as a DSH reasoning block when enabled.
Layer 2 — cordis rows (cordis.patch.yml)
| Row | What | Why |
|---|---|---|
time-context |
per-step clock | kills hallucinated dates |
compaction-basic |
auto-compact at 60% | bounds prefill; the 08-18 incident fix |
tool-result-pruner |
drop stale tool results | keeps them off later requests |
command-compact |
manual /compact |
escape hatch |
session-title-llm |
title via cloud model | keeps the local slot free for the interactive step |
Layer 3 — text-toolcall guard (plugins/text-toolcall-guard)
Vendored from @deepseek-ai/dsh-text-toolcall-guard (built lib/ + src/).
When a step closes with no native tool calls but the assistant text carries
<tool_call> / <function=…> markers at line starts, the guard steers a
corrective message so the model re-issues the call natively. Bounded to
maxCorrections (2) per turn per agent; marker detection requires line-start
placement so prose that merely quotes the format is not corrected. Peers are
resolved at runtime from the dsh installation's profiles/node_modules
fallback (the designed out-of-tree-plugin path), so no registry fetch needed.
Installation
# from this directory
dsh plugin --profile web add file:$(pwd)
# verify the layer joined the stack
dsh --profile web --dump-config | grep -A3 "dsh-local-perf"The web profile's cordis.patch.yml should then only hold rows this bundle
does NOT own (currently: none — everything moved into the bundle).
Updating the bundle
git pull # or edit locally
cd plugins/text-toolcall-guard # rebuild the vendored guard if its src changed
pnpm exec tsc -p tsconfig.json --outDir lib --declarationDir lib/types
# reinstall the layer
dsh plugin --profile web add file:$(pwd)Publishing (GitHub / dsh-plugin ecosystem)
Published at https://github.com/flowingboy/dsh-local-perf (public, main,
topics: dsh-plugin). To re-publish after local edits:
git add -A && git commit -m "dsh-local-perf: ..."
git push origin main
# topics (once)
gh repo edit dsh-local-perf --add-topic dsh-pluginLayout
cordis.patch.yml the perf layer (all tuning rows)
settings.example.yaml model-config template (copy to ~/.dsh/settings.yaml)
plugins/text-toolcall-guard/ vendored guard plugin (lib + src)
README.md this document — the preserved process