dsh-vision-bridge
Agent 与会话 活跃维护

dsh-vision-bridge

ShiraGawaAnri/dsh-vision-bridge

会话插件包,使用不支持图片的模型时,不再弹出功能拦截提示,自动调用工具完成视觉识别,无需手动切换模型也无需额外配置即可处理含图需求。

0
Stars 标星
0
Forks 分支
0
Watchers 关注
0
Open Issues
JavaScript
主要语言
MIT
开源协议
23 KB
仓库大小
1 个月前
最后推送
一键安装扩展 / 插件指令
dsh plugin --profile web add github:ShiraGawaAnri/dsh-vision-bridge
git clone https://github.com/ShiraGawaAnri/dsh-vision-bridge.git
git clone git@github.com:ShiraGawaAnri/dsh-vision-bridge.git
README.md main

中文 | English

dsh-vision-bridge

A DSH profile bundle for sessions running models that cannot accept image
input (DeepSeek and similar). Instead of the native "the current model does
not support images" rejection, the plugin runs the following pipeline:

  1. materializes the image into the session workspace (.dsh-vision-bridge/);
  2. rewrites the user message (via the sanctioned agent/pre-step channel) into
    a [图片桥接] instruction carrying the image paths;
  3. registers a system-prompt rule (order 150) requiring the model to call
    the configured vision tool (default vision_glance, fuzzy-selectable such
    as glance / ground / detect, matched against the agent's actual
    toolset) and answer from its analysis — the tool-call card and its result
    appear in the conversation;
  4. patches llm.streamWithRegistration — the dispatch funnel behind both
    llm.stream and the agent loop's prepared-call path — to strip image blocks
    (including images nested inside tool results), replacing them with a
    placeholder before a text-only adapter serializes the conversation.

Sessions whose agent preset matches excluded presets (default minimal,
the two-tool preset) are NOT bridged: their images are replaced with the
original "model does not support images" notice text. The preset a session
runs is resolved the same way DSH itself resolves it: the newest
agent-preset/selected event wins, with the creation-time header as fallback.

Why the native rejection happens (DSH itself)

  • dsh-host-apiproxy's prompt RPC rejects image messages before they enter
    the session when llm.resolveModelInfo().inputModalities lacks image
    (that is the banner shown in the UI).
  • The DeepSeek adapter throws UNSUPPORTED_CONTENT on any message containing
    image blocks at serialization time.

This plugin intervenes at three layers:

Layer Mechanism
Admission Patches llm.resolveModelInfo so bridged models report an image input modality
Current message agent/pre-step materializes the image and injects the vision-tool instruction
History Patches llm.streamWithRegistration (the shared dispatch funnel) to strip image blocks before adapter serialization

Settings (Settings → 视觉桥接)

Setting Default Meaning
Enable vision bridge on Off = restore native DSH blocking entirely
Vision tool (fuzzy input) vision_glance The tool the model must call; fuzzy-matched against the agent's real toolset
Custom prompt (optional) empty Extra instruction passed to the vision tool
Bridge models deepseek-official*,deepseek* provider[:model] patterns, * wildcards
Excluded session presets minimal Multi-select + extra wildcard patterns; matched sessions are NOT bridged (original blocking text)

Configuration is persisted through the DSH settings service (namespace
vision-bridge); it survives restarts.

Platforms: WSL/Ubuntu and Windows

  • Image bytes are written via ctx.shell with base64 on stdin (no huge
    command lines): bash (base64 -d) on Linux/WSL, PowerShell
    ([IO.File]::WriteAllBytes) on native Windows — the shell flavor is
    auto-detected once (Write-Output probe + resolveExecutable('pwsh')
    fallback).
  • Shell commands carry the session-resolved sandbox policy (like the bash
    tool): danger-full-access sessions run without a sandbox backend; confined
    sessions run inside the workspace fence when a backend exists.
  • The vision tools themselves are native DSH tools (vision-tools skill); their
    runtime already handles cross-platform execution.

Requirements

  • The session must have the vision-tools skill loaded (or another vision
    plugin) so a vision tool exists in the agent's toolset; otherwise the bridge
    degrades to a self-heal instruction (load the skill first) or a clear notice.
  • Standard/cordis presets have the skill tool; the minimal preset does not
    (it is excluded by default).

Installation

Unzip the bundle, then add it to a profile:

unzip dsh-vision-bridge.zip -d dsh-vision-bridge
dsh plugin --profile web add ./dsh-vision-bridge

(dsh plugin forwards to pnpm in the profile directory. Alternatively, copy
the extracted folder into the profile's node_modules/@dsh-external/ and
reconcile plugins.)

Known limitations

  • The admission patch is process-wide, so excluded sessions cannot get the
    native error banner — the same reason is rendered as text instead.
  • Multiple sessions share the plugin's global effects (llm patches, pre-step
    listener, system rule) while the plugin is mounted.
  • Bridged models never receive image blocks (the adapter forbids them); the
    attachment stays available in the session.