dsh-vision-tool
Agent 与会话 活跃维护

dsh-vision-tool

visail/dsh-vision-tool

支持用户将图片粘贴至聊天框即可让纯文本多模态模型识别图像内容,自动改写粘贴的图片,analyze_image工具默认路由至Kimi视觉模型,无需额外配置即可补全纯文本多模态接口的视觉能力。

1
Stars 标星
0
Forks 分支
1
Watchers 关注
0
Open Issues
JavaScript
主要语言
MIT
开源协议
14 KB
仓库大小
1 个月前
最后推送
一键安装扩展 / 插件指令
dsh plugin --profile web add github:visail/dsh-vision-tool
git clone https://github.com/visail/dsh-vision-tool.git
git clone git@github.com:visail/dsh-vision-tool.git
README.md main

dsh-vision-tool

Image routing for text-only models in DeepSeek Harness (DSH).

Text-only models (e.g. deepseek-v4-flash) cannot see images. This bundle gives
them eyes in two coordinated steps:

  1. vision-prompt — shadows POST /api/session.prompt. When the active
    session model does not support image input, pasted images are persisted as
    content-addressed attachments and rewritten in place into text prompts that
    carry the full attachment reference JSON. Any other request (no image, or a
    model that already supports images) is forwarded unchanged, with the
    official /api trust fence (DNS rebinding / cross-site defense) reimplemented.
  2. vision-tool — registers a global analyze_image tool. The model calls
    it with the attachment reference (or a local file path); the tool routes
    the image to a vision model and returns its text answer.

Verified end-to-end: paste an image → attachment persisted → model calls
analyze_image → vision model describes the image.

Install

Requires the dsh CLI (see package and install a plugin).

# from npm (once published)
dsh plugin --profile <name> add dsh-vision-tool

# straight from a git host (pin a commit; no build step needed — pure ESM)
dsh plugin --profile <name> add github:<you>/dsh-vision-tool#<sha>

# or from a local tarball
pnpm pack
dsh plugin --profile <name> add ./dsh-vision-tool-0.1.0.tgz

The bundle's cordis.patch.yml inserts two rows: vision-tool and
vision-prompt. Restart the profile afterwards:

dsh --profile <name>

Verify the layer landed without booting:

dsh --profile <name> --dump-config   # look for the "# == dsh-vision-tool" layer

Configuration

Credential (required)

The tool resolves KIMI_CODE_API_KEY — from $DSH_HOME/.credentials.yaml or an
environment variable of the same name. Get the key from your
Kimi Code subscription page (sk-kimi- prefix).

# $DSH_HOME/.credentials.yaml
KIMI_CODE_API_KEY: sk-kimi-...

Switching vision models

The defaults target kimi-for-coding at https://api.kimi.com/coding/v1.
Override the row in your profile's cordis.patch.yml (a patch replaces the
whole config, so restate every key you keep):

- id: vision-tool
  name: dsh-vision-tool
  config:
    baseURL: https://api.kimi.com/coding/v1
    model: kimi-for-coding
    apiKeyEnv: KIMI_CODE_API_KEY
    maxImageBytes: 20971520
    timeoutMs: 120000

Note: kimi-for-coding only accepts temperature: 1 (anything else is
rejected with HTTP 400). The tool hard-codes temperature: 1 as its default
and is not configurable for this model. Other OpenAI-compatible vision
endpoints generally work as long as they accept temperature: 1.

Supported inputs

  • attachment — full reference JSON injected by the paste-rewrite mechanism
    ({"attachmentId":"sha256:...","mediaType":...,"bytes":N,"width":N,"height":N}).
    Pass it verbatim; do not strip fields.
  • path — local image file (absolute, or relative to the session cwd).
  • Formats: png / jpg / jpeg / webp / gif. Local files up to
    maxImageBytes (default 20 MB). Attachments are bounded by the harness
    attachment store limits.

Security

  • vision-prompt reimplements the official /api trust fence: loopback /
    trustedHosts host check, sec-fetch-site and Origin checks.
  • Request bodies are capped at 160 MB (413 otherwise), matching the harness
    http-bridge default.
  • Any failure degrades to passthrough — the original request is forwarded
    unchanged, never swallowed or mangled.
  • The tool only reads the attachment you reference and your configured
    credential; it never stores prompt or image content beyond the attachment
    store the harness itself maintains.

Diagnostics

Both plugins append to $DSH_HOME/vision-trace.log:

handle: rewrite result = REWRITTEN
vision-tool: execute: resolve KIMI_CODE_API_KEY -> source=file len=72

License

MIT