dsh-vision-bridge — DSH Web GUI dynamic vision bridge

English | 中文

A cordis plugin for the DSH (DeepSeek Harness) Web GUI that enables pasted
images: the UI accepts image attachments, and the main model (DeepSeek) decides
on demand which vision model to call — qwen3-vl-flash for visual Q&A and
qwen-vl-ocr for plain text extraction — via Alibaba Bailian's OpenAI-compatible
endpoint.

Features

• Pasted images in the GUI — registers a deepseek-vision wrapping route
  declaring inputModalities: ['text', 'image'], so the chat box accepts image
  paste.
• Dynamic vision (zero pre-translation) — pasted images are not
  pre-transcribed; the image block is replaced with a hint (sha256 attachment
  id) and the main model writes the vision prompt per question/context before
  calling the tool. No extra calls, no upfront guessing.
• Two vision tools:
  • describe_image(image, prompt) — visual Q&A (qwen3-vl-flash), prompt
    written by the main model on each call.
  • extract_text(image) — OCR (qwen-vl-ocr), extracts all text in reading
    order.
• Three image input forms — http(s) URL / local file path / pasted
  attachment id (sha256:…).
• Robustness — MIME sniffing (PNG/JPEG/GIF/WebP), 20 MB cap, explicit
  401/403 hints, timeout and empty-response guards.

What it does

Architecture

The plugin is a single-file cordis native plugin that wraps the official
deepseek-official adapter into a deepseek-vision route, and calls Bailian
directly from the tool implementations.

flowchart LR
  U[User pastes an image] --> G[GUI allows it<br/>inputModalities text+image]
  G --> W[deepseek-vision route<br/>wraps deepseek-official]
  W --> H[hintBlocks: image block →<br/>text hint with sha256 id]
  H --> M[Main model DeepSeek<br/>sees the hint]
  M -->|visual Q&A| D[describe_image<br/>qwen3-vl-flash]
  M -->|extract text| E[extract_text<br/>qwen-vl-ocr]
  D --> B[Bailian compatible-mode<br/>/chat/completions]
  E --> B
  B --> R[Text result back into chat]

Key design point: the vision prompt is fully decided by DeepSeek on every
call, based on the user's question and conversation context — asking about the
"amount" vs. the "tax number" on the same invoice sends different prompts,
maximizing information and minimizing calls.

Usage

Drop the plugin into your DSH Web profile plugin directory:

cp vision-bridge.js ~/.dsh/profiles/web/plugins/

It depends on the llm and tools services and requires the official
llm-deepseek adapter (deepseek-official) to be loaded — the plugin wraps it
to register the deepseek-vision route.

Configure your Bailian API key in ~/.dsh/.env (either works):

DASHSCOPE_API_KEY=sk-xxxx
# or
VISION_API_KEY=sk-xxxx

Get a key from the Alibaba Cloud Bailian console.

Development

No build step — the plugin is a single plain-JS file. Edit, restart the web
server, done. The model names are constants at the top of the file
(VISION_MODEL / OCR_MODEL); swap them for other Bailian models (e.g.
qwen-vl-max, qwen3-vl-plus) and restart.

Notes

• Attachments must have been sent in the session before they can be read (the
  plugin looks them up by sha256 in the DSH attachment store).
• This plugin only solves seeing images — the main DeepSeek model has no
  vision input, so DSH's built-in read_image tool is not usable on its own.

License

MIT