cortiq-router
其他 活跃维护

cortiq-router

infosave2007/cortiq-router

依托allaigate语义路由器,自动识别提示词的任务类型与复杂度,将LLM请求智能调度至最适配的模型,无需手动配置路由规则即可获得最优推理效果。

1
Stars 标星
0
Forks 分支
1
Watchers 关注
0
Open Issues
JavaScript
主要语言
MIT
开源协议
176 KB
仓库大小
1 个月前
最后推送
一键安装扩展 / 插件指令
dsh plugin --profile web add github:infosave2007/cortiq-router
git clone https://github.com/infosave2007/cortiq-router.git
git clone git@github.com:infosave2007/cortiq-router.git
README.md main
# Cortiq Router **Smart LLM request router for the DeepSeek Harness.** Every prompt is classified by task and difficulty, then answered by the model that fits it. [English](https://github.com/infosave2007/cortiq-router/blob/main/README.md) · [Русский](https://github.com/infosave2007/cortiq-router/blob/main/README.ru.md) · [中文](https://github.com/infosave2007/cortiq-router/blob/main/README.zh.md) [![npm](https://cdnimage-cache.doubi.ren/?url=https://img.shields.io/npm/v/cortiq-dsh-llm-router)](https://www.npmjs.com/package/cortiq-dsh-llm-router) [![license](https://cdnimage-cache.doubi.ren/?url=https://img.shields.io/badge/license-MIT-blue)](https://raw.githubusercontent.com/infosave2007/cortiq-router/main/LICENSE)

What it does

You stop choosing a model. The plugin reads each prompt, asks the
allaigate semantic router what kind of task it is
and how hard it is, and sends the call to the model your rules name for that
combination.

Your prompt Router says Answered by
"Привет, как дела?" chitchat, low a fast, cheap model
"Translate to French: Hello world" translation, low a fast, cheap model
"Write a Python one-liner that reverses a string" code, low a fast, cheap model
"Design a lock-free MPMC queue in Rust and prove linearizability" code, high your strongest model

Simple work stops paying premium prices, hard work stops being answered by a
model that cannot do it.


Install

dsh plugin --profile web add cortiq-dsh-llm-router

No global dsh? Use npx -p @deepseek-ai/dsh dsh plugin --profile web add cortiq-dsh-llm-router.

One prerequisite. The router picks among the models your other provider
plugins already serve
llm-deepseek, llm-pi-ai, or any other registered
provider. Keep at least one configured: this plugin never talks to a model
vendor itself, it only decides who should answer.


Quick start

  1. Get a key. Sign up at allaigate.com and copy your cortiq_… key.
  2. Store it. In the harness: Settings → Models → Cortiq Router, paste the key.
    Or export it: export CORTIQ_ROUTER_KEY=cortiq_…
  3. Pick the model Cortiq Router · Auto in the chat model selector.

That is the whole setup. The defaults route across DeepSeek models; point them at
your own model ids when yours differ (see Routing rules).


How it works

your prompt
    │
    ▼
┌──────────────────────────────┐
│ 1. Text extraction           │  only the last user turn by default —
└───────────┬──────────────────┘  the rest of the conversation stays local
            │  POST /v1/route
            ▼
┌──────────────────────────────┐
│ 2. Semantic router           │  task = code, complexity = 0.78 → high
└───────────┬──────────────────┘  router.allaigate.com
            │
            ▼
┌──────────────────────────────┐
│ 3. Your routing table        │  code @ high → ["your-strong-model", …]
└───────────┬──────────────────┘  an ordered chain, not a single name
            │
            ▼
┌──────────────────────────────┐
│ 4. The provider that serves  │  llm-deepseek, llm-pi-ai, …
│    that model answers        │  streamed back untouched
└──────────────────────────────┘

Nothing here is a single point of failure. No key, an unreachable router, a
timeout — the call still goes out, on the fallback model. A candidate whose
provider refuses (missing credential, unknown model) is stepped over and the
next candidate answers. Only a provider that has already started streaming keeps
the call.


Model picker entries

Four entries, all of them routing. The last three pin the policy for that
conversation without a trip to settings:

Entry Behaviour
Auto Uses the policy profile from settings
Auto · cost saver Prefers the cheap end of every chain
Auto · balanced The middle ground
Auto · quality first Escalates to strong models sooner

Routing rules

Two layers, checked in order: a per-task rule first, then the tier chain.

llm-cortiq-router:
  # Where your low / medium / high tiers sit on the router's 0…1 score
  complexityBands:
    low: 0.35
    medium: 0.65

  # Fallback chains per tier — tried in order
  globalTiers:
    low:    [deepseek-v4-flash]
    medium: [deepseek-v4-pro]
    high:   [deepseek-v4-pro]

  # Per-task overrides, applied before the chains above
  taskRules:
    code:
      low:  [deepseek-v4-flash]
      high: [claude-opus-5]
    translation: [deepseek-v4-flash]

  # Last resort: router unreachable, unpaid, or nothing else registered
  defaultModel: deepseek-v4-flash

Use the model ids your provider actually advertises. An OpenRouter route
names DeepSeek models deepseek/deepseek-v4-flash, while llm-deepseek names
the same model deepseek-v4-flash. A bare id is looked up in every registered
provider's catalog; write provider:model (e.g. openrouter:deepseek/deepseek-v4-pro)
to name a route exactly.


Settings

Setting Default What it does
apiKeyEnv CORTIQ_ROUTER_KEY Credential holding the allaigate key
routerUrl https://router.allaigate.com Classifier endpoint; point it at your own deployment
routerProfile balanced cost-saver / balanced / quality-first
textStrategy last_user What gets classified: last user turn, + system prompt, or the whole thread
maxChars 4000 Hard cap on characters sent to the classifier
timeoutMs 15000 Hard prompts escalate to an oracle (~10 s); on timeout the call runs on the fallback
complexityBands 0.35 / 0.65 Where low / medium / high sit on the score
globalTiers, taskRules, defaultModel see above Which model answers
language auto Copy language: en, ru, zh, de, fr, es, tr
echoRouting true Log every decision: task, tier, score, chosen provider/model

Self-hosting the classifier

Point routerUrl at your own deployment and prompts never leave your
infrastructure. The endpoint must serve POST /v1/route over TLS that Node
accepts — a private CA works when NODE_EXTRA_CA_CERTS names it; a self-signed
certificate does not (Node's fetch has no per-request opt-out).


Links

License

MIT