dsh-skillflux

Dynamic Skill Runtime Manager for
DeepSeek Harness.

SkillFlux keeps the full Skill pool outside the model-facing catalog. For each
task, it selects a small set of relevant Skills, mounts them for the current
turn, and releases the mounts when the turn ends.

When the local pool has no good match, SkillFlux can search skills.sh and the
public GitHub SKILL.md corpus live, then rank candidates with relevance-first
quality and 30-day repository activity signals before proposing a pinned mount.

Status: v0.2 for DeepSeek Harness 0.1.1-rc.2. Harness is still a
developer preview, so this project follows the current RC API.

中文文档

Why SkillFlux

A growing Skill library should not make every request carry a growing catalog.
Large catalogs consume context and make Skill selection less predictable.

SkillFlux acts as a runtime layer between the Agent and its Skill pool. The
automatic catalog contains no more than maxActiveSkills selected Skills
(three by default), while the official DSH Skill Registry remains the source
of truth. Explicit /skill-name invocations remain available separately.

Quick start

You need Node.js 22.20.0 or later and a DeepSeek Harness profile on the
0.1.1-rc.2 package line.

1. Install SkillFlux into the profile you use:

   dsh plugin --profile web add github:YiyuZh/dsh-skillflux

   Replace web with another profile name, such as headless, when needed.

2. Restart that Harness profile.

3. Verify the runtime from a DSH conversation:

   /skillflux status

For a reproducible deployment, pin a commit:

dsh plugin --profile web add github:YiyuZh/dsh-skillflux#<commit-sha>

The repository includes built lib/ artifacts, so GitHub installation doesn't
run a prepare script. The bundle patch disables the official tool-skill
consumer and mounts SkillFlux under the unique skillflux loader ID. It keeps
the official skill Registry and skill-filesystem provider active.

How it works

User task
  -> ordered rules + deterministic lexical router
  -> optional bounded usage-based tie-breaking
  -> optional embedding fallback for unfilled slots
  -> local Registry + persistent cache + online multi-source discovery
  -> relevance-first quality ranking with 30-day activity signals
  -> select and mount within the skill-count and optional catalog-token budgets
  -> Agent calls a mounted Skill
  -> unmount at turn/end
  -> keep downloaded files until explicit cleanup

A mount is scoped to the receiving Agent. Unmounting removes the runtime
registration from future catalogs; it doesn't delete cached files or text
already stored in session history.

Capabilities

• Route by ordered rules, then deterministic English and Chinese lexical scores.
• Optionally use successful Skill loads as a small, decaying ranking boost for
  candidates that already pass the lexical relevance threshold.
• Optionally fill unmatched catalog slots with semantic similarity from a local
  Ollama or OpenAI-compatible embedding endpoint.
• Limit the model-facing catalog with maxActiveSkills.
• Optionally enforce a conservative estimated-token budget for the Skill
  catalog prompt.
• Discover candidates from the DSH Registry, the SkillFlux cache,
  skills.sh, and authenticated GitHub SKILL.md code
  search.
• Re-rank remote matches by task relevance, marketplace adoption, repository
  activity, stars, forks, license metadata, and configured trusted owners.
• Resolve remote candidates to immutable GitHub commit SHAs.
• Register cached Skills through the current Agent's ctx.skills scope.
• Verify cached content with a SHA-256 manifest before every load.
• Support per-remote-mount, per-repository/session, and automatic approval
  policies.
• Expose model tools for loading, searching, and mounting Skills.
• Expose /skillflux commands for status, routing explanations, usage, and
  cache cleanup.

Routing behavior

SkillFlux routes a task in this order:

1. Preserve an explicit user invocation such as /pdf-reader and exclude that
   name from automatic routing.
2. Group model-invocable Registry and cached candidates by Skill name. Registry
   entries represent a name first, while same-name cache entries remain
   available as fallbacks.
3. Apply matching routes in configuration order.
4. Score the remaining name representatives with deterministic lexical
   matching and reject scores below minRouteScore.
5. When adaptiveRouting is enabled, add a bounded, time-decaying usage boost
   only to candidates that already passed minRouteScore. Rules keep priority,
   and history cannot make an irrelevant candidate cross the threshold.
6. In hybrid mode, use embeddings only when rules and lexical matching leave
   catalog slots unfilled. Semantic results never displace those earlier matches.
7. Mount the selected names until maxActiveSkills is reached. When
   catalogTokenBudget is enabled, skip a candidate that would make the
   estimated catalog prompt exceed that budget. If a candidate can't load,
   try its same-name fallbacks in candidate-pool order.

The lexical score is:

For equal scores, Registry candidates rank before cached candidates, and cached
candidates rank before remote candidates. Cache ties prefer higher install
counts and then a stable source/name order.

Semantic fallback uses cosine similarity, filters results below
minEmbeddingSimilarity, and reports a rounded 0-100 score. It runs only after
the ordered rule and lexical stages.

Online quality discovery

Remote discovery is live, not a bundled catalog:

1. skills.sh supplies marketplace matches and install counts.
2. When GITHUB_TOKEN or GH_TOKEN is available, GitHub Code Search finds
   matching public SKILL.md files outside the marketplace. SkillFlux fetches
   and validates each matched frontmatter before accepting it.
3. GitHub repository metadata supplies the immutable HEAD commit, stars,
   forks, license, archive state, owner type, and last push time.
4. SkillFlux rejects zero-relevance, archived, disabled, below-star, and
   below-quality candidates, then returns the highest-quality matches.

The quality score is capped at 100. Relevance is a gate and the largest single
component, so a famous but unrelated repository cannot outrank an exact new
match merely because it has more stars.

remoteRecentActivityDays defaults to 30. Activity inside that window receives
the full freshness contribution; older maintained projects decay gradually
instead of being discarded. Add owners you have independently vetted to
remoteTrustedOwners; being an organization or having many stars is not itself
treated as verification.

GitHub code search requires authentication. Start DSH from a shell that exposes
one of the standard variables, for example in PowerShell:

$env:GH_TOKEN = gh auth token
dsh web

Without a token, SkillFlux continues to search skills.sh and enrich those
results through the public GitHub REST API. It simply skips the broader GitHub
code-search provider.

Configuration

SkillFlux accepts these plugin options:

maxActiveSkills: 3
minRouteScore: 8
approvalPolicy: always       # always | session | automatic
remoteDiscovery: automatic   # automatic | on-demand | off
remoteProviders: [skills.sh, github]
remoteSearchLimit: 5
remoteSearchTimeoutMs: 30000
remoteMinQualityScore: 35     # 0-100
remoteMinStars: 0
remoteRecentActivityDays: 30
remoteTrustedOwners: []       # e.g. [anthropics, openai, vercel-labs]
catalogDescriptionMaxLength: 160
catalogTokenBudget: 0              # 0 disables; otherwise 64-1000000
maxSkillFiles: 1000
maxSkillBytes: 10485760
installTimeoutMs: 300000
routerMode: lexical                # lexical | hybrid
embeddingProvider: ollama          # ollama | openai-compatible
embeddingEndpoint: http://127.0.0.1:11434/api/embed
embeddingModel: embeddinggemma
embeddingApiKeyEnv: SKILLFLUX_EMBEDDING_API_KEY
embeddingTimeoutMs: 5000
embeddingCandidateLimit: 128
embeddingCacheSize: 512
minEmbeddingSimilarity: 0.45
usageTracking: true
usageMaxEntries: 1000
adaptiveRouting: false
adaptiveMaxBoost: 6
adaptiveMinUses: 2
adaptiveHalfLifeDays: 30
routes: []

Add ordered rules when a known task must prefer specific Skills:

routes:
  - matchAll: [pdf, analyze]
    skills: [pdf-reader, document-parser]
  - matchAny: [react, frontend]
    skills: [react-specialist]

A rule can contain matchAll, matchAny, or both. Rule results keep their
declared order, skip unavailable Skills, and still respect
maxActiveSkills.

Hybrid embedding router

Embedding routing is opt-in. The recommended private setup uses Ollama:

ollama pull embeddinggemma

routerMode: hybrid
embeddingProvider: ollama
embeddingEndpoint: http://127.0.0.1:11434/api/embed
embeddingModel: embeddinggemma

For an OpenAI-compatible embedding service, select the protocol and point at
its exact embeddings endpoint:

routerMode: hybrid
embeddingProvider: openai-compatible
embeddingEndpoint: https://provider.example/v1/embeddings
embeddingModel: provider-embedding-model
embeddingApiKeyEnv: SKILLFLUX_EMBEDDING_API_KEY

Set the named environment variable in the process that launches DSH. SkillFlux
never stores that value. Candidate vectors are kept in a bounded in-memory LRU
and disappear when the plugin stops. An unavailable, malformed, timed-out, or
reconfigured embedding endpoint fails open to the lexical result.

Usage statistics and adaptive routing

Usage tracking records successful mounts and successful skill({ name })
loads by exact candidate ID. It is enabled by default, while adaptive routing
is opt-in:

usageTracking: true
adaptiveRouting: true
adaptiveMaxBoost: 6
adaptiveMinUses: 2
adaptiveHalfLifeDays: 30

Records are stored atomically in
$DSH_HOME/storages/skillflux/usage.json (normally
~/.dsh/storages/skillflux/usage.json) and bounded by usageMaxEntries.
The file also has a hard 2 MiB limit; least-recently-useful records are evicted
first when either bound is reached.
SkillFlux stores only the candidate ID, Skill name, origin, source, counters,
and timestamps. It does not store task text, Skill instructions, or resources.

The boost is capped by adaptiveMaxBoost, requires at least
adaptiveMinUses successful loads, and halves after
adaptiveHalfLifeDays without use. A telemetry read or write failure falls
open to normal routing. Set usageTracking: false to disable persistence; in
that case adaptiveRouting must also remain false.

Catalog context budget

catalogTokenBudget is an optional second bound in addition to
maxActiveSkills. The default 0 keeps existing behavior. A nonzero value
preflights each mount and rejects only the candidate that would exceed the
budget, allowing later smaller or same-name fallback candidates to continue:

maxActiveSkills: 3
catalogDescriptionMaxLength: 160
catalogTokenBudget: 512

The estimate covers the complete replacement-form Skill catalog prompt after
description truncation. It uses ceil(UTF-8 bytes / 3): intentionally
conservative for typical English and close to one token per CJK character, but
it is not a model-specific tokenizer result. /skillflux status reports the
current estimate and /skillflux explain marks rejected candidates as
budget-skipped.

Approval policies

If approval is unavailable, rejected, or canceled, the remote mount fails
closed.

Model tools and user commands

The model can use:

• skill({ name }) to load instructions for a Skill already mounted this turn.
• skillflux_search({ query, remote? }) to search installed, cached, and
  immutable remote candidates.
• skillflux_mount({ candidateId }) to mount a candidate from the current
  SkillFlux discovery state.

You can use:

/skillflux status
/skillflux explain
/skillflux usage
/skillflux cache list
/skillflux cache clean <cache-id>
/skillflux cache clean all

explain shows each candidate's router stage, score, base score, adaptive
boost, and whether it was selected, successfully mounted, or skipped by the
catalog budget.

Cleanup skips cache entries that are still mounted. At turn/end, SkillFlux
unregisters runtime mounts but retains downloaded files for later reuse.

Security and trust

• Remote discovery accepts only public GitHub repositories from skills.sh or
  authenticated GitHub SKILL.md code search.
• GitHub-discovered SKILL.md files are bounded to 256 KiB and must pass the
  same supported frontmatter parser before they become candidates.
• The SHA-256 hash of a GitHub search preview must match the SKILL.md selected
  by the installer, preventing a same-name Skill elsewhere in the repository
  from silently replacing the reviewed match.
• Archived and disabled repositories are rejected. Repository popularity,
  activity, and license metadata are ranking evidence, not a security verdict.
• Each remote result is resolved to a 40-character commit SHA before SkillFlux
  creates its candidate ID.
• Installation downloads that immutable GitHub codeload archive through the
  pinned skills@1.5.23 CLI.
• Transport extraction is capped at 5,000 files. The selected Skill is
  separately capped at 1,000 files and 10 MiB by default.
• SkillFlux checks paths, symlinks, frontmatter, file counts, byte counts, and a
  SHA-256 content manifest before mounting.
• SkillFlux caches scripts as resources but never executes them.
• Automatic discovery sends a bounded keyword query instead of the complete
  user message.
• Hybrid routing sends at most 1,000 characters of the direct task and at most
  1,000 characters of each candidate's name, whenToUse, and description to
  the configured embedding endpoint. It never sends Skill bodies or resources.
• Usage records never include task text or Skill content and are bounded to
  usageMaxEntries entries in the DSH storage directory.
• GITHUB_TOKEN or GH_TOKEN is optional. It enables broad GitHub code search
  and batched repository enrichment; SkillFlux doesn't persist it.

Skills are external instructions and can be malicious. Approval is a trust
decision, not a sandbox. Keep DSH permissions, sandboxing, and tool approvals
enabled.

Evaluation

Run the versioned routing corpus without an API key or network access:

corepack pnpm eval

The suite contains 36 lexical cases, 4 adaptive safety cases, 8
provider-independent semantic-vector cases, 7 catalog-budget cases, and 8
remote-quality pairwise cases covering English, Chinese, normalization, rules,
thresholds, capacity, ranking, deduplication, semantic top-k, context budgets,
freshness, trust, adoption, and negative rejection.

These results verify the deterministic router and vector-ranking contracts
against checked-in inputs. The semantic vectors are synthetic, so these results
do not measure a particular embedding model, third-party Skill quality, or the
final answer from an online model. Read the
evaluation corpus guide for the case format and limitations.

Known limitations

• Hybrid quality depends on the configured embedding model. SkillFlux does not
  download or manage that model.
• Semantic fallback considers at most embeddingCandidateLimit local
  candidates in current Registry/cache order.
• Catalog token counts are portable estimates, not exact counts from the
  configured chat model. They exclude loaded Skill bodies, tool schemas, and
  other session history.
• GitHub Code Search is unavailable without GITHUB_TOKEN or GH_TOKEN; the
  skills.sh provider remains available.
• Quality scoring is evidence-based triage, not a code audit. Inspect the exact
  pinned candidate and keep approval/sandbox controls enabled before mounting.
• Unmounting can't remove text already committed to session history.
• A new upstream commit creates a new immutable cache entry. Old entries remain
  until explicit cleanup.

Development

External contributors should follow CONTRIBUTING.md for the
fork, topic-branch, offline quality-gate, live GitHub discovery test, evaluation,
and pull-request review workflow.

corepack pnpm install
corepack pnpm check
corepack pnpm eval
corepack pnpm test:discovery-live
corepack pnpm test:embedding-live
corepack pnpm pack --dry-run

test:discovery-live runs a real skills.sh query and also GitHub Code Search
when GITHUB_TOKEN or GH_TOKEN is present. Override the task with
SKILLFLUX_DISCOVERY_QUERY, add comma-separated trusted owners with
SKILLFLUX_TRUSTED_OWNERS, or set SKILLFLUX_REQUIRE_GITHUB=1 to fail when the
GitHub provider is unavailable.

test:embedding-live expects the configured Ollama model to exist. Override
the defaults with SKILLFLUX_EMBEDDING_MODEL, SKILLFLUX_EMBEDDING_ENDPOINT,
and SKILLFLUX_EMBEDDING_PROVIDER when testing another endpoint.

The test suite covers routing, DSH catalog virtualization, explicit invocation,
remote response validation, cache integrity, lifecycle cleanup, bounded usage
storage, adaptive-threshold safety, and the bundle patch. Read
CONTRIBUTING.md before submitting a change.

License

MIT