Skill Set Optimization Scorecard

Baseline

• Canonical registered skills: 82, sourced from content/skills/ and content/skills/registry.yaml.
• Current gate model: Q1-Q4 labels exist in workflow contracts and task routing.
• Current domain model: domain profiles are injected at runtime.
• Current eval model: offline academic quality fixtures score broad quality dimensions.

Retired Root TODOs

The former root-level planning files TODO_ROADMAP.md and todo_academic_skills_next.md were retired after the repository restructure. They mixed completed milestone history, stale pre-restructure paths, and active ideas that now belong in focused maintainer docs or implementation specs.

Use this scorecard for skill-set optimization priorities, and use docs/development/repository-structure.md for source-layout ownership. Future roadmap updates should be added to focused docs under docs/maintainer/, docs/development/, or a dedicated implementation spec instead of recreating a root TODO file.

Next Optimization Targets

Target	Baseline	Next Bar
Executable Q1-Q4 semantic gates	Q1-Q4 semantic gate report schema, semantic_checks, and quality-gate-report.md audit exist	Gate reports generated by Q1-Q4 owner skills include evidence anchors and blocking actions in real task-run traces
Economics and finance method packs	Method templates with assumptions, diagnostics, artifacts, and failure modes	Audited method anchors, Q1-Q4 relevance, local diagnostic artifacts, failure triggers, and subject-install visibility
Offline eval expansion	6 broad cases	Gate-failure and domain-method cases included
Skill routing precision	Registry inputs/outputs plus task routing	Gate and domain-pack requirements visible in task prompts
Release confidence	Structural validation	Structural + semantic contract validation

Semantic Gate Milestone

The Q1-Q4 gate layer now requires semantic_checks in quality-gate-report.md. This moves the baseline from status-only reporting to evidence-anchored checks:

• q1_rq_method_alignment
• q2_claim_evidence_traceability
• q3_reporting_completeness
• q4_reproducibility_baseline

The next bar is task-run evidence: real Stage C/F/G/I runs should write gate reports whose evidence anchors resolve to project artifacts.

Real-Agent Smoke Roadmap

Real-agent smoke should stay a maintainer-only opt-in acceptance check; it is not a default CI gate. Its purpose is to prove that local Codex, Claude Code, and Antigravity task-run paths can produce auditable Q1-Q4 gate reports without polluting the maintainer's normal installation.

The smoke design should isolate the runtime environment before launching local agents:

• Create a temporary project workspace and an isolated HOME.
• Set isolated CODEX_HOME, CLAUDE_CODE_HOME, ANTIGRAVITY_HOME, and related cache/config paths.
• Materialize the current checkout package into that isolated environment.
• Run Stage C/F/G/I owner tasks against a small fixture topic.
• Audit the generated quality-gate-report.md with scripts/audit_quality_gates.py --strict.
• Leave the check opt-in through an explicit maintainer command or environment variable, because local agent availability, login state, and model behavior are not deterministic enough for routine CI.

Economics And Finance Method-Pack Milestone

Economics and finance method packs now move beyond general diagnostics. Each audited method should provide:

• canonical_references as compact literature anchors for why the method belongs in the pack.
• gate_relevance so Q1-Q4 owners know where method evidence matters.
• diagnostic_artifacts that name concrete RESEARCH/[topic]/... files.
• failure_triggers that block unsupported causal, risk-adjusted, predictive, pricing, or structural claims.

The full CLI install remains one complete Qiongli workflow. Subject packages and active_subject: auto select a specialization layer on top of the canonical workflow, and installer tests should keep verifying that enhanced economics and finance method-pack fields are visible in materialized subject payloads.

Recommended Next Optimization

After the economics and finance method-pack pass, the next implementation pass should focus on real task-run traces: Stage C/F/G/I runs should show that method pack diagnostics feed Q1-Q4 semantic gate reports rather than remaining static YAML metadata.

Measurement Commands

Current baseline commands:

uv run python scripts/validate_research_standard.py --strict
uv run python scripts/audit_skill_sections.py --strict
uv run python evals/runner/run_suite.py

Post-optimization commands, available after Task 1/2 create the audit scripts:

uv run python scripts/audit_quality_gates.py --strict
uv run python scripts/audit_domain_method_packs.py --strict