Skip to main content

Gitmoot v0.4.0

First official (non-beta) release. Headlines the SkillOpt evaluator-hardening work (#345): Gitmoot now records how the LLM judge's accept/reject verdicts compare to the human's decision at the manual gate, reports calibration, and carries per-task_kind judge prompts in the contract — the substrate for tuning the judge against real human agreement.

Highlights​

Judge↔human outcome capture + judge-report​

  • Every candidate promote/reject at the manual gate records all four judge↔human directions (agree_accept, agree_reject, judge_accept_human_reject, judge_reject_human_accept) into a local skillopt_judge_outcomes table, tagged with the judge prompt version, evaluator id, and prompt hash. Capture is measurement-only and never overrides a decision; reports with no judge signal are skipped.
  • gitmoot skillopt judge-report [--template <id>] renders a confusion matrix, agreement rate, Cohen's κ, soft-score calibration bands, and per-dimension disagreement.

Per-task_kind judge prompt in the contract​

  • The exchange contract additively carries evaluator_profile.judge.config.judge_prompt_templates[task_kind] and judge_prompt_version, so a tuned judge prompt can be selected per task kind (no contract_version bump).

Judge-prompt optimization (companion: gitmoot-skillopt v0.3.0)​

  • The captured outcomes feed an offline, frozen-and-alternated, per-task_kind judge-prompt optimization loop in gitmoot-skillopt v0.3.0 (optimize --judge-prompt-optimization), gated on held-out human agreement. On a real held-out set it lifted judge↔human agreement on the correctness task kind from 0.33 → 0.67 (gate-accepted), with non-improving variants correctly rejected. See the judge-prompt optimization guide.

Validation​

  • go build / go vet / go test ./... and the -race workflow suite are green; the judge-outcome capture was validated end-to-end on a live Codex run across all four directions.

Install / update​

gitmoot update                                   # existing installs
curl -fsSL https://gitmoot.io/install.sh | sh    # fresh install

Assets: gitmoot_linux_amd64, gitmoot_darwin_amd64, gitmoot_darwin_arm64, sha256sums.txt.