# Shepherd fixes the PR, its gate approves it
By Jag Reehal, 2026-10-09. Canonical: https://arrangeactassert.com/posts/shepherd-fixes-the-pr-its-gate-approves-it/
Shepherd takes a pull request from opened to approved.
Its review loop runs in your coding agent, reviews the change, fixes what it can and repairs the build.
Its gate runs in CI and decides whether the result earns an approval.
I give approval to a separate reviewer. You keep the decisions that need a person.
PostHog's [list of AI coding mistakes](https://posthog.com/newsletter/ai-coding-mistakes) argues that lint, types and tests matter more once an agent writes the code. Shepherd runs those checks inside the pull request loop.
I built [Shepherd](https://github.com/jagreehal/shepherd) in two parts. The loop prepares the change and can push a fix. The gate decides approval, and only the gate can grant the approval that lets you merge.
```mermaid
---
alt: "Shepherd runs review rounds inside a coding agent: swarm reviews, triage fixes or defers threads, and simplify checks the result. After a round with no changes or new findings, Shepherd repairs CI and reads the gate’s verdict. The loop and the gate record their work on GitHub."
---
flowchart LR
subgraph inner["Shepherd: in your coding agent"]
S["Swarm reviews the diff"] --> T["Triage fixes or defers threads"]
T --> SI["Simplify"]
SI --> S
end
inner --> C["CI repair"]
C --> ST["Shepherd's gate in CI checks + model review approves or refuses"]
ST --> GH[("GitHub threads, commits, statuses, verdicts")]
inner --> GH
```
Figure 1: Shepherd's loop gets the PR ready, and its gate decides whether it gets approved.
## How Shepherd works
Shepherd is a set of skills that runs inside a coding agent such as Claude Code or OpenCode. You install it with `bunx @jagreehal/shepherd install`, or with `npx skills add jagreehal/shepherd` for any agent the skills CLI supports, and run `/shepherd 42` on a PR. One iteration works like this:
1. **Swarm** runs the repository's own lint, typecheck and nearby tests first, plus any scanners you have installed, such as `gitleaks`, `semgrep` and `osv-scanner`. A cheap model then reads the whole diff and hands the risky hunks to specialist lenses (correctness, security, simplicity, maintainability, slop, and any your team adds), which run in parallel. For a PR that closes an issue, a spec lens checks the diff against what the issue asked for. A second model checks each high-severity finding against the code before it posts.
2. **Triage** works through the open review threads. It fixes the concrete ones, resolves the ones it can show are mistaken, and leaves the decisions that belong to the author. Threads another person has joined stay with that person.
3. **Simplify** checks the result for code the change no longer needs. A fix gets another review round.
4. After a round with no changes or new findings, **CI repair** brings the branch up to date and fixes what the PR broke. Shepherd then reads the gate's verdict. The loop allows up to four review rounds before handing unresolved work back to you.
Wrap it in `/loop 5m /shepherd 42` and you can leave it running. The loop never replies to people and never approves.
## Set the checks and the fix mode
Your team can set the checks, reviewers and fix permissions in `.shepherd/config.yml`. A fresh `shepherd init` writes that file, the gate's policy in `.shepherd/gate.yml` and the workflows, and starts in review-only mode. Choose `init --preset local-fix` to leave fixes in your working tree, or `init --preset push` to let Shepherd commit and push them.
You can extend an organisation's policy pack and override settings for one repository. A pack from another repository needs a full commit SHA. Shepherd reads team policy from the default branch, so a PR author cannot use their change to remove a reviewer or relax the fix rules.
Before merging a policy edit, run:
```bash
bunx @jagreehal/shepherd config validate
bunx @jagreehal/shepherd preview --working-tree --recipe pr
```
The preview lists the checks and models, the steps that can write code or touch GitHub, and the decisions left to a person. You can keep using the older `.shepherd/lenses.yml` configuration.
## How the gate decides
The gate runs in CI, and `shepherd gate ` runs the same review on your machine. It checks deterministic gates first: sensitive paths, size limits, credentials on added lines, test deletions and skip markers, and who wrote the PR. A model review with read-only access to the checkout judges what the gates allow, and reads the issues the PR closes to check the diff does what they ask. The model can refuse a change the gates passed, and only the gates can make a change eligible. The gate posts an approval as a GitHub review pinned to the reviewed commit, so it counts toward branch protection. Before posting, it checks that the head and base match the change it reviewed.
The model review runs through the [AI SDK](https://ai-sdk.dev/), so you choose the provider with one variable, the same one the loop reads: `SHEPHERD_MODEL=bedrock/openai.gpt-oss-120b-1:0`, `opencode-go/kimi-k3`, `openrouter/moonshotai/kimi-k3`, or a Claude model on Anthropic.
The model reads the checkout through three tools, `read_file`, `grep` and `glob`, confined to the repository, and ends the review by calling `submit_verdict`, whose input schema is the verdict. Every provider I tested supports tool calling, so one verdict format works across them. On Bedrock, the gate signs with an API key or your AWS credentials, including a named profile. In my tests the models read the code before deciding and returned a valid verdict through the tool. A model that answers in prose gets one follow-up call that must be `submit_verdict`.
The gate records each decision at the point it makes it, as I argued in [You Can't Support What You Didn't Record](/posts/you-cant-support-what-you-didnt-record/), so every review leaves a run record. A Kimi K3 review of a one-line change made five tool calls, read 22,656 of its 33,580 input tokens from cache, and took 57 seconds. Limits from [autotel](/posts/logging-sucks/)'s guard bound each review: $2 of spend, three million tokens, a spin-loop rule, a tool-call cap, and a 15-minute timeout that cuts off a model call in flight. The gate prices each call from the `pricing:` your team sets in `.shepherd/gate.yml`, and each price carries the date you read it. `shepherd init` writes a dated list price for the model you choose. Set an OTLP endpoint and each review becomes a trace, with one span per model call and tool call, each cost tagged with its price date.
Teams that prefer Claude Code or Codex can run those as the reviewer instead. The gate runs them in a pinned container with a read-only copy of the checkout and only their own credential, on a private network whose one route out leads to their own model provider. The egress proxy opens a tunnel only when the TLS handshake inside it names that provider's host, so a reviewer can't reach another site that shares the provider's CDN address.
## Read what the tests checked
Repositories that use [executable-stories](https://github.com/jagreehal/executable-stories) give Shepherd more than a test exit code. Shepherd reads each failed scenario's steps, expected and actual values, and `covers` paths. It reruns the affected scenarios after a fix. For a skipped scenario, triage restores the test and fixes the code if that test fails.
Shepherd compares runs of the same scope. A focused run leaves out tests it did not target, so you can't read a test missing from it as one the PR removed. For an incident or a linked issue, Shepherd runs `executable-stories goal` with the issue's ticket against full current and base runs. The command checks that the required scenarios pass and flags scenarios the change dropped, skipped or shortened.
The gate can read the review report CI uploads:
```yaml
# .shepherd/gate.yml
stories:
workflow: ci.yml
artifact: executable-stories
file: index.review.json
```
The report needs executable-stories-formatters 1.27 or later to record the commit where the tests ran. The gate checks that commit against the PR head, or a merge whose parents match the reviewed base and head. It labels filtered runs, and reports stale or missing evidence. A comparison label such as `headRef` cannot establish where tests ran.
The PR author wrote those tests. The gate keeps the report inside its untrusted input and uses it as review context. A passing scenario, a declared `covers` path or a trace grants no exemption from a gate and supplies no independent reviewer approval.
## See where each PR stands
Shepherd sets a `shepherd` commit status on the head at the end of each iteration:
- `pending` while work remains or a verdict hasn't arrived
- `failure` when only the author can move the PR, with the reason in one line, such as `gate refused: ...` or `3 threads need you`
- `success` once the gate has approved the current head, CI is green and no review thread remains open
The status lives on GitHub, so it outlasts the session that set it. You can come back to a PR hours later and read what it needs from you without opening a transcript. A push Shepherd hasn't seen arrives with no status at all, so a branch rule that requires the status keeps the PR from merging until Shepherd has looked at it.
The gate handles description edits in a separate workflow, so the `shepherd-gate / review` check on each commit shows the verdict that counts.
On a re-review, the gate withdraws its standing approvals before fetching the PR or taking a skip path. One exception keeps an approval when a base merge leaves the PR's diff byte-identical and today's policy allows it. After posting, the gate checks for a changed head or base and a verdict from a later-started run. An older run cannot leave its approval standing over a newer refusal.
You can trace a Shepherd iteration through [autotel](/posts/logging-sucks/). With an OTLP endpoint configured, Shepherd exports a span for the iteration and one for each recorded stage, including the wait for the gate. Repository, PR and commit fields let you correlate that work with the gate's review. Separate processes can link their spans through traceparent values. The CLI records stage timing and outcomes; your coding-agent harness must supply model usage.
The [gardener](/posts/your-coding-agents-need-a-gardener/) can read those traces from OTLP JSON exports, Grafana Tempo or Datadog. It flags failed stages, repeated retries, long approval waits and high gate review cost for investigation. It matches each sample to a known PR commit and reports gaps in collection or pricing. You can follow a proposal back to its trace ID and GitHub evidence; the trace alone cannot count a fix as accepted.
## Run the loop before you open a PR
You can run the same review on your machine before GitHub sees the change. `/shepherd --local` captures your unpushed commits, staged and unstaged edits and untracked files, then runs the same lenses, verification and fix rules as the PR loop. Fixes land in your working tree. Shepherd never commits, pushes, comments or sets a status in this mode, so you review the fixes and commit them yourself.
A clean report needs `shepherd local finish` to say `complete`. That check confirms every lens finished, every changed line range reached a reviewer, and the files didn't change mid-review. It proves each line went in front of a reviewer, which falls short of proving the reviewer understood it. A complete review writes a receipt keyed to the exact content, and `shepherd local status` tells you whether the files on disk match it. Put that in a pre-push hook if you want one.
Local mode ends at "local checks passed". Merge readiness stays with the PR loop and the gate. To see it work, `bunx @jagreehal/shepherd local demo` builds a repository with seven planted bugs, one for each way a change reaches the review, including a deleted check.
## One checklist for review and approval
A service team cares about observability and API compatibility. A frontend team cares about accessibility and its design system. Shepherd lets your team add its own reviewers on top of the built-in lenses:
```bash
bunx @jagreehal/shepherd lens new observability --from observability --applies 'src/**'
```
That command copies Shepherd's bundled observability pack into `.shepherd/lenses/observability/SKILL.md` and registers it in your team configuration, `.shepherd/config.yml` when present, or the older `.shepherd/lenses.yml`. Drop `--from` and you get a blank skill to fill in. The `## Review` section lists rules with IDs, so a finding names the rule it applies: `[observability/error-context]` means a caught error lost its cause or the IDs you need to find the request. The pack's seven rules check for the telemetry [Instrument Before You Know the Question](/posts/instrument-before-you-know-the-question/) says you need before production breaks. They cover outbound calls with no span or log, unstructured logs, secrets in telemetry, silent data drops and unbounded metric labels. The `## Fix` section tells triage to use the logger and tracer the repository has, and to hand you any change that adds a telemetry dependency.
Shepherd reads the lens file from the default branch, so your team decides who reviews a PR and the PR itself has no say. You can try a lens on a real PR with `/swarm --preview`, which posts nothing.
The gate reads the same file as a rule pack. Give the lens a `gate:` setting and add `rules_from` to `.shepherd/gate.yml`, and the gate's reviewer applies those rules to matching files, then refuses, escalates or notes each broken one:
```yaml
# .shepherd/config.yml
lenses:
observability:
skill: .shepherd/lenses/observability
applies_to: ['src/**']
gate: escalate
# .shepherd/gate.yml
rules_from: { shepherd: {} }
```
Your team edits one checklist for both the fixer and the approval reviewer. A pack can only add checks. The gate reads it from the default branch, and a PR that edits a configured pack goes to a person, because later reviews trust its text.
## Catch an agent silencing a check
An agent chasing a green build can silence a rule and leave the code as it was. Swarm lists each suppression comment a PR adds, such as `oxlint-disable`, `@ts-ignore` or `nosec`, as a finding and reads the code it covers. The gate flags the same comments for its reviewer. The reviewer refuses a suppression that hides a finding the change introduced and approves a narrow one that gives its reason.
A PR that loosens lint, test or scanner config gets the same treatment. A linter can't see its own config change, so the gate's reviewer reads that diff and refuses a disabled rule, a widened ignore or a lowered coverage threshold. It approves tightening.
The gate's tests check holds a PR that deletes a test file, renames it out of the runner's collection pattern or adds a recognised skip or focus marker. Conditional skips and rewritten assertions reach the reviewer. With that split, a model can't explain away a disabled test the gate can identify.
## After a failure reaches production
Run `/incident ` with a red default-branch build or a production failure. Shepherd tries to reproduce the symptom in a test, checks which PR introduced it, and records whether the gate approved that PR. It opens a fix PR after verifying the fix. If it cannot reproduce the failure, it stops without claiming a fix.
The [gardener](https://github.com/jagreehal/shepherd/tree/main/skills/garden) reads those incidents alongside review history. For a regression the gate approved, it can propose a check that flags the culprit change and allows the fix. You review the proposed rule before it affects later PRs.
## Work alongside Claude Code Review
[Claude Code Review](https://code.claude.com/docs/en/code-review) posts findings on a PR and leaves the approval to you, which is the decision the gate makes. The gate waits up to five minutes while the **Claude Code Review** check run on the head is queued or running, so those findings reach its reviewer.
Triage fixes 🔴 Important and 🟡 Nit findings like any bot finding and leaves 🟣 Pre-existing ones to you as follow-ups. Shepherd posts no `@claude review` comment, because each one starts a billed review; its summary tells you when a review failed or ran out of budget. Code Review reads `REVIEW.md` from the default branch, so the gate treats a PR that edits it like any change to review rules and refuses one that skips paths, lowers severities or caps findings.
## Spend money where judgement matters
Your team configuration sets the models. A `ladder` lists models from cheapest to strongest, and `models` pins a lens or a runner to one model, from any provider your coding agent can reach:
```yaml
ladder: [opencode-go/deepseek-v4-flash, opencode-go/glm-5.3, opencode-go/kimi-k3]
models:
security: opencode-go/qwen3.8-max
```
In one swarm review through OpenCode, the router on DeepSeek V4 Flash cost $0.013, two specialist lenses on GLM-5.3 cost about $0.16 each, and the orchestrating session on Kimi K3 cost $0.71. The orchestrator ran up most of that bill, so I would tune its model first.