Report-only, in development
Sentinel
Watches a live app and finds the mistakes that look fine: a contact form that says your message was sent when it never was, a filter that quietly returns everything, a page showing one person another person's data.
Finds the bugs that never throw, proves each one is real, and says plainly what it could not check.
- Platform
- Web, GitHub App, Scheduled runs
- Status
- Report-only, in development
- Demonstrates
- An agent system built to stay quiet
02
What it is for
Conventional monitoring answers two questions: is the app up, and did it throw. Neither catches the class of failure that matters most, which is output that looks correct and is wrong. A form that submits and drops the message. A filter that silently returns every row. A cache serving one user another user's data. Nothing throws, no alert fires, and the app is quietly wrong for three weeks.
That class is also what an LLM-assisted build produces most often, because the code compiles and reads plausibly. The target user follows from that: someone who built an app largely by prompting has, by construction, less of a mental model of their own codebase than someone who wrote it line by line. They cannot perceive their own bugs, cannot write a test that would catch one, and cannot read a stack trace. So every finding is translated into plain language, and the design question for any feature is whether it helps that person.
03
Why it exists
The scope is deliberately narrow and written down: Sentinel does health, meaning is the app doing what it is supposed to do. It does not do performance optimisation, security auditing, SEO, accessibility compliance, uptime monitoring, or observability dashboards. Each of those is a category with incumbents, and a watchdog that drifts into all of them is a dashboard nobody reads.
It is also not an incident responder. Those are reactive, they serve teams that already run an observability stack, and they assume somebody is on call. Sentinel wakes up because time passed rather than because someone committed, brings its own sensing rather than consuming a telemetry pipeline that does not exist, and reports to one person who may not know the bug is possible.
04
Screens
05
What it does
Proof before report
A suspicion is not a finding. Each candidate enters a confirmation loop against the live system, and a judge separate from the detector rules on the evidence. Suspicions that cannot be confirmed produce silence.
Honest coverage map
Every check area is listed with its real state and, where it is not being watched, the exact missing piece. Nothing is reported healthy because it was never looked at.
Separated powers
Hunter senses read-only, Fixer may only write to a branch, Verifier re-runs the checks, and Judge alone decides. The component that proposes a change never decides whether it worked.
Regression memory
A dismissed finding stays dismissed. A watchdog that re-raises something you already judged is one you stop reading, so the dismissal is remembered rather than re-derived.
Blast-radius gates
Changes are tiered. Trivial, sensitive and out of depth carry different permissions, and pull requests on sentinel branches are never merged automatically.
Cost as architecture
A per-run ceiling is armed before any model call, deterministic checks gate model calls, and cheap models gate frontier ones. Token economics decide what the product can charge.
06
What it runs on
Engine
- 01
- TypeScript
- 02
- Claude Agent SDK
- 03
- Anthropic
- 04
- OpenAI
Service
- 01
- Node
- 02
- Postgres
- 03
- PGlite
- 04
- Supabase
Client
- 01
- Next.js
- 02
- React
Sensing
- 01
- Playwright over CDP
- 02
- GitHub App
- 03
- Webhooks
07
How it's built
Three layers, and the line between them is the design. The engine is generic and knows nothing about any specific application: the orchestrator, four subagents, the fixpoint loop, model tiering and the sensing tools. Playbooks are per-app knowledge held as data rather than code, one audit file per target with its journeys, oracles and baselines. The harness binds the two and is the only place a concrete target is named. The engine declares the shapes it needs; the harness supplies the values.
The four subagents have separated powers. Hunter senses and is read-only. Fixer may write to a branch and nothing else. Verifier re-runs the checks, so the component proposing a change is never the component deciding it worked. Judge is the sole ship authority. The fixpoint rule governs the loop: a minimum of two iterations, a maximum of four, stop when a pass produces zero new findings and zero new edits, and escalate to a human rather than shipping something that has not settled.
The eleven-step run flow is ordered by cost rather than by logic. Cheap deterministic checks that spend no tokens run before any model does, a cheap model triages before a frontier model is asked, and the work is scoped to the diff so nothing that did not change is audited twice. Writing is fenced by blast radius: trivial changes may eventually auto-merge with a rollback armed, sensitive ones require a pull request and explicit approval and are never auto-merged, and anything out of depth is reported and not touched.
08
A piece of the code
engine/src/restraint/pipeline.ts
for (const c of candidates) { const policy = policies[c.severity]; // Stage 1 — confidence (bar is per-severity) const lowConf = confidenceGate(c, policy.confidenceBar); if (lowConf) { suppressed.push({ candidate: c, reason: lowConf }); continue; } // Stage 2 — down vs. broken vs. deliberate (ALWAYS applies) const downDeliberate = downDeliberateCheck(c, ctx.env); if (downDeliberate) { suppressed.push({ candidate: c, reason: downDeliberate }); continue; } // Stage 3 — flake quarantine (SEVERITY-GATED: critical skips the patience).}Excerpt — trimmed for reading, not a full file.
09
The hard parts
01 — Problem
A monitor that reports suspected bugs is worse than no monitor. One false positive costs a debugging session on working code, and after two or three the tool gets ignored. That is the real failure mode of every product promising to find your bugs. The bar is not noticing something odd, it is being right often enough to be worth reading.
Approach
Detection and assertion are separated by a proof step, and the judge is structurally distinct from the detector rather than a second prompt to the same one. The harness carries a test for the opposite case as well: that a suspicion which cannot be confirmed produces silence rather than a hedged report.
Tradeoff
Expensive, and most of the spend is on findings that are correctly discarded. It also adds latency between a bug existing and being reported, which is the wrong trade for an outage and the right one for a bug nobody has noticed in three weeks. The system optimises for trust over speed.
02 — Problem
An agent that can edit a repository is one bad judgement from being the incident. The failure is not a wrong patch, which review catches, but a plausible patch merged at three in the morning by something with no stake in being wrong.
Approach
Writing is fenced structurally rather than by instruction. Fixer can only reach a branch, Judge alone decides, pull requests open on sentinel branches and are never merged, and a run against anything not confirmed to be a non-production environment refuses to write at all. The proofs assert that the repository's HEAD is unchanged and that every throwaway branch was cleaned up.
Tradeoff
It cannot fix anything unattended, which is the feature people ask for first. The bet is that a tool trusted to open a pull request is worth more than one that was allowed to merge once and then turned off.
03 — Problem
Every candidate finding costs model calls, and a naive design spends the most on the app that changed the least. Token economics decide whether the product can be priced at all, which makes cost an architectural constraint rather than an optimisation to do later.
Approach
The run order is the cost model. Deterministic checks that spend nothing run first and gate the model work, a cheap model triages before a frontier model is called, the work is scoped to the diff so unchanged code is not re-audited, and a per-run ceiling is armed before the first token is spent.
Tradeoff
Diff-scoping means a bug in code that has not changed can go unseen for longer, which is a real gap in a product whose whole premise is silent failures. The periodic full pass is what pays for it, and it is the expensive one.
Ask about Sentinel
Answers are drawn from the notes and source behind this case study, and cite what they read.
10
What it did
50,520
Lines of TypeScript
112
Proof suites
18
Product surfaces
4
Subagents, separated
Measured Measured 27 August 2026