Monitor
Find the answers and agent runs that need a person, see why each was flagged, and approve or reject held steps.
Monitor shows everything Agent Lens scored for your project, with the items a person should look at first. Use the Prompts | Agents switch at the top to move between answers and agent runs.
Prompts
Requests lists every scored answer, newest first. Each row shows the app, the start of the prompt and answer, the verdict, the highest risk and which probe it came from, the flagged sentence, and the review status. You can filter by time, app, probe, verdict and review status, or search the text.
Open a request to see:
- the prompt, and what the model was given (records, documents, tool results);
- the answer, with the words that drove a probe highlighted and the sentence that drove the score outlined;
- Why was this flagged? This re-reads the answer with each sentence removed and shows how much each one moved the risk (for example, removing sentence 2 takes data leak from 0.59 to 0.00). Re-reads are not counted as traffic;
- every probe on the 0–1 scale with its threshold;
- Review: mark the flag as a Confirmed failure or a False alarm, pick the error type (data leak, deception, …) and add a note;
- Copy as API call, to reproduce the request;
- the record: decision id, input digest, probe release, thresholds and the signature.
To review lists flagged answers nobody has decided on yet, highest risk first, with Confirmed failure and False alarm in each row.
Insights shows the flag rate by day, which probes fire, which apps, and how precise each probe has been on the flags you reviewed.
If Keep text is off in Settings, Monitor shows the verdict, risks and sentence positions but not the words.
Agents
Runs lists each agent run with its agent, number of steps, the strongest verdict and whether a step is waiting for a person. Open a run to see its trace:
- every step in order: the tool, its arguments (credentials removed), what it returned, the verdict, and the probes that fired;
- the decision-trace matrix, with one column per failure mode;
- held steps with Approve and Reject. The agent waits on your answer, and the integrations resume or refuse when you click;
- the seal check for each decision.
To link to a run, use /monitor?side=agents&run=<run_id>.
Open issues lists held steps and flagged runs that still need someone, oldest first. Review opens the run at the held step. Suggest a fix writes a debug report, proposes a change to the agent's prompt, and re-runs the case to check it.
Insights shows traffic by agent and tool, and the checks agents skipped.
Labels make the numbers better
Every Confirmed failure and False alarm is stored as a label for your project. Insights uses the labels to show each probe's precision on your traffic, and they are the evidence for adjusting a threshold in Settings. Agents never see labels.
From Monitor to Eval
From a run, Save as eval scenario turns the trace into a test case for Eval: the task, the tools, what each tool returned, and the rule the agent broke. Credentials and redacted values are removed.