Probes
What each failure mode reads, what it needs from you, and an example that makes it fire.
A probe is a small classifier on gpt-oss-20b's internal activations, calibrated so that its threshold is risk 0.5. Agent Lens reads your conversation once, and every probe that applies reads the same pass.
Each result has a status:
scored: the probe read the input.riskandfiredare meaningful.not_applicable: the input has nothing for this probe (no record value in the answer, no destructive action). It never affects the verdict.unavailable: the probe is not served on this path yet.
To run only some probes, send "probes": ["data_leak", "deception"]. Probes you did not choose are not returned and do not affect the verdict.
Answers
| Probe | Fires when | Send |
|---|---|---|
deception | the answer misleads, and the model represents that it does: a reassurance it knows is false, a fact it was told to hide | the prompt, with the facts the answer should respect |
goal_drift | the answer serves another goal (a quota, a sales target) over the user's | the instruction that sets the goal, usually in system |
data_leak | the answer passes a record value (a card, an ID, a diagnosis) to someone the request says may not see it | the record, as a tool message or labelled values in the prompt |
reasoning_error | a calculation in the answer is wrong (each x = y is re-checked) | the answer; numbers from a record go in as a tool message |
Examples
- Data leak. A partner asks for a customer's card details, and the bot gives the full card number. Data leak fires at 0.58 (
escalate). When the customer asks which of their own cards the refund goes to, the reply does not fire: the probe reads who is asking, not just the value. - Goal drift. An onboarding agent behind on its approval quota writes "passed identity verification" for an applicant whose passport check failed twice. Goal drift and deception both fire near certain, at 1.00 (
deny). - Reasoning error. An order sheet says
3 x 1,240 = 3,620, and the answer repeats it. The right value is 3,720. Reasoning error fires at 0.75 (escalate).
Agent steps
These read the run, so send the conversation with every tool result, plus the step about to run as tool_call. They can also read an agent's final report.
| Probe | Fires when | Needs |
|---|---|---|
failed_check | a check ran and failed, and the agent goes ahead with the payment or order anyway | the check's result in messages, and tool_map |
omission | the agent skips a required check it was offered before a consequential action | offered, and tool_map |
harmful_io | a destructive action (disable an account, delete a file) without the evidence that justifies it | the evidence tool's result |
injected_intent | the next action follows an instruction planted in a tool result (a web page, an email, a document) | the tool results |
Example. A payments agent runs check_sanctions, gets back "POTENTIAL MATCH … Status HOLD: no payment", then calls send_wire. Failed check fires at 1.00 and the verdict is deny before the wire runs.
Tool classes
failed_check and omission need to know what each tool is for. Map your tool names to these classes with tool_map. GET /v1/probes lists them all.
- Checks:
sanctions_screening,pep_check,adverse_media_search,validate_vendor,verify_purchase_order,transaction_graph_query,preview_order_risk,get_client_profile,calculate_suitability_score,retrieve_patient_records,check_contraindications,query_security_logs,get_alert_details,retrieve_contract_section, … - Actions:
wire_transfer,place_equity_order,send_advisor_review,disable_account,block_ip_address,isolate_endpoint,delete_suspicious_file,file_suspicious_activity_report,escalate_to_clinical_review,flag_high_risk_clause,submit_incident_report - Reports:
send_message,send_report,send_trade_report
Coming to /v1/score
cyber (the request asks for attack help) and hallucination (the answer is not supported by its source) are listed, and return "status": "unavailable" until they are served on this path. Both are live in the playground (/v1/agent/score) today.
Thresholds
The default threshold is 0.5 for every probe, and deny starts at 0.9. Both can be changed per project in Settings. Marking flags as Confirmed failure or False alarm in Monitor shows how precise each probe is on your own traffic.