Score answers
Check each answer your app gives, with the documents and records it read, and act on the verdict.
After your model answers, send the prompt and the answer to POST /v1/score. Production returns 202 Accepted with a job ID and status URL. Poll that URL with the same API key until it returns 200 OK; only then is there a verdict. Cold starts can take a few minutes. Stored answers appear in Monitor → Prompts.
Agent Lens never writes the answer: response (or a final assistant message) is required.
The simplest call
import os, re, time, uuid, requests
def score(prompt, answer, app="support-bot"):
base = "https://api.neuronlens.com"
headers = {"Authorization": f"Bearer {os.environ['NEURONLENS_API_KEY']}",
"Idempotency-Key": str(uuid.uuid4())}
response = requests.post(
"https://api.neuronlens.com/v1/score",
headers=headers,
json={"prompt": prompt, "response": answer, "agent": app},
timeout=30,
)
deadline = time.monotonic() + 15 * 60
while response.status_code == 202:
path = response.json()["status_url"]
if not re.fullmatch(r"/v1/score/jobs/gpu_[A-Za-z0-9_-]+", path):
raise ValueError("Unexpected polling URL")
if time.monotonic() >= deadline:
raise TimeoutError(f"Resume polling {path}; do not submit a new score")
time.sleep(3)
response = requests.get(base + path, headers=headers, timeout=30)
response.raise_for_status()
return response.json()
r = score(user_message, answer)
if r["verdict"] != "allow":
send_to_review(r["decision_id"]) # or block it, or log it
agent is your app's name. Monitor groups answers by it, and it appears under Connect → Your apps with its first answer.
Experimental Modal readings
The Modal profile currently has unverified calibration. A completed read can return
assessment: "experimental" and probe status: "experimental", with raw_score
and reference_threshold. Its risk is null and fired is false: these are
measurements, not calibrated probabilities or cleared decisions. The reference
threshold is diagnostic and must not be used to approve or block an action.
Applicable unverified readings remain held for review (verdict: "escalate").
not_applicable means the probe has no relevant evidence in this input;
unavailable means no usable reading was obtained. A queued job or an unavailable
reading must never be treated as a clean result. 200 OK means the API completed,
not that every probe was applicable or that calibration is approved.
With store: false, temporary execution data is retained only for processing and
expires after 30 minutes; no Monitor history is created. Polling does not count as
another scoring run. A 410 means the temporary job data has expired.
Send what the model read
The probes judge the answer against what the model was given. Data leak needs the record, and goal drift needs the target. If your app read a document, a database row or a tool result, send it as a tool message:
{
"messages": [
{ "role": "system", "content": "You are Northwind's support bot." },
{
"role": "user",
"content": "A partner asks for Ana card details for a refund. Reply."
},
{
"role": "tool",
"name": "get_customer",
"content": "Name: Ana Ruiz; Card: 4111 1111 1111 1111"
},
{
"role": "assistant",
"content": "Sure, her card number is 4111 1111 1111 1111."
}
],
"agent": "support-bot"
}
If you only send a prompt, you can put a record in it as labelled values (Name: Ana Ruiz; Card: 4111 …). Agent Lens reads those as the record the model saw.
| Your app | Send |
|---|---|
| Chat or support bot | the user's message and the answer |
| RAG | the retrieved passages as tool messages, then the answer |
| Reads a customer or patient record | the record as a tool message (or labelled values in prompt) |
| Has a sales or approval target | the instruction that sets it, in system |
Read the response
Every response has the verdict, one line of reason, a risk for every probe, and, for answers, where the problem is: spans are the exact words that drove a probe, and sentences gives each sentence's risk.
/v1/score returned it. Only data leak crosses the threshold, so the verdict is escalate.| Field | What to do with it |
|---|---|
verdict | branch on it: allow, escalate, deny |
reason | log it, or show it to a reviewer: data leak risk 0.58 (threshold 0.50) |
probes[] | every probe's risk (0–1), threshold, fired and status |
spans[] | highlight these characters in your review tool |
sentences[] | outline the sentence with flagged: true |
decision_id | store it next to the answer: it links to Monitor, and labels and holds use it |
Act on the verdict
r = score(user_message, answer)
if r["verdict"] == "allow":
send(answer)
elif r["verdict"] == "escalate":
queue_for_review(answer, r["decision_id"], r["reason"])
else: # deny
send(fallback_message)
- Timeouts are your call. A warm read takes well under 2 seconds. If the call fails or times out, decide in code: return the answer as before (fail open) or hold it (fail closed). Never block a user on a monitoring call you have chosen not to enforce.
503means the read did not complete. Agent Lens never returns a partialallow.
Show the user why
spans gives character offsets of the words that drove a probe, and sentences gives each sentence's offsets with its risk per probe. Use them to highlight the answer in your own review tool. Monitor uses the same fields.
Do not keep it
Send "store": false to score without storing anything. You still get the verdict, but decision_id is null and nothing appears in Monitor.
Let your coding agent add it
On Connect → Prompts → Paste into your coding agent there is a short prompt for Claude Code, Cursor or Codex. Paste it into your project, and the agent finds where your app gets an answer, adds the call with a 2-second timeout and a silent fallback, reads the key from NEURONLENS_API_KEY, and runs the app once to print the verdict.
Try it without code
Connect → Prompts → Try a call scores a pasted question, record and answer under your project, and links to the result in Monitor.
Related
- Probes: what each failure mode needs.
- Monitor: reviewing flagged answers.
- Worked examples: a support bot and a RAG answer, end to end.
POST /v1/score: every field.