Skip to content
AgentStat

Health score

Each agent gets a 0–100 health score, recomputed every half second. The two token-rate parts are always computed. The two signal parts, hallucinationRisk and latencyTrend, are null until you provide the signal, and are then left out of the score.

Read it with onHealthChange or ref.current?.getHealth(id):

<AgentStat agents={agents} onHealthChange={(id, health) => console.log(id, health.score)} />

The parts

FieldMeaning
score0–100 composite of the parts below, minus penalties for active anomalies.
tokenEfficiencyHow close the recent average rate is to the agent's expected band (default [5, 25]; see below). 100 = inside it. Always computed, from a rate of 0 if none was reported.
stabilityHow steady the token rate is relative to its own level. 100 = steady. Always computed.
hallucinationRisk1 − confidenceScore as a percentage, or 100 while the status is hallucinating. null until you send a confidenceScore (0–1) or that status.
latencyTrendimproving, stable or degrading, from latencyMs reports or the time to first token of events. null until there are four.

AgentStat can't detect hallucinations from telemetry. hallucinationRisk only reflects a signal you send, for example a grounding or eval score:

// hallucinationRisk becomes 18
ref.current?.updateAgent('writer', { confidenceScore: 0.82 });

How it's computed

base  = weighted mean of the parts present
          tokenEfficiency          × 0.35
          stability                × 0.25
          (100 − hallucinationRisk) × 0.30  (if not null)
          latency score            × 0.10   (if not null)

score = base − anomaly penalties, clamped to 0–100

The latency score is 100 for improving, 50 for stable and 0 for degrading.

The weights of the parts present are renormalized, so an agent with only a token rate can still reach 100. An agent that has reported nothing scores 88: a rate of 0 against the default band gives tokenEfficiency 80, and stability is 100.

  • Rate window: token-rate parts use the last 5 seconds.
  • tokenEfficiency falls off with the distance outside the band, relative to the band's upper edge: 100 − distance ÷ upper edge × 100. It means the same for a 10 tokens/s agent as for a 1,000 tokens/s one. While an agent is waiting, no output is expected and it isn't penalized.
  • stability is 100 − the average step between samples ÷ the average rate × 100.
  • latencyTrend compares the newer half of the last 20 latency reports with the older half. A change within ±15% is stable.
  • Anomaly penalties: 15 points for a warning, 30 for a critical one, times 1.0 for a stall, 0.9 for thrash and 0.7 for a spike.

The score is eased so it doesn't flicker; a drop of more than 15 points shows immediately. onHealthChange fires when the score moves by a point or more, or when hallucinationRisk or latencyTrend changes.

Calibrate the band

The default band is 5–25 tokens/s. Set one per agent that matches your model, so tokenEfficiency means something:

import { createAgent, type Agent } from '@dan-build/agentstat';

const agents: Agent[] = [
  { ...createAgent('coder', 'Coder'), config: { expectedTokensPerSec: [40, 120] } },
];

For agents added by a stream, use stream.describe(id, { config: { expectedTokensPerSec: [40, 120] } }).