Pisama¶
Process-level failure detection for multi-agent LLM systems. Catch loops, state corruption, persona drift, coordination breakdown, and convergence failures across Claude Managed Agents, LangGraph, n8n, Dify, OpenClaw, and Semantic Kernel. Open source. MIT licensed.
-
Python SDK
pip install pisama, then detect failures in 3 lines. No server needed. -
Full Platform
Dashboard, tiered detection, self-healing, REST API. Docker Compose.
-
API Reference
REST API for traces, detections, healing, and integrations.
-
Detection Reference
Per-detector documentation with F1 scores and accuracy benchmarks.
-
Cookbook
Integration examples for Claude Managed Agents, LangGraph, n8n, Dify, OpenClaw, and Semantic Kernel.
-
OSS vs Cloud
Compare the free SDK with the full platform.
-
Auto-Instrumentation
One line of code. Patches Anthropic + OpenAI SDKs automatically.
-
Chaos Engineering
Inject failures to test agent resilience. 6 experiment types with safety controls.
What Pisama detects¶
LLM agents fail silently. A coding agent loops for 40 minutes. A research agent hallucinates citations. A support agent drifts from its persona. Standard monitoring misses all of it. Pisama automates the error analysis loop: it reads every trace, names the first upstream failure, and reports binary failure modes with calibrated confidence, not generic quality scores.
Pisama ships 87 detectors across the MAST failure taxonomy, spanning planning, execution, verification, safety, and 5 agent frameworks. 54 are measured against real and synthetic traces, and 25 are externally validated at production grade (real-trace F1 of 0.80 or higher, mean 0.93). The rest are in active calibration.
| 84 detectors | 6 externally validated on real traces (F1 >= 0.80): completion, context, decomposition, hallucination, over_refusal, specification |
| 5-tier escalation | Hash ($0.00) state delta embeddings LLM judge ($0.02) human review |
| 5 frameworks | Purpose-built detectors for LangGraph, n8n, Dify, OpenClaw, and Claude Managed Agents |
| $0.05 avg/trace | 90%+ resolved at Tier 1-2, zero LLM cost |
| OTEL native | OpenTelemetry with gen_ai.* semantic conventions |
| Self-healing | Fix generation, approval workflows, rollback |
Supported frameworks¶
LangGraph · n8n · Dify · OpenClaw · Semantic Kernel · Claude Managed Agents · OpenAI Assistants · Bedrock Agents · Claude Code · Any OTEL source
Quick links¶
- SDK Quickstart:
pip install pisamaand detect failures in 3 lines - Cookbook: Framework integration examples
- Installation: Full setup guide
- Configuration: Environment variables
- Detection Reference: All detectors with F1 scores
- Deployment: Production deployment
Research notes¶
- Verifier Calibration in RL Environments explains why grader lineage, class-aware agreement, and regression gates should travel with RL environment reward functions.
- Pisama Fix-Efficacy Rerun Environment makes the datasheet executable: it re-runs a failing agent with a proposed fix applied and measures whether the failure is actually gone, instead of simulating it.