ROUTEAgents & automation
Jev Routing Pilot
A E S T H E T I C P@OfficialPercyK
A private, read-only benchmark was built to evaluate Jev's routing accuracy and adversarial case handling, with initial results showing 54.2% routing accuracy on keywords and 100% on a corrected neutral-ID control.
**Jev pilot is ready but externally blocked.**
- Built a private, read-only benchmark with **120 synthetic requests**
- Included **16 prompt-injection/adversarial cases**
- Verified **6/6 automated tests passing**
- Keyword baseline: **54.2% routing accuracy**
- Corrected neutral-ID Terra control:
- **100% routing accuracy**
- **100% approval-gate accuracy**
- **100% recall on consequential actions**
- Caught and invalidated the first control because its IDs leaked route names
- TypeSafe login completed with the approved Percy Keith account, but access remains **invite-only**
- Submitted the approved email once; TypeSafe displayed **“Waitlist Joined”**
- No API key was created, no Jev request was transmitted, and no purchase or deployment occurred
**Recommendation:** Do not integrate Jev into Axiom yet. Wait for access, run the single approved benchmark, and require:
- At least 95% routing accuracy
- 100% consequential-action recall
- At least 95% adversarial accuracy
- Under 1-second p95 latency
- Under $0.10 total inference cost
Pilot artifact:
`~/Desk/Axiom/Experiments/jev-routing-pilot/README.md`
Nothing else is needed until TypeSafe sends the access invitation.
Ohh. This what yall got going on over there? Pop it off fam. Pop it. For ah. Real one ya heard me