LLM Injection Defense Tool Migration to Jev

Sep 20

This project migrated an LLM-injection defense tool from GPT-5 nano to Jev, replacing static analysis and LLM stages with the `systemone` API and a multi-tier scoring mechanism, resulting in significantly faster performance and improved scoring.

SignalI ditched my static analysis stage and the LLM stage for the `systemone` API, using a multi-tier scoring mechanism.

Today I converted one my LLM-injection defense tools away from GPT-5 nano to @typesafeai 's Jev. I ditched my static analysis stage and the LLM stage for the `systemone` API, using a multi-tier scoring mechanism. So far it not only runs several hundred times faster but also scores better on my existing test harnesses. The LLM version ran 90-180 seconds (depending on reasoning) on my subsample of ~700 cases, but the jev version runs ~40 seconds (earlier runs were quite a bit speedier on jev, may be some slowness on the provider). Until I actually gave this a go, I wasn't sure what to expect on results. This new thing is.. weird. Not bad, but meaningfully different from using LLMs and most of my LLM-esque expectations don't hold. I think mostly this is going to dominate a lot of common classification and routing cases we see go to LLMs now. I suspect figuring out and integrating this new style of model is going to be a skill differentiator on building this new class of semantic routing/classification app. My next experiment is to bake it into a full LLM harness and allow the LLM to pre-run jev as a source of information. If folks are interested in either the injection detection or the LLM-using-jev work, let me know and I'll do a deep dive (maybe even in video?)

Original post