FILTERSearch & retrieval
Jev for LLM Routing
Reid Fletcher@LxKus
1:13Jev is used to filter candidate files for generative LLMs, reducing the number of files sent to the model, tokens processed, and overall cost, while maintaining recall.
Using generative LLMs for routing is a mistake.
We put @typesafeai Jev in front of Claude to filter candidate files in @aitionapp:
• -33% files to model
• -25% tokens | -23% cost
• $0.037 to score 543 files (0.38s/file)
• 0 loss in recall x.com/LxKus/status/2100611468580323414/vid