CHOOSEAgents & automation
TypeSafe Computer Use
Yash Chaudhary@YashChaudhary
This project enables a Mac to achieve plain-English goals by combining OCR and the Accessibility tree to drive actions, avoiding large model loops for screenshots.
typesafe-computer-use (Show HN): drive a Mac toward a plain-English goal without screenshot→frontier-model loops.
• OCR (Vision) + Accessibility tree → TypeSafe Jev Choice classifier → click/type; writer model only for free text
• Author-measured vs Claude Opus 5 bare screenshot: ~$0.0002/decision vs ~$0.032 (~155×); ~1.5s vs ~5.5s end-to-end step
• Realistic loop with history: ~$0.0002 vs $0.035–$0.08 (~170–390×); ~12-step task ~$0.003 vs $0.40–$0.90
• MIT · macOS 14+ · Python 3.12+ · never sends screenshots to a large model
github.com/awlevin/typesafe-computer-use