← Index

CONTROLAgents & automation

TypeSafe Jev Desktop Interaction

Aaryan@aaryantwt
Project photo 1

This project enables computer use within a Dockerized Linux environment by using the accessibility tree as 'eyes' and TypeSafe Jev to answer typed questions for the next action, with a11y invoke or xdotool serving as 'hands'.

Computer use without shipping screenshots to a frontier model (thanks to @typesafeai ): a Linux desktop in Docker → accessibility tree as eyes (50ms) → TypeSafe Jev answering typed questions for the next click (0.9s/step) → a11y invoke or xdotool as hands. Fresh VM per run. noVNC for humans. github.com/maskjelly/TiVM @typesafeai @tedkalaw @hackgoofer @CompleteSkeptic
Original post Code