I recently came across an interesting program called Screph.
It is a visual tool for working with GUI automation. You can mark interface elements on screenshots, describe clicks, keyboard input and transitions between different UI states, and then use this information to create automation scripts or provide context to AI coding agents.
It seems potentially useful for desktop applications without a convenient API, as well as for QA, computer vision experiments and automation of legacy software.
Has anyone here tried Screph or worked with a similar visual approach to GUI automation?
- Website:
