[Release/Open Source] FireFly – AI game automation framework tested on Cosmic Client

[Goombax]

I’ve been working on FireFly, an open-source framework for training game agents from gameplay. It can use computer vision to detect platforms, monsters, NPCs and portals, record game observations + player inputs, automatically generate training data, and train behavior-cloning policies from demonstrations.
Current features:
- YOLO object detection
- UNet segmentation / platform geometry
- Template tracking and OCR
- Automated dataset collection
- Behavior-cloning training
- ONNX real-time inference
- Visual state/collection/policy graphs
It’s still under active development, so I’m interested in feedback from people who have worked on game automation before.
Github:

Discord:
andrew0943