I’ve been working on FireFly, an open-source framework for training game agents from gameplay. It can use computer vision to detect platforms, monsters, NPCs and portals, record game observations + player inputs, automatically generate training data, and train behavior-cloning policies from demonstrations.
Current features:
- YOLO object detection
- UNet segmentation / platform geometry
- Template tracking and OCR
- Automated dataset collection
- Behavior-cloning training
- ONNX real-time inference
- Visual state/collection/policy graphs
It’s still under active development, so I’m interested in feedback from people who have worked on game automation before.
Github:
[Only registered and activated users can see links. Click Here To Register...]
Discord:
andrew0943
Current features:
- YOLO object detection
- UNet segmentation / platform geometry
- Template tracking and OCR
- Automated dataset collection
- Behavior-cloning training
- ONNX real-time inference
- Visual state/collection/policy graphs
It’s still under active development, so I’m interested in feedback from people who have worked on game automation before.
Github:
[Only registered and activated users can see links. Click Here To Register...]
Discord:
andrew0943