So I'm gaining an interest in artificial intelligence again. With hellmouth slowly being rebuilt, this is something I'd like to PROPERLY add (not the simple test versions I've coded before) this for the open beta stage of the server (month or so away minimum)
My thoughts.... If you can limit the FUNCTION of AI and limit the SCOPE of AI then it shouldn't be very difficult to create a near player like experience.
As far as function and scope goes we really only need to look at 1 thing...
Function: Provide a pvp experience which rivals that of a human opponent
Scope: Pvp ONLY
By focusing on what exactly it is we want to create it will allow us to avoid issues entirely which would otherwise ruin the goal of the project (bots which recreate reasonably 'inteligence' pvp gameplay). We will NOT be worrying about chat logic or anything outside the realm of "kill enemies, protect and help allies"
My current idea for coding this is to use a separate logic server. The logic server will be updated with entity variables (internal and external) and the logic will translate this (through the filter of desires, short term and long term memory) into a decision. The logic server then will send the resulting 'choice' to the game server and enter what could be considered a 'listening mode' in which it will use information being returned to modify short term memory (which hopefully in some cases can be translated into more abstract rules for long term memory... that's a bit more advanced though).
Eg: Jump to X1/Y1 and FB X2/Y2. Server then returns that you have killed player Y, this would serve to reinforce the factors used to make this decision... but if it resulted in the bot being killed or taking large amounts of damage, there should be a negative feedback to the decision making processes.
Anyways, once I get some time I'll start working on turning it into a working model vs simple theory I'm hoping to be able to break down the function of every single thing required before I actually code the server though.. with this size of a project, if you lose focus of what a system is responsible for, you can get screwed over with surprising speed.
<Update: June 15>
Not done anything more on this. Got LOADS to do before a AI/Bot system becomes remotely applicable (working on a proxy for latest version of co so I can log packets and then work on a fully custom 5250+ source... so yah... gonna be a while lol)
I did get bored thismorning though and started again on the Logic Server. Wrote the communication for GameServer<->LogicServer as well as some of the structure for how to actually handle the data transfer. Should make the final version go much quicker when it's needed.
Ooh: I'm also thinking that once I'm done logging packets with the proxy, I may hook it up to the logic server and have it be a proxy for botting on real co. That would allow for some serious AI for the autohunt/loot and make it much more fluid.
<Updated: May 27>
Released the crappy test source I had this version of the bot in. Feel free to muck around with it and test the duelist bot.
Note: Very poorly coded version. it was just me testing out how to best go about features and I'm re-coding and organizing everything much nicer in the new version. This was just a testbed.
<Updated: May 24>
Video of the test version of the duelist
Finished getting basic version working. I'm re-writing everything from scratch though. Making it far more streamlined and self contained so that I can transfer it between sources (and eventually release) easier.
Temporarily using Hellmouth as my test source simply because there are less bugs I don't know about as compared to ElitecoEmu (such as the gear calculations). This will just make things alot easier for the time being.
Main Challenge Atm:
I need to find a logical/scalable way to build my AI system. My current plan is to build a connectionalist network to handle the bots motivations and choices. That will allow for a real AI system to be put in place in the future and allow for simple adaptation for now. Basically I wanna do 3 layers to the network.
External Stimulus:
All world environments that the bot should know about including but not limited to...
- Number of monsters in the area (screen)
- Health of monsters in that area
- Closest monster to bot/distance to it
- Distance to 'master'
Internal Conditions:
All internal variables that the bot has including but not limited to
- Jump Variables (last jump time, last jump try, last jump distance/time before next, can bot jump, etc)
- Walk Variables (last walk time, last walk direction, time between steps, walk target (x/y), can bot walk
- Attack Variables (current target, attack type, last skill used, last skill time, last attack time, etc)
- Statistics Variables (health, mana, stamina, defense and all that good stuff)
Actions Layer:
Will take all input and determine what action should take place
- Jump (type of jump, where to)
- Walk (where to)
- Attack (what type/skill/atk what)
- Wait (Do nothing)
The way neural networks work means that you feed in data and they 'trickle down' through nodes in the network eventually resulting in a decision. The resulting decision is compared with what "should" happen (learning) and the weights given to different nodes/connections is altered changing future outcomes.
The way I see it is that if I do something along those lines, you can set default values for nodes/connections to fit approximately with how you want the bot to behave. Then add in a training mode which causes the bot to ask if its decision was correct after each action. The weights are then fixed in place when training is completed and little/no changes in behavior occurs after that.
Add in a bit of random chance and you will have a very 'intelligent' bot that can provide alot of uses ingame.
I still am working on structure for the network before I actually do any coding but if anyone wants to look at it and criticize my ideas... I really do welcome it.
here's a link about neural networks in C#. Mine would be a similar concept but not quite the same (wouldn't have a solution to compare it to, rather would choose 1-2 actions from the result and act on them.)
PROGRESS ON ACTUAL BOT:
- Skills now work: So far all I've tested is fastblade. (linear attack function works though so that covers all of those skills. Also means I can convert all the existing skill functions EASILY)
- Damage now works (Player>Bot and Bot>Player both deal proper damage(ish))
- Behavior types added. This allows for different types of bots not to be running the same damn checks and wasting cpu power. I split it off into duelist and companion bots for now but I have a few more types planned including PVE, Boss and Team Pvp types. Just makes adding stuff in the future easier
- Added more functionality to the walk/jump functions making them more fluid
- Added Duelist jump type (doesn't follow the player like the other one)
- Transferred over large amounts to hellmouth source and re-wrote it as I went making it FAR more streamlined.
So as we are right now... we have duelist bots working rather nicely (they jump around, attack you, take damage, die, revive themselves and keep going)
Companion bots are very basic. They have walk and jump functionality to follow the player and that works very well. I need to modify it some so when the player goes through close spaces the bot doesn't freak out and stop following.. that's due to the random coords it generates being turned down by the dmap handler (although I have fixed most of that). They also have a nice command structure to enable/disable jumping and walking, attack a specified player with fb (only does it once as it is right now. Easy to change though), remove bot from world, re-do the bots equipped gears and some other basic stuff...
</Update>
-------------------------------------------------------------------------------------------
So I've made some progress. It's actually far easier than I thought so you can disregard most of my speculation in this post although I still welcome better ways of doing things. Right now I'm not aiming for efficiency as much as I am getting something out there that's WORKING. I'll end up re-writing a streamlined version if I ever intend it for deployment or release.
Progress:
- Linked the companion to the clientsocket (not required, I can easily create a separate one. It was just easier for testing AND allows for a true companion system later down the road)
- Re-wrote some packets and game systems to accommodate BASIC functionality such as displaying the loaded bots to clients in range and some movement (Jump and view gears work)
- Added structure to store all loaded bots on the server
- Added new thread system to run through all loaded bots to perform any future actions and AI.
- Added/adding new calculations dealing with bots (jump distance, walking, etc) to allow for future implementation
- Tested threading system. It all works! Bots now follow their owner around like lil puppies lol.
- Linked Bots with their "owner" (NOT required! Still able to have bots without owners... it just makes features easier in the future)
- Posted SOME BASIC info on how to add them to your own server. It's not a release, you still need to know what you are doing to add them successfully! If you want it, it's a few posts down.
For the record I'm using CoEmu as I'm damn familiar with it. I'll probably convert it to impulses if I ever get it to a completely working state.
The idea is this... I was originally wanting to try to create a bot/clone character for use in pvp or to test server loads (it's alot easier to load up a hundred fake clients all doing some simple pre-programmed things than it is to try to get a bunch of people on at the same time when I don't have internet lol) After starting a little on that I realized you could do ALOT more with them than just that... You could actually completely change the game if you chose to and make them alot more intelligent and useful than guards.
Use 1: Sidekicks. You could give it out a ton of different ways to players... something for 3rd/4th reborn... donation... super hard quest... or however you wanted. Basically it would create a permanent entity that would have some basic artificial intelligence coded into it that would follow you around and could perform a wide range of tasks such as
Companion system Features:
- Looting
- Killing monsters (using skills/xp skills as well in a somewhat intelligent manner
- Summon/dismiss the companion with a custom skill
- Intelligent movement from the companion including walking and jumping (Not ontop of you instantly like many guards do now but rather random delay between actions and using some AI to determine where it should go as well as random chance)
- Customizable gear (I'm thinking I'll use a separate warehouse to control the inventory for the companion and use the AI to determine what the companion chooses to wear from its inventory. I've already done a separate warehouse before when I did hellmouth)
Other uses for the same system
- Pvp drones (think Acids fb bots)
- Testing server loads
- Filling out server (Think about it.. somewhat intelligent players running around playing the game... would make some maps feel alot less empty)
- Intelligent teamates for quests: Have quests that require a full team and fill in any missing slots with bots.
- Custom events/quests where bosses are bots and actually display some real intelligence and fight back properly using skills/movement
Again.. high hopes that I'm sure I probably won't actually finish but I do wanna see how far I can go with the idea as it's not something anyone has ever really released or gone far with before.
Ooh and before I get asked: Yes, if I get the system to a semi working state I will most likely release it. Most likely not with all of the features but if it works I'll post some version of it for ppl to mess around with. Hell if someone wanted to give me a hand I definitely wouldn't mind either lol.
So far I got very little functionality done. Re-wrote the view equipment so that it would also apply to bots.. Pot calculates off cause I was using eliteCoEmu to test it out and I don't think it calculates it properly.
