[Dev] Hellmouth Revival

[_tao4229_]

Originally Posted by pro4never

Strange, when I sent ANSWER_OK (from SYSTEM, to ALLUSERS, same as I do for normal login), it just sticks at Loading...


Then again I AM having some strange mini problems so could just be my source not wanting to cooperate. I'll log full char creation sequence later. Will save me some time prob

Thanks though.

Make sure you're using the right chat type
I have that for login it's 0x835, yet when I send it for creating a character it's 0x834.

[Arcо]

Updated
.Arco is out of the project.
Got a job, won't be able to contribute as much to this project.
Wish you best of luck Chris.

[pro4never]

Had a bit of time after work today.

Entity spawn packet functional. Not all values are logged but I got enough to do full char gears and such. I'll leave 'full' structure till I finish things like view gears at which point it will actually make a difference.




Before I add in spawning other objects to players I'm debating how I want to handle it. After taking a peek at korv's UG project the idea of doing a PROPER system for it does sound infinitely superior.

What is the opinion of those in the know on spawning surroundings to clients? Looking for something efficient but preferably easy to implement.


For those who don't know what I mean...

many/most public sources use something along the lines of..

foreach(KeyValuePair<uint, Clients> Target in Game.OnlinePlayers)
{
if(Client.Map == Target.Value.Map)
{
if(Calcs.Distance(Client.X, Target.Value.X, Client.Y, Client.Value.Y) < 16)
{
Client.Send(SpawnClient(Target.Value));
}
}
}

Or something similar...

Now this is all well and good from the standpoint of 'it works'.... but think for a second... you are doing two or more calculations PER sever object... every time ANYTHING changes.

This gets processor intensive very quickly. Say you have something like 10,000 monsters loaded into the server, 500 npcs and 100 players... plus every single dropped item on the map... every single time any of those players jump you are going to run something like 20,000 calculations simply to spawn what's on their screen (because you are running through 2-3 calculations for every single object in the entire server). For a higher end server, this is simply not acceptable.

The alternative (as I understand it from korvs basic explanation) is to divide the maps into sections. Each section containing information about what it contains (mobs, npcs, players, items, etc). You then assign the character to the 'section' it belongs to and simplly spawn objects that server contains

pseudo example...

foreach(object A in Client.MapSection)
Spawn(A);

On movement you would then calculate which direction you are moving/how close to the border of your section you are and will then switch to the new section you are in.

Again, that's a very poor explanation of his system and I've had almost no chance to look into it.... but what is the best way to handle EFFICIENT spawning of server objects to a client? I am really hoping to be able to refine this server as I go so that I don't need to come back and re-code huge sections of it after the fact (most likely breaking other things that rely on the original code). It's far easier to just do it right the first time.


<edit>

So I just was thinking... I'm probably not thinking things through here and it would take up alot of ram but would doing a per cell system work well?

Eg: you have your map loaded into the source and for each cell you have values

Accessibility, height, MobObjects, itemObjects, ClientObjects, NpcObjects

These structures are stored in basically rows/columns associated with their X/Y values.

Any time a server object moves/is created their cell is changed to that location and you could do something to load into the character it's surrounding based on a 16x16 or w/e grid surrounding them and then spawn these items to it (so you only have to do a for loop through the nearby tiles to check the cell values for entities/items)

All I can think of for this though is heavy ram usage (storing that much data in memory for every cell/map....)

Alternatively a binary file could be used/read for the maps to reduce ram usage while maintaining speed and ease of access...


Guuhhh so many options and I can't think what would be a good one.

Advice appreciated as always.

[Nullable]

Your best bet is spatial partitioning, ex. for co implementing a Quadtree, there are different implementations but here is a summary & example of the whole concept

Edit:
those too
Space partitioning - Wikipedia, the free encyclopedia
Quadtree - Wikipedia, the free encyclopedia

[Korvacs]

The system that pro was describing is a form of spatial partitioning, however a much simplified version, you query an area, the idea of the sectors is to reduce the amount of sectors that need to be queried (ie, if query area fits within 1 sector, just query one sector, if not then ask map object to query neccassery sectors and return objects) quadtrees have the advantage of being able to split into more and more peices when the number of objects increase, but you'll find the more they split the more work the server has to do to insert new objects into the quadtree (it has to go down through more levels), which is fine to a point, but with 20k monsters spawned it'll become a trade off between insert speed and query speed.

Cell Partitioning is certainly an idea, but unfortunately its just not realistic in terms of RAM, storing 1 byte(bool) for access uses about 30mb, so thats 30 million cells, adding a second will double it, and thats just 1 byte, adding many many other arrays and objects is going to pile memory on top of that.

As for spawning things around the client go, ive yet to see a system that works correctly, theres an issue with the client where you can send a spawn packet to early, and i cant seem to find the correct area to query. Ive looked at various sources and none of them seem to do it correctly or efficiently.

[CptSky]

The loop in each object can work for little server. For sure, QuadTree will be very good, but with Co, you can use other way. Just splitting the entities in each map will be correct. I'm sure that you will never have 20'000 entities to spawn on the same map, except for really big map.

[pro4never]

Originally Posted by CptSky

The loop in each object can work for little server. For sure, QuadTree will be very good, but with Co, you can use other way. Just splitting the entities in each map will be correct. I'm sure that you will never have 20'000 entities to spawn on the same map, except for really big map.

While I agree it's not super important at the second... but if we do get this server to a playable point where we have most of the features done and are ready for open beta, I'd really not want to have to go back at that point, re-write all the spawning code and then go through every feature that requires spawning and update them to use quadtree or a similar spacial partitioning system.

If it's going to be efficient and save me re-writing significant sections of code in the future, I'd much rather do it now.

Looking at existing servers out there... if we get to public beta level (giving myself at least 3 months or so before that happens, we'll see how things go) and if noah can finish up the side project then I'd say top 3-5 server is going to be rather easy between running fully custom source that won't be ******* up every 2 seconds, running the latest tq patch and project indigo... it's not that much of a stretch to imagine that it will be SLAMMED with players (if not in the long term.. there will be a large influx when it's first released of ppl wanting to see it). Thousands of objects per map is not that much of a stretch when you factor in

Ground items
Monsters
Npcs
Players

I mean bird island alone will probably have what? couple thousand monsters in total + all the items dropped by them? That's a shitload of objects to be iterating through and then spawning

[CptSky]

Originally Posted by pro4never

[...]

Yes, BI will have a lot of entities, but not enough. I'm sure that you will never reach the 10'000 entities on the BI map. You will have something like 2K or 3K of monsters and you will never reach 3K of objects. Co isn't a really big game. Splitting the entities for each map will work fine and you will not have to recode the spawn system. A loop in this will not really affect the gameplay.

[Nullable]

Originally Posted by Korvacs

The system that pro was describing is a form of spatial partitioning, however a much simplified version, you query an area, the idea of the sectors is to reduce the amount of sectors that need to be queried (ie, if query area fits within 1 sector, just query one sector, if not then ask map object to query neccassery sectors and return objects) quadtrees have the advantage of being able to split into more and more peices when the number of objects increase, but you'll find the more they split the more work the server has to do to insert new objects into the quadtree (it has to go down through more levels), which is fine to a point, but with 20k monsters spawned it'll become a trade off between insert speed and query speed.

It is true that with more nodes branching, inserting new objects becomes CPU intensive. However as CptSky is saying, co's maps are fairly small to contain 20k entities at the same time. IMO a quadtree would be a proper strategy to use in such case.

Originally Posted by Korvacs

As for spawning things around the client go, ive yet to see a system that works correctly, theres an issue with the client where you can send a spawn packet to early, and i cant seem to find the correct area to query. Ive looked at various sources and none of them seem to do it correctly or efficiently.

As for that part, the area to query seems to be random on official servers (or maybe dependent on the speed of objects/client).Judging by the results of a few hours of logging spawn packets and calculating distance, it seems to be in the range of 18 ~ 35. As my client in some cases receives the packet when a player reaches a distance of 34 steps and in other cases when a player reaches a distance of 20 steps.

[Korvacs]

Ive been experimenting with a bounding box around the character object of various sizes more around the 26x26 -> 32x32 none of them seem to be correct, sometimes i send a spawn packet and the client just never spawns the target object, other times it does, but in all cases the server sends the spawn packet correctly. Its an irritation that ive had for a long time, i can never find a decent value.

[pro4never]

Yah I'm definitely thinking that I'll attempt some sort of quadtree setup at some point but for now I may just end up doing splitting by map (thanks for that idea).


VERY strange problem. One of the coders is trying to test out the source to test features he's been coding but the client was crashing when he would try to log in. I setup the server and had him log into that to see if there was some problems with his side and it turns out as soon as I send the general data packet to finish up the login there is a huge memory leak in conquer till it eats up all memory and crashes...

It honestly ran up to 1.7 gig ram on his computer and then gave "system out of memory" error and such. I commented out the packet and it doesn't do it so it's obviously just the one.

Note: I'm not using time stamp currently but it's caused no problems from my side... Also tried replacing his conquer.exe in case it was some issue with the client.

[Santa]

Not only trying to login to your server, tried logging into TQs server and the same thing happened, got up to 2gigs that attempt.

Re-installed the client and it works, with the same **** patches...

[pro4never]

Minor progress... mostly very behind the scenes work were we aren't coding new features... we are doing the base systems to support them in the long run.


Update Wed Sept 8 2010
Schmidty has been awesome with his work on npcs.

We now have full npc system loading from text files (using full C# syntax. His current version is using microsoft.compile afaik. He was using Csscript before that)

I've added a database for items and npcs and am loading them on server start. They all spawn properly and options are working. Simple stuff but the way we are running things means they will be completely seperate from the main source meaning no re-compiling or restarting to see changes... hell not even a need to reload anything into the source! it simply pulls the text file when the npc is used and executes the code inside.



We've decided he's gonna stick with some.. further npc work. As part of the umbrella of Project Indigo, he will be creating tools for admins to be adding and modifying npcs on the fly from external computers without ever restarting the computer or changing anything.


We will obviously also have database commands added to place npcs in the server instantly and update script id's, meshes, and all that good stuff so that basicly, we code the system and then when we go closed beta, the approved members will be able to build the actual ingame world properly.


So basically when done this will mean fully customizable npc systems from any computer with net access instantly. No restarts, no reloading stuff.. NOTHING.

It's actually kinda cool if I can say so myself ^^.




Other progress: Noah has started on actual development of project indigo. Not sure how far he got as I had a bunch of stuff I needed to do with school today.. changing schedule, osap, etc.


NOTE: FORUM IS ONLINE!!



We've thrown up a crappy free forum to track stuff and let developers share code snippets/get help from eachother. It's shitty but w/e. Forwarded old domain name temporarily. This will be what we use until we get a proper vps to host when it goes closed/open beta.


NOTE: Entry page will be added soon using the hellmouthco.com domain name which will allow users to sign up for closed beta and a mailing list to keep up to date with things eg: we will use this mailing list to let people know when beta is opened up and to give out pre-beta slots as we need more stress testing. That's a week or two off but it will be there. Aka use the forwarded url vs the end point.





Spawning other characters from server (using temp eq as I still haven't bothered doing full loading of char inventory/equipment from database)

Spawning npc from database

Npc script from .npc file loaded on the fly

[-Fáng-]

Awesome =] I'm working on NPCs in my source too. I was going to do a full database system so that I don't have to restart the server or anything! =]

[pro4never]

problem with that is it's not user friendly to use sql for an entire system (think about it... all those different checks based on char stats?)

This system uses full C# syntax and formatting just loaded in through .npc files on use.

It also allows full access to our GameClient variables so we can do the whole.

If (Client.Level > 120)
{
Talk(Blah, Client);
Finish(Client);
}

And yes... syntax is almost identical to CoEmu meaning users will find it very simple to help us write scripts.