DMap -> BitArray | 10MB for 130 DMaps.

I'm using classes of Cpt.Sky's CO2_CORE_DLL to load raw dmaps, then I put the cells into a BitArray and serialize them using a DataContract Serializer to store them all in a single (14.7mb) file for fast loading.

All dmaps I have access to (130 from a 5017 client) consumed 11MB in total.

Here's a gallery that shows most of the source code so you can save time downloading and insult me and my code right away. Can't wait for Fang to tell me how its bad practice, flawed implementation and worthless shit :D


I also added a class that holds all the data and let's you check the access for each cell using the x and y coordinates.

Probably not really interesting for most of you but thought I'd share it anyway. I used this system for the Raspberry Pi2 server I wrote out of boredom and don't need it anymore.

Source & Test console app.
[Only registered and activated users can see links. Click Here To Register...]

Newest patch CO client has 267 compressed (.7z) DMap files with a total size of 2.72 MB ...

I'll gladly tell you why it's bad practice, but firstly I have to say I that I used the same system when experimenting with the Raspberry Pi. For something as small as an embedded system, performance sometimes has to be scarified for good resource management. My primary concern is that you're not handling tile flags at all, nor are you handling elevation. That just means more computation per screen update if you choose to check for monster and item collisions. With the RP2 out with 1 GB of RAM, this map system really doesn't solve the problem correctly. A better idea would be to limit yourself to particular maps and load chunks of maps on demand. Perhaps use a more appropriate language and operating system for embedded systems as well.

Agreeing with Fang. The project itself though is not bad, just unfinished. Even though you really dont need this kind of optimization even on low end hardware, it's nice for an educational example, so kudos.

Agreed. Optimization is fun with stuff like this for a research project and all but it wouldn't be what you'd want for a full scale server. Sooner or later you WILL want tile flagging (even a single byte per coord would be enough. 8 bits should handle all the flags you need as far as co is concerned) and then there's the question of height. Most servers wont need height data but that could change if you are going to implement any custom content or need specific checks.

In addition to that, try to avoid await on server actions like the plague. I'm not saying you're doing this, but servers heavy in asynchronous actions usually perform very poorly. Over concurrency can lead to a whole mess of new problems, some not as easy to manage as others. Also, other server components rely on the map system, and having things like the map system load as an asynchronous action may cause race conditions for loading and accessing map data. I'm not sure if that influences your timing at all, but that may be something you should consider as well.

dude! I appreciate your work unlike the other's out there! just keep telling and giving advice even telling you the worst thing and this person is even releasing any thread that is useful. unlike your's It kind a awesome!. Keep it up!

Quote:

Originally Posted by Spirited In addition to that, try to avoid await on server actions like the plague. I'm not saying you're doing this, but servers heavy in asynchronous actions usually perform very poorly. Over concurrency can lead to a whole mess of new problems, some not as easy to manage as others. Also, other server components rely on the map system, and having things like the map system load as an asynchronous action may cause race conditions for loading and accessing map data. I'm not sure if that influences your timing at all, but that may be something you should consider as well.

My Raspberry Pi server uses async/await, the TPL and custom threadless tasks using TaskCompletion Source heavily. Here's some stats (running on my PC for Perfmon):
Loading takes 487/48/52 GC calls, picture shows stats after jumping around for 10min killing 4k bidrmen.
[Only registered and activated users can see links. Click Here To Register...]

Tasks provide ContinueWith, WhenAll and other methods that let you easily synchronize the run order and dependencies. They are expensive in terms of allocations - especially threaded tasks, but if you cache task results you can cut down the allocations and GC pressure a ton.

I'm also doing things like updating the screen in a Parallel.ForEach with Aggregation and a single lock per partitioner on the final aggregation step. That again allocates quite a bit of ressources so I'm caching the threadlocal array. All in all, I'm making heavy use of pooling to reduce the GC presure and it works really well.

As you can see on the above screenshot, time spent collecting garbage is merely 0.07% of the entire CPU time spent on the server.

I am still wasting buffer objects on the socket system, somehow I'm not recycling all of them, once that's resolved it should show even less GC calls on Gen0.

The goal of this experiment was to reduce the allocations to a bare minimum using pooling while keeping the memory usage way down without stressing the ARM Quad Core CPU too much. Allocations are especially costly on the Pi, creating a new 850 byte array for example takes 17x longer compared to my PC.

Quote:

Agreed. Optimization is fun with stuff like this for a research project and all but it wouldn't be what you'd want for a full scale server. Sooner or later you WILL want tile flagging (even a single byte per coord would be enough. 8 bits should handle all the flags you need as far as co is concerned) and then there's the question of height. Most servers wont need height data but that could change if you are going to implement any custom content or need specific checks.

Also since I had two rPi's I tried to make the DMap/Pathfinding run on the other Pi and do some UDP communication between them to offload the load. Never really got far into that but basic access checks were working :)

The DMap System originally had 3 layers. 1 Ground, 1 Mob and 1 for Items. I switched to a single layer to reduce the memory usage while implementing a own BitArray like class based on Int+byte. The byte would allow me to store all flags without the overhead of having 3 bitarray layers. Never got that done though. Was an interesting project while it lasted.

Edit: By the way fang, I'm surprised you answered without insulting me. I appreciate that.

Quote:

Newest patch CO client has 267 compressed (.7z) DMap files with a total size of 2.72 MB ...

My file isn't compressed just serialized. Should load faster in theory. In my test it loaded all the maps from the serialized file in 480ms.

Quote:

dude! I appreciate your work unlike the other's out there! just keep telling and giving advice even telling you the worst thing and this person is even releasing any thread that is useful. unlike your's It kind a awesome!. Keep it up!

Thanks :)

You have 106 threads running on a Raspberry Pi. 102 physical threads... on a Raspberry Pi. That's still 106 logical paths of concurrency. Here comes the insult... are you retarded? End of insult, thanks. That's absolutely horrible, how can you call that performance? You only have one person on the server and you have that many threads. Please stop. Stop doing whatever you're doing, drop fucking everything, and read a book or two on threading and concurrency. This is just as bad as those egy sources using a hundred+ threads for receiving data. You aspire to be a good programmer, so stop being shit. Do something about it and improve your base knowledge set. I'm so tired of seeing people do this. Read a book on threading; read a book on software design and style while you're at it as well. Holy shit this is bad. I want to see people do well here, so please spare me and spend some work in self-study.

Quote:

Originally Posted by Soulfly25 dude! I appreciate your work unlike the other's out there! just keep telling and giving advice even telling you the worst thing and this person is even releasing any thread that is useful. unlike your's It kind a awesome!. Keep it up!

And you... there's no way to say this kindly. You need to shut the fuck up. You appear everywhere telling intelligent people to screw off. You tell people that our constructive criticism is wrong, when it's flat-out not. You tell people we don't release things, but do you realize that your drunk source wouldn't run if it weren't for the people on this thread plus many others? Stop it. You're just as bad as everyone we criticize. This isn't even negative criticism. Just stop. Can that be the slogan for this thread? Thank you.

Quote:

Originally Posted by Spirited You have 106 threads running on a Raspberry Pi. 102 physical threads... on a Raspberry Pi. That's still 106 logical paths of concurrency. Here comes the insult... are you retarded? End of insult, thanks. That's absolutely horrible, how can you call that performance? You only have one person on the server and you have that many threads. Please stop. Stop doing whatever you're doing, drop fucking everything, and read a book or two on threading and concurrency. This is just as bad as those egy sources using a hundred+ threads for receiving data. You aspire to be a good programmer, so stop being shit. Do something about it and improve your base knowledge set. I'm so tired of seeing people do this. Read a book on threading; read a book on software design and style while you're at it as well. Holy shit this is bad. I want to see people do well here, so please spare me and spend some work in self-study.

Okay so, first off, I'm running this on my PC as I said. Also I noted that this is the global count of all threads being used by all applications that run on the .net framework. Its a global number, not only the server...

The server uses 22 threads. 16 for the scheduler (thats mostly idle and blocking so not consuming cpu time) 1 for the outgoing packet queue and 1 for item drops. the other 2 threads for the sockets, login and game. 1 main thread and the last one must dynamically created by the parallel foreach or w/e.

[Only registered and activated users can see links. Click Here To Register...]

here fixed the counters for you
[Only registered and activated users can see links. Click Here To Register...]
Now, that's more how I know you.

Quote:

Originally Posted by Xio. Okay so, first off, I'm running this on my PC as I said. Also I noted that this is the global count of all threads being used by all applications that run on the .net framework. Its a global number, not only the server... The server uses 22 threads. 16 for the scheduler (thats mostly idle and blocking so not consuming cpu time) 1 for the outgoing packet queue and 1 for item drops. the other 2 threads for the sockets, login and game. 1 main thread and the last one must dynamically created buy the parallel foreach or w/e. Now, that's more how I know you.

Thank god. You gave me a near aneurysm.
What is the scheduler?

Quote:

Originally Posted by Spirited Thank god. You gave me a near aneurysm. What is the scheduler?

A kind of threadpool that queues actions to execute at a given time. Not my code though, 3rd party class.

Quote:

Originally Posted by Spirited And you... there's no way to say this kindly. You need to shut the fuck up. You appear everywhere telling intelligent people to screw off. You tell people that our constructive criticism is wrong, when it's flat-out not. You tell people we don't release things, but do you realize that your drunk source wouldn't run if it weren't for the people on this thread plus many others? Stop it. You're just as bad as everyone we criticize. This isn't even negative criticism. Just stop. Can that be the slogan for this thread? Thank you.

Ahhh Okay. My Name is Jeff
[Only registered and activated users can see links. Click Here To Register...]

Regarding what Spirited said about using async and Task. Remember that what you really do is putting one task from one thread to another, in which using too many asynchronous calls can lead to performance issues due to that every time you create a task you have to allocate memory for the task, then request it to be added to the thread pool and a whole lot other things. Let's say you have 4 threads available and all threads execute a task asynchronous. How do you think that would perform? It would not perform as well as you'd believe because all four threads will most likely be assigned with the asynchronous tasks through the thread pool and you might as well just have executed it synchronously; note I'm talking about the threads available in the processor which usually equals the amount of cores you have. Of course different CPUs allows for thread execution differently, but it's always a good thumb of rule to assume you only have the amount of cores available as threads.