couple questions about custom source (NHibernate,networking socket,dmap reading)

[go for it]

this is my first try for a custom source, im almost ready to finish everything but i got some questions that bothers me

why NHibernate instead of custom class to read/write from/to database ? about efficiency, implementation and compatibility

why would you call a socket class better than other ? i've had reversed infamousnone socket class dll, saw impulse socket class at trinity and at pp, saw 2 more , all sounds the same, i know any would do but i want to code something decent

dmap reading, i've heard alot how the way of reading dmap files taking too much memory at impulse base, i've also seen cptsky dll and the dll pro4never used at his 5095 base, which is better and why ? is the structure of this files available ? any information ?

does a thread per client the best design you may have ? is there a better way?

thanks for reading, please kindly help me out with information i need, always remember you don't have to answer all of them just tell me what you know, also if you have any relative information about coding a server please kindly share it with me, you may always pm me
thanks and have a wonderful day

[pro4never]

Database management: Why write your own code when dozens of people with more knowledge than you have done your job for you?

Personally I used nhibernate because it was a well documented and robust framework that handles all aspects of database threading, querying and more.


Socket systems: Boils down to efficiency, threading and how it handles packet fragmentation. The best example I've seen publicly would be hybrid's latest source where they read just the packet length and then wait until they've received the entire packet body. This avoids the need for packet splitting and results in non fragmented packets.


Dmap reading: There's many options but the main concern is that the straight dmap files contain more information then you need. For example they use a uint value to store access information (4 bytes) when the values they are storing never go beyond ushort. As a result the files are almost twice bigger then they should be.

Most servers also do not care about map objects (besides their impact on access information) so by parsing these files, compacting them to their smallest/most efficient size then you use less memory and place less strain on your server. Tinymaps is again, someone else doing a very good system to save you the time of doing it yourself. It's nothing super fancy but it's solid and does its job well. It's also open source so you can make any changes you need to (eg I added portal information to the tinymap files because I needed the portal id's to look up from tq binary database)


Threading system: Never use thread per client. it's not scalable and is completely useless. There's a bunch of different options but it really boils down to how you want your game's logic to flow.

Personally I'm a fan of using management threads which call a tick event for server manager (map manager, player manager, etc) which then ticks any sub elements that manager is tracking based on the current time.

This creates a fairly robust threading system for your entire server's delayed events all by using a handful of permanent threads.

That being said, threadpools are another great way of handling things but I've avoided them for the most part preferring to do a small amount of threads basically handling the game loop.

[Spirited]

Woah, that's a lot of questions. I'm going to try to answer than in the order that was received. Hopefully I can help clear a few things up (though you'll have to look up the terms I'm going to mention - I can't explain essays worth of material).

1. NHibernate is an object-relational mapper, meaning it allows us to compare sql data types with C# data types. This is helpful because structured query language database management systems can only store and manipulate scalar values such as integers and strings organized within tables. Using NHibarnate, we can convert the object values into groups (objects... like classes). It's like stored procedures, but with objects. Hopefully I just explained that correctly. Jack (Korvacs) would be a better person to talk to about that.

2 / 4. There are different types of socket systems. There are blocking, non-blocking, and asynchronous socket systems. Blocking means that the server runs on one thread - blocking the thread to process one client at a time. Non-blocking (the model you refer to in question 4) means that one thread is created per client, not to block processing. This is problematic for many connections though because only 1 thread can be processed by one physical/logical core at a time on your processor. Asynchronous is the best option, creating an I/O Completion Port for managing a thread pool. The threads created process asynchronous requests, optimizing your server for multiple client connections. Instead of having 200 threads for 200 clients, you'd have 16 (for example) processing them.

3. Dmaps are just data maps. Creating a map system is just a way of loading that data and storing it efficiently. Most servers now convert dmaps to versions that only contain tile access data. That allows for faster loading times. As far as storing the data, that's completely up to you. One way would be a byte array, but you really don't that many bits to just store one bit for the access type. If you want to include elevation into it, then you just add another short to keep track of per tile.

[go for it]

well i would like to ask 3 more questions, may i ? thanks

first, should i use interface if im using a class inheriting other classes including the methods i need to use
for example, it would be useless to create maybe interface/abstracted class for packets if im going to create a class that inheriting other packet classes a private byte[] and methods that use this byte[], so it would be a waste to use interface/abstracted at this case, right ?

second, does removing all namespaces from classes to make them global effect efficiency ?

is it a good programming practice to have all constants and enums at there own classes or may i move them to special class to clean things up ?

any more tips about the design ?

thank you guys for that awesome information you gave me, that's so useful

[Smaehtin]

Originally Posted by Fang

2 / 4. There are different types of socket systems. There are blocking, non-blocking, and asynchronous socket systems. Blocking means that the server runs on one thread - blocking the thread to process one client at a time. Non-blocking (the model you refer to in question 4) means that one thread is created per client, not to block processing. This is problematic for many connections though because only 1 thread can be processed by one physical/logical core at a time on your processor. Asynchronous is the best option, creating an I/O Completion Port for managing a thread pool. The threads created process asynchronous requests, optimizing your server for multiple client connections. Instead of having 200 threads for 200 clients, you'd have 16 (for example) processing them.

Let me just stop you there. Non-blocking sockets do not create a thread for every connection made. Non-blocking simply means that functions like connect, recv, etc. don't block the thread when being called.

[Spirited]

Originally Posted by Smaehtin

Let me just stop you there. Non-blocking sockets do not create a thread for every connection made. Non-blocking simply means that functions like connect, recv, etc. don't block the thread when being called.

Yes, you are technically correct. Traditionally non-blocking socket systems are synchronous, but asynchronous socket systems are also non-blocking and can be referred to as a non-blocking socket system.

Originally Posted by go for it

well i would like to ask 3 more questions, may i ? thanks

first, should i use interface if im using a class inheriting other classes including the methods i need to use
for example, it would be useless to create maybe interface/abstracted class for packets if im going to create a class that inheriting other packet classes a private byte[] and methods that use this byte[], so it would be a waste to use interface/abstracted at this case, right ?

second, does removing all namespaces from classes to make them global effect efficiency ?

is it a good programming practice to have all constants and enums at there own classes or may i move them to special class to clean things up ?

any more tips about the design ?

thank you guys for that awesome information you gave me, that's so useful

I'm not sure I understand your first question. Using an interface would give you the ability to process two similar classes as one data type. For example, classes Cat and Dog share an interface IAnimal. In IAnimal, methods are defined that Cat and Dog both need to implement. When processing Dog as IAnimal, only those methods are available until IAnimal is converted back to Dog. When using inheritance, you're inheriting base methods from another class. For example, class Animal is the base of class Pig. Pig contains all public and protected variables and methods defined in Animal.

Regarding namespaces, unused namespaces include libraries into your executable that you're not using. You could remove those references to make the file smaller, but otherwise it really doesn't effect performance that much, especially on modern computers.

Finally, organization is completely up to you. I don't normally put enums in their own file or class, but if you find that enum definitions are making your class definition disorganized, you can use partial to split the class definition up into two files. It's not all that normal to do that though. Partial is usually used to split up large classes based on methods.

[Smaehtin]

Originally Posted by Fang

Yes, you are technically correct. Traditionally non-blocking socket systems are synchronous, but asynchronous socket systems are also non-blocking and can be referred to as a non-blocking socket system.

Still, I don't think I've ever seen a non-blocking socket system that would spawn a new thread for every connection made though.

[go for it]

Originally Posted by Fang

Yes, you are technically correct. Traditionally non-blocking socket systems are synchronous, but asynchronous socket systems are also non-blocking and can be referred to as a non-blocking socket system.



I'm not sure I understand your first question. Using an interface would give you the ability to process two similar classes as one data type. For example, classes Cat and Dog share an interface IAnimal. In IAnimal, methods are defined that Cat and Dog both need to implement. When processing Dog as IAnimal, only those methods are available until IAnimal is converted back to Dog. When using inheritance, you're inheriting base methods from another class. For example, class Animal is the base of class Pig. Pig contains all public and protected variables and methods defined in Animal.

Regarding namespaces, unused namespaces include libraries into your executable that you're not using. You could remove those references to make the file smaller, but otherwise it really doesn't effect performance that much, especially on modern computers.

Finally, organization is completely up to you. I don't normally put enums in their own file or class, but if you find that enum definitions are making your class definition disorganized, you can use partial to split the class definition up into two files. It's not all that normal to do that though. Partial is usually used to split up large classes based on methods.

thanks once more for helping me out
what i meant at the first post is
if chat packet and attack packet share same interface then they can both be treated as that interface, would be useful to have something like
send (interface ipacket){}
but do you think there is a way around this ? or atleast how this could be that important later on ?

at this particular case i've figure out that using an interface means you are already having something in common between all classes that is going to share this interface, so you may always get around this using the common objects/things they shared at the very first place (for this example it would be the byte[]), but could that be a general rule ?
i honestly don't find a reason to use them, but would an empty interface do the trick ?

[Spirited]

Originally Posted by go for it

thanks once more for helping me out
what i meant at the first post is
if chat packet and attack packet share same interface then they can both be treated as that interface, would be useful to have something like
send (interface ipacket){}
but do you think there is a way around this ? or atleast how this could be that important later on ?

at this particular case i've figure out that using an interface means you are already having something in common between all classes that is going to share this interface, so you may always get around this using the common objects/things they shared at the very first place (for this example it would be the byte[]), but could that be a general rule ?
i honestly don't find a reason to use them, but would an empty interface do the trick ?

Check out Impulse's system on sending packets. He uses a packet interface to do that. Another way of doing it is having an abstract class that has implicit operations for converting itself to the same type (byte array). Then, you could just inherit that abstract class (that might contain methods for writing and reading the packet as well). I find that to be a lot cleaner and user-friendly.

[CptSky]

For DMaps, my implementation in my DLL is based on the fact that I wanted to expose as much as possible the structure of the format to allow a potential of creating / editing DMaps. I skip parts as it is useless for a server... It is not a good implementation for loading all DMaps in memory as there is too much info. In my C++ source, I still load portals, access and altitude. Originally, I though that it could be interesting to put the access (true/false) in a bit set, but well... You ALWAYS use the DMap info. Does the overhead to access the bit worth the saved memory ? No. When you easily have 250 MB accessible for your server, you can afford loading access information as boolean in memory. I personally prioritize performance vs memory, because you have your reserved memory, while you don't have your reserved CPU cycles.

Also, I know few servers are rewriting DMaps to a custom format for faster loading. But eh, in theory, your server will run for few days without rebooting, no ? So, saving few seconds at launch by day for all the job of converting the DMaps ? Not worth it

----

For the socket system. If you do something optimized for CO2, Hybrid's way to receive based on the first two bytes is really good. But it is based on the protocol, while a TCP/IP socket should be standard. For high efficiency, it's an acceptable thing. A good TCP/IP socket must not use a lot of resources. It must handle gracefully peak load. If you can handle bunch of players without lag. It's perfect

----

Thread per client is ****.

----

Abstract Msg class with implicit conversion is nice for packets.

[pro4never]

Originally Posted by Smaehtin

Still, I don't think I've ever seen a non-blocking socket system that would spawn a new thread for every connection made though.

It's what coemu used if I remember correctly.

It was a sync socket system which spawned a thread for each client so that it did not block the main thread.

I think that's probably what he was referring to.

[go for it]

thanks for everyone for helping me out have a wonderful day

[Smaehtin]

Originally Posted by pro4never

It's what coemu used if I remember correctly.

That's so sad.

[Korvacs]

Originally Posted by Smaehtin

That's so sad.

We're talking about a source that was developed 7 years ago, and that some of us very rapidly succeeded in superseding. Back in what can now be described as the good ol' days of conquer development when people weren't dicks about what got released and where, as a result development moved quickly and openly.