Hey everyone,
Recently, I was asked about optimal threading and socket system models for a game server; optimal in respect to efficiency and player load (or scale). Before I start, I should clarify that there are many threading and socket models that I will not address in this thread; I will only be addressing the fundamental threading and socket models. First, let's talk about threading. Recall that all program instructions are processed by the CPU in sequence, meaning one line at a time (pre-threading). If we visualized this process, we could imagine it as a single lane of traffic on a highway, where each car represents a line of code in the sequence of instructions, or thread.
Now, as you know, most highways don't have just a single lane of traffic, they have multiple where two or more lanes of traffic allow cars to travel. Well, a CPU works in a similar way, where it can process multiple threads (sequences of instructions) at the same time. This is called multi-threading. Just like how a highway doesn't have just one lane of traffic, a CPU doesn't have just one core processing instructions (most of the time). Just like how multiple lanes on a highway allows for more traffic, the amount of cores the CPU has determines the amount of threads that can be processed at the same time (where one core = one thread being processed, assuming no hyper-threading).
So, let's go into an example: let's say you have a single-core processor (1 core, where only one thread can be processed at once). Let's say you're rendering a video for youtube, and your CPU is near 100% usage. You might notice that though your computer might slow down, you can still do multiple things at once. How does this work? Well, let's go back to the highway example. Let's say you have a one lane highway that can only accept one line of cars at a time. As the first line of cars travel along the road, let's say a second line of cars from an on-ramp request access to the highway. Well, since the highway can only accept one line of cars at a time, we need to cut the two lines of cars up and merge the lines. Assuming the drivers are polite, a few cars at a time from each line (alternating) will merge into the final line of cars. In other words, the cars will merge into one lane of traffic that can travel on the highway. The CPU uses the same technique (usually handled by the operating system) to process multiple threads on one core, where sequences of instructions are cut up to be processed on one core. As you may imagine, this slows down the execution of instructions (just as it slows traffic on a highway).
Now, let's talk about threading in regards to applications. Let's say you're reading it data from hundreds of files. Well, you can speed up this process using multi-threading, where groups of files are given to each thread to read in. Using multiple threads, you can reduce the amount of time it takes to read in files by ~200% - 800%. So we should create as many threads as we can to process files, right? Well, let's think back to the traffic example. The more on-ramps (or multiple lines of cars) leading to the same highway, the more cars on the road and the slower traffic becomes. So to answer the question, more threads than the CPU can process is bad. So, how many threads are too many? It depends on the type of work, but for I/O and database processes (like socket systems and game servers), no more than four times the amount of cores available is best. Twice the amount of cores for packet receiving seems to be the best strategy (which leaves the remaining threads for sever maintenance and operating system threads).
Phew. Now let's talk about socket systems. Obviously, a one thread socket system isn't optimal. With a model like that, only one person can be connected at one time (blocking the socket system up). This type of model is a blocking socket system. Now, let's add a thread for accepting new players and a thread per player to receive data. At first, this seems like a great solution that won't block the socket system up (a non-blocking socket system), but after a few players, this server model will slow down due to the amount of threads requesting processing. When will it slow down? It depends on the amount of data packets requesting processing. If we're talking about Conquer Online, maybe around 25 - 75 active players (depending on the system, of course).
The next option, which is popular on these forums, is using an asynchronous model using the built in asynchronous calls in .NET. Depending on the configuration of the setup, an asynchronous model can make a strong, scalable socket system. The model first allows clients to have their own threads, then shares threads between players as traffic increases. The main disadvantage of this model is that as the volume of data processing increases, the amount of overhead caused by asynchronous calls through Windows and managed threading through .NET also increases (creating and destroying threads). It's far from the worst socket model, but is still far below the final option that I will discuss. In regards to Conquer Online, this model can support around 35 - 300 players (again, depending on the system).
Finally, let's talk about overlapped I/O (otherwise known as IOCP / I/O Completion Ports). With this thread model, a dedicated amount of worker threads are created, all listening on a completion port. This eliminates the need to create and terminate threads upon connect and disconnect. All I/O (data packets) are queued to the completion port. When a packet is queued to the completion port, the last associated thread is released and the packet is processed almost immediately. This type of model doesn't use windows events, which speeds up packet processing significantly. In regards to Conquer Online, this model can support around 50 - 1000+ players (again, depending on the system).
Keep in mind, the player counts are hypothesized based on the socket's threading implementation. I hope this answers the original question that was asked. If you have any further questions on threading and socket systems, send a comment or private message my way. Excuse any grammatical errors (I didn't proof read this).
Cheers,
Spirited
Recently, I was asked about optimal threading and socket system models for a game server; optimal in respect to efficiency and player load (or scale). Before I start, I should clarify that there are many threading and socket models that I will not address in this thread; I will only be addressing the fundamental threading and socket models. First, let's talk about threading. Recall that all program instructions are processed by the CPU in sequence, meaning one line at a time (pre-threading). If we visualized this process, we could imagine it as a single lane of traffic on a highway, where each car represents a line of code in the sequence of instructions, or thread.
Now, as you know, most highways don't have just a single lane of traffic, they have multiple where two or more lanes of traffic allow cars to travel. Well, a CPU works in a similar way, where it can process multiple threads (sequences of instructions) at the same time. This is called multi-threading. Just like how a highway doesn't have just one lane of traffic, a CPU doesn't have just one core processing instructions (most of the time). Just like how multiple lanes on a highway allows for more traffic, the amount of cores the CPU has determines the amount of threads that can be processed at the same time (where one core = one thread being processed, assuming no hyper-threading).
So, let's go into an example: let's say you have a single-core processor (1 core, where only one thread can be processed at once). Let's say you're rendering a video for youtube, and your CPU is near 100% usage. You might notice that though your computer might slow down, you can still do multiple things at once. How does this work? Well, let's go back to the highway example. Let's say you have a one lane highway that can only accept one line of cars at a time. As the first line of cars travel along the road, let's say a second line of cars from an on-ramp request access to the highway. Well, since the highway can only accept one line of cars at a time, we need to cut the two lines of cars up and merge the lines. Assuming the drivers are polite, a few cars at a time from each line (alternating) will merge into the final line of cars. In other words, the cars will merge into one lane of traffic that can travel on the highway. The CPU uses the same technique (usually handled by the operating system) to process multiple threads on one core, where sequences of instructions are cut up to be processed on one core. As you may imagine, this slows down the execution of instructions (just as it slows traffic on a highway).
Now, let's talk about threading in regards to applications. Let's say you're reading it data from hundreds of files. Well, you can speed up this process using multi-threading, where groups of files are given to each thread to read in. Using multiple threads, you can reduce the amount of time it takes to read in files by ~200% - 800%. So we should create as many threads as we can to process files, right? Well, let's think back to the traffic example. The more on-ramps (or multiple lines of cars) leading to the same highway, the more cars on the road and the slower traffic becomes. So to answer the question, more threads than the CPU can process is bad. So, how many threads are too many? It depends on the type of work, but for I/O and database processes (like socket systems and game servers), no more than four times the amount of cores available is best. Twice the amount of cores for packet receiving seems to be the best strategy (which leaves the remaining threads for sever maintenance and operating system threads).
Phew. Now let's talk about socket systems. Obviously, a one thread socket system isn't optimal. With a model like that, only one person can be connected at one time (blocking the socket system up). This type of model is a blocking socket system. Now, let's add a thread for accepting new players and a thread per player to receive data. At first, this seems like a great solution that won't block the socket system up (a non-blocking socket system), but after a few players, this server model will slow down due to the amount of threads requesting processing. When will it slow down? It depends on the amount of data packets requesting processing. If we're talking about Conquer Online, maybe around 25 - 75 active players (depending on the system, of course).
The next option, which is popular on these forums, is using an asynchronous model using the built in asynchronous calls in .NET. Depending on the configuration of the setup, an asynchronous model can make a strong, scalable socket system. The model first allows clients to have their own threads, then shares threads between players as traffic increases. The main disadvantage of this model is that as the volume of data processing increases, the amount of overhead caused by asynchronous calls through Windows and managed threading through .NET also increases (creating and destroying threads). It's far from the worst socket model, but is still far below the final option that I will discuss. In regards to Conquer Online, this model can support around 35 - 300 players (again, depending on the system).
Finally, let's talk about overlapped I/O (otherwise known as IOCP / I/O Completion Ports). With this thread model, a dedicated amount of worker threads are created, all listening on a completion port. This eliminates the need to create and terminate threads upon connect and disconnect. All I/O (data packets) are queued to the completion port. When a packet is queued to the completion port, the last associated thread is released and the packet is processed almost immediately. This type of model doesn't use windows events, which speeds up packet processing significantly. In regards to Conquer Online, this model can support around 50 - 1000+ players (again, depending on the system).
Keep in mind, the player counts are hypothesized based on the socket's threading implementation. I hope this answers the original question that was asked. If you have any further questions on threading and socket systems, send a comment or private message my way. Excuse any grammatical errors (I didn't proof read this).
Cheers,
Spirited
