Server Architecture Rant

[Spirited]

Hey everyone,

I've been thinking for a while about server architectures. I've been thinking about what TQ Digital did incorrectly when they maintained a game company. I mean, yes... the game concept flaws are apparent throughout all of their games, especially in Conquer Online due to its past popularity. We all know that the game was designed to be small and limiting... but it was designed a decade ago, so we really can't be harsh on TQ, right? Well... looking at how they managed their popularity, they did the most cost effective solution and kept adding new servers. Was it the right thing to do as far as potential growth? Mmm... I don't think so, and let me explain why.

It does seem like Conquer Online was never really build to succeed. Even now a decade later, the game concept hasn't really changed much. Seems odd that it has received the popularity it has, right? Well, there's no doubt about it: TQ made a successful, hit MMORPG. What I want to ask is, what if when they saw the popularity... when technology was quickly improving... what if they brought the game through the ages? I mean, you can run their game on an iPhone... what does that say about their client's quality?

So, here's where things get interesting. What if (a few years ago) they invested more in the game they created? What if they kept up with the ages? Well, let's say they fixed maps size limitations; they fixed the limitations on server growth the game is currently presented with. More maps, large large maps, more monsters, more players, larger PVP events... more server processing... a crap ton more of server processing. We're talking about expanding the game's maps to all of China and beyond the borders. Better 3D models, etc. Then, let's say people had such an interest in the game that they flooded the servers like they did with World of Warcraft. Is the server architecture TQ holds true valid in such a scenario? Obviously not... so how would they have matched this ungodly amount of traffic?

Well... I've thought about this and I've mapped situations out quite extensively. Here's what I came up with with for handling such huge amounts of traffic on one server (and I would like your feedback). I get that this doesn't matter in a game like Conquer Online, but let's go with it. I started out with my Phoenix server design: an account server, a message server, and a map server. This design is flawed and unnecessary. I mean, it works and it works well, but it's still unnecessary. It's just icing on the cake in an attempt to support what Conquer Online currently doesn't support: cross-server communication. Anyways, we'll return to that later... that's not what I wanted to rant about and this new design would support it as well. Back to the server design.

Let's say we kept the account server. All players logging in connect to an account server of their selected regional settings. I know, that's a lot of traffic... and TQ's already doing this, but let's keep going. No game content delivered, not a problem. The message and map servers are the problems; they complete the game server concept.

So, let's say that instead of having the client connect to one server, it has the ability to connect to multiple (as it currently does with the anti-bot server). Instead of having the message server handle a large portion of the game server concept, let's make it an operator (a bit weird, I know, but stay with me). Just like a telephone operator, you call in... it checks a directory for you... and redirects you. In more details, it accepts the client, reads in the player's character location from the MySQL database (the directory), and sends back the response of what map server to connect to. It also holds the MySQL database for unifying the map servers.

The client then disconnects from the message server and connects to the appropriate map server. Each map server hosts a region or collection of maps. This splits up work very effectively, as it doesn't go through a message server or any other server to deliver content. It's just the map server at this point. When players teleport to another map out of the region, the client disconnects and recontacts the message server / operator for the new map server.

I was quite content with this, so I started challenging the design with problems. The first was chatting with other players. Chat is huge in any multiplayer game - and so I came up with two servers that would handle this appropriately. The first server is the chat server. It connects directly to players. If a whisper goes out, it is handled by the chat server. If a regional message goes out, it is handled by the map server the client is connected to. If a world message goes out, it is handled by the message sever. A bit messy, I know, but most conversation is done in private... whispering, party chat, guild chat, friend chat... etc. All of those private chats contain pools of players to send information to - so most of it is managed by that chat server. Then, for cross-server chat, there's a chat router server that routes cross-server conversations or brings in IM services like AIM or Skype.

Wow. Trying to explain this design has made me realize how overly complicated it really is. What are your thoughts on this? I know it's a weird concept to think about, but I thought it may be interesting. If you have any challenges to present the design, please let me know. I'd love to try and disprove the effectiveness of the design. Thanks for your time.

Regards,
Spirited

[Super Aids]

Tbh. I don't think Conquer was ever meant to reach outside of China.

There are some flaws in your system though, let me demonstrate how I would do it using a single picture.

Edit: Actually went ahead and fixed the pic.

1. The client would establish a connection to one of the auth servers. These auth servers would respond with the address for whatever gameserver you're trying to reach.

2. A connection would be established to the game server, which is connected to the message server, map server and database server. Each game server would have their unique message, map and database server.

3. A connection would be established to the message server which handles all messages this counts for world messages too, as it's connected to the game server it could reach out to all players as well.

4. A connection would be established to the map server which handles all map related things (You know the drill.)

5. Each server (Game, message and map) will have a connection to the database server which has a connection to the global database server. The global database server transfers data from itself into each servers database to avoid too many connections to the same database server. Ex. when you login all your player information will be imported from ex. Server2's database server into Server3's database server through the global database server which will store the data for backups.

That is how I would approach writing it.

That means basically a client would have 4 connections. One temporary to the auth server and 3 to the game server, message server and map server. The game server, message server and map server would all have a unique connection to the database server. The database server would have a unique connection to the global database server.

This system would technically not be possible for a private server, unless you attached a proxy (ofc. running local) that would redirect packets to their respective destinations.

A more indepth login process:
Client
1. Client connects to auth server
2. Auth server authenticates the client.
3. Auth server writes the login details of the account whether the login was okay or not to the global database.
4. The global database sends a request out to all game servers whether the client is already logged in somewhere. If it's not logged in then it will allow it to be logged into a game server.
5. The auth server then responds to the client with the address of the game server.
6. The client disconnects from the auth server OR the auth server disconnects the client. A simple timeout system would work.
7. The client then connects to the game server.
8. The game server sends a request to the global database for the players information and whether it was authenticated properly.
9. The game server responds to the client that it was authenticated properly.
10. The client connects to the message server.
11. The client connects to the map server.
12. The login process is proceeded. The game server for player info and map server for locations, surroundings etc.

[Best Coder 2014]

Originally Posted by Spirited

what Conquer Online currently doesn't support: cross-server communication.

Regards,
Spirited

It's funny that you post this right after TQ announces: (The "Cross-server Adventure - PvP Feature" part).

There has been cross-server chat in CO for a while now, too. I'm not sure if that's what you're talking about when you say cross-server communication though, but it's still something.

[Spirited]

Sorry, I'm on a phone and it's a bit tricky to type. I don't see the flaws with the design I wrote up in the rant. You didn't mention them either. Also... that server design you came up with... you might want to think on it more. I'll explain tomorrow when I'm at a keyboard.

[GameHackerPM]

I'm pretty sure that these words are important, i read some! But they are really a lot xD.
Thanks anyway! will complete reading later!

[KraHen]

Originally Posted by GameHacker-PM-

I'm pretty sure that these words are important, i read some! But they are really a lot xD.
Thanks anyway! will complete reading later!

Thank you for meaningfully joining the conversation, your input matters a lot to us.

@Fang : I totally see where you`re coming from, distributed servers are THE thing to do to handle many many clients, but tbh, no one is really doing it (aside from a very select few), including game studios themselves. For instance, a WoW emulator can hold 4-5k players just fine, 4-5k players is much even in WoW (it really creates the illusion of the fully populated world), Blizzard used the same (well, wildly different but similar) technique : just add more servers. There is absolutely no problem with this implementation, its been the "standard" for quite some time now. It`s just how the game is designed (whether it`s bad or good, thats another story), and the architecture is built around the game design. What you are mentioning works flawlessly for other situations, like cloud hosting, file sharing, etc, but not for games (at least, not for the games we see today), and I can tell you exactly why : time. Time is of the essence when it comes to gamedev, especially at studios, we have to make compromises, and this architecture works for these games just fine (and if it`s not broken, don`t fix it).

[Spirited]

Originally Posted by KraHen

Thank you for meaningfully joining the conversation, your input matters a lot to us.

@Fang : I totally see where you`re coming from, distributed servers are THE thing to do to handle many many clients, but tbh, no one is really doing it (aside from a very select few), including game studios themselves. For instance, a WoW emulator can hold 4-5k players just fine, 4-5k players is much even in WoW (it really creates the illusion of the fully populated world), Blizzard used the same (well, wildly different but similar) technique : just add more servers. There is absolutely no problem with this implementation, its been the "standard" for quite some time now. It`s just how the game is designed (whether it`s bad or good, thats another story), and the architecture is built around the game design. What you are mentioning works flawlessly for other situations, like cloud hosting, file sharing, etc, but not for games (at least, not for the games we see today), and I can tell you exactly why : time. Time is of the essence when it comes to gamedev, especially at studios, we have to make compromises, and this architecture works for these games just fine (and if it`s not broken, don`t fix it).

Thanks for sharing your experience with me. JP and I talked about a cluster architecture for services like cloud computing and media providing services; but I *think* we both agreed that the architecture I posted would be a simple and clean solution. It really isn't necessary for a game like Conquer Online, but it's still fun to rant about.

[Konvict-]

Wonder if TQ will EVER change their architecture. (Conquer 3.0 or w/e)

[CptSky]

Originally Posted by Spirited

Thanks for sharing your experience with me. JP and I talked about a cluster architecture for services like cloud computing and media providing services; but I *think* we both agreed that the architecture I posted would be a simple and clean solution. It really isn't necessary for a game like Conquer Online, but it's still fun to rant about.

Yes, but I still agree with KraHen. If you want to handle tons of players (100k), a cluster architecture will be needed. If you want to handle few players (5k), a simple architecture like TQ or most MMO will do the job, and honestly, at some point, you want to split them by servers, instead of having everyone on the same.
A cluster architecture would be useless for 5k and wouldn't be efficient. You architecture design is like between the two, but do you really need something between the two designs ?

[Super Aids]

Originally Posted by Konvict-

Wonder if TQ will EVER change their architecture. (Conquer 3.0 or w/e)

You're going to get disappointed.

[Konvict-]

Originally Posted by Super Aids

You're going to get disappointed.

Haha, nah. I am very familiar with TQ. They would never do such a thing. They seem to do the minimal work to just get the job done. (On all their games)

[KraHen]

Originally Posted by Konvict-

Haha, nah. I am very familiar with TQ. They would never do such a thing. They seem to do the minimal work to just get the job done. (On all their games)

I don`t see how that is a bad thing, EVERY studio does that. Although that "minimal work" might be a LOT more for other games, simply because there`s more content or the bar of quality is set higher. The effort TQ is showing is more than enough for CO, the playerbase is there, their monetization works really well (mainly for them, not for the players), and will work (at least on short term). If you`re familiar with F2P monetization techniques, you can see what TQ is doing (and doing well) : periodic content updates squeezing out the most cash out of them as possible (for a short time at least, until the next update). This is opposed to other market models (such as Riot`s or Path of Exiles`) which aim on fidelity and long term small payments (but for that you have to have a GREAT game to work, TQ knows that CO isn`t one of them).

[Super Aids]

Originally Posted by KraHen

I don`t see how that is a bad thing, EVERY studio does that. Although that "minimal work" might be a LOT more for other games, simply because there`s more content or the bar of quality is set higher. The effort TQ is showing is more than enough for CO, the playerbase is there, their monetization works really well (mainly for them, not for the players), and will work (at least on short term). If you`re familiar with F2P monetization techniques, you can see what TQ is doing (and doing well) : periodic content updates squeezing out the most cash out of them as possible (for a short time at least, until the next update). This is opposed to other market models (such as Riot`s or Path of Exiles`) which aim on fidelity and long term small payments (but for that you have to have a GREAT game to work, TQ knows that CO isn`t one of them).

Conclusion: The stupidity really lies with the players.

[Konvict-]

Originally Posted by KraHen

I don`t see how that is a bad thing, EVERY studio does that. Although that "minimal work" might be a LOT more for other games, simply because there`s more content or the bar of quality is set higher. The effort TQ is showing is more than enough for CO, the playerbase is there, their monetization works really well (mainly for them, not for the players), and will work (at least on short term). If you`re familiar with F2P monetization techniques, you can see what TQ is doing (and doing well) : periodic content updates squeezing out the most cash out of them as possible (for a short time at least, until the next update). This is opposed to other market models (such as Riot`s or Path of Exiles`) which aim on fidelity and long term small payments (but for that you have to have a GREAT game to work, TQ knows that CO isn`t one of them).

I agree to a point. I feel like conquer could have been even bigger then it was. Now I feel like their doing the right thing, but previously they could have done much more and got more out of it.

[Aceking]

Originally Posted by Konvict-

I agree to a point. I feel like conquer could have been even bigger then it was. Now I feel like their doing the right thing, but previously they could have done much more and got more out of it.

The only thing they are doing the right thing by at this moment is their bank account.

They were doing the right thing before CPs, BP, Lotto etc was introduced.
Once they were out, you were FORCED to pay to play essentially, otherwise its useless.