[Release] PBDO Bot - Official Thread

Quote:

How we manage traffic spikes and some info on the backend of PBDO! Just to keep you updated as it seems like people around here like this. We actually just invested in a new server for hosting our bot client update files. Most of you may have noticed the speed in which the forum has been running over the last 24 hours and this is due to an absolutely MASSIVE spike in network traffic to our dedicated forum host due to literally hundreds of thousands of people downloading our bot either for the update, or for free trial time whilst the other bots are down. This has caused us to prepare for the future when we could very well be handling over 100,000 bot client downloads every update. Just to show you what this spike looked like, this is a screen grab of the network traffic over the last day. Please note that the forum server has only a 100mbps connection. [Only registered and activated users can see links. Click Here To Register...] Just to give a little bit of info on what that image is showing. The top graph shows bandwidth usage in bps (bits per second). The scale down the left hand side indicates "M" so its million bits per second, or mbps (megabits per second). The green line indicates outgoing traffic. What this means is how much data our server is "uploading" to the internet. Whenever you download something from us, be it a image for the forum, a page on the forum, or a bot client update, our server "uploads" the file to you who then "downloads" it from us! So basically, the green line shows how much data people are downloading from our server. The blue line shows the opposite. Its the measure of incoming traffic. What this means is how much data our server is "downloading" from the internet. Whenever you try and load a page up, you send a "upload" a small request to our server, whenever you request a bot client update, the same thing happens! Because this is a web server, incoming traffic is almost ALWAYS smaller than outgoing traffic. The red line is the average traffic over that timeframe. Now that that's out of the way. Lets start explaining what the graph shows. As you can see on the graph, at the very start (Sunday 6am Germany Time) both incoming and outgoing traffic was pretty average. They are both sitting at around 10mbps which is what is normal for our forum server in inactive time (eg. no update out). This goes on til around 10am when people being to check the forums for a possibility of an update being released. This goes on until around 18:00 Germany time when we drop forced update *.70. You can see the traffic pretty much spiking upto 100mbps, the maximum our connection allows. Thats from all the bots auto-updating. Over the next two hours there is still a lot of activity on the forums with people checking what others are saying, people complaining on bugs and people downloading the starter package or exe only packages. Then at around 19:30 Mr.Smith drops forced update patch *.71 to fix most of the bugs. Again the traffic spikes at almost 100mbps. During these forced update times (those spikes last around 20 minutes) the site becomes extremely slow and unresponsive and people downloading updates become extremely slow due to the massive amount of traffic. Due to us being the only bot online at the moment, plenty of people have also been trying the bot our and taking advantage of the trial time whilst its still on! Assuming all of these users decide to buy a license, traffic is going to be massive! Thats the first graph done! Now onto the bottom one. The colours are the same as for the top. This graph shows, in Gigabytes (GB) how much traffic is uploaded and downloaded on any specific date. Green is outbound, blue is inbound. As you can see, today we have already gone well over 100GB of uploads (as of right now we are sitting very close to 152GB of uploads). This is absolutely massive! Here is a graph of the entire week for our web server: [Only registered and activated users can see links. Click Here To Register...] Now. When you compare it to a graph of last months traffic you will see why this is such a big deal to us! (Please note the data usage graph at the bottom is not from the previous month) [Only registered and activated users can see links. Click Here To Register...] The first thing you will realise is the peak traffic usage. Last month it didn't peak over 22mbps, this month it maxed out at 100mbps. Thats more than a 4 times increase in traffic in less than a month! If this growth continued, we'd be in serious trouble. The next thing you will realise is the average traffic usage. Last month it was around 5mbps, this month its around 13mbps. Even there thats more than double the average traffic from last month, of which most of the time (this month) we had no online bot. So after reviewing those graphs, Mr.Smith and I both thought we should invest in a slightly faster new server for hosting files off of. As the forum doesn't use much (we route most of our traffic via CloudFlare which takes almost 50% off of our traffic each month) we decided to keep it on the same machine (its been faithful so far) with the same 100mbps connection. The new file server has a 1Gbps connection capable of pushing out files at around a maximum of 125Mb/s (thats the speed you see on your download window in the bot). This kills two birds with one stone. It allows us to take the pressure of updates off our forum server to keep it nice and snappy, and also speed up downloads for updates. Users around Dallas Texas should also see a very nice speed increase as the new server is based there. Whilst our forum server has a hefty 5TB of traffic a month, our new file server will have 10TB allowing us to expand immensely whilst keeping bills (slightly) low. Once we implement AnyCast it will allow us to distribute this traffic right around the world and to speed things up a lot for you guys, but that costs a fair bit of money to setup and will probably be coming in the near (or distant) future! I hope this helps you guys realise how much work even managing the distribution of the bot, and our forums is, and why it costs money to produce something like PBDO-Bot. I'd love to do these sorts of posts more often, but I lack serious time. Anyway, peace out! -jD