[WIP] Ducksoup - c# filter

[b0ykoe]

- Description is a bit more fancy there!

What is DuckSoup?
DuckSoup is a C# packet filter for v188 Silkroad Online. The main focus is on quality, maintainability and stability. The base is fully written by myself with some help of friends and google. The only part I got inspired from was the PacketHandler part which has some links to Chernobyls PacketHandler.

Why is DuckSoup so special? DuckSoup does not use `.NET Framework`. It uses `.NET Core` which is by far more porefull, advanced and it can be used on Linux for some extra performance and security! Also DuckSoup is developed under the concept "quality over quantity".

Motivation
Since florian0 released as a client addition, Devsome released as a website for Silkroad so I thought it would be time to release a public, open source filter.

Requirements
Windows Windows 7/Vista/8.1/Server 2008 R2/Server 2012 R2
- [Microsoft Visual C++ 2015 Redistributable Update 3]()
- [KB2533623]()
>

Windows Server 2016, 2019 and Windows 10 should be able to run netcore apps nativly.

Linux (Ubuntu / Debian)

$ wget [url]https://packages.microsoft.com/config/debian/10/packages-microsoft-prod.deb[/url] -O packages-microsoft-prod.deb
dpkg -i packages-microsoft-prod.deb

$ sudo apt-get update; \
  sudo apt-get install -y apt-transport-https && \
  sudo apt-get update && \
  sudo apt-get install -y dotnet-sdk-3.1

Download
You can download the latest version from here

Installation
The following steps will be the same for Windows & Linux.

Extract the folder you've just downloaded. Create at least 3 folders, one for each (Gateway, Download, Agent).

Go into each folder and open up DuckSoup.exe (or dotnet DuckSoup.dll), after that open `GlobalSettings.json`.
Set your Database settings and enter the Remote (Server) address, same as Port. Also add a Bind address, this is the IP of the filter server. Also set a Port for the component. I usually use 4000 for the Download Server, 4001 for the Gateway Server and 4002 (4003, 4004 and so on) for the Agent Servers.

Numbers for the "ServerType" option:

0 = None
1 = DownloadServer
2 = GatewayServer
3 = AgentServer
4 = ManagementServer

In the DownloadServer & AgentServer folder you dont have to do anything more.

In the GatewayServer folder you have to edit the

GatewaySettings.json

If you want dont to remove the Captcha you can turn off "RemoveCaptcha". For that replace ` "RemoveCaptcha": true,` with ` "RemoveCaptcha": false,`. If you have any other captcha than `0` you have to replace the captcha here. Note that it only works with a static captcha.

Now you have to edit the `RedirectionRules`. There will always be a default rule you can copy & change. You should get the hang if you look at the example below.

Now all thats left is to patch your client to the filter IP and to the Gatewayport you've set in the

GatewayServer/GlobalSettings.json

On Windows: Doubleclick the "DuckSoup.exe" in all folders
On Linux: `dotnet DuckSoup.dll` in all folders
> Hint: You might wanna use `screen` on Linux.

Wanna use 2 Agent Servers?
Copy the `AgentServer` folder, change the Ports and IPs `GlobalSettings.json` and add a RedirectionRule in your `GatewaySettings.json` in the GatewayServer folder! Example below.

Extras
GlobalSettings.json :
- ServerName is just a name, you can change it to whatever you like
- ServerType has a few options (0 -> None, 1-> DownloadServer, 2-> GatewayServer, 3->AgentServer, 4-> ManagementServer)
- If your players get Disconnects you can upper the ByteLimitations
- DebugLevel will show different stuff in log, more messages = more lag. If you set it to 7 it will show everything below it
- DebugLevel has a few options (0 -> nothing, 1 -> Fatal only, 2 -> adds Fatal, 3 -> adds Fatal, 4 -> adds Info, 5 -> adds Connections, 6 -> adds Debug, 7 Everything)
> Hint: None does absolutely nothing and ManagementServer is not ready yet

Examples

  "RedirectionRules": [
    { // DownloadServer
      "RemoteAddress": "111.111.111.111", // server IP, will be hidden
      "RemotePort": 15881, // real downloadserver port (please use your port here this is a example value!)
      "RedirectAddress": "222.222.222.222", // filter IP, will be shown
      "RedirectPort": 4000 // filter port that will be redirected - please set correctly in the DownloadServer Folder (GlobalSettings.json)
    },
    { // AgentServer 1
      "RemoteAddress": "111.111.111.111", // server IP, will be hidden
      "RemotePort": 15882, // real agentserver1 port (please use your port here this is a example value!)
      "RedirectAddress": "222.222.222.222", // filter IP, will be shown
      "RedirectPort": 4002 // filter port that will be redirected - please set correctly in the AgentServer Folder (GlobalSettings.json)
    },
    { // AgentServer 2
      "RemoteAddress": "111.111.111.111", // server IP, will be hidden
      "RemotePort": 15882 , // real agentserver2 port (please use your port here this is a example value!)
      "RedirectAddress": "222.222.222.222", // filter IP, will be shown
      "RedirectPort": 4003 // filter port that will be redirected - please set correctly in the AgentServer2 Folder (GlobalSettings.json)
    }
  ],

Features
- Async Server handling
- Async Database handling
- Exploit Protection
- Redirection Rules for Gateway -> Agent and Gateway -> Download

Planned Features
- a centralized management server which handles requests regarding max IP, max hwid, synchronized actions and events
- silk per hour
- HWID restrictions
- IP restrictions
- sheduled notices (for all players & individual players)
- max plus
- plus notice
- and so on.. The basic filter stuff

Performance
Testsystem: Dedicated Server - Ryzen 5 3600 - 64 GB Ram - 1Gbit connection
Testcase: Ghostuser connects a amount of clients and then sends all 5 seconds a movepacket on x users (configured) to the client, directly one after another. This is not a real test case, since this will NEVER happen (there isn't simply the case that those mass on packets come in at in a 1ms timeframe, as long as you're not electus!). There wont be the case that 300 clients will send movement packets at ONCE (ONCE ONCE, not with a timespan of like a few ms between them).

Conclusion: The filter performs way better on Linux. Windows is doable and you will be able to handle more players on more cores. The higher CPU usage is good since we can handle more workload faster.

DuckSoup:
Login Test:
Windows:
1 Gateway - 2 Agent - 650 Users, CPU spiked up to 95~% (on all 12 threads), all cients connected with a noticeable amount of lag (normal delay 1.5 - 2.5 sec)

Linux:
1 Gateway - 2 Agent - 650 Users, CPU spiked up to 45~% (on all 12 threads), all clients connected with lesser noticeable amount of lag (short delay 0.5 - 1.5 sec)

Movement Packet Test:
Windows:
2 Agents 300 Users, 300 moving, CPU spiked up to 20~% (on all 12 threads), client lag 0.5 - 1 seconds
2 Agents 650 Users, 300 moving, CPU spiked up to 70~% (on all 12 threads), client lag 1.5 - 2 seconds

Linux
2 Agents 900 Users, 450 moving, CPU spiked up to 40~% (on all 12 threads), client lag 1.5 - 2 seconds

Isolines Filter (SR_PROXY)
Login Test:
1 Gateway - 1 Agent - 650 Users (actually only 325 users connected on the Agent duo to 2 Agent setup) , CPU spiked up to 65~% (first 6 threads), not all clients connected duo to a timeout issue with a big amount of lag (big delay random between 1 - 4 sec)

Movement Packet Test:
Windows
1 Agent 300 Users, (650 connected overall), 150 moving, CPU spikes up to 25~% (first 6 threads), client lag 2-3 seconds
1 Agent 300 Users, (650 connected overall), 300 moving, CPU spikes up to 60~% (first 6 threads), client lag 4-7 seconds

Errors
Q: My Server is stuck at starting!
A: Check your Database connection, if it has no connection it won't start since its loading LauncherNews, Skills, Items from the DB.

Q: I can't connect!
A: Check your Ports.

Q: It does not redirect to the AgentServer!
A: Check if your `RedirectionRules` in your `GatewayServer/GatewaySettings.json` are correct.

Q: My client won't patch!
A: Check if your `RedirectionRules` in your `GatewayServer/GatewaySettings.json` are correct.

Special Thanks
- qqdev
- for the kind words, the motivation and that you helped me on my questions
-
- for the kind words, the motivation and ideas
- his Silkroad Laravel project can be found
-
- for the kind words, the motivation and ideas
- his SRO_DevKit can be found
-
- for the original released SilkroadSecurityAPI (originaly released )
-
- for the SilkroadDocs (can be found )
- Chernobyl
- for the idea and the PacketHandler
- Everyone thats contributing with ideas, testing, comments!
License
This code has been licensed under the *DON'T BE A DICK PUBLIC LICENSE*.

[Devsome]

Nice work, as always

[XxGhostSpiriTxX]

good release thank you very much for sharing keep going friend

[Hercules*]

Good work thank you.

[concucu]

thanks

[mudik]

Cooles Projekt

[vietnguyen09]

Great work bro.

[JellyBitz]

Your packet handlers are using a HashTable, this has been discussed way too much before about Dictionary collection being faster.

Every piece of performance is good for a network connection where the latency needs to be lower as possible.

Also I checked this piece:

PHP Code:

// accepts client connections
      while(!_exit) {
           await _tcpServer.AcceptTcpClientAsync().ContinueWith(OnAccept, TaskScheduler.Default); // What about DDoS?
           await Task.Delay(3);
      } 


I'm not sure how TCPListener really works, but looks very vulnerable.
The awaiting queue looks like a synchronous method for me =.=

- - -
I'm still looking for a method to handle multiple handlers for a certain opcode.

Even the Dictionary looks like it's generating me a tiny delay (TryGetValue specifically, or it's because it's empty? or it's because the object creation on every packet? I had to check that) so I had to choose add global handlers at the moment, just like your _default handler but a list.. u.u

[b0ykoe]

Originally Posted by JellyBitz

Your packet handlers are using a HashTable, this has been discussed way too much before about Dictionary collection being faster.

Every piece of performance is good for a network connection where the latency needs to be lower as possible.

Also I checked this piece:

PHP Code:

// accepts client connections
      while(!_exit) {
           await _tcpServer.AcceptTcpClientAsync().ContinueWith(OnAccept, TaskScheduler.Default); // What about DDoS?
           await Task.Delay(3);
      } 


I'm not sure how TCPListener really works, but looks very vulnerable.
The awaiting queue looks like a synchronous method for me =.=

- - -
I'm still looking for a method to handle multiple handlers for a certain opcode.

Even the Dictionary looks like it's generating me a tiny delay (TryGetValue specifically, or it's because it's empty? or it's because the object creation on every packet? I had to check that) so I had to choose add global handlers at the moment, just like your _default handler but a list.. u.u

Heyo, thanks for your feedback. This is my first real c# program, which totally isn't a excuse. I'll update the HashTable(s) in the next update to Dictionarys and improve the Packethandler.

I didnt think about DDoS or connectionspam in the first place, actually true. I thought starting it as a Task would be enough. You're right that needs further investigation and improvement!

The work on the PacketHandler was mostly from chernobyl, I took it from here (dont want to have the credits for something I didn't do). I've just improved it a little and 'll look again over it to get rid of that list.

Edit - 07. August 2020

I've added a buildserver and edited the main post a little. I'm greateful for every hint, for every comment, for every testing (failed or successful, doesn't matter) and every idea. I'll try to find some time to implement all the ideas I've got.

About features, they're postponed until a unknown date. I want to provide a stable base which is performant and secure.

Download
You can download the latest version from here
I've add

[dR. [d3V][GM][PR0] 39YP7]

btw Task.Delay(3) is not going to wait for 3 ms (on windows)
also use generic collections to avoid boxing/unboxing


for the accept connection part
you could have an acceptance queue that pushes out a socket every 10-15ms and remove the Task.Delay that way you'll avoid cpu spikes due to high connection count and you'll still be able to accept a reasonable amount of clients per second

Session
Initialize _clientBuffer and _serverBuffer after a successful connection
would also recommend using a pool you're using .net core

why is packet flood check inside the foreach loop?

exit the loop after Destroy()?

the client references the server? uhh

your RefObjCommon would benefit from being a dictionary instead of the linq query you perform on every spawn


Finally:

Why is DuckSoup so special? DuckSoup does not use .NET Framework. It uses .NET Core which is by far more porefull, advanced and it can be used on Linux for some extra performance and security! Also DuckSoup is developed under the concept "quality over quantity".

How do you take advantage of .net core's features?

the performance numbers do not reflect the "quality over quantity" to be honest.

Good luck and I wish you the best.

[irockalone]

Originally Posted by b0ykoe

Heyo, thanks for your feedback. This is my first real c# program, which totally isn't a excuse. I'll update the HashTable(s) in the next update to Dictionarys and improve the Packethandler.

I didnt think about DDoS or connectionspam in the first place, actually true. I thought starting it as a Task would be enough. You're right that needs further investigation and improvement!

The work on the PacketHandler was mostly from chernobyl, I took it from here (dont want to have the credits for something I didn't do). I've just improved it a little and 'll look again over it to get rid of that list.

Edit - 07. August 2020

I've added a buildserver and edited the main post a little. I'm greateful for every hint, for every comment, for every testing (failed or successful, doesn't matter) and every idea. I'll try to find some time to implement all the ideas I've got.

About features, they're postponed until a unknown date. I want to provide a stable base which is performant and secure.

Download
You can download the latest version from here
I've add

reupload

[b0ykoe]

Originally Posted by irockalone

reupload

Fixed. Seems like gitlab removed the possibility to download the latest artifacts from a branch.

[devdash]

@

I am really confusing reading this, I will appreciate if you help me understanding this.
What is this exactly? i have read the post, but still.

Is it a free open source filter that we can download and start use with limited features, that we can simply add to the filter if we know how to code in C# ?

I mean if so, i can start learn C# if that will make me someday have my own filter, that i always dreamed about, if not, then what is this, because i always wanted to have a filter for my own, but i never had the chance, so maybe this is the chance, but i do not understand

I will appreciate if you learn me more about this, thanks in advance, and if this what i think, then you're doing a really great job, because as a member in the new community, we're suffering for having no filter, they target us with attacks when we do have 50 players because we're weak when there're servers with thousands get no attacks but they have their own filters.

Thanks again.

[b0ykoe]

Originally Posted by alydandy

@

I am really confusing reading this, I will appreciate if you help me understanding this.
What is this exactly? i have read the post, but still.

Is it a free open source filter that we can download and start use with limited features, that we can simply add to the filter if we know how to code in C# ?

I mean if so, i can start learn C# if that will make me someday have my own filter, that i always dreamed about, if not, then what is this, because i always wanted to have a filter for my own, but i never had the chance, so maybe this is the chance, but i do not understand

I will appreciate if you learn me more about this, thanks in advance, and if this what i think, then you're doing a really great job, because as a member in the new community, we're suffering for having no filter, they target us with attacks when we do have 50 players because we're weak when there're servers with thousands get no attacks but they have their own filters.

Thanks again.

This is a open source filter yes. You can modify it as you want and you can add stuff as much as you want. When I find some time I'll work on it and fix the issues other tell me or replace some garbage code with some better code.

Also keep in mind that this is a Silkroad packet filter, not a DDOS Shield or something like this in any way. It will only prevent bad packets and will execute stuff you wrote (ex. events, features, you name it). This will not magically make ddos disappear

[devdash]

Originally Posted by b0ykoe

Also keep in mind that this is a Silkroad packet filter, not a DDOS Shield or something like this in any way. It will only prevent bad packets and will execute stuff you wrote (ex. events, features, you name it). This will not magically make ddos disappear

Maybe now, but later we can make it a shield against DDOS and also FLOOD and be fully protected using it? or is it impossible to make this filter prevent such DDOS or FLOOD attacks?

Because i think safety come first, i mean if we're not protected then what ever feature we add it will be useless since the attackers will keep us offline.

Thanks for the early replay <3