Register for your free account! | Forgot your password?


Go Back   elitepvpers > Popular Games > Silkroad Online > SRO Coding Corner
You last visited: Today at 05:03

  • Please register to post and access all features, it's quick, easy and FREE!

Advertisement



[Discussion] Is any other Silkroad Security API out there?

Discussion on [Discussion] Is any other Silkroad Security API out there? within the SRO Coding Corner forum part of the Silkroad Online category.

Reply
 
Old   #1
 
elite*gold: 290
Join Date: Oct 2013
Posts: 261
Received Thanks: 189
[Discussion] Is any other Silkroad Security API out there?

Hi guys,

I've done some overall researches last few days about packet handle for SRO filter and what I've found that almost filter (standalone application) out there is used SilkroadSecurityAPI from @.

While I'm thankful to him for make a simple/lightweight API but I still wondering is any other "better" API out there in nowadays (sorry @, no offence)? seems the best one, but it is like abandoned since this community likely dead and it requires really high skill and so long time to get on with it.

A release that seems great recently is from @ with .NET Core with some really decent test cases but I'm just wondering can we discuss about weak / strong part of each API in real environment server?

Any idea?
vietnguyen09 is offline  
Old 08/02/2020, 22:03   #2
 
elite*gold: 0
Join Date: Sep 2018
Posts: 260
Received Thanks: 352
SilkroadSecurityAPI does all you need to handle the blowfish/handshake pretty well even on asynchronized states, and has nothing to do with a bot/filter performance at my opinion.
JellyBitz is offline  
Thanks
3 Users
Old 08/03/2020, 18:58   #3
 
elite*gold: 290
Join Date: Oct 2013
Posts: 261
Received Thanks: 189
Quote:
Originally Posted by JellyBitz View Post
SilkroadSecurityAPI does all you need to handle the blowfish/handshake pretty well even on asynchronized states, and has nothing to do with a bot/filter performance at my opinion.
Due the test that @ just did, can you tell me what you think about it?
vietnguyen09 is offline  
Thanks
1 User
Old 08/03/2020, 19:22   #4
 
elite*gold: 15
Join Date: Sep 2011
Posts: 745
Received Thanks: 199
Quote:
Originally Posted by vietnguyen09 View Post
Due the test that @ just did, can you tell me what you think about it?
Well in my filter its just a little modified to work with .Net core honestly. I didnt changed the code at all. Also I've resized the buffers a little since 8k seemed a little much. 4k was plenty for my case.
b0ykoe is offline  
Thanks
2 Users
Old 08/04/2020, 14:04   #5
 
elite*gold: 0
Join Date: Jan 2009
Posts: 278
Received Thanks: 573
Drew's SilkroadSecurityAPI is great at what it does but was probably never intended to be used in a scenario where it becomes part of the horrible choke point we know as filters today.

X-Filter is using regular SilkroadSecurityAPI with a few QoL changes so I wouldn't really call it a different API but there are probably also people who have written another one from scratch.


In order to write a better API you'll first have to understand and then where the weaknesses are in SilkroadSecurityAPI and how it connects to C# in particular.
There are some who ported the API to C++ for performance reasons but in reality they're just offsetting the weaknesses instead of working on them. Creating more headroom for mistakes made in the higher application code.

Before we analyze some of the weaknesses of SilkroadSecurityAPI we have to address why C# plays a big role in this.

C# or rather the CLR (Common Language Runtime) uses a Garbage Collector that is responsible of cleaning up all the objects (Sessions, Packets, Strings) you've created when they're no longer needed. So every time you allocate something (on the heap) it needs to be collected later. This magic does not come for free. So we'll have to design around the GC to write high performance code in C#.

We can achieve this by:
- Pooling Memory
- Pooling Objects
- Avoiding unnecessary copying
- Allocating on the stack instead of heap
- Using value types when appropriate

Here are some links for techniques you can use to tame the GC.
-
-
-


1. Blowfish

Code:
        public byte[] Encode(byte[] stream, int offset, int length)
        {
            if (length == 0)
                return null;

            var workspace = new byte[GetOutputLength(length)];

            Buffer.BlockCopy(stream, offset, workspace, 0, length);
            for (int x = length; x < workspace.Length; ++x)
                workspace[x] = 0;

            for (int x = 0; x < workspace.Length; x += 8)
            {
                uint l = BitConverter.ToUInt32(workspace, x + 0);
                uint r = BitConverter.ToUInt32(workspace, x + 4);
                Blowfish_encipher(ref r, ref l);
                Buffer.BlockCopy(BitConverter.GetBytes(r), 0, workspace, x + 0, 4);
                Buffer.BlockCopy(BitConverter.GetBytes(l), 0, workspace, x + 4, 4);
            }

            return workspace;
        }
If we look at the Encode method of the the Blowfish there are several issues.

1. Allocates a new buffer for the output (workspace).
2. Copies the input (stream) into the output (workspace).
3. Clears the array where no data has been copied over. (unnecessary as arrays are always default initialized in C#)
5. BitConverter.GetBytes allocates the result array which is then copied back into the output (workspace)

This could be improved by using unsafe pointers but .NET Core has brought us Span<T> so we can avoid a lot of these issues while writing safe code.

Span<T> allows us to access the same memory sequentially as any blittable data type with little to no overhead, thus we encipher without converting and copying back between parts of the byte[] and uints.

Code:
        internal int Transform(TransformMode transformMode, in ReadOnlySpan<byte> input, in Span<byte> output, int length)
        {
            var input32 = MemoryMarshal.Cast<byte, uint>(input);
            var output32 = MemoryMarshal.Cast<byte, uint>(output);

            int blockIndex = 0;
            for (int i = 0; i < length / 8; i++)
            {
                var left = input32[blockIndex + 0];
                var right = input32[blockIndex + 1];

                if (transformMode == TransformMode.Encode)
                    this.Encipher(ref left, ref right);
                else if (transformMode == TransformMode.Decode)
                    this.Decipher(ref left, ref right);

                output32[blockIndex + 0] = left;
                output32[blockIndex + 1] = right;

                blockIndex += 2;
            }
        }
final block transforming omitted for simplicity

With this method you can also easily encrypt into the the same buffer which gets rid of the output (workspace) allocation and copying input to output which is done by the transform anyway.


2. Packet

The Packet class is a nice general purpose class which you can use to read and write your packets data, no matter if massive or not, but not without flaws.
I don't have any fancy tables to prove my point but some of the write and read methods are rather slow. Further more the underlying MemoryStream in PacketWriter has a default buffer size of 256 and will grow when running out of space by allocating a new buffer that is double the size of the old buffer. The older buffer is also copied over to the new buffer. Spawn packets can easily break this threshold.

Thus it's probably not a good idea to pool a MemoryStream as you can't shrink it's growing buffer. So with enough time every pooled instance of a Packet could've been a spawn packet once and will have a rather large buffer wasting memory when the majority of the packets are small. But you can try to use the .

I'd suggest creating a Packet class that works on a fixed 4096 byte buffer which has been allocated from a Pool (ArrayPool<T> / MemoryPool<T>) and is living on an object pool as well. But since it can't grow there is no way to create a massive packet anymore. For this you'll need to use multiple regular packets and read and write across them thus it simply contains a list of packets.

3. Security
I intendet to write this section in more detail but it turned out more exhausting than expected.
In essence it comes down to a full rewrite as you'll need to carefully avoid copying and allocating where possible.
- Don't allocate a new list everytime you TransferOutgoing/TransferIncoming

With a fixed 4096 byte message you can:
- Receive directly into the the fixed message buffer.
- encrypt/decrypt into itself avoiding allocations and coyping found in Recv and FormatPacket.

Pointing out every small issue would make an obscene large list and it's not all purely performance focused.
The SecurityFlags class for example can be easily swapped with a [Flags] enum to improve readability and reduce complexity.

4. Other
I'm not saying you should all now go and try to rebuild SilkroadSecurityAPI but if you truly care about performance by all means give it a try.
But it won't magically solve all your issues as the Network and Threading design equally important.

Another problem with filters I've seen is that nobody seems to think about which data structures to use.
Things like List<T> for white/blacklists are easy to avoid where a Dictionary<TKey, TValue> or HashSet<T> is appropriate. Please educate yourself about the so you understand how to judge the complexity of such collections and choose accordingly.
DaxterSoul is offline  
Thanks
13 Users
Old 08/05/2020, 06:56   #6
 
elite*gold: 290
Join Date: Oct 2013
Posts: 261
Received Thanks: 189
Quote:
Originally Posted by DaxterSoul View Post
Drew's SilkroadSecurityAPI is great at what it does but was probably never intended to be used in a scenario where it becomes part of the horrible choke point we know as filters today.

X-Filter is using regular SilkroadSecurityAPI with a few QoL changes so I wouldn't really call it a different API but there are probably also people who have written another one from scratch.


In order to write a better API you'll first have to understand and then where the weaknesses are in SilkroadSecurityAPI and how it connects to C# in particular.
There are some who ported the API to C++ for performance reasons but in reality they're just offsetting the weaknesses instead of working on them. Creating more headroom for mistakes made in the higher application code.

Before we analyze some of the weaknesses of SilkroadSecurityAPI we have to address why C# plays a big role in this.

C# or rather the CLR (Common Language Runtime) uses a Garbage Collector that is responsible of cleaning up all the objects (Sessions, Packets, Strings) you've created when they're no longer needed. So every time you allocate something (on the heap) it needs to be collected later. This magic does not come for free. So we'll have to design around the GC to write high performance code in C#.

We can achieve this by:
- Pooling Memory
- Pooling Objects
- Avoiding unnecessary copying
- Allocating on the stack instead of heap
- Using value types when appropriate

Here are some links for techniques you can use to tame the GC.
-
-
-


1. Blowfish

Code:
        public byte[] Encode(byte[] stream, int offset, int length)
        {
            if (length == 0)
                return null;

            var workspace = new byte[GetOutputLength(length)];

            Buffer.BlockCopy(stream, offset, workspace, 0, length);
            for (int x = length; x < workspace.Length; ++x)
                workspace[x] = 0;

            for (int x = 0; x < workspace.Length; x += 8)
            {
                uint l = BitConverter.ToUInt32(workspace, x + 0);
                uint r = BitConverter.ToUInt32(workspace, x + 4);
                Blowfish_encipher(ref r, ref l);
                Buffer.BlockCopy(BitConverter.GetBytes(r), 0, workspace, x + 0, 4);
                Buffer.BlockCopy(BitConverter.GetBytes(l), 0, workspace, x + 4, 4);
            }

            return workspace;
        }
If we look at the Encode method of the the Blowfish there are several issues.

1. Allocates a new buffer for the output (workspace).
2. Copies the input (stream) into the output (workspace).
3. Clears the array where no data has been copied over. (unnecessary as arrays are always default initialized in C#)
5. BitConverter.GetBytes allocates the result array which is then copied back into the output (workspace)

This could be improved by using unsafe pointers but .NET Core has brought us Span<T> so we can avoid a lot of these issues while writing safe code.

Span<T> allows us to access the same memory sequentially as any blittable data type with little to no overhead, thus we encipher without converting and copying back between parts of the byte[] and uints.

Code:
        internal int Transform(TransformMode transformMode, in ReadOnlySpan<byte> input, in Span<byte> output, int length)
        {
            var input32 = MemoryMarshal.Cast<byte, uint>(input);
            var output32 = MemoryMarshal.Cast<byte, uint>(output);

            int blockIndex = 0;
            for (int i = 0; i < length / 8; i++)
            {
                var left = input32[blockIndex + 0];
                var right = input32[blockIndex + 1];

                if (transformMode == TransformMode.Encode)
                    this.Encipher(ref left, ref right);
                else if (transformMode == TransformMode.Decode)
                    this.Decipher(ref left, ref right);

                output32[blockIndex + 0] = left;
                output32[blockIndex + 1] = right;

                blockIndex += 2;
            }
        }
final block transforming omitted for simplicity

With this method you can also easily encrypt into the the same buffer which gets rid of the output (workspace) allocation and copying input to output which is done by the transform anyway.


2. Packet

The Packet class is a nice general purpose class which you can use to read and write your packets data, no matter if massive or not, but not without flaws.
I don't have any fancy tables to prove my point but some of the write and read methods are rather slow. Further more the underlying MemoryStream in PacketWriter has a default buffer size of 256 and will grow when running out of space by allocating a new buffer that is double the size of the old buffer. The older buffer is also copied over to the new buffer. Spawn packets can easily break this threshold.

Thus it's probably not a good idea to pool a MemoryStream as you can't shrink it's growing buffer. So with enough time every pooled instance of a Packet could've been a spawn packet once and will have a rather large buffer wasting memory when the majority of the packets are small. But you can try to use the .

I'd suggest creating a Packet class that works on a fixed 4096 byte buffer which has been allocated from a Pool (ArrayPool<T> / MemoryPool<T>) and is living on an object pool as well. But since it can't grow there is no way to create a massive packet anymore. For this you'll need to use multiple regular packets and read and write across them thus it simply contains a list of packets.

3. Security
I intendet to write this section in more detail but it turned out more exhausting than expected.
In essence it comes down to a full rewrite as you'll need to carefully avoid copying and allocating where possible.
- Don't allocate a new list everytime you TransferOutgoing/TransferIncoming

With a fixed 4096 byte message you can:
- Receive directly into the the fixed message buffer.
- encrypt/decrypt into itself avoiding allocations and coyping found in Recv and FormatPacket.

Pointing out every small issue would make an obscene large list and it's not all purely performance focused.
The SecurityFlags class for example can be easily swapped with a [Flags] enum to improve readability and reduce complexity.

4. Other
I'm not saying you should all now go and try to rebuild SilkroadSecurityAPI but if you truly care about performance by all means give it a try.
But it won't magically solve all your issues as the Network and Threading design equally important.

Another problem with filters I've seen is that nobody seems to think about which data structures to use.
Things like List<T> for white/blacklists are easy to avoid where a Dictionary<TKey, TValue> or HashSet<T> is appropriate. Please educate yourself about the so you understand how to judge the complexity of such collections and choose accordingly.

It is so lucky me and this section to have your opinion, I don't need to tell how valuable in your information you've just wrote here, learning so much. Thanks for your knowledge.

I've read your SilkroadSecurity so many times and the Silkroad packet document. Thanks again for what you've done for us, the beginer.
vietnguyen09 is offline  
Thanks
1 User
Reply


Similar Threads Similar Threads
Is There Any Emulators for games systems out there
06/13/2009 - Consoles - 4 Replies
hi i am trying to find emulators for game systems so i can play the old nentindo and the n64 on my pc if u have any please post the site on this topic thank u
Is there any working hacks out there?!
01/05/2009 - WarRock - 0 Replies
I hav'nt found one hack that was working. Either they crash the whole thing or they just dont do anything. The only hack who does work (and it works excellent) is insano7's hack WRplus. BUT that hack cost, but according to me, you can't find a better hack and will never be able to. So, is there any free working hack who got more then no spread and no recoil (couse that is not much of a hack) feel free to pm it here, or post link to a good hack ty!
Is there any English FLYFF servers out there?
09/22/2008 - Flyff - 3 Replies
like topic says
is there any Agbot compatible auto buffer that switches weapons out there?
04/28/2008 - Silkroad Online - 0 Replies
just as the title says.
in botjail. is there any other way out other than.
08/04/2006 - Conquer Online 2 - 2 Replies
wondering if there is anything to get me out of bot jail without dbs. also ... is there a newer version for the hunt bot? the "cracked" one. i&#39;ve been banned for the second time now. or is there another bot out there like that one that tq wont detect?



All times are GMT +2. The time now is 05:03.


Powered by vBulletin®
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.
SEO by vBSEO ©2011, Crawlability, Inc.
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Support | Contact Us | FAQ | Advertising | Privacy Policy | Terms of Service | Abuse
Copyright ©2021 elitepvpers All Rights Reserved.